Reference Build: Z8NA-D6 "Computing" Rig

**Jmtyra** · 09-11-10, 10:57 PM

NOTE: This thread will see very *FEW* updates moving forward. Due to having zero overclocking ability, the board is NOT recommended vs. an SR-2 for best ppd/$$.

However, if you are looking for NO OC and are not concerned about performance/$$ then read on.

Latest news:
--> Price Drop! Newegg has the Z8NA-D6C (MIO) for $249 + S&H. (9/11/10) Thanks to pik4chu for the notice!
--> 'Open Box' Z8NA-D6 (w/Remote Mgmt) boards for $210 +S&H @ Newegg. (9/12/10)
--> Thread is being discontinued (10/14/10)
--> Updating PSU, CPU, Thermal, and Performance sections (10/14/10)

HAI!

Within the Folding/SETI/Rosetta team(s), there appears to be quite a bit of interest in the dual-socket 1366 Xeon 'Z8NA-D6C' board by ASUS. (Including the Z8NA-D6 as well.) My experience with this build is scattered across multiple threads, which isn't very helpful to those trying to glean information in our community. This is an attempt to consolidate and provide a single point of reference for those wanting to create a similar build.

ASUS's website: Z8NA-D6C and Z8NA-D6
Newegg's website: Z8NA-D6C and Z8NA-D6

Once the build is 100% complete, this thread will be less of a 'build log' and more of a 'reference build' to be made available for all distributed computing teams.

Starting out, I'll have this broken down to the following categories: CPUs, RAM, PSU, Cases, Thermal (heatsinks/fans/cooling), Performance (PPD and such).

Feel free to provide any feedback or suggestions!

Z8NA-D6C Baseline information and specs:

CPUs

List of supported CPUs from ASUS's site. NOTE the required BIOS version! As of this writing, Newegg is not shipping the latest BIOS! (9/11/10).
Temporary solution: pik4chu has found a vendor on eBay selling and shipping the chip quicker than ASUS. Just purchased one myself. /sad panda (9/11/10) Additional note: This online BIOS vendor does the trick.

Z8NA-D6:

Z8NA-D6C:

RAM

You CANNOT use "standard" desktop RAM on this board! Learn from my mistake!

Officially supported RAM "Qualified Vendor List": PDF Link to ASUS's website

Here are your RAM choice types:
(Note: 'Registered' and 'Buffered' are the same thing. Including both for completeness.)

ECC + Registered/Buffered
Non-ECC + Registered/Buffered
ECC + Non-Registered/Unbuffered
Non-ECC + Non-Registered/Unbuffered (I thought this was = 'normal' desktop memory. Apparently not.)

PSUs

If you have a PSU validated/used on this board please let me know

Corsair 750HX (Jmtyra)
NZXT Hale90 850W (Jmtyra)

Cases & Enclosures

This may be a very short section. Possibly removed if no interest.
These boards use the standard ATX form factor, so the options are virtually unlimited.

Things to note, however: Make sure you have sufficient airflow, as you have roughly twice the heat load of a standard single-socket system. Additionally, the Intel 5500 NB supposedly runs hot.

Enthusiast or "high airflow" cases should suffice.

Will verify NB heat range and dissipation once the rig is up and running.

Thermal: Heatsinks, Cooling, Fans, Etc

This board does not grant the luxury of space. The CPU sockets are right next to each other.

Confirmed to work:

Cooler Master H212+ (IF you ARE NOT using RAM w/heat spreaders) (Jmtyra)
Corsair H50 (Jmtyra)
Corsair H70 (Jmtyra)

Confirmed NOT to work:

Cooler Master H212+ (IF your ARE using RAM w/head spreaders) (Jmtyra)

Performance (PPD/kWh/$$$)

Using 2x Xeon E5620s, the rig pulls roughly 220w~225w continuously while folding
(No expansion cards, but using 4x HDDs; 2x 7.2k & 2x 10k HDDs)

F@H Numbers, pulled from our very own HFM.NET using v0.5.1 r198
Running SMP2 console v6.30 (Released: 2010-07-27) w/ "-bigadv -smp 16 -verbosity 9"

Code:

Project ID: 2685
Core: GRO-A3
Credit: 8955
Frames: 100

Number of Frames Observed: 70

Min. Time / Frame : 00:37:52 - 26,430.3 PPD
Avg. Time / Frame : 00:38:16 - 26,016.9 PPD

Number of Frames Observed: 44

Min. Time / Frame : 00:37:52 - 26,430.3 PPD
Avg. Time / Frame : 00:38:18 - 25,983.0 PPD

Code:

Project ID: 2686
Core: GRO-A3
Credit: 8955
Frames: 100

Number of Frames Observed: 12

Min. Time / Frame : 00:36:12 - 28,276.4 PPD
Avg. Time / Frame : 00:36:32 - 27,890.3 PPD
Cur. Time / Frame : 00:36:33 - 26,993.1 PPD
R3F. Time / Frame : 00:36:38 - 26,931.7 PPD
All Time / Frame : 00:36:32 - 27,005.4 PPD
Eff. Time / Frame : 00:38:58 - 25,319.0 PPD

Code:

Project ID: 2692
Core: GRO-A3
Credit: 8955
Frames: 100

Number of Frames Observed: 43

Min. Time / Frame : 00:34:11 - 30,815.2 PPD
Avg. Time / Frame : 00:36:07 - 28,374.3 PPD

Code:

Project ID: 6060
Core: GRO-A3
Credit: 481
Frames: 100

Number of Frames Observed: 100

Min. Time / Frame : 00:02:48 - 19,913.0 PPD
Avg. Time / Frame : 00:02:49 - 19,736.5 PPD

Code:

Project ID: 6061
Core: GRO-A3
Credit: 481
Frames: 100

Number of Frames Observed: 100

Min. Time / Frame : 00:02:52 - 19,222.4 PPD
Avg. Time / Frame : 00:02:54 - 18,892.0 PPD

Code:

Project ID: 6062
Core: GRO-A3
Credit: 481
Frames: 100

Number of Frames Observed: 35

Min. Time / Frame : 00:02:47 - 20,092.1 PPD
Avg. Time / Frame : 00:02:53 - 19,056.0 PPD

Code:

Project ID: 6077
Core: GRO-A3
Credit: 481
Frames: 100

Number of Frames Observed: 9

Min. Time / Frame : 00:02:55 - 18,730.3 PPD
Avg. Time / Frame : 00:03:06 - 17,093.5 PPD

The following was using two SMP clients both configured as follows:
Running SMP2 console v6.30 (Released: 2010-07-27) w/ "-bigadv -smp 8 -verbosity 9"

Code:

Project ID: 6701
Core: GRO-A3
Credit: 921
Frames: 100

Name: CPU0
Number of Frames Observed: 1

Min. Time / Frame : 00:12:06 - 5,930.5 PPD
Avg. Time / Frame : 00:12:06 - 5,930.5 PPD
Cur. Time / Frame : 00:12:06 - 4,774.8 PPD
R3F. Time / Frame : 00:00:00 - 0.0 PPD
All Time / Frame : 00:12:06 - 4,774.8 PPD
Eff. Time / Frame : 00:18:40 - 3,095.1 PPD 

Project ID: 6701
Core: GRO-A3
Credit: 921
Frames: 100

Name: CPU1
Number of Frames Observed: 1

Min. Time / Frame : 00:12:13 - 5,845.8 PPD
Avg. Time / Frame : 00:12:13 - 5,845.8 PPD
Cur. Time / Frame : 00:12:13 - 4,729.2 PPD
R3F. Time / Frame : 00:00:00 - 0.0 PPD
All Time / Frame : 00:12:13 - 4,729.2 PPD
Eff. Time / Frame : 00:18:40 - 3,095.1 PPD

Code:

Project ID: 6701
Core: GRO-A3
Credit: 921
Frames: 100


Name: CPU0
Number of Frames Observed: 10

Min. Time / Frame : 00:11:41 - 6,250.6 PPD
Avg. Time / Frame : 00:11:52 - 6,106.3 PPD


Name: CPU1
Number of Frames Observed: 10

Min. Time / Frame : 00:11:43 - 6,223.9 PPD
Avg. Time / Frame : 00:11:50 - 6,132.1 PPD

Backup Slot 1 of 3

Backup Slot 2 of 3

Backup Slot 3 of 3

**harlam357** · 09-12-10, 01:03 AM

Now THIS is what I'm talking about!!!

Can't wait to see the finished product!

**pik4chu** · 09-12-10, 01:54 AM

Me too! this is excellent. Makes me want to get my board ASAP so I can start contributing lol.

Might also be worth allowing people to post suggestions or questions on possible equipment/combination to be tested. Or make a sister thread for that to keep this one solely for information.

Also a note on your RAM comment. while you cannot use regular desktop memory for this board it does not have to be fully buffered or registered RAM, it just has to be ECC. allowing for cheaper solutions if you want to.
Such as, a 2*2Gig kit of DDR3 1333 ECC unbuffered is only ~$105 compared to ~$150 for fully buffered.

**harlam357** · 09-12-10, 02:01 AM

Exactly!!! I'm looking at a 3x2GB kit of Kingston Value RAM... ECC Unbuffered. I do suppose you have to have a set of RAM per populated CPU socket correct?

I would think so since the NB is on the CPU itself. So if I wanted to get one CPU/memory right off the bat I'd have to buy another set of "matching" memory for the second CPU when I get the funds. That's something to keep in mind for those of us looking to get into this on the "cheap".

Edit: As far as heatsinks go... a lot of folks seem to be using these: http://www.newegg.com/Product/Produc...82E16835203002

**pik4chu** · 09-12-10, 02:13 AM

Originally Posted by harlam357

Exactly!!! I'm looking at a 3x2GB kit of Kingston Value RAM... ECC Unbuffered. I do suppose you have to have a set of RAM per populated CPU socket correct?

I would think so since the NB is on the CPU itself. So if I wanted to get one CPU/memory right off the bat I'd have to buy another set of "matching" memory for the second CPU when I get the funds. That's something to keep in mind for those of us looking to get into this on the "cheap".

Edit: As far as heatsinks go... a lot of folks seem to be using these: http://www.newegg.com/Product/Produc...82E16835203002

you need at minimum the first slot populated with ram. and keep in mind you dont need a triple set for this board. it runs triple channel but you don't have to put in 3 sticks iirc. and I think that's true with most triple channel boards these days if I'm not mistaken.

I think this is what I'm looking at now. Going to go the fully registered route since this first server will also be my NAS and secondary domain controller. Going to see if I can get away with 4gigs to start. Another thing of note regarding RAM but if you choose unbuffered you are limited to 24gigs. Which does matter if you are wanting to run multiple VMs on the equipment.

Ya that heatsink is the most common choice since it what Intel says to use but people are also saying they are pretty noisy. And if I'm paying 30bucks for something like that Id prefer something more efficient or more powerful I think. hopefully we will get some good results from this thread. What I'm looking for first is the ppd, if the ppd looks good I'm moving forward ASAP and will help provide more data as well.

**Jmtyra** · 09-12-10, 02:30 AM

Originally Posted by harlam357

Now THIS is what I'm talking about!!!

Can't wait to see the finished product!

Still working on getting the pictures setup. Smugmug was not working out, so I'm getting my hosting tied to my domain to link the pictures via FTP (for upload). Hope you're not on dial-up!

Originally Posted by pik4chu

Me too! this is excellent. Makes me want to get my board ASAP so I can start contributing lol.

You're more than welcome to join in on the fun.

However, don't buy it from Newegg if you want to run the 5600 series or processors. I'm still dealing with that issue.

Originally Posted by pik4chu

Might also be worth allowing people to post suggestions or questions on possible equipment/combination to be tested. Or make a sister thread for that to keep this one solely for information.

I'm hoping they post here. That's why there's so many posts in this thread prior to the first reply, for the organization of data, along with a few 'spare' spots for info. I could be overly hopeful, but if so I guess it'll be a learning experience for me hehe.

Originally Posted by pik4chu

Also a note on your RAM comment. while you cannot use regular desktop memory for this board it does not have to be fully buffered or registered RAM, it just has to be ECC. allowing for cheaper solutions if you want to.
Such as, a 2*2Gig kit of DDR3 1333 ECC unbuffered is only ~$105 compared to ~$150 for fully buffered.

K -- I'll update it now. I originally purchased 'normal' DDR3, but now I'm running (or, rather, going to run) EEC buffered RAM.

Originally Posted by pik4chu

you need at minimum the first slot populated with ram. and keep in mind you dont need a triple set for this board. it runs triple channel but you don't have to put in 3 sticks iirc. and I think that's true with most triple channel boards these days if I'm not mistaken.

It's both slots closest to the CPUs, two slots total; one above and one below the CPU sockets. Dunno about other triple channel boards, but this one will work with just one stick per CPU socket.

Originally Posted by pik4chu

I think this is what I'm looking at now. Going to go the fully registered route since this first server will also be my NAS and secondary domain controller. Going to see if I can get away with 4gigs to start. Another thing of note regarding RAM but if you choose unbuffered you are limited to 24gigs. Which does matter if you are wanting to run multiple VMs on the equipment.

LOL, we think so alike it's scary. Once this is up and running, it will replace my current "low power" Hyper-V/NAS/WHS server. Accordingly, I'll be running ECC Buffered RAM. It's holding 8GB of it ATM, as I want to be on the safe side. Considering FAH + Hyper-V machines + OS + NAS + etc. I'm also trying to limit the number of headless 'folding only' boxes laying about. Combining uses and all that!

Originally Posted by pik4chu

Ya that heatsink is the most common choice since it what Intel says to use but people are also saying they are pretty noisy. And if I'm paying 30bucks for something like that Id prefer something more efficient or more powerful I think. hopefully we will get some good results from this thread. What I'm looking for first is the ppd, if the ppd looks good I'm moving forward ASAP and will help provide more data as well.

I'm in the same boat. While I don't mind Intel 'stock' coolers, I do mind them trying to mimic a shop vac. No thanks.

**pik4chu** · 09-12-10, 02:40 AM

Ya its a shame newegg has the board for cheap right now, have you seen any place that sells the board with the correct BIOS? I haven't really looked around for it yet.

*edit* what we need to do is like have an OCForums, community E55xx chip to pass around lol. Get an older chip to be used to flash the bios as I was googling around and looks like buy.com and amazon both have old BIOS version as well. There are some jackholes selling the free BIOS chips for $15-20 on eBay, which may sadly be worth it since it means you will get the chip quickly. Still pretty friggin lame.

**Jmtyra** · 09-12-10, 03:23 AM

LOL, a 'community' chip, exactly!

Too bad the cheapest ones that can be used on that board are still ~$200.

I may go with the eBay vultures. While ASUS is supposed to be giving free replacement chips, you have to twist their arm plus wait on hold and deal with their tech support. Just like any other company, tech support is a time sink. If you pay, it's like $10 for the chip and another $5 (or $10?) for shipping. So, it's the same as the eBay jerks. But, I can buy one of those hassle free and it'll be here when I get back home on Friday. (Ships from the same state!).

Edit: There goes $19.96 for a BIOS chip. Not sure if it's ASUS or Newegg I'm pissed at...Newegg not verifying what they are marketing, or ASUS for not having this on the original BIOS. Or both. Whatever, I just want this rig up and running. I have to catch sfu!

**pik4chu** · 09-12-10, 03:35 AM

Originally Posted by Jmtyra

LOL, a 'community' chip, exactly!

Too bad the cheapest ones that can be used on that board are still ~$200.

I may go with the eBay vultures. While ASUS is supposed to be giving free replacement chips, you have to twist their arm plus wait on hold and deal with their tech support. Just like any other company, tech support is a time sink. If you pay, it's like $10 for the chip and another $5 (or $10?) for shipping. So, it's the same as the eBay jerks. But, I can buy one of those hassle free and it'll be here when I get back home on Friday. (Ships from the same state!).

Edit: There goes $19.96 for a BIOS chip. Not sure if it's ASUS or Newegg I'm pissed at...Newegg not verifying what they are marketing, or ASUS for not having this on the original BIOS. Or both. Whatever, I just want this rig up and running. I have to catch sfu!

Ya I discovered the same thing, 200 bucks for a chip. Really need to find one used and it might be an option (like 100bucks or something). I think I will probably end up going the eBay route as well sadly. As for the epic failure Id say I blame Asus more since they are the ones who told Newegg what it will support but didn't make a note about BIOS, and then the BS you have to go through to get one. But some fault is still on Newegg for a) not verifying the support for the CPUs and b) not making note of it on the product page which is pretty major in my book.

**Jmtyra** · 09-12-10, 03:46 AM

Fair enough, I'll be pissed at both of them.

I think I'll stick it to Newegg...I'm buying one of their open box versions of the Z8NA-D6 (the more expensive one w/the remote management chip) for $210 vs the $290 I paid for the retail version. Once I get the open box and verify it works, I'll return the retail one as defective since, technically, it is. That'll more than make up for my $20 loss of having to buy a @#$(! BIOS off of eBay + lost time + irritation.

Wow, I feel better already!

Shopping cart -> buy -> CLICK!

**pik4chu** · 09-12-10, 04:06 AM

Been doing some even deeper digging about cooling for this board and came across a few tidbits.

This is from a review on the board itself.

Other Thoughts: Just wanted to add something to my previous review. Two Xigmatek S1283V Dark Knight coolers fit on this motherboard. I was using Dynatron T667s, but had to ditch them since the CPUs were running way too hot. With the Dark Knights the CPUs are cool and quiet and I'm am able to use the machine as a workstation.

So it looks like another confirmation for the dark knights I mentioned in the other thread so that is good.

There was another post that said they were able to get a pair of Cooler Master Hyper 212's with the 1366 adapter to fit the board.

Ill keep looking around for what I can but at this point there are more or less 3 groups of people that got this board

group 1: Used the regular old Intel stock HSF for $30 each and are ok with it
Group 2: used aftermarket ones with mixed success
Group 3: are completely pissed about the cramped board, the super hot NB and/or the BIOS bs, lol. Or got a board that was DOA or partially DOA.

**harlam357** · 09-12-10, 12:34 PM

Originally Posted by pik4chu

you need at minimum the first slot populated with ram. and keep in mind you dont need a triple set for this board. it runs triple channel but you don't have to put in 3 sticks iirc. and I think that's true with most triple channel boards these days if I'm not mistaken.

I think this is what I'm looking at now. Going to go the fully registered route since this first server will also be my NAS and secondary domain controller. Going to see if I can get away with 4gigs to start. Another thing of note regarding RAM but if you choose unbuffered you are limited to 24gigs. Which does matter if you are wanting to run multiple VMs on the equipment.

Ya that heatsink is the most common choice since it what Intel says to use but people are also saying they are pretty noisy. And if I'm paying 30bucks for something like that Id prefer something more efficient or more powerful I think. hopefully we will get some good results from this thread. What I'm looking for first is the ppd, if the ppd looks good I'm moving forward ASAP and will help provide more data as well.

Cool... you probably just changed my mind and sent me into the fully registered memory route.

My plan for my version of this box, beside the FAH use, will be use as a NAS. I have a ~~4x750GB~~ (I meant) 4x640GB Matrix RAID-5 array in the box (case I plan to use) now powered by a P35/ICH9R setup. That array is coming over to this new platform as is... data in tact.

I haven't seen much talk about PSUs yet... what are your plans... both of you? I have either a 650TX or 750TX here that I could swap in. I don't plan on running any GPUs in this board, purely CPU, so I don't need the extra PCI-E leads on the 750TX, but thought I might need the extra power/headroom.

My biggest concern about my build will be cooling. The case setup just won't allow me to use the taller, 120mm fan, based heatsinks. I need something smaller in the range of a 92mm fan. Any suggestions?

Edit: Just found this one (based on 92mm fan): http://www.heatsinkfactory.com/noctu...xeon-5500.html

I, like you, am waiting to see what kind of PPD this baby will generate. Once I see a few WUs benchmarked and things look good I'm straight into the buyers seat too and will help with providing more data as well. It's really killing me... I want to go ahead and buy soooooo bad...

**ihrsetrdr** · 09-12-10, 01:06 PM

Ryan,

My guess is that since it will be running @stock, any HSF matching stock specs will be sufficient.

**pik4chu** · 09-12-10, 01:28 PM

Originally Posted by harlam357

Cool... you probably just changed my mind and sent me into the fully registered memory route.

My plan for my version of this box, beside the FAH use, will be use as a NAS. I have a 4x750GB Matrix RAID-5 array in the box (case I plan to use) now powered by a P35/ICH9R setup. That array is coming over to this new platform as is... data in tact.

I haven't seen much talk about PSUs yet... what are your plans... both of you? I have either a 650TX or 750TX here that I could swap in. I don't plan on running any GPUs in this board, purely CPU, so I don't need the extra PCI-E leads on the 750TX, but thought I might need the extra power/headroom.

My biggest concern about my build will be cooling. The case setup just won't allow me to use the taller, 120mm fan, based heatsinks. I need something smaller in the range of a 92mm fan. Any suggestions?

Edit: Just found this one (based on 92mm fan): http://www.heatsinkfactory.com/noctu...xeon-5500.html

I, like you, am waiting to see what kind of PPD this baby will generate. Once I see a few WUs benchmarked and things look good I'm straight into the buyers seat too and will help with providing more data as well. It's really killing me... I want to go ahead and buy soooooo bad...

In regards tp PSU I have an OCZ 750W or so or perhaps a Corsair 720W around that I will be using to start with. For the case I probably will just pick up another antec 300 or an HAF692 locally (microcenter deals ftw) I would really like to start off with the norco 4xxx but I would rather spend that money on getting the rig setup initially than on the case and it will be some time before I expand past 13 drives anyways. Plus once I have the NAS built and if Im happy with the ppd then I will be getting a second one to replace my primary server with as well. Beyond that however I think I will be following sfu's path for the farm side of it and go for cheap i7 rigs with multiple gts 240/250/460s in them for maximum ppd per case footprint. And the heat will also be less of an issue for those as I will have a dedicated room for rigs (I didnt even ask, the g/f suggested it lol, god I love her

) It will be slightly slow going at first though since there are some house expenses but the first server will happen very soon once we get some ppd confirmation from jmtyra.

In regards to the HSF, I will be buying one of the intels and one of the black knights to try out as those are the HSF Ive seen people say work the most (3 mentions of the Xigmatecs, about 20 of the intels)

**Jmtyra** · 09-13-10, 10:35 PM

Quick note -- not forgotten, crazy work/travel week. Hope to be back home late Friday/mid-Saturday; will have the eBay BIOS and 'backup' board to ~~throw together~~ finish this rig. From there I'll do a single WU for ppd/kWh/$$ and post results accordingly.

Sorry for the delay...will respond when I can.

**harlam357** · 09-13-10, 11:15 PM

Tis ok... I've calmed down now.

I'm trying to sell a couple GTX260-216s here in the classies (shameless plug) to help pull together the funds for my build. So I'm going to try to wait until they sell. Giving it this week at least... plus it's the GFs birthday next week, so I gotta budget for that too.

**Jmtyra** · 09-18-10, 11:03 PM

Quick update (and teaser). Got the 'open box' board, had bent CPU pins so it's going back. F that. Received the eBay BIOS ~4hrs ago via USPS, and got to work. Should have some numbers soon, but it'll be a bit as it looks like a tornado and Radio Shack hit my room right now. Plus I'm flying back in ~11hrs, so, ya.

Just grabbed the kill-a-watt and the latest SMP client. -bigadv -smp -verbosity 9 just got started. Pulling 220w AC (external), and that's with 2x H50s + 2x Velociraptors + 3x 1.5TB HDs. So, the ppd/kWh looks very promising.

Any-who, didn't want anyone to think I forgot. Just busy as h3ll I promise!

**harlam357** · 09-18-10, 11:33 PM

WHAT!?!?!? NO PPD!?!?!?!? You're killing me bro!!!

**Jmtyra** · 09-18-10, 11:50 PM

Setup HFM.NET to pull stats for those interested. It's showing a TPF of 3m05s, downloaded/started 53m ago with an ETA of 4h25m. Roughly ~5.5h/WU? Project P6077 (R0, C171, G74). (Including a few reboots for configuration and such).

Anyone know of a FAH calculator we can use to "guesstimate" the rough PPD for that TPF? Then we can extrapolate the PPD/kWh.

K -- I have to pack and clean up the mess.

**Jmtyra** · 09-18-10, 11:51 PM

Originally Posted by harlam357

WHAT!?!?!? NO PPD!?!?!?!? You're killing me bro!!!

Sorry it's the first run...machine was only up for 10 minutes.

Just posted more info though.

Thread: Reference Build: Z8NA-D6 "Computing" Rig

Thread Tools

Search Thread

Posting Permissions