- Joined
- Sep 19, 2009
- Location
- Crestview, FL
"It's clobberin' time!!!" This is my official build thread for my Fantastic 4p Folding rig... first post will be updated with pics/info stats, and subsequest threads will deal with questions, comments and input from you and I.
Set up (prices will fluctuate over time):
SuperMicro H8qgl-If+ (Back order, comes from SuperMicro) - $628.00 (Awaiting arrival still, ordered on Jan 26th)
x4 Opteron 6174 - ~$1740when I got them
G.SKILL Ripjaws X Series 8GB pack (x4) - $239.96
Noctua NH-U9DO - $303.96
OS will be Native Linux with a Distro recommended by TC: went with Ubuntu 10.10 and [H]ardforum install, One stop shop to add Kraken/langouste with all the terminal codes too.
(Win Server for a day or two so I can do the Bench Marks for I.M.O.G and our team)
Total: ~$2912 for upgrade
Building into (already owned):
HAF 932 CoolerMaster tower
Modular SilverStone 1000Watt (Will upgrade eventually to an 80+ Gold Corsair)
Special thanks to ALL our community members that have helped direct me and school me in this 4p stuff... And especially to my Wife for her allowing me to pretend I can hang with the big boys... well, letting me spend money!
Lessons Learned: (will try to update as the thread grows so you dont have to dig!!)
-When Installing Ubuntu (what ever version, I used 10.10) ENSURE you use EXT3 for the format of the partion you install to. If you keep the default of EXT4, you WILL have write issues after the W/U is "...Done" after it compresses. It will litterally sit there for 1 1/2 hours, wasting PPD time!!
- Do not do an "Update" when you install, leave it "stock" as that can create supposed write issues. Currently I am experience a PPD loss due to load imbalance after re-installing to EXT3. I will update this line after the worker unit finished and I can test out ONLY UPDATING the Kernal from the one that came with 10.10. i.e. TPF pre EXT3 and original install, with all updates to include the Kernal was 14:08 on p6903. Now that I do not have the write issues and being on EXT3, I see my TPF on p6903 is 14:45... that could be due to the older Kernal.... Stay tuned for updated results (added lesson 5 March 2012)
- Update 6 Mar 2012: Updating Kernel does not do much... infact my TPF increased by 5-10sec, but that very well could be W/U variance.
- If you have this board, during setup, I noticed that the NUMA is enabled by Default, and Node Interleaving is disabled by default... In your settings, you can "load optimal settings" and you are good to go... If you are running TPC to monitor temps, you will have to DISABLE PowerNow... That is Enabled by default... Key points: Load Optimal settings, DISABLE PowerNow for the TPC install, then good to Fold!
-Top right CPU gets HOT... Figure a way to cool that sucker (pic below, with explanation caption, for what I currently did)... One thing you can do, and I should have done (or may still!!), If you have a Water Cooling system from your old rig but dont want to Water cool all 4 CPUs, Just water cool the top right CPU! Unless you think it would look to ugly The temp spread from coolest CPU to the hottest should stay within 10C... the 10C higher CPU is your top right CPU pictured below. Mine reached 17-20C before I rigged the 2 extra Noctua fans to the side!!!
Pics
The beginning
9 standoff mobo mounts used in total... I omitted the very top row
I REVERESED 3 of the top offsets and used the motherboard hold down screw, to screw the stand off to the case, then used a brass 6/32 nut on the screw that originally would screw to the case... next three pics will show that process
Almost there!!! Sleepy time now
Made it!!!
After a day of testing and burn in... Temps on the #2 processor SUCKED... a good 17-20C higher than the lowest CPU!!!! This is a TEMP set up of the extra fans... I used the clips that come with the Noctua as hangars for the two fans... they are both blowing to the back of the case. If I can swing it, I will mount them evenly between both heat sinks, getting a more direct flow to the back of the case. Either way, this set up the CPU temps range 35 to 45C on all CPUs rather 35-55! Untill I figure out "flonomics" of air, the right two CPUs will run a little hotter, as the static pressure with one fan on each of the sinks does not give the static pressure needed to move the air like the left two CPUs
Set up (prices will fluctuate over time):
SuperMicro H8qgl-If+ (Back order, comes from SuperMicro) - $628.00 (Awaiting arrival still, ordered on Jan 26th)
x4 Opteron 6174 - ~$1740when I got them
G.SKILL Ripjaws X Series 8GB pack (x4) - $239.96
Noctua NH-U9DO - $303.96
(Win Server for a day or two so I can do the Bench Marks for I.M.O.G and our team)
Total: ~$2912 for upgrade
Building into (already owned):
HAF 932 CoolerMaster tower
Modular SilverStone 1000Watt (Will upgrade eventually to an 80+ Gold Corsair)
Special thanks to ALL our community members that have helped direct me and school me in this 4p stuff... And especially to my Wife for her allowing me to pretend I can hang with the big boys... well, letting me spend money!
Lessons Learned: (will try to update as the thread grows so you dont have to dig!!)
-When Installing Ubuntu (what ever version, I used 10.10) ENSURE you use EXT3 for the format of the partion you install to. If you keep the default of EXT4, you WILL have write issues after the W/U is "...Done" after it compresses. It will litterally sit there for 1 1/2 hours, wasting PPD time!!
- Do not do an "Update" when you install, leave it "stock" as that can create supposed write issues. Currently I am experience a PPD loss due to load imbalance after re-installing to EXT3. I will update this line after the worker unit finished and I can test out ONLY UPDATING the Kernal from the one that came with 10.10. i.e. TPF pre EXT3 and original install, with all updates to include the Kernal was 14:08 on p6903. Now that I do not have the write issues and being on EXT3, I see my TPF on p6903 is 14:45... that could be due to the older Kernal.... Stay tuned for updated results (added lesson 5 March 2012)
- Update 6 Mar 2012: Updating Kernel does not do much... infact my TPF increased by 5-10sec, but that very well could be W/U variance.
- If you have this board, during setup, I noticed that the NUMA is enabled by Default, and Node Interleaving is disabled by default... In your settings, you can "load optimal settings" and you are good to go... If you are running TPC to monitor temps, you will have to DISABLE PowerNow... That is Enabled by default... Key points: Load Optimal settings, DISABLE PowerNow for the TPC install, then good to Fold!
-Top right CPU gets HOT... Figure a way to cool that sucker (pic below, with explanation caption, for what I currently did)... One thing you can do, and I should have done (or may still!!), If you have a Water Cooling system from your old rig but dont want to Water cool all 4 CPUs, Just water cool the top right CPU! Unless you think it would look to ugly The temp spread from coolest CPU to the hottest should stay within 10C... the 10C higher CPU is your top right CPU pictured below. Mine reached 17-20C before I rigged the 2 extra Noctua fans to the side!!!
Pics
The beginning
9 standoff mobo mounts used in total... I omitted the very top row
I REVERESED 3 of the top offsets and used the motherboard hold down screw, to screw the stand off to the case, then used a brass 6/32 nut on the screw that originally would screw to the case... next three pics will show that process
Almost there!!! Sleepy time now
Made it!!!
After a day of testing and burn in... Temps on the #2 processor SUCKED... a good 17-20C higher than the lowest CPU!!!! This is a TEMP set up of the extra fans... I used the clips that come with the Noctua as hangars for the two fans... they are both blowing to the back of the case. If I can swing it, I will mount them evenly between both heat sinks, getting a more direct flow to the back of the case. Either way, this set up the CPU temps range 35 to 45C on all CPUs rather 35-55! Untill I figure out "flonomics" of air, the right two CPUs will run a little hotter, as the static pressure with one fan on each of the sinks does not give the static pressure needed to move the air like the left two CPUs
Last edited: