The Thing

[Norcalsteve]

"It's clobberin' time!!!" This is my official build thread for my Fantastic 4p Folding rig... first post will be updated with pics/info stats, and subsequest threads will deal with questions, comments and input from you and I.

Set up (prices will fluctuate over time):
SuperMicro H8qgl-If+ (Back order, comes from SuperMicro) - $628.00 (Awaiting arrival still, ordered on Jan 26th)
x4 Opteron 6174 - ~$1740when I got them
G.SKILL Ripjaws X Series 8GB pack (x4) - $239.96
Noctua NH-U9DO - $303.96
OS will be Native Linux with a Distro recommended by TC: went with Ubuntu 10.10 and [H]ardforum install, One stop shop to add Kraken/langouste with all the terminal codes too.
(Win Server for a day or two so I can do the Bench Marks for I.M.O.G and our team)

Total: ~$2912 for upgrade

Building into (already owned):
HAF 932 CoolerMaster tower
Modular SilverStone 1000Watt (Will upgrade eventually to an 80+ Gold Corsair)

Special thanks to ALL our community members that have helped direct me and school me in this 4p stuff... And especially to my Wife for her allowing me to pretend I can hang with the big boys... well, letting me spend money!

Lessons Learned: (will try to update as the thread grows so you dont have to dig!!)
-When Installing Ubuntu (what ever version, I used 10.10) ENSURE you use EXT3 for the format of the partion you install to. If you keep the default of EXT4, you WILL have write issues after the W/U is "...Done" after it compresses. It will litterally sit there for 1 1/2 hours, wasting PPD time!!

- Do not do an "Update" when you install, leave it "stock" as that can create supposed write issues. Currently I am experience a PPD loss due to load imbalance after re-installing to EXT3. I will update this line after the worker unit finished and I can test out ONLY UPDATING the Kernal from the one that came with 10.10. i.e. TPF pre EXT3 and original install, with all updates to include the Kernal was 14:08 on p6903. Now that I do not have the write issues and being on EXT3, I see my TPF on p6903 is 14:45... that could be due to the older Kernal.... Stay tuned for updated results (added lesson 5 March 2012)
- Update 6 Mar 2012: Updating Kernel does not do much... infact my TPF increased by 5-10sec, but that very well could be W/U variance.

- If you have this board, during setup, I noticed that the NUMA is enabled by Default, and Node Interleaving is disabled by default... In your settings, you can "load optimal settings" and you are good to go... If you are running TPC to monitor temps, you will have to DISABLE PowerNow... That is Enabled by default... Key points: Load Optimal settings, DISABLE PowerNow for the TPC install, then good to Fold!

-Top right CPU gets HOT... Figure a way to cool that sucker (pic below, with explanation caption, for what I currently did)... One thing you can do, and I should have done (or may still!!), If you have a Water Cooling system from your old rig but dont want to Water cool all 4 CPUs, Just water cool the top right CPU! Unless you think it would look to ugly The temp spread from coolest CPU to the hottest should stay within 10C... the 10C higher CPU is your top right CPU pictured below. Mine reached 17-20C before I rigged the 2 extra Noctua fans to the side!!!

Pics

The beginning


9 standoff mobo mounts used in total... I omitted the very top row


I REVERESED 3 of the top offsets and used the motherboard hold down screw, to screw the stand off to the case, then used a brass 6/32 nut on the screw that originally would screw to the case... next three pics will show that process




Almost there!!! Sleepy time now


Made it!!!



After a day of testing and burn in... Temps on the #2 processor SUCKED... a good 17-20C higher than the lowest CPU!!!! This is a TEMP set up of the extra fans... I used the clips that come with the Noctua as hangars for the two fans... they are both blowing to the back of the case. If I can swing it, I will mount them evenly between both heat sinks, getting a more direct flow to the back of the case. Either way, this set up the CPU temps range 35 to 45C on all CPUs rather 35-55! Untill I figure out "flonomics" of air, the right two CPUs will run a little hotter, as the static pressure with one fan on each of the sinks does not give the static pressure needed to move the air like the left two CPUs

['Cuda340]

Looking forward to it.

Good luck Steve

[harlam357]

Looking good Steve! Wish I could have scored my MCs for that price.

How many sticks of RAM did you get? Looks like 8... you'll need 16 for full memory bandwidth.

[Norcalsteve]

How many sticks of RAM did you get? Looks like 8... you'll need 16 for full memory bandwidth.

Thats a x4 pack of 2 gigs each for 8 gig's, 4 packs total, 16 sticks. Would be nice to only need 2 sticks in each slot on this board though... either way, i saved some money on the board.

[Macaholic]

Nom. Magnyfique!

[sniper_piper]

Appreciate the action figure. It's clobberin' time!

[capttripppp]

Ok, so I'm a little jealous! Looks to be something saweet

[ihrsetrdr]

Veeery nice!

[Adak]

At last.... Steve's 4P, has come along... His 1P days are over.... And Folding is like a song. I foresee ahead!

[Norcalsteve]

It will be nice not seeing anyone on my threat list again when I get it up and running... Until then...

[Norcalsteve]

Updated with the order date of the motherboard to give a rough estimate of transit time to arrival. I'll let ya know when I get it still...

[Leonardo]

It will be nice not seeing anyone on my threat list again when I get it up and running

Oh, just wait!

You'll blaze through the ranks for a while,
then settle into a lofty perch,
stretch out your arms and exclaim,
how wonderful the world really is,
looking down on all those below you fighting amongst themselves.
Then there will be the gentle knock,
followed by a bang; one intruder will pin you against the wall
as several others rush past.

And you will start thinking about the next upgrade.

But of course, I am speaking from personal experience.

Looking forward to pics of your build.

[Norcalsteve]

So poetic... I love it! I was gonna say something about that when I was typing, but yes, you are right... I am high enough in the team ranks to see any of our teammates build one or two of these, plus other rigs, and pass me. Plus I am sure some people will join the IB-E Xeon band wagon when the SR3 comes out... Then the true money baller's will come out and mop the floor...

[Jmtyra]

Very nice, sub'd! Added a new section to the 4P thread for "Build Logs" and added your thread.

[Norcalsteve]

Sweetness, should start seeing more updates within a week if my board comes in

[Posidon42]

The prices on the CPUs seems to be going up not down. Best price I can find for the 6174s are 490 a piece.

[dfonda]

Sweetness, should start seeing more updates within a week if my board comes in

I'm guessing we are in the same boat...everything but the Mobo...
My RAM and H/S 212+'s arrived today.
The only thing I have gotten is an order confirmation, very sad.

[Norcalsteve]

Did you order from the same place I did? There customer service is quick to respond, but have not gotten the tracking to me yet

[Robert17]

Very nice Steve. Yours makes mine feel like this:

[Leonardo]

Yours makes mine feel like this:

What, completely out of focus? Maybe we could recommend a couple good books.