Twenty cores on the cheap - earlier-gen Xeon build

[David]

So for work I carry my laptop back and forth, but it's not quite got the power I need for some jobs. I find myself doing a lot more computationally-intensive work these days.

I decided to build a new workstation for the office. The aim was to strike a good balance between horsepower and cost.

With that in mind, I perhaps took a bit of a risk and got a dual socket X99 board off ebay for ~£100. I then managed to get my hands on the following:

• Two E5-2660v3 chips (2.6 GHz, 20 cores), for £70 each
• 64 GB (4 x 16 GB) DDR4 ECC RDIMMs for £140 total
• AMD WX2100 for £90

I bought the following stuff new:

• 2x Noctua NH U9-DX for £120
• 512 GB Gigabyte SSD for ~£100

I already had a case and PSU that were unused (Antec Dark Fleet unit that I reviewed many years ago, but that comes with quite a beefy 1kW PSU).

Total cost is about £600 - £650.

Currently setting it up. Seems to work fine, although the motherboard came with no manual, no CMOS battery, and not even a little note about the front panel pin layout!

I've just put Ubuntu 20 on it. Next step is to do some stress testing and see what temperatures are like.

David

 

[bigtallanddopey]

You can always get old server/workstation parts on the cheap as companies factor the price in over something like over 5 years. Once the 5 years is up they just get rid of them and buy new. I picked up my old workstation, 8 core Xeon and 16GB ram for £50 from my work. They took the HDD out but that’s all I had to provide, now a nice little home sever.

It’s crazy sometimes as the 8 core Xeon with 32GB of ram workstation that replaced it cost 5k.

 

[David]

Yeah, depreciation on this kind of kit is crazy. About 18 months ago I bought a 2x12 core Xeon Gold / 96 GB RAM / 1 TB SSD machine and it was about £8k. It's undoubtedly a fair bit faster than the machine I've just built - in terms of # of cores, clockspeed, and IPC - but it's also >10x the price! This rig should do fine for a desktop that sees some heavy loads.

At the moment I've got folding@home running - it's all running on the bench, open to the air, but one CPU is at a little over 50 deg. C and the other is around 60 deg. C. The coolers were a very tight fit though - I can't get both fan clips on so they're sitting slightly squint with one clip per fan. I think I can get some cable ties and cable tie the two fans on each side together to stop them slipping off.

 

[EarthDog]

Nice build... and cheap!

I believe those xeon's are sandybridge-EP based from around 2012. I don't think you'd be able to get more for less in modern(ish) processors, but it's likely a single 5900x or 10900k similar beats it since IPC and clockspeeds are a lot higher. That said, itself would have cost you double+ to build that. Well done!

 

[DaveB]

Nice build... and cheap!

I believe those xeon's are sandybridge-EP based from around 2012. I don't think you'd be able to get more for less in modern(ish) processors, but it's likely a single 5900x or 10900k similar beats it since IPC and clockspeeds are a lot higher. That said, itself would have cost you double+ to build that. Well done!

They are Haswell-EP from 2014.

 

[David]

I believe those xeon's are sandybridge-EP based from around 2012. I don't think you'd be able to get more for less in modern(ish) processors, but it's likely a single 5900x or 10900k similar beats it since IPC and clockspeeds are a lot higher. That said, itself would have cost you double+ to build that. Well done!

I was imprecise in the first post - they are 2660v3 chips, so late 2014. They have the AVX2 extensions (vs AVX) which make a difference to some software I run. They are also 10 core rather than 8 core chips.

But yes, I reckon a 5950X would give it a good run for its money!

 

[EarthDog]

They are Haswell-EP from 2014.

The first post originally said "e5-2660"... Sandybridge EP is what that returns (see below). He has now clarified and edited the first post.
https://ark.intel.com/content/www/u...0-20m-cache-2-20-ghz-8-00-gt-s-intel-qpi.html

...the updated v3 is Haswell-EP, correct... though if I paid attention to the clock speeds, I probably would have caught that.

Regardless, it doesn't change the content of my post. It's a cheap build that rivals today's horsepower for a lot less.

 

[David]

Brief update:

• Switched to Debian Sid, because I like to live dangerously. Still need to finish setting some software up, including some specialist software (TopSpin, Mercury, Gaussian, Gaussview, Orca, AIMAll etc.) that isn't available through apt.
• It's now in the Antec Dark Fleet case. Under my desk, under full load (20 cores FaH) the chips sit at ~47 degrees C. Very impressive. I don't know if this is just the type of chip or if the coolers are just very, very good. My other two machines run much hotter - the E5645s sit at ~75 degrees C under load, and the 6136G chips at 80 - 90 degrees C. The latter are using the standard cooling solutions that came with the Precision towers, although the E5645s were swapped in by me so the thermal paste job is up to scratch. The other machine is new enough that I don't want to risk poking around inside it yet.
• The USB3 ports on the board don't kick in until the OS loads, so I was having issues getting into the BIOS when the keyboard was in the USB3 ports.

 

[ihrsetrdr]

Brief update:

Switched to Debian Sid, because I like to live dangerously.

Lol, I remember someone over on the Debian forums making the comment: "nice thing about running unstable is that if it breaks, you get to keep the pieces".

 

[habbajabba]

Nuclear Magnetic Resonance imaging and Computational Chemistry Using the Quantum Theory of Atoms in Molecules.
Finally, someone is using the video card for something other than gaming.noice

 

[David]

Nuclear Magnetic Resonance imaging and Computational Chemistry Using the Quantum Theory of Atoms in Molecules.
Finally, someone is using the video card for something other than gaming.noice

All on the CPU I'm afraid. The types of calculation I do are not well implemented on GPUs yet, although that might change.

I have some colleagues who do coarse-grain molecular dynamics simulations (so not far from the types of theory that Folding@Home uses) and these absolutely scream on GPUs. They're upgrading the University cluster with some new GPU nodes next month - apparently we're getting 8 x nvidia A100 (ampere) GPUs.

 

[habbajabba]

Awesome. This is why computers are made and why spending thousands is worth it right now.
I like Debian myself (MX). It has no system.d by default and so far has been good to me. I'm currently looking at gettin a wifi6 card from qnap for MX just cause they're pretty cheap but since the pc is right near the router it's rather not needed atm. PC's of the future will literally have organic components and make the fastest ones we have today look like they're standing still. Cluster computing is the best to be had right now but that will change. GPU/CPU schmeepeeyou later.