Water Cooling a Workstation PC

[Iron_Planter]

Greetings all,

I'm building a workstation PC and installing a custom water cooling loop. I need some help figuring out how to liquid cool my Firepro s9100 series server GPU's (see components list for models). I have a request out to EK and αcool for custom cooling solutions. I wondering if the community has any suggestions. I've done some comparing to the R9 290 series and the W9000 series cards. Unfortunately, the s9000 series is longer and the mounting holes don't line up. I'm looking in to a processor only waterblock and sticking air cooling heatsinks on the HBM and VRM modules with something like the Kraken G12 shroud and a high CFM fan as a secondary solution. Before I pull the trigger on that I want to make sure I've exhausted my other options.

Components:

• Motherboard: ASRock ROMED8-2T
• CPU: AMD EYPC 7402P
• RAM: 2x eventually 8x NEMIX RAM 32GB Replacement for Samsung M393A4K40BB2-CTD DDR4-2666 ECC RDIMM
• 2x AMD Radeon Firepro s9150
• 2x AMD Radeon Firepro s9170
• PSU: EVGA SuperNOVA 1300 G2 120-G2-1300-XR 80+ GOLD 1300W

Cooling:

Positive pressure fan configuration:

• 3x Apevia 412L-RGB fans
• 2x Thermaltake TT-1425 fans

Custom water loop thus far:

• EKWB EK-CoolStream Classic SE 360 Slim Radiator
• Thermaltake A1 TT RGB DDR3/DDR4 Water Block CL-W239-CA00SW-A
• Alphacool 12950 Eisblock XPX Aurora PRO
• Alphacool 13324 Eisball Digital RGB - Plexi (incl. Eispumpe VPP755 V.3) Water Cooling Reservoirs
• Mayhems Ultra Clear Tubing (3/8 - 1/2) 10/13mm (soft) Tubing
• Compression Fittings
• Distilled Water
• GPU cooler: ?

Case:

• Thermaltake Level 20 HT CA-1P6-00F1WN-00

I'm hoping to get this build done by the end of July 2020.

Any help is greatly appreciated. I'll post build progress pictures as it progresses.

 

[maxfly]

I don't know anything about your gpus but ill do my best to help.

There are much better rads available. Eks se rads are average performers. Hardwarelabs and alphacool make very solid thinner rads in the 360x30mm range. Tbh if your going dual GPU i would look at 45-60mm rads. You may also want to look at adding a 240 or 280 to the loop. HWL sr2 and gtx rads are top of the heap but they are pricey.
Skip the memory blocks. Memory generally doesnt get hot enough to warrant wcing. Not to mention the blocks are extremely restrictive.
The xpx aurora pro seems decent but i havent seen any reviews. It may be wise to check out heatkiller, bitspower, ek and xspc cpu blocks. Epyc blocks arent my strong suit unfortunately.
I would go with the vpp655 vs the 755 pump if you can get the res by itself. The 755 was known to have many issues upon release. I wouldnt trust it(v3 may be improved but i cant say). Any d5 from a trusted brand like swiftech, xspc, ek, koolance etc. will work well in its place. Altho it doesnt look like the eisball comes as a standalone?
Mayhems tubing is solid but i only run primoflex lrt for soft tubing due to its resistance to leeching and fading over time. Its been bulletproof for me.
The best compression fittings seem to be bitspower but they are expensive. Alphacool,ek and barrow are solid as well. Xspc compression fittings a solid but their rotary fittings are junk.
Distilled with petras biocide(not the copper sulphate) is what i have run for ages problem free but there are solid premixed options as well. Distilled alone will inevitably result in nasties growing in your loop. Biocides are cheap insurance.
GPU block...obviously a full coverage block is going to perform best but i dont know where to get any for those cards. I think your on the right track in going with a universal block and kraken g12 shroud/fan but is there a block that will line up properly? Koolance might have a block that could work but im not certain.
Hope that helps and GL!

 

[maxfly]

Your in luck! 9100 blocks are available for pre-order from EK.

https://www.ekwb.com/shop/ek-pro-gpu-wb-wx9100-ni-inox

- - - Auto-Merged Double Post - - -

Available on the 23rd.

- - - Auto-Merged Double Post - - -

https://www.techpowerup.com/268763/ek-expands-professional-portfolio-with-new-water-blocks

 

[Zerileous]

Agree with most everything above.

You can also possibly run a a block without needing to use the Kraken adapter. https://www.overclockers.com/forums...My-New-Vega-56-Cooling-Mod?highlight=vega+mod. For cooling the VRM and VRAM, high CFM is not necessary, just airflow.

Regarding radiators, no single 360mm rad is going to be enough to cool that behemoth of a machine. A general rule is 1x120mm of radiator per 100W produced. Going by TDP alone (which may be an under-estimation), you've got a 180W CPU, 2x275W GPUs and 2x235W GPUs. This means you'll need to be able to handle 1200W. You may want to consider the Tower 900 as it has room for even larger radiators. Even then you'll be cutting it close with 2x 560mm rads. Really you should consider something like an external rad such as the Mo-RA or two (there is a newer version but it appears to be out of stock https://modmymods.com/watercool-mo-ra3-420-pro-stainless-steel-25121.html).

Is your system already purchased? I believe you will get much better performance and value out of threadripper, unless your application specifically benefits from 8 channels of memory vs 4 or 128 PCIe lanes vs the 48 of TR. I'm not sure but I bet the GPUs will do fine with 8 lanes each, leaving 16 left for storage. Again this is all speculative, I haven't really put time into the platform. The TR parts have higher base and boost clocks (and higher TDP). Just a thought. You would also need a 1600W PSU as the increased TDP of the TR parts will put you over 1300W.

edit: regarding the post about the 9100 blocks. It appears that the WX9100 is a 3 y/o vega-based card https://www.techpowerup.com/gpu-specs/radeon-pro-wx-9100.c2989 while the Firepro S 91X0 are older parts https://www.techpowerup.com/gpu-specs/firepro-s9150.c2608

Edit2: how do you handle video output? None of the cards have video out, nor does the motherboard / CPU as far as I know. Will this be run headless?

Edit3: I just saw these, might be a good alternative to the Mo-RA https://www.overclockers.com/alphacool-releases-five-new-radiators/

 

[Iron_Planter]

I appreciate the feedback!

@maxfly

• Thanks for the input. I'm definitely getting a biocide to deal with the possibility of growing, nasties, haha.
• I know the RAM blocks are totally unnecessary, but I like the aesthetic. If they restrict the flow too much I can always remove them.
• The aphacool eisball does come without a pump, but I opted for the combo due to pricing and hoping that the V3 fixed any issues with the pump. I may add a second pump for reliability.
• The ek wx9100 blocks are sadly for the workstation card variant which don't have the same PCB.

@Zerileous

• I wasn't sure that the single 360mm was enough for everything. After looking at your input I've considered replacing one of the glass side panels with a side panel rad something like https://www.alphacool.com/shop/radi...xt45-full-copper-1080mm-nova-radiator?c=20539 350 x 350 x 45 mm rad as the side panels on the HT 20 are about 410 x 480 mm.
• The core components of the system are already purchased.
• I spent a long time debating Threadripper vs Epyc. The GPUs should saturate a 16x PCIe 3 lane under full load. If I can split out the PCIe 4 to two 8x lanes I will probably double the amount of GPU's in the system. I went with PCIe 4 so I had more wiggle room for PCIe bifurcation and storage. Unfortunately I need the 128 PCIe lanes.
• Bifurcation Products:
  • http://www.ameri-rack.com/ARC2-PELY423-C7_m.html
  • http://www.ameri-rack.com/ARC2-733-Xm.html
  • https://www.amfeltec.com/minipcie-carriers/
• The 8 channels of RAM aren't necessary but will be nice for higher throughput.
• The MOBO has a vga output for initial configuration. The system will be run headless.

This system is being built as a dev environment for GPU accelerated data bases and machine learning. Adding it to an existing cluster to do heavy lifting. Hoping to virtualize a CAD workstation or two during the day and run compute jobs at night.
Also, if EK or alpahcool aren't interested in making custom waterblocks, someone from the GamersNexus discord is willing to give it a go if I grab/draft CAD files for the PCB. This is something I'm seriously considering if the big boys say no.