6 GPU Folding Rig Build

[torin3]

Finally installed the new motherboard today. After a couple of reboot cycles, I'm still only getting 2 GPUs.

......


Then I went into the BIOS and set the PCI-E settings to where they looked like all the slots should be active.

Aaannndd....

 

[harlam357]

Nice Torin! Playing catch up... is this half of the target? Three more GPUs to come?

PPD on that 3080 Ti looks a bit low. Just a low producing WU?

 

[torin3]

Nice Torin! Playing catch up... is this half of the target? Three more GPUs to come?

PPD on that 3080 Ti looks a bit low. Just a low producing WU?

Yep, planning on adding more. Just need to get the 2nd PSU Y-splitter and a few more riser cables. Figure full power in a week or two. I'm planning on adding 1 card at a time and making sure it works well.

Low PPD was due to the computer being off while I was switching the motherboards.

 

[WhitehawkEQ]

WU 18445 is a low point WU anyway, I get about 4mil+ in windows with that WU on my 3080's.

 

[torin3]

I have been too busy to add more cards, but I just came out to the garage, and saw this, so it does look like I'm getting normal production for all the cards.

 

[WhitehawkEQ]

 

[harlam357]

Yeah, that seems a little slow for that WU torin. Here's what my 3070 Ti does on P18213.

 Name: Ryzen 5800X Slot 02
 Proc: GPU / GeForce RTX 3070 Ti
 Number of Frames Observed: 300

 Min. Time / Frame : 00:01:10 - 5,878,103.0 PPD
 Avg. Time / Frame : 00:01:14 - 5,407,999.0 PPD

 

[torin3]

I haven't updated this much since I've been very disappointed. You need x8 data width to fold with the current generation of cards without loosing PPD. If I try and boot up with more than 4 cards at x8 data width, this is what I see:



Oh, and I passed Macaholic finally, and I'm now in the #3 position for the team for overall points.

 

[HayesK]

This is a long shot, but if all the slots on cpu-0 are occupied and the resource test occurs before the x16/x8 slot auto switches to x8, then 48 lanes would be required. Manually setting the x16 slots to x8 might help, but could still be too late. May need to avoid using one of the slots on CPU-0.


five slots:
CPU-0, slots 5+7 at x16 and slot 2 at x8. Uses 40 lanes. (slot 6 not used)
CPU-1, slots 1+4 at x16. Uses 32 lanes. (slot 3 not used)

six slots:
CPU-0, slots 5+7 at x16 and slot 2 at x8. Uses 40 lanes. (slot 6 not used)
CPU-1, slot 1 at x16 and slots 3+4 at x8. Uses 32 lanes.

Alternates:
CPU-0 slot 5 at x16 and slots 6+7 at x8. Uses 32 lanes. (slot 2 not used)
CPU-0 slot 5 at x16 and slots 2+6 at x8. Uses 32 lanes. (slot 7 not used)
CPU-0 slots 2+6+7 at x8. Uses 24 lanes. (slot 5 not used)



EP2C612 WS
PCIe Expansion Slots (SLOT7 close to CPU)
PCIe x 16
- SLOT7: PCIe3.0 x16, auto switch to x8 when SLOT6 is occupied [CPU0]
- SLOT6: PCIe x16 (PCIe3.0 x8) [CPU0]
- SLOT5: PCIe3.0 x16 [CPU0]
- SLOT4: PCIe x16 (PCIe3.0 x8) [CPU1]
- SLOT3: PCIe3.0 x16, auto switch to x8 when SLOT4 is occupied [CPU1]
- SLOT2: PCIe x16 (PCIe3.0 x8) [CPU0]
- SLOT1: PCIe3.0 x16 [CPU1]

E5-2650 v4
PCI Express Revision 3.0
PCI Express Configurations ‡ x4, x8, x16
Max # of PCI Express Lanes 40

 

[superducky]

Unfortunately Haswell CPU's won't auto RE-BAR. This is a platform limitation that you won't be able to work around without decompiling your own bios. NVIDIA cards are notoriously bad about requiring a bunch of RAM below 4G in the BAR.

 

[WhitehawkEQ]

BAR, RE-BAR? What is this BAR, RE-BAR you speak of? Me not knowing this.

 

[dfonda]

BAR, RE-BAR? What is this BAR, RE-BAR you speak of? Me not knowing this.

Resizable BAR essentially makes the entirety of the graphics frame buffer accessible to the CPU at once; where it could once sip, it now guzzles. The idea is that once textures, shaders and geometry are loading in faster, games should run faster with higher frame rates

To enable Resizable BAR, you can enter your motherboard's BIOS menu (usually by pressing a key like Delete at boot), and enabling the Resizable BAR setting—you may have to hunt around to find it. If you see an Above 4G Memory setting, enable that as well. Be sure to save your changes when you exit.