Torin3's 4P F@H build

[torin3]

I found the dead stick. But now it shuts off within 2 minutes of starting folding. I was running the temp command every few seconds and nothing was above 28°C when it shut off.

I'll leave it off until I get the new memory added in tomorrow.

 

[orion456]

I found the dead stick. But now it shuts off within 2 minutes of starting folding. I was running the temp command every few seconds and nothing was above 28°C when it shut off.

I'll leave it off until I get the new memory added in tomorrow.

I would put 2 sticks in each bank and run memtest for 24 hours. A lot of times it takes a while for memtest to identify random errors.

If the 2 sticks work, I would try to fold on 2 sticks, thats plenty of memory for FAH to work.

I assume these aren't matched sticks. That alone can cause problems with small timing errors.

 

[torin3]

The memory did arrive today! Whoo-hoo!

Also, I got a 6pin PCI-E to 8pin EPS converter. I've been using a 2 molex to 8pin EPS converter, but I've read that it may have problems, and it is better to use a PCI-E converter instead.

And while we are supposed to be getting 6-12" of snow today, I'll be picking up fresh batteries for my 1500VA UPS on the way home.

Hopefully, this will resolve everything.

 

[dfonda]

Nothing like good steady power to cure your ills!

 

[torin3]

Well, I think I'm good to go.

I've got 32GB of memory in the blue slots. The 8pin EPS plugs are connected, one directly and one with an adapter from a 6pin PCI-E plug. The UPS has fresh batteries. And I'm pulling about 520W from the wall on a 1KW PSU. The CPUS are running between 22°C and 27°C after 10% of an 8104 WU.

Now the two things that I think were causing my problems.

1, the drive the folding appliance was running from died right after I put the RAM in. BIOS no longer sees it. That may have been my entire problem. Failing drive crashing. I've replaced it with another drive and a fresh install of the folding appliance. Only weird thing is I seem to be unable to change the password. When I get a chance, I'll make a new account and run from that.

2, when I had less than 16 sticks of RAM, this is what part of the BIOS boot sequence looked like:



And now that it does have 16 sticks, it looks like this:



The only thing I'm wondering is why the different speeds? I'm assuming this is RAM speeds, and all the sticks are 1600 and faster.

But, I'm running at about 490K ppd for the 8104, and almost 700K ppd total.

I'll give it a week, then I'm going to mine with the video cards for a month or so to buy a new card, or maybe two. That should get me to nearly 1M ppd.

 

[ihrsetrdr]

Looking reall good Torin!

I'm using the folding appliance that they built out of Ubuntu over at [H].

http://hardforum.com/showthread.php?t=1769083

Nice and easy to set up with good instructions.

I'm a little behind the times- have not heard of this appliance. I liked Notfred's, but he discontinued his project for some reason.

 

[WhitehawkEQ]

And now that it does have 16 sticks, it looks like this:



The only thing I'm wondering is why the different speeds? I'm assuming this is RAM speeds, and all the sticks are 1600 and faster.

Is the ram non-ECC Registered mem? If so then it won't run at 1600MHZ, I have this Server ram 8GB kit (4x2GB) from Kingston and it runs at 1600MHZ in the MB.

 

[torin3]

Is the ram non-ECC Registered mem? If so then it won't run at 1600MHZ, I have this Server ram 8GB kit (4x2GB) from Kingston and it runs at 1600MHZ in the MB.

I'll have to look through the manual again, but I didn't pick up my first time through that ECC Registered would do 1600. I was aware that my RAM would only do 1333 in this board. But what I was saying I was surprised at was the ones doing 1067.

Thank you for the heads-up!

 

[WhitehawkEQ]

I'll have to look through the manual again, but I didn't pick up my first time through that ECC Registered would do 1600. I was aware that my RAM would only do 1333 in this board. But what I was saying I was surprised at was the ones doing 1067.

Thank you for the heads-up!

NP I had gotten 32GB of non-ECC Registered Desktop ram and max speed I got was 1333 MHZ, I was having other problems to but soon as I put the server ram in, it ran at 1600.

 

[OptyTrooper]

That beast put my 2P 12 core system to shame. Nice set-up!

 

[torin3]

Well, it has gone through two WU without any problems. An 8104 and an 8103.

That is a very nice 24 hour point total!

I'm currently running an 8101 and it is quite a bit slower.

I was getting about 8 minutes per frame with the 8104 and about 490Kppd.
I was getting about 11 minutes per frame with the 8103 and about 470Kppd.
With the 8101, I'm getting about 14 minutes per frame with about 300Kppd.

So, I'm wondering, for Bigadv, does RAM speed make a lot of difference with ppd? And if so, could I run regular RAM and ECC-Registered on the same board, as long as I only have one type or the other per CPU bank?

I will be flashing the BIOS to the overclockable one in the next week or two.

 

[Adak]

Check your mobo manual of course, but you can also test it yourself. Stop FAH, go into the BIOS, and switch your memory settings, from interleaved to non interleaved, and restart FAH. Compare your folding times now, with what you had previously. (You must have all the same type of memory installed, to test this.)

I've always found interleaved to be the best, for any computer, including the 4P, but you have the most recent edition of the mobo and BIOS, so your optimum settings might be different. You might check the 'optimum' settings option in your BIOS, and then see what it has set for your memory.

CPU clock speed is the main thing with folding, imo.

When you installed the [H] Ubuntu folding appliance, didn't it mention what the optimum settings for your BIOS should be, to use it effectively?

 

[WhitehawkEQ]

So, I'm wondering, for Bigadv, does RAM speed make a lot of difference with ppd? And if so, could I run regular RAM and ECC-Registered on the same board, as long as I only have one type or the other per CPU bank?

I would not mix the 2 types of ram, ECC (Error Checking & Correction) ram is not the same as desktop ram. There is an option in the BIOS to turn off ECC so it might work but still I would not try it. And for the money I put in this monster I don't want to risk killing it.
But then again who knows, it might work, I just don't know let me know if you do it and if it works.

 

[torin3]

Check your mobo manual of course, but you can also test it yourself. Stop FAH, go into the BIOS, and switch your memory settings, from interleaved to non interleaved, and restart FAH. Compare your folding times now, with what you had previously. (You must have all the same type of memory installed, to test this.)

I'll check through the manual again. It is actually pretty big at 87 pages.

I've always found interleaved to be the best, for any computer, including the 4P, but you have the most recent edition of the mobo and BIOS, so your optimum settings might be different. You might check the 'optimum' settings option in your BIOS, and then see what it has set for your memory.

CPU clock speed is the main thing with folding, imo.

When you installed the [H] Ubuntu folding appliance, didn't it mention what the optimum settings for your BIOS should be, to use it effectively?

That CPU clock speed is king was my recollection as well.

As to installing the folding appliance, I don't remember seeing anything about optimum BIOS settings. It said to let it run, so I left it to do so, checked back in about 10 minutes and it was done and at the login prompt.

The [H] overclocking BIOS does have instructions about letting it select the optimum settings, but I haven't flashed to it yet. I was trying to see if I could do it when dfonda had time to come over and watch me so he could do it to his system.

I would not mix the 2 types of ram, ECC (Error Checking & Correction) ram is not the same as desktop ram. There is an option in the BIOS to turn off ECC so it might work but still I would not try it. And for the money I put in this monster I don't want to risk killing it.
But then again who knows, it might work, I just don't know let me know if you do it and if it works.

Well, I've pretty much blown my budget for folding for a while, unless I do really well with mining I'm not going to be able to make any big purchases for a while. Though I've got some hardware I might sell.

Though, since I'm pretty sure my problem was due to the dead drive, not the memory, I may go ahead and buy 4x2GB sticks and run with 1 stick per CPU if it isn't too much of a hit. Then a month or two down the road, get another 4, & repeat until I'm back up to 16 sticks of identical ECC RAM.

Thanks again for the help everybody!

 

[Cobrabill93]

You will take a hit with one stick in each section. You want the 4 channel mode for sure.
I would not mix eec with regular either.
Keep the same type in each section as best you can. All 1gb together all 2gb all together.
Can you manually set the ram to the slowest speed that you have now?
That way they will all run the same? Faster ram can run slower.
Slower ram can sometimes run faster.

 

[torin3]

You will take a hit with one stick in each section. You want the 4 channel mode for sure.
I would not mix eec with regular either.
Keep the same type in each section as best you can. All 1gb together all 2gb all together.
Can you manually set the ram to the slowest speed that you have now?
That way they will all run the same? Faster ram can run slower.
Slower ram can sometimes run faster.

All the RAM I have in it is 2GB sticks of 1600 or faster DDR3. The board maxes out non-ECC RAM at 1333.

I guess I will be building up my ECC RAM until I have enough to switch.

 

[torin3]

I don't mind the questions, but I just finished taking off the SR-2 motherboard and watercooling. I'm not too happy with the cats right now. It looks like one of them walked across the board after I took out the x5675 to sell to my work and bent 2 pins in the socket. Now I'm going to have to test it with the CPU I have left in that socket to see if it will still work right. If not, I'll have to sell it as a 1P board. Though with those 7 PCI-E 16X slots, it will make a nice mining rig for someone, even with just one CPU.

Good news. I got a chance to put the board under a microscope and was able to get the 2 pins back into place with no damage. I'll be able to sell it as a 2P board.

 

[dfonda]

Good news. I got a chance to put the board under a microscope and was able to get the 2 pins back into place with no damage. I'll be able to sell it as a 2P board.

McGyver

 

[Adak]

Add one round of !

 

[torin3]

Well, I'm having some problems. Ran for about 3 days without a problem, then yesterday it shut off twice in about an hour. I've previously had problems with my UPS, but I thought replacing the battery resolved it. So I switched it over to the surge protection only outlets, and it lasted another day. It shut off again this afternoon.

So, I put it back on the UPS outlets. Still only drawing a little over 500W from the wall. I also dropped it down to 1 DIMM per CPU. Lost about 2 1/2 minutes per frame. Moderately large hit, but I should still complete the units in plenty of time.

Manual indicates I should be able to run uDIMMs at 1600 speed, but even setting the manual settings to the maximum speed, it only does 1333. So, I'm going to start getting ECC RDIMMs. I'll get about 8GB at a time and add them in as I can. I'm going to be selling of some hardware to pay for it so I can be up and running at full speed faster.

There is a slight possibility that it may be the replacement hard drive, as it is the same vintage and size as the one that died on me. It might be going bad as well.

I might pick up a USB stick with the ECC RDIMMs and install the appliance on it to rule out the hard drive issue.

And I think I'm going to go ahead a flash the bios in the next day or so to see if I can't make up some of the point difference with a faster CPU speed.