Planting the seeds for a farm

[gulp35]

I am considering building a folding farm. Like any good farmer, I want to plan my plantings so I get the biggest harvest with the smallest investment/ best utilization of my farmland and my time. I have some ideas, but what I need is some feed back if my ideas are correct or not, and any practical advise (like, I’ve used that board and its great/crap/etc.)

First, I’d like to present my assumptions I used in developing this “crop rotation”. Later when commenting, please stay within my “property lines” (no farming on the neighbors plot!). I do not want to hear “run a dual megabucks” when I put a limit of most economical, unless you can present some reason why they are truly the most economical. Likewise, please feel free to tell me why my assumptions are wrong...don’t worry, I can take it. The reason I am posting this is I want to hear some other opinions.

Objective: Develop folding farm that is most economical on points per day/ per dollar basis. System should be painless (i.e. not needing constant attention. It should run trouble free.).

Assumptions:
1. Build all layers the same (server and folding layers). Server will also (I think) have a monitor, keyboard, mouse, and moderate HDD (20Gig). I am thinking build them all the same so once have the pattern, I can easily duplicate. Because of this, I will probably buy most parts from vendors, not e-bay, etc.

2. Moderate overclock to maximize economics. Not so far that the system become unstable. I’m thinking mobile Xp2500/2600 to say 2500 MHZ, or A64 to 2450 MHz. Therefore, I want a Motherboard/CPU that can take over clocking in stride. However, I don’t want “the best”, I want what will meet this criteria relatively painlessly (i.e. no trying to solder wires onto the MB, etc.) and no more. I am guessing I need a board that allows CPU voltage changes in the BIOS, along with BIOS FSB changes. I picked some motherboards below based on user feedback at Newegg. But I’d really like to hear from you all on this.

3. Operating system. Here I’d like some advise. I am the last one to support Bill G., but I hear all kinds of people on these forums struggling to keep headless Linux systems running. I am thinking buy some used hard drives (say 5-10 Gig units), slap on XP or 2000 and go. At some point, a $10 -$15 hard drive is worth it to cut out the struggle to keep a system without a hard drive running. In addition, I hear that headless units loose work during power failures or if the server goes down. Finally, I hear nasty rumors that Linux does not fold as fast as Windows. Feedback/advice welcome on this point.

4. Work Units – All this talk of QMD cores has me a little worried, but for now, I’d like to aim at “farming” big WU’s. That what I run most of the time on most of my current machines. So I am guessing 256 Meg of memory. I was thinking a single stick so if later the I need more I can added it.

5. I have an A64 and it is fast...320 ppd (on big WUs) typically running at 2450 MHz. However, I am assuming a Mobile XP 2500/2600 running at the same speed is only 10 to 15% slower (Anyone have some data either way on this?). If the A64 is only 15% faster, it really does not justify the added cost it seems.

6. A Motherboard with onboard video. I know you can by cheap video cards, but then I worry about them making the system unstable (my current T-bird machine has a funky video card that takes it down periodically). I know you may not need a video card all the time, but again I want painless (I have promised my wife that the farm will not be a huge time investment). I want to be able to hook a keyboard; mouse and monitor up to any layer and be able to work on it with out sticking in cards, etc.

7. Heat sinks. Yeah, it would be cool to have a row of the latest monster heatsinks on all the farm animals, but when you look at the numbers, it gets hard to justify a $45 heat sink when a $15 does just as good in some cases. So I want the cheapest heat sink which will cool enough to allow an overclock of say 2400-2500 MHz. This farm will be in my basement, so it will usually be at 60 to 70°F ambient. I’d really like to hear from anyone out there running a nice cheap HS doing this...yeah, its not sexy, but it is cost effective!

8. Motherboard FSB. I was thinking of buying a motherboard that can support 400 MHZ FSB out of the box. Then just step the standard FSB from 266 towards 400 and the CPU should be the limiting constraint.

9. Memory – same thing, get PC3200 memory, and 400 FSB should be no problem, so again the CPU should be the limiting factor. And since it really should not take over 400 FSB, even “cheap” PC3200 should do fine. Again, I’d like to hear your experiences here, good and bad.

10. Power supply. I know you can split them, but again this looks like it is also a source of hassle for people. I’d like the cheapest PS that will get the job done.

11. Ignore Power consumption. I know it is a factor, but I think hard facts as to what power systems actually consume (vs. specs, estimates, etc.) are few and far between. So my economics are strictly on purchase price.

To summarize, I want to build identical systems, operate headless, but with video output available and hard drives. Overclock a reasonable amount.

Here are some ideas ( just a starting point really)

CPU

Mobile XP 2500 http://www.newegg.com/app/ViewProductDesc.asp?description=19-103-401&depa=1 $85

Mobile XP 2600 http://www.newegg.com/app/ViewProductDesc.asp?description=19-103-436&depa=1 $95

Motherboard

Biostar M7VIZ http://www.newegg.com/app/ViewProductDesc.asp?description=13-138-231&depa=1 $44

ASRock K7S41 http://www.newegg.com/app/ViewProductDesc.asp?description=13-157-024&depa=1 $49

Asus A7V400-MX http://www.newegg.com/app/ViewProductDesc.asp?description=13-131-509&depa=1 $52

Abit VA-20 http://www.newegg.com/app/ViewProductDesc.asp?description=13-127-191&depa=1 $53

Memory

Roswell 256Mb http://www.newegg.com/app/ViewProductDesc.asp?description=20-223-024&depa=1 $22

Wintec 256 Mb http://www.newegg.com/app/ViewProductDesc.asp?description=20-161-614&depa=1 $23

Buffalo 256 Mb http://www.newegg.com/app/ViewProductDesc.asp?description=20-150-618&depa=1 $23

Heatsink

Dynatron BH-625 http://www.newegg.com/app/ViewProductDesc.asp?description=35-114-012&depa=0 $10.50 +$5 shipping

Masscool Falconrock http://www.newegg.com/app/ViewProductDesc.asp?description=35-150-023&depa=0 $11 +$5 shipping

Power Supply

Powermax 480W http://www.newegg.com/app/ViewProductDesc.asp?description=17-163-025&depa=0 $12+$7shipping

Roswell RV 300 http://www.newegg.com/app/ViewProductDesc.asp?description=17-182-001&depa=0 $14+$5 shipping

Basic layer – CPU $85
Motherboard $50
Memory $23
Heat Sink $16
Power supply $20
E-bay Hard drive $15
Price/layer =$193

Points per day =320*.85 = 272 (est. based on my current A64)

1.40ppd/$

I welcome your comments and ideas.

 

[@md0Cer]

I have to go within 30 seconds or so, so I can't read your whole post, but in this thread http://www.ocforums.com/showthread.php?t=372369

Some things were considered like cost/performance of different approaches, electricity, cooling, etc. I will be back to look at this more in depth. Ta ta for now

 

[AMD_OCER]

I was thinking of the same thing last night looking at those same parts on newegg.

I just setup my farm in my sig a week ago and so far it is stable. But I am thinking of upgrading already.

One word of advice make sure the motherboards will lan boot, otherwise you will need to add floppy drives to your cost. The motherboards I used say they had a LAN boot option but it does not work. There is no provision for flashing a boot rom anywhere and it appears that there is not one installed by the manufacturer.

Cluster007

 

[gulp35]

Why LAN boot..

Help me her...why do you need a LAN boot if each layer has a small (2-5 GB) harddrive to boot off of? I want to avaoid all these hardware issues people seem to have. I have even thought Linux might work better if you had it boot off a hard drive rather than try and boot off a CD temporaily hooked up to the layer.

In theory, booting off a LAN is a great idea..saves alot. But in practice it seems to be a lot harder to make it work consisently.

Please tell me if this is incorrect.

 

[Arkaine23]

I'd think using XP-Mobiles (OC to around 2.2 - 2.4 GHz) is the cheapest vs performance. The modding required would be for adjusting multiplyer to a lower value by inserting snips of wire into the cpu socket if you wanted to run at 200 FSB, or alternately just running mobile 2400+s at 166 FSB instead of 200. (2241mhz) OCing much more than that on cheapo HSF's is probably not a very good idea anyway.

You can also split 420W thermaltake PSU for 2 mobo which deliver more power than 2 cheapo PSU's together. Not to mention, the thermaltake PSU last longer, generate less heat and more efficient >70% than the 65% efficient than say a Codegen.

Look for the best farming motherboard in this forum. I think it is biostar nforce2 400 which allow you to OC in the bios.


Linux folding is not any slower than windows folding, especially when looking at gromacs units. Also you forgot the $ for a Windows license for each machine and only listed the cost for the HDD.... shame on you. I'd recommend using Overclockix LTSP 2.1 for a diskless farm. Its not that difficult to get it up and running. You'd want 256MB for each client, 512MB for the server if you have more than half a dozen clients. The server's HDD need be no larger than say 3GB, but a decent 20GB 7200 rpm drive would probably be prefereable.

If spending a little extra for small HDD's is no problem, than Linux can still save you the cost of a windows license for each layer. There are easily installable (takes 20 minutes) distros with built-in folding. And since I have built just such a distro, and many ppl here have experience using it, you'd find no lack of support.


Each layer should produce around 250-275 PPD on big packets, and cost roughly $175-225 to build, depending on what corners you cut and deals you seek out.

CPU- $75
Motherboard- $50
decent PSU- $30 (not split) add $15 for a splitter and consider it $45 per 2 layers
Ram- $30
HSF- $15
cheapo video card- if not already onboard $10
HDD optional $15

 

[dark_15]

I also found this guide to help you setup a Linux farm...
http://overclockix.octeams.com/How_to_setup_a_Folding_Farm.htm

From Arkaine and Overdoze

 

[Helsyeah]

Well, I had a whole post written up to show what kind of performance/price ratio can be had for a dually xeon setup, but then the power kicked off .

So here i go, again

This dually setup is spec'd in the smp section, but i thought i would summarize it here. There are a couple caveat's however with it however.

The processors (1.6 lv xeons) that work the best are currently sold on Ebay, BUT, i know of more than a couple people who have contacted the seller (lacompute) personally and bought them directly from him instead of going through ebay. Also it appears that availability could be an issue. The purchase price seems to be around $120 per set of two. So this is the biggest catch, setting up a deal with lacompute.

With the setup im going to spec, you would need to get a cheap pci card. If this is really an issue then here's sticking point #2. I have run a couple different vid cards in my dually setup and have yet to have stability issues that can be attributed to the video cards themselves though.

As for what to expect, say you get a very conservative 2.4ghz per processor with 150x16 to go easy on the fsb. This should be reachable with around 99% of the 1.6 lv's and the mobo i will be listing. Each processor will then kick out, conservatively 250 PPD, 500PPD total for each layer. Chasr would probably have a more solid set of numbers, but i will err on the save side here.

Layer Summary: (mostly Pulled from the dually xeon cruncher thread in the SMP section)

CPUs: 2x1.6ghz lv 1.3v xeon processors - $120 from lacompute

MOBO: ASUS PC-DL 875 chipset dual xeon mobo - $189 @ zipzoomfly .

Power Supply: AMS Mercury 460W - $54 @ directron

Memory: 2x512 PC3500 Kingmax - $118 @ newegg

Heatsinks: Intel Wind Tunnels - 2x$12 @ acmemicro

Vid Card: Geforce mx400 $33 @ zipzoomfly

Layer Price Summary:

Cpus - $120
Mobo - $189
PS - $54
Memory - $118
Heatsinks - $24
Vid Card - $33
Ebay HD - $15

Layer TOTAL - $553

Price Per Point conservative Estimate 500PPD / $553 = .9 PPD/$.

EDITED:

Granted there are several possible hangups with the setup. Erring on conservative side, the PPD/$ is WORSE than the athlon layer setup outlined earlier, but these layers would probably take up a bit less room than an amd farm.

 

[AMD_OCER]

Help me her...why do you need a LAN boot if each layer has a small (2-5 GB) harddrive to boot off of? I want to avaoid all these hardware issues people seem to have. I have even thought Linux might work better if you had it boot off a hard drive rather than try and boot off a CD temporaily hooked up to the layer.

In theory, booting off a LAN is a great idea..saves alot. But in practice it seems to be a lot harder to make it work consisently.

Please tell me if this is incorrect.

IMHO, in a farm it would be better to lan boot with a n.i.c. rom bootloader on a floppy than try to rely on old cheap hard drives. Floppy's are only used for booting that is it... using less power, less chance of having an old hard drive taking a dive. If you want data reliability then do something like RAID in your server.

If a Linux n00b like me can do this on a ten layer farm in about 4 hours, you could too.

Cluster007

 

[gulp35]

Excellent feedback...

Thanks everyone for the excellent feedback.

I have a few questions...

I assume you can boot a Linux folding layer of a harddrive hooked directly to ...is that correct? It seems booting off a hard drive 1) elinimates the issues with needing to boot from a LAN - which sems to become an issue for some 2) reduces the chance of loosing data when the server/HDD crashes.

As far as the dual xeon...thanks for the excellent write up...but I am a little confused...the dual xeons are 1.11 ppd/$ while the mobile XP seem to come out at 1.4 ppd/ $ (more point per dollar invested). Am I missing something?

 

[Helsyeah]

As far as the dual xeon...thanks for the excellent write up...but I am a little confused...the dual xeons are 1.11 ppd/$ while the mobile XP seem to come out at 1.4 ppd/ $ (more point per dollar invested). Am I missing something?

lol, thats me trying to do math without thinking it though too much... was thinking less ppd/$ was better... doh. .

the figures are correct, but also very conservative. So based on current projections, the amd system would be the better value. At this point it is just another alternative with the benefit of the dually setup probably requiring less physical space to get the same amount of PPD compaired to an amd farm.

Original post has been edited. So next time ill pay a bit more attention before going off with listing alternatives ... like i said, consider it an alternative now.

 

[Arkaine23]

The XP layers are more point efficient per cost, however the dual xeon should be able to do closer to 600PPD with some modding to 2.8-3.2 Ghz. And the actual cost of the XP layers is probably going to be on the high side of the range I gave... $225, especially if you need to get a HDD for each.

And about terminal services and pxe-booting... All layers have to be rebooted if the server goes down, but linux is very stable. Pick the right board from the get-go (i.e. one with an option for 133/166/200 FSB with onbaord LAN and a PXE-friendly onboard LAN, for around $50. Then you'd keep the cost close to the $200 mark.

Further deal-hunting might net you an excellent deal on appalbred durons with motherboard for around $60, which would be excellent folders if using a HDD for each layer. LAN-booting might not be available with their onabord LANs or in some cases the onbaord LAN may not have a linux driver (depends on the board) and an additional NIC would be needed).

 

[Ven0m]

I'm not an expert here, but I'd suggest to avoid old cheap hdds, and that's why:
1. that adds cost of rig
2. they are noisy
3. they use some extra power
4. they can die any day without warning
5. computer don't crash (except for power outage)

Eventually it would be better to get one good HDD, put it in one computer and make other comps use it via 100mbit/1gbit LAN, or simply to have everything in RAMdisk, but this way you loose data on reboot. Anyway no/one hdd solition is more reliable.
Another thing is that I think that some mobos don't need a graphics card to boot. I guess that even mine can do it. This way you use graphics card only to prepare Linux to boot. That results in less cost and less power consumption and you've still got control using SSH terminal.

And if yuo decide to go with Athlons, 1 PSU for many comps is better. Such computers, especially without HDDs and graphics cards wouldn't draw much power, so single better PSU could serve 2-3 computers easily. Other thing is that PSUs have max efficiency with higher load, so power consumption would be less than with running separate PSU for each computer. This trick might work with Xeons too, but I'm not sure... You'd need some better PSU, so it might be not too cost-effective.

 

[gulp35]

If we (both me and my dad are using this account name, he has written all of the above posts) are going to use linux on the boxen, both of us need to get accustomed on how to use it. Are there any good guides on the basics/intermediates of linux?

 

[samuraisam]

We have an alternative OS forum here with many very knowlegable members. I'm not currently active there, though I do run *nix so you could always PM me for info too.

If you are interested in *nix, books are always nice to have. There are many different flavors and many different books for those flavors (also called distro's or distrobutions). HERE is a good online Linux how-to.

-Sam

 

[AMD_OCER]

If you want an Linux distro that will be easy to setup and maintain than I vote for Arkaine23's Overclockix LTSP. He has done a great job on this one, good instructions, and there are plenty of people who can help you use it here.

And Arkaine has said what I was trying to say, it's a PXE bootable motherboard... Basically that's what I was trying to say above. "I was just checking the specs on the inline rotary girder.. AHH I AM RETARDED."

And even if you don't have a PXE bootable motherboard like I do, you can put the PXE boot rom on a floppy disk and boot it that way. No CD-ROM involved here.

Also I can reboot my server and not loose any work that was saved automatically onto the server's hard drive every time a portion of a frame was completed... every few minutes in my case. I also can reboot any node and not loose more than a few minutes of work, the same as rebooting the server. This is also beneficial in data security as you can put a mirrored RAID array on your server and have an efficient and automatic data backup.

RAID is going to be my next server upgrade.

I am sure some one here can correct my terminology if i have said anything wrong.

 

[fireburster]

I built my farm of 10 pcs basicly from newegg refurbs. Grabbing mobos and cpus when i could. I run overclockix and xp on them. I had some 20-40gb harddrives around which i share out mp3s and movies on to make it worth having them run a pc. Grabbed a 4 port kvm on a buy.com deal and have that hooked to the overclockix ones since i was too lazy to setup vnc on them.

Run emIII and just check on them when it shows their is an error.

Most of the pcs are in a rack in the basement that i scrounged from work. I had just mobos on mounts on racks but i bought cheap cases and just have them stacked on the side.

I think 4 pcs are boards that use ddr or sd so i used sd since i have 4gb in 128s it seemed.

Ask friends for video cards and other stuff like that to help cut the cost also.

 

[Arkaine23]

Sounds like you got it right, except I didn't build the LTSP version of Overclockix. I made all other versions of Overclockix but that one. LTSP was done by overdoze, who used to be my boss.

Notes- Installation and configuration is pretty simple. The server runs two folding clients by default, the client machines run one foilding client, its a terminal server so it sends out an OS image for each client to run in ram, it also serves folding directories from its HDD to the client machines through NFS file-sharing, and it acts as a DHCP server to give out IP addresses. It runs samba file-sharing so that you can use EM3 from a windows computer to monitor all the folding clients.

Its easy to remove the second folding client from the server or add a second folding client for any of the layers in case of hyperthreading or SMP systems. Each folding client has a "flags" file where you can set the flags that are used. All folding clients run as background processes, like a service install in windows. The server boots into graphical mode and has a folding monitoring utility and tons of typical desktop/office software. The server also runs VNC so you can remotely access it graphically from another computer, so it won't need a keyboard, monitor, or mouse. The layers all boot into text-mode and can be accessed from the server using SSH to run commands.

There's documentation available on the CD and on Overclockix's website.

 

[Ven0m]

That sounds cool & easy
I haven't known that there's so simple way. I assumed that you need to install linux normally, set it up etc. Buit that seems to be click & go

 

[gulp35]

OK, lets say we do Linux...

What motherboard would you suggest (this is a rhetorical question, not to any specific person out there). Do any of the MBs listed above have proven PXE ability? Have any of you overclocked Mobile XP's on these board? Any recommendations or horror stories you'd care to share? Is there a better "cheap" Motherboard out there that is ideal for an overclocked farm?

 

[Arkaine23]

ASRock "K7S41GX" SiS741GX Chipset Motherboard for AMD Socket A CPU -RETAIL

There should be similar boards on the same or a closely related chipset. I'm sure there was even one with DDR400 capability..... just keep in mind these probably don't have much/anything in the way fo voltage options, and FSB is probably gonna be very limited... maybe just 100/133/166/200 (most don't have the option for 200). The plus is that the Sis LAN shoud do fine for PXE-booting, and they have onboard video. The other nice alternative is gonna be based on nforce2 and a bit more expensive. Pretty sure those will work....

166 FSB would be good enough for a Mobile 2400+ though with a cheapo HSF unit.