folding on a cluster

[OCn00b]

Now that I've got linux running, I'm considering putting together a cluster. I realize folding won't benefit from that but what I need to know is can each node still fold individually or being a part of a cluster mess it up? If folding can run fine in a cluster, it will probably mean an additional 20-30ghz for the team :)

[Fast420A]

Try it and findout is all I can say. The idea has always been shut down since folding isn't multithreaded but you should give it a go.

[overdoze]

it should work fine. However which cluster software are you talking about? I'm trying to do the same thing but right now it does not run on smp box.

[Gasoline]

Imagine a Beowulf Cluster of.... of waitaminute, this ain't Slashdot :p

You could just run multiple instances on the Linux cluster as if it were a big SMP machine, just give each instance a separate "machine ID" number (1 thru 8) in each instance's respective client.cfg file. An easy way to facilitate that would be to create a separate logon user with a separate home directory for each instance (i.e. fah1 with /home/fah1, fah2 with /home/fah2, . . . up to fah8 with /home/fah8) and install the client in each of those users' home directories. To launch the clients, just sign on to each fah# user and "nohup" the FAH4-Linux console so you can then logoff and leave the stuff running in the background. This is what I do on my HT (pseudo-smp) Linux borgs... I configured my fah1 and fah2 users so you cannot directly logon to them remotely... only locally or also another more privileged user may "su" to them, in the name of security. This method would work great if you only want to run 8 or fewer instances on the cluster (that 8-limit on the machine ID number in the client.cfg file), and you want to run them in a true cluster environment for academic reasons.

If you have more than 8 machines in the cluster and want to use them all, and have a separate Internet-routable ip address for each one, or NAT or a proxy server, you could just treat them all as a farm and run a separate client instance purely locally on each machine, but that wouldn't give you the "coolness" of the "cluster experience". ;)

Stanford's FAQ gives a little insight to running on a cluster:

http://www.stanford.edu/group/pandegroup/folding/faq.html#run.cluster

[OCn00b]

The cluster software I've heard most about is Mosix so I figured I would give that a try as it seems pretty easy to set up. As far as folding on the cluster, I'm not too concerned about how it's set up as long as each node can fold without problems. I'm off to check that stanford link now.

[axlecrusher]

You don't have to give different machine numbers if are you are running on multiple machines. But I guess you might have to if you use clustering software. I'm kinda interested in seeing what happens. Will Mosix try to move processes to different computers bassed on load? If it tries to move instances of folding across CPUs, or in this case computers, it will probably be slower because network transfer is very slow compared to just switching CPUs on an SMP system. I don't really know, I'm just thinking.

Axle

[overdoze]

I have already tried folding on clusterknoppix which has built-in mosix. It can not migrate folding job due to share memory on different threads. It will migrate the folding benchmark to the fastest node in the network and then it will fold locally... If you set up to fold on each node it should work fine with mosix except you might get misleading benchmark result.....
Let me know, I'm interested in your progress.

[OCn00b]

If the benchmark is wrong, that's fine. As long as it will fold right I'll be happy :) I need to format one of my rigs so I'll probably try to put linux on that one and try out a 2 rig cluster to make sure everything goes well. If everything turns out ok, it's off to the races :burn:

[LiGhTBoY]

Why don't you use Arkaine's overclockix for your farm. A farm is not exactly cluster, but I believe it's a farm you want to set up. It will migrate the jobs to the rigs and all the progress will be saved on the main server of the farm. Also search for the yatta-monster howto, I don't remember where I found it.

[OCn00b]

I've used overclockix but a farm isn't what I'm trying to set up, I already have one of those :). I'm trying to set up a cluster.