What distro and what load management sw for a cluster?

I'm hoping someone here has some real experience with clustered computing. A friend and I are trying to plan out a couple of clusters (8x PS3s, 8x Core2Quads, or one of each) as part of a small business (ad)venture.
My question to anyone who has experience (or research) is what distro should we run and what load management software for actual cluster performance?

I'm thinking Gentoo, since it's so powerful and customizable. Also I've been looking at SGE (Sun Grid Engine) since it's technically free (we'd probably purchase a support plan, though, since they are priced reasonably and it's nice to have backup).

Advice?

P.S. If you have advice or comments on the hardware choices please direct them here (General Hardware thread) to keep this thread on topic.

well i dont know a heck of a lot about distribution clustering but I am apart of an R&D team that tests different configurations for cluster storage.

Are you planning on having cluster storage as well or just clustering the cpus?

Well we need to know a little more about what kind of cluster you want. Just a MPI cluster? Or a SSI cluster? For SSI there are only 4 options, Kerrighed, OpenSSI, OpenMosix and LinuxPMI.

I don't think any of them have been ported to anything but he x86 architecture. Kerrighed as a AMD64 version in progress. OpenMosix is only 2.4 kernel, it never got fully ported to 2.6 kernel. LinuxPMI is a continuation of OpenMosix after OpenMosix project was stopped.

Shelnutt2 said:

Well we need to know a little more about what kind of cluster you want. Just a MPI cluster? Or a SSI cluster? For SSI there are only 4 options, Kerrighed, OpenSSI, OpenMosix and LinuxPMI.

Click to expand...

Well, not really sure yet. Just doing MPI makes sense since we want versatility (to be able to use this for rendering as well as big number crunching.) SSI sounds nice but I'm thinking it would be good to not go diskless, to keep the versatility, right?

Or, we could do one MPI one SSI, heh.

I guess there's probably nothing other than YellowDogLinux for PS3...? (And the price for their Y-HPC is kinda... meh.)

EDIT: Ok, so obviously x86 is a better choice since it's more universal. No matter what distro we run, we need more software.
Anyone have experience with Sun Grid Engine or TORQUE? We like SGE...

We use platform ROCKS and Lava.

http://opportunity.neu.edu/