Looks like they might have it

Ready to send is going up and waiting for validation is going down. And only 12 or so hours after a nearly 3 day down period.

Hopefully we will now have long periond w/o problems and unexpected outages.
Berkeley team really did a good job.

September 11, 2005 - 19:00 UTC
After weeks of dealing with stymied servers and painful outages we're back on line and catching up with the backlog of work. It was a month in the making, but it was always the same problem - dozens of processes randomly accessing thousands of directories each containing up to (and over) ten thousand files located on a single file server which doesn't have enough RAM to contain these directories in cache.

Since this file server is maxed out in RAM, our only immediate option was to create a second file server out of parts we have at the lab. So the upload and download directories are on physically separate devices, and no longer competing with each other. The upload directories are actually directly attached to the upload/download server, so all the result writes are to local storage, which vastly helps the whole system.

While this all very good news, this isn't the final step. The disks on the new upload file server are old - we'd like to replace this whole system at some point soon (something with bigger, newer, faster disks and faster CPUs).

Looks like it....Validation backlog down from 600.000+ to some 40.000

Mictlan and I have got some 3-4000CS these last 2 days


Anyway...-> Kenco, good to see you back in business. Both in crunching and here at the forums. Been a while if I remember correct??

Dk Jedi Allianc said:

Anyway...-> Kenco, good to see you back in business. Both in crunching and here at the forums. Been a while if I remember correct??

Click to expand...

Thanks. Yes it's been 8 or 10 months or more. Life got complicated for a while and was demanding all my attention in other areas. It's nice to be back.