Revised plans for BigAdv (BA) experiment.

[Macaholic]

Blog post here;

by VijayPande » Wed Jan 15, 2014 11:17 am

We've put a lot of time into reading the comments about the BA experiment and have come to some conclusions regarding how we should proceed. BA was originally conceived as an experiment to push FAH as close to what you could run on a traditional supercomputer as possible, doing calculations that most researchers thought could never run on a distributed computing platform. In order to make this possible, the requirements for BA would have to be pretty extreme and constantly updating (much like how supercomputers are constantly being updated to the latest hardware). In recognition of this extreme set of requirements came a very large PPD.

BA was embraced by donors with powerful machines. It also encouraged donors to buy and build powerful machines, which are naturally expensive, in turn leading to donors naturally becoming upset when requirements change. However, updating BA is important. Without updating to keep BA amongst only the top machines in FAH, there would be a huge point inflation (turning off non-BA donors) and also limit what we could do with FAH outside of BA-needed projects.

In the most recent announcement of an update in BA, the reaction was particularly negative. This was far from our intent -- we're here to push our research forward and also to help bring donors together in this important cause we're all fighting for. The BA update did the opposite. It turned people off from Folding@home and served as an obstacle to our ability to push our research forward.

We've listened to the donor comments (including those running BA and those who are not) and come to the following plan to be sensitive to their concerns but also to avoid this sort of issue continuing over and over in the future.

1) The posted change in BA requirements will be revised. The only change in requirements going forward will be to require 24 cores (with according changes in deadlines) and that will occur on May 1, 2014. Why change the minimum core count? In general, FAH works best when we have fewer longer trajectories rather than more slower trajectories, which is why we need to change the BA requirements periodically. Given the amount of BA work we would like to do, the cutoff has to be raised above the current level. While, 32 cores would be ideal but we can still get work done at 24 and, in recognition of donor needs, we will set the level there.

2) The BA experiment will permanently end on January 31, 2015. On that date, the servers will be set to accept only and we will have no plans for future BA WUs. This would allow donors to continue to use their machines and recoup more of their investment than the previous plan. This decision would also work to avoid future issues and strife within the FAH community associated with BA. We understand that many donors will be very disappointed about this decision. This was a judgement call I had to make and this decision is, in my opinion, the right thing to do for the long term good of FAH, even though I know there will be many upset donors right now.

In leading FAH, my approach has been to push the limits, try new experiments, but also keep an eye to the future such that FAH outlasts and out performs other distributed computing projects. Starting with establishing FAH itself over 13 years ago, to pushing to GPUs, true SMP, Playstation 3, and most recently to supercomputer like nodes with BA, we have constantly been trying new directions to see how we can further and further advance our research. All experiments come to an end, sooner or later. I think my team and I have learned a lot from reading the recent posts and it's time for us to concentrate on the core parts of FAH and improve them and not bite off too much.

Going forward, the next steps will include a discussion of the change of the QRB formula and possibly an update of the benchmark machine. Our plan is to seek more input from donors for both changes. While a distributed computing project cannot be run effectively through polls, I think there is a lot of room for us to improve in terms of connecting to donors and incorporating their concerns. I'm very excited about the future of FAH. I think my team has learned a lot with BA and hopefully we can take that going forward to make FAH even better.

Thanks to all for their contributions and participation in FAH. Working together we have done and will continue to do great things!

 

[torin3]

Now, admittedly, this is coming from the viewpoint of someone just getting a 4P rig set up....

But I'm a bit disappointed that they are dropping the BA program.

Feels like 'Don't like our decision? Fine, here is a candy for you. Now don't pay attention while we take our ball away.'

 

[madhatter256]

Just switch to primecoin or other CPU minings if you already have a 4P setup...

 

[orion456]

Some how seems contradictory. BA is important, updating BA is important, so we are not pushing the update so hard and then we are ending BA completely.

I guess BA is not that important after all. I must be missing something here.

If the reaction was negative before at [H] I wonder how this news will satisfy their strike? [update: looks like the reaction is the finger]

Meantime - fold on, no doubt the Universe is unfolding as it should.

 

[Adak]

I think VJ was overwhelmed by the negative reaction of the threshold update to BA, again. Rather than continue these huge reactions, he just decided it was best to end the BA experiment.

Which is probably best. He was modeling the BA program after the way supercomputers are handled, but failed to note that everyday folders are not getting paid for folding on their BA rigs - so there's a HUGE difference in how a required update is perceived.

Some [H]'ers are quitting - Musky had some 4 letter words for PG, that rhyme with "tuck", but others are folding on, now that the dust has begun to settle out.

 

[Cobrabill93]

I want to see what they do with the SMP and QRB situation.
They still need to make it worth peoples electric and investment to run SMP.
That was part of the problem in the first place. Now many want to run them if the reward is too low. A 24 or 48 core can return SMP WUs in a just a couple hours. That should come with an extra bonus.

 

[orion456]

I think VJ was overwhelmed by the negative reaction of the threshold update to BA, again. Rather than continue these huge reactions, he just decided it was best to end the BA experiment.

Probably right, but it seems he went from a debate on core requirements to ending the whole thing (maybe by design). Could he possibly think that won't provoke an even bigger reaction? It sounds like another relations disaster - of which PG has a long history.

The relative peace of Rosetta is looking better and better.

 

[orion456]

I want to see what they do with the SMP and QRB situation.
They still need to make it worth peoples electric and investment to run SMP.
That was part of the problem in the first place. Now many want to run them if the reward is too low. A 24 or 48 core can return SMP WUs in a just a couple hours. That should come with an extra bonus.

Well they already have opened another kettle of fish with the BA like points given to GPUs. If they scale those back ppl will be up in arms, and if they don't then who would fold on a 4p machine, getting 1/3 the points they had before, when they can switch to GPUs (beware of course PG often changes the terrain in unexpected ways.)

I think we need a longer term road map so we aren't taken by surprise again and again.

 

[Robert17]

F@H has a much longer history than my time around T32. So my comments should be taken with a "2 cents worth" frame of mind.

I volunteered for a project many years ago. During the effort I both got into some poison oak from which I still bear scars today, and was bitten by another volunteer's doberman pincher; drew good blood and left another scar. When I complained about the dog not being controlled properly I was told that if I didn't like it I could leave. I left.

Perhaps VJ hasn't volunteered, gotten bloodied and scarred, and therefore doesn't personally make the connection with the volunteers enough to "get it". Maybe I'm wrong? Still, there is work to be done. It's just a matter of priorities and perspective on everyone's part. The middle ground shouldn't be that hard for him to visit.

 

[dfonda]

We could see this coming a mile away...with the fastest 4P rigs pushing almost 1mill a day. Something had to give. Glad I'll get about 3 years out of mine.

Who knows maybe they'll change their minds again.

Just IMHO its not Panda who's taking the ball home...He owns the court and has just changed the rules. Its the donors who want to take their balls and go home.

I have never pretended to be in it for science. That's an excuse I make to my wife so I can upgrade. I'm in it for team comraderie, and the points. Its like a game for me.

I don't mind folks complaining when the rules change but don't pretend your in it for science because if we were , we wouldnt give a crap about the points!

I think he is doing us a favor! My electric bill is already through the roof. Its like gamblers being kept out of a casino.

 

[Nathan_P]

It's not just musky quitting. We have already lost about 10 4p machines with a load more likely to be shut down over the next 12 months by donors who will not switch to SMP and more than likely will leave and never return to F@H

 

[Adak]

[H]'s FAH team loss, should be offset by their gain on the [H] WCG team, hopefully. It would be a shame to lose those very powerful folders, completely.

With the end of the BA project, the EVGA team may have a resurgence, and be able to give [H] some stiffer competition.

 

[RoXQi3x]

(...) don't pretend your in it for science because if we were , we wouldnt give a crap about the points! .

Ahem, please allow me to have multiple reasons for folding. For me, it is points/game/competition, it is hardware/performace/bargain hunting, it's those I lost/doing good and it's even "believe in something"/science.

 

[Cobrabill93]

There are a bunch of EVGA folders not folding too.
Their BIONC WCG team is growing leaps and bounds.
There are other teams that have slowed down and have members stop folding.

I just can't see how PG said it was so important to upgrade BA servers, but it is not important enough to keep the BA program.
All he had to do is keep it at 24 core and raise the QRB for SMP a bit. Wn win for everyone.
I hope PG does not think everyone is just going to switch to SMP. They need to fix QRB, and fix the scaling issues with high core count systems.

I think PG flipped us the bird.

 

[Adak]

There are a bunch of EVGA folders not folding too.
Their BIONC WCG team is growing leaps and bounds.
There are other teams that have slowed down and have members stop folding.

I just can't see how PG said it was so important to upgrade BA servers, but it is not important enough to keep the BA program.
All he had to do is keep it at 24 core and raise the QRB for SMP a bit. Wn win for everyone.
I hope PG does not think everyone is just going to switch to SMP. They need to fix QRB, and fix the scaling issues with high core count systems.

I think PG flipped us the bird.

That is the perception many folders have, and that is precisely why PG was smart to get OUT of the BA program. WAY too divisive. VJ said it was modeled after a supercomputer center, which seems odd to me.

Supercomputer centers earn a lot of money for their upgrades - folders never earn money by folding.

Folders get very stressed when you start changing the points system around a bunch. Nobody likes having their nose grabbed, and being lead around by pulling on it. Make a good points system, and leave it alone. If it has to be changed, make the changes gradually. It's the big and sudden changes that really cause stress in the folders.

I recall the QMD project - they were great point wu's, but you could only participate if you had an Intel cpu**! This project was announced very shortly after I had JUST bought two AMD based PC's for folding.

I'm in it for the science, but I enjoy the points as well, so I stuck with it, but I was NOT happy about the sudden announcement of the QMD project starting up. At least with the BA ending announcement, we have a year's notice.

Hopefully, the drama will continue to decrease, and we can get back to focusing on folding.

** It required the Intel numerical library run-time.

 

[Cobrabill93]

I guess it depends on what they to to the return times and QRB SMP.
Is this a calm before the storm? It's been too quiet I think.

If there are still BA work they will give it a new number 8103 will now be a 7788 (or some number) and they will give it to large count server class hard ware, for less points, call it SMP, and no one will know the difference!!!! I would not doubt that for a second.

 

[orion456]

I guess it depends on what they to to the return times and QRB SMP.
Is this a calm before the storm? It's been too quiet I think.

If there are still BA work they will give it a new number 8103 will now be a 7788 (or some number) and they will give it to large count server class hard ware, for less points, call it SMP, and no one will know the difference!!!! I would not doubt that for a second.

There in is the problem. PG announces a major change out of the blue (again) rendering BA equipment worthless and they fail to do the thinking necessary to complete the picture and tell us what will happen to the QRB and return times.

I don't believe BA will disappear they will do just as you said and incorporate it into SMP. Unfortunately for them, a lot of the big guns will be annoyed and be gone.

Now we have another period of uncertainty....thanks PG.

Just to show the confusion happening in PG, on Dec 20th, VJ wrote in relation to BA: "beyond the ramp up Dr. Kasson discussed, I expect that there will be another change in requirements closed to the end of 2014, to keep up with changes in hardware." A short time later he is writing that BA is cancelled Jan 2015. How can we be expected to understand what PG is doing when they can't keep the story straight.



We have already lost a lot of folders and I am afraid, more are to follow. It's really very disheartening and very unnecessary. Talk about abusing your biggest donors.

 

[Adak]

I believe there will be no more BA work units, period. There were none before the BA program, so I expect there will be none, after it either. VJ said before the BA program, work of that size went to a supercomputer cluster, not to FAH.

That could be the case, again or they could break any BA sized work unit into a smaller size, and make it an SMP or GPU work unit, as they chose. Either way, it wouldn't take a large number of cores and a long time to process.

Since they can query the folding box to see how many cores it has, it's possible that the assignment server could send BA work units to the fastest folding boxes, and simply not call the wu BA, anymore. As was speculated above. The fastest boxes would fold them handily, and earn a nice amount of points (not as much as a BA wu), and we'd probably not notice much difference, or complain if it happened to us. "We got a big wu, and it was worth a lot of points".

I'm hoping VJ doesn't come up with a new "program", for FAH, anytime soon. They're very disruptive.

 

[dfonda]

Make a good points system, and leave it alone.

Pretty much describes Rosetta....

 

[ihrsetrdr]

The anxiety caused by changes in BA are certainly understandable- I myself am all too familiar with that as well. But, I can't control was PG does so I'll just continue folding with existing hardware, at a level of electrical consumption that I can afford.

I trust that Harlam will take good care of my #10 team position. ;-)