Reliability Dilemna

I think we’ve had a bit of a burn-in effect, haven’t had the failures to boot into OSs. However, I’m faced with a bit of a dilemna judging this (or anything else).

If I just go with personal experience, I know that at least a few aren’t doing so well. Do I just ignore that?

If I poke around forums a lot, I get a broader (and I think better) perspective, but it will hardly be “scientific” or “objective.” On the other hand, there is no measurement out there, not even our CPU
Database, that meet that criteria. Trying to predict general reliability (as opposed to performance) based on a sample of one is not scientific or objective at all.

There’s no realistic way I can think of that will get you a “scientific” or “objective” reading on general reliability based on user experience. Even if you could get one, what’s the point of lab results when you want to know what happens in the jungle?

Do you see the problem? I know the first isn’t good enough, from both a theoretical and experimental standpoint. Nobody can realistically do the third, and it probably would be useless even if you could.

That leaves some variation on the second option. I can understand people not liking the explanation “because Ed says.” However, what I don’t think people realize is that setting up some sort of reliability database is essentially no better than Ed poking around. The only real improvement you get is being able to verify what “Ed says.”

Not that this wouldn’t be a good or desirable thing to do, but presuming Ed is a good poker-arounder, it boils down to the same thing. Either way has the same fatal flaws from the perspective of statistical validity.

In short, outside of greater verification, the end results wouldn’t actually be significantly better, it would just look to be better. Somebody still has to interpret the data, and somebody still has to sift out a ton of statistical static.

Ironically, “we can’t take Ed on blind faith” is a far better argument than believing a database would somehow be far more objective.

Even more ironically, far from being insulted, Ed would be even more delighted to see some sort of reliability measurement in database form than you’d be, but that’s because he knows how tough it is to try to even guesstimate this. All a database does is turn Mohammed going to the mountain into the mountain going to Mohammed.

Email Ed