Do You Think You Are Getting Screwed By Reviews?

“It is interesting how these motherboards get such rave reviews and then shit all over us when we buy them !!!!!”

Yes, it is, isn’t it?

Before you think this is a “bash the other websites” article, that quote came from our forum and referred to a motherboard we very much liked.

Wanted: Reviewer. Must Be Son of God

Right now, most computer hardware sites are essentially review sites. They get products, they test them out for a while, they say something about them. Then they go on to the next product.

Is that good enough anymore?

Recently, I spent a lot of time reading various forums and it seemed that the people who actually buy the equipment live in a different universe than the ones who review it.

This is not good.

The point of this article is not to raise the question, “Do review sites do a good enough job?” but rather “Can review sites do a good enough job?”

If the answer to the second question is “No,” then it really doesn’t matter why the answer might be “No” for the first. If the job can’t get done by anybody under current circumstances, then we need to look more at how the job gets done than who’s doing it and why they’re doing it badly.

We think the answer to the second question is “No,” and think it would take Jesus to be able to do a perfect job given the nature of the beast.

Many of you have already asked Him for tech support, as in, “Jesus Christ, why doesn’t this work?” I believe He is assigned other duties, that’s why He hasn’t answered. 🙂

Jesus, Save Me!

I get, say a motherboard. It’s one motherboard. At the least, there are thousands of the same model motherboard being manufactured out there. Probably hundreds of thousands in some cases.

I’m supposed to take this one motherboard, and from this sample of one tell you how any of these motherboards are going to work under any and all circumstances.

If you believe that, then verily I say unto you, “You do not have faith the size of a mustard seed, but that of a planet.”

We couldn’t promise you that and keep a straight face based on what we’ve seen the last six months. We’ve had motherboards catch on fire. We’ve had motherboards do much worse than the average; we’ve had motherboards do much better than the average. We’ve had two of the same motherboard, and the two of them managed to cover both ends of the spectrum.

Those umpteen thousand motherboards will go into umpteen thousands of computers, with even more scores of thousands of hardware combinations, more thousands of tweaks and settings, and probably millions of different application combinations. All of these things interacting with each other under the completely clear, easy-to-operationally-follow guidance of Microsoft Windows.

And I (or anyone less than Jesus) am supposed to tell you whether or not all these God only knows how many combos work or not? Ahead of time?

Even to answer a simple question like “how far can I overclock this motherboard?” is in reality an impossible question to answer with an exactitude. First, I probably don’t have your equipment. Second, even if I got the same makes and models, I’d probably do something a bit differently than you. Finally, even if I did absolutely, exactly the same as you with the same models of equipment, I’d bet I get different results than you most of the time.

If even going that far doesn’t work (and just how many of you are out there?), what chance do I or anybody else have with one motherboard and one set of equipment to predict anything you might come across?

This is a job for Jesus.

Now if I had Him, this would be easy. I’d just ask Him, and He’d use His Divine Diagnostics to check out all possible combinations on all possible equipment and tell me just what I, you, or anybody and everybody else would get.

Has anybody seen this Divine Diagnostic program on sale yet? I’ll even take a beta. 🙂

Quality Control Crapshoot

Let’s assume, for argument’s sake that 10% of the owners of Brand Xs motherboards have a particular problem.

If I or Review Site A or Review Site B all get samples of the 90% that don’t, what problem? We could test our sample of one forever, and never see this problem. Not only don’t we find it, we can’t find it. Unless we can call in Jesus, of course.

I’ll tell you one problem a lot of review sites, including ours, have had with recent mobo testing. Most of the multiplier problems have been occurring with 1200s. We and the others didn’t use 1200s to test. Ergo, no problem.

Sure, you can say, “Go get a 1200.” That’s fine, but even if we could play hardware Napster at the biggest computer component store available, we couldn’t come up with every combo, never mind have the time to test all of them. Jesus, where are you?

I Can’t Pretend To Know All, And You’d Have to Pretend To Say Otherwise

Reviews have certain valuable functions. It can inform you about features. It can tell you sometimes if certain promised features actually work or not. It can usually identify absolute dog products. It may identify problems inherent to the product, and alert you to shortcoming the manufacturer downplays or ignores.

So they have some value, and I don’t need Jesus to help me figure that out.

However, I think a lot of people think reviews do a lot more than that, including a bunch of the things they just can’t do that people think they can.

Keep in mind that we overclockers are generally pushing equipment to the limit. Lots of weird things can happen that don’t under strict default circumstances. Computers can do a lot more things than toasters, and the price of that versatility are a myriad of things that can go wrong, and a even bigger number of reasons why.

And nobody can singlehandedly determine all this without divine intervention, and anyone who tells you otherwise either doesn’t comprehend the situation, or is not telling the truth.

Maybe we need to put on our thinking caps and figure out a better way to do this without Godly guidance.

Is This Our Problem or My Problem?

I personally think the review system as it currently exists as the last be-all-and-end-all word for a product doesn’t work, if it ever really did. Whether or not it worked in the past, it doesn’t seem to provide enough of a guide now, with equipment getting quirkier and quirkier. Any system that needs Jesus on the job to make it work is not too realistic.

But, like my opinion of a product based on a sample of one, it may not matter what I think. What matters is what most of you think.

The status quo right now seems to be “let the review sites review, let the users in the forums and newsgroups support each other.” I don’t really think that works too well, either, for reasons I’ll talk about tomorrow, but for right now, let’s just deal with reviews.

First, I need to know if this is just all in my head, or in yours, too.

What I’d like you to do is write me a letter telling me how you really go about selecting equipment (I’m not taking names, if you just ask your friends, that’s fine, the only bad answers are dishonest ones). Tell me what your experience has been (if any) using review sites as a guide to selecting equipment, and whether you’ve been satisfied or not by it. Tell me whether or not it’s been a help or a hindrance to you (if you want to vent in the process, that’s OK :)).

If you want to mention places you’ve found particularly good or particularly bad, mention that if you like. (I’m not going to quote anybody, and I’m not going to keep track of who or how many like or dislike a place, just at most general impressions.)

Say just what it is you expect out of a review, and whether or not you generally get it. Say how important or unimportant reviews overall or specific reviews by specific sites are to you.

Say What You Want, But Don’t Be Peter Pan

If you could change the whole review mechanism, how would you? Be realistic about this, though. Don’t say, “I think a thousand combinations should be tested.” That’s about as realistic as expecting Jesus to start testing for us. Unless, of course, you’re ready to write out a check to pay for it, or you can get Jesus to help us out. Nor should you say, “I think the combination I want should be thoroughly tested” unless you’re ready to do likewise.

Are you willing to pay a price for those changes, and how much? For instance, if you want more thorough reviews from a particular place, would you accept far fewer product reviews from that place? Or would you just spend less time at the more through places, and more time at others?

Would you rather have review sites focus on fewer products and pay more attention to them after the initial reviews, or do you prefer having a lot of choices and reading about a lot of different products?

To put it bluntly, let’s say you’re on a desert island and you had to choose between five thorough reviews, or twenty five reviews done like they are now, which would you prefer? One or the another. Sorry, folks, “both” is not a valid answer. More indepth coverage means less choice when ad revenue is static or declining. If somebody asked you to work three times as many hours for a pay increase of 10% (which is probably what your ideal would work out to be), what would you tell him?

Unless, of course, you’d actually be willing to pay for get both. (This is not something we’re thinking about doing at all, just testing the price you’re willing to pay.) Would you be willing to pay, say, five dollars a month to get extensive, intensive reviews from any website, and if you are, what would you expect for your money?

It’s easy to choose “Give me more” over “Give me less.” It’s a harder (but more realistic) choice to choose between “Give me this” or “Give me that,” and that’s the choice folks like us would have to make, whether it’s us or Tom’s Hardware. For us, we’re pretty much pushing the maximum doing what we do now. We’d have to hire more people to get appreciably more work done, and unless Jesus starts blessing our bank account like the wine jugs at Cana so it would never run out, we can’t do it at the moment. Right now, we feel like we’re doing a fair facsimile of Jesus feeding thousands with just a few loaves of bread. 🙂

Doing more product support would have to mean doing less of other things for us, and I’d bet that would be true for virtually every other website.

I really want you to think about this for a while before answering, think about what’s really important to you, and what isn’t.

Email Ed

Be the first to comment

Leave a Reply