The other day, SG Overclockers wrote a piece questioning the general integrity of the hardware review system.
We certainly don’t disagree with anything said in the piece, but we think some more points need to be made.
It’s Time and Money
The article dwells quite a bit on the financial aspect of reviewing; that websites cannot afford to buy the equipment they test, and because they can’t, they’re more or less tainted.
While this is true, the problem is bigger than that. It’s not just money, if anything, time is a bigger factor.
It’s not inconceivable that an organization could be put together which would buy all the equipment they review. After all, Consumer Reports does just that.
However, such an organization would be horribly handicapped by time. If it waited until a product became available, then spent a good deal of time testing, they would be hopelessly behind the curve in an industry with short product cycles and even shorter levels of audience patience.
If you want to be the first kid on your block with something (and increasingly, this is becoming literally true), you’re not going to wait patiently for somebody to meticulously test equipment. You’re going to buy on the early (or at least fairly early) buzz.
By the time Mr. Tortoise shows up, attitudes about the product, good or bad, have already been set. So long as being first is more important to most than being right, guess what’s going to heppen?
Give Me A Buzz!
That’s just a symptom of the bigger problem, though. There’s an unspoken assumption in the SG piece that the audience is composed of Vulcans who calmly, coldly assess all the data available on X and its competitors, then rationally decide.
Well, there are people like that, but the reality is, those people stick out like a non-pon farred Spock at an orgy. Well, a handful of Vulcans, anyway.
(Actually, a lot of people buy equipment like they’re Spock during pon farr, except that they want to mate with silicon lifeforms.:))
The reality is most people find their hobby a source of pleasure, and want to get more pleasure out of it. It gets them high, and they want to get higher. Looking at and buying equipment is, for the most part, an emotional decision.
Unlike most drugs, though, one need not necessary pay to get a thrill. One can get at least a somewhat satisfactory buzz just reading about new products, even products the person will never be able to afford.
The point is not the product, the point is the buzz, and if you have the nerve to interrupt that buzz, you’re often treated like the bartender who cuts you off. You don’t get gratitude; you get resentment.
The fact is, most people don’t like to hear negatives (except, of course, when they’re uttering them; then anything goes.:)). They don’t like it when you speak negatively about products, they don’t like it when you speak negatively about those who review products. To such people, facts and truth mean nothing. Getting the buzz is what counts.
Nor are most people necessarily too meticulous in carefully weighing available information to come to a decision. Rather than seek information to decide, they decide, then seek information that backs them up. In other words, ready, shoot, aim.
Early in my tenure here, I used to regularly get people who would order equipment, then ask me about it. Apparently, they hoped I would confirm their wisdom, unfortunately, I wasn’t often able to oblige them.
I often get emails complaining that I’m too negative about something or everything. To me, being positive or negative is irrelevant; what’s important is whether what I say is or isn’t. I have an opinion based on my reading of the facts; an article will normally follow the pattern of “Here’s what I think, and here’s why I think it.”
What I’ve found over the course of time is that people will only rarely contest my facts or reasoning. What they will often do is complain about the negativity, and by the end of the email, it’s clear that what they’re really objecting to is spoiling their high.
The point of all this is not to complain, but to point out that, on the whole, people don’t do this to get a better screwdriver, but to meet emotional and psychological needs, and the most popular websites are not those who do the best, more objective reviews, but rather best meet their audience’s needs.
Can Only Real People Be Trusted?
It’s been suggested that the only place where you can get a real, truly unbiased review is from real people in real forums.
Well . . . . that’s sometimes true. User reviews are good for finding out real negatives, provided you take the Family Feud approach to it. If lots of people have the same problem with the same feature, and nobody has a solution to the problem (outside of a flat-out workaround), it’s pretty safe there’s a problem.
Otherwise, they’re not too much good taken too literally. For openers, you have little idea whether any particular person knows what he or she is doing. Second, at least some people will be loathe to admit they picked a lemon. This is especially the case when it comes to giving equipment a thumbs up/down. I’ve seen some places compiling user experiences where trash got about an 80% approval rating.
One can certainly benefit from reading a lot of comments about a piece of equipment, but that can be time-consuming and often headache-inducing. Raw data is just that, raw, and unfortunately a lot of people aren’t too good at processing it.
Let Me Decide!
A review is inherently an act of judgment based on limited circumstances. No one person can test all possible hardware and software combinations, no one review can give personalized advice.
Over the course of time, I’ve gotten more than a few, “Humanity needs this (my) configuration tested.” We’re not too self-centered. 🙂
Somewhat more often, I see “Don’t you dare tell me what to think! Give me the facts and let me decide!” like I took their credit card hostage and they can only order through me.
I’d be rather more inclined to shut up if I saw that everyone was a silicon Solon out there, but unfortunately, those who yell the most about personal judgment tend to give little sign they have any.
All I (or anyone else) can do is offer fairly generic advice to help most people make a reasonable if not wise buying decision. However, if somebody’s way of thinking is inherently illogical, there’s not much you can do about it.
Of course, if you have more fanatical allegiance to a particular company than the company’s CEO, well, why even waste words? I’ve had cases of people so wound up against something I said that they couldn’t even tell me what I said that got them so wound up, even after being asked.
We Get The Reviews We Deserve
If this piece seems a bit disjointed, it’s not me, it’s just the diverse ways people react to the situation.
It’s not that SG Overclockers is wrong in anything they say, in many ways, the system is broken, and many people know that.
Problem is, ask those people what’s broken, and they’ll give you a million answers. Often it’s a version of people saying, “Congress is terrible, but my Congressperson is good.” I’ve spoken about certain places critically without naming names on occasion, and I’ve gotten emails telling me, “Great article! Why can’t places be good like _____ ,” and ______ was the place I was talking about.
Unfortunately, we’re not a bunch of Spocks. So long as we aren’t, and the votes we cast with our mice are cast largely for extrarational reasons, we’re not going to get rational objectivity, nor will it much matter.