Numbers Numskullery Sets Up Numbers Skullduggery
Doing this job and reading emails over the course, you get to see patterns in how people think.
A very common trait of those interested in computers is the demand for concrete data. People don’t like abstractions. They don’t like extrapolations. They don’t like educated guesstimates.
They want hard, cold numbers, thinking that a number is a safe and easy basis upon which to make judgments. They want what they think is certitude.
There’s one big problem with that. It just ain’t so. Numbers are not rocks. They can be as fake as Oz. If you believe in numbers as the be-all and end-all, you probably think that’s smart.
Unfortunately, to those whose job it is to make those numbers Oz, you’ve figuratively bent over and asked for it.
We’ve spoken in the past about Intel doing that with MHz. Today, we take a look at ATI.
As a few sites have recently pointed out ( a good description can be found here, ATI has been cooking the books when it comes to Quake benchmarks.
Essentially, they’ve set things up so that when a Quake benchmark is run, the system automatically goes into a lower quality mode to generate higher numbers.
Now why did they do that? Because you bent over and asked for it.
The folks at ATI must have essentially decided, “These numbnuts are so stupid that they won’t notice anything but a number. So that’s just what we’ll give them.”
Don’t get mad at me. I didn’t make up those drivers, ATI did. They were the ones who talked like they were kissing your butt while doing something quite different in that region. I’m just noticing.
Look at how ATI handled this. They threw out very short-term loaners to a lot of review sites and gave them very little time to do more than crank out some numbers. Most didn’t notice anything amiss. To be fair, under those circumstances, we probably wouldn’t have either.
ATI got its share of glowing reviews from places that just cranked out numbers, and probably sold a lot of cards to people who just wanted to look at numbers.
The point is not to criticize the review sites or people, but to point out a big weakness to the approach. If you get fixated on a number, people can and will take advantage of it.
Cheating on video benchmarks is almost as old as video cards. I recall companies doing things like encoding the video benchmark data in ROMs to improve the numbers, and that was years and years ago.
That’s newbie stuff, though, in comparison to the concept of “make sure the number they look at is good, and to hell with everything else.” That’s probably as old as the Pyramids. Pharoah’s bureaucrats probably cooked their papyrus books to come up with the number the Big Boss wanted.
Whenever you see a highly processed number (and most benchmarks are precisely that), the number itself is pretty meaningless unless you understand just how they got that number.
A highly processed number contains lots and lots of invisible elements. Some can be dubious, some can be downright evil, but you’ll never know from just looking at the end result.
It’s not that computer hardware enthusiasts are village idiots. This is a very, very common human trait all of us are guilty of in at least some aspect of our lives.
A single number is easy. The total reality is harder, sometimes a lot harder, and those trying to promote what they want, whether it be you buying a video card or a political position, rarely have an interest in presenting all sides of the story.
Ignorance is not bliss.
If you don’t do this, this wasn’t meant for you. There obviously are many for whom it does apply, otherwise, ATI wouldn’t have done it. If this does apply to you, the point is to open your eyes, as anonymously as possible. I have no idea who this applies to unless you tell me.
Get mad at the people who are sneaking up behind you and whispering sweet little nothing while taking advantage of you, not the one who notices.
Tomorrow, how the “Family Feud” approach to reading reviews can be manipulated, too.