Video Quality

Got this note this morning:

Ed, I have to agree with you about the numbers game. 3Dmark and Quake3 tunnel vision by the hardware sites may have caused the demise of 3dfx almost single-handedly.

I have a V5, and it is still my favorite card. I also have a GF2 GTS and a GF3. The GF3 is a bit of a disappointment [a GF2 does better on lower-performing systems]. For flight sims, nothing touches a V5 using FSAA.

. . . A V5 running Falcon 4.0 at 1024×768 w/ 2x FSAA looks like video. On a GF3 it looks like crap. Benchmarks don’t tell you that, and many sites don’t either.

The point being, as you said, a number from a benchmark tells you very little unless you qualify what that number means.

This note illustrates an old program in a new arena.

It’s easy to measure frames per second. It’s easy to run an automated benchmark which spits out a number. It’s not so easy to measure quality.

This has been a pretty big reason why we’ve been loathe to really look at video cards. Neither Joe nor I can exactly spot an incorrect pixel from fifty feet, if you know what I mean.

Even if we could, our eyes are not your eyes. What we think is good may well be bad to you. For instance, the lines on Trinitron monitors don’t bother me at all. For those it drives mad, not only would they call me biased, they’d actually have good reason to be.:)

While these are more than quibbles, we probably could come up with reasonable answers.

But if we did, would most of you care?

Let me put it this way, if there were two new “hot” video cards being compared and you had to choose between a video quality benchmark and a Quake benchmark, which would you pick?

I think most would take the Quake even though any difference between the two cards would be meaningless in any real-life usage. After all, that’s just what they’ve been doing.

Let’s ask another, more realistic question. Let’s say in the comparison, Video Card #1 has a noticeably but not dramatically better picture than Video Card #2, and gets 9000 in 3DMark2001. Video Card #2 gets 10500.

Which do you pick? So far, the majority answer seems to be Video Card #2, so much so that most people aren’t even very interested what Video Card #1 has to offer.

Do you know why I feel pretty safe saying that? If I were incorrect, there would be plenty of image quality video software and benchmarks out there, and plenty of discussion about it. Where is it?

In fact, there’s very little video image quality software out there. BapCo had Video 2000. Is there a Video 2001? No.

There’s DisplayMate software, and that seems to be pretty much it.

Why Is That?

I think there’s two reasons:

The Instinctual Drive To Competition, No Matter How Meaningless All this talk about quality matters is the purpose of buying
a video game is to improve the gaming experience.

Is it? Or is there another game being played?

If most people aren’t really playing a game, but are more interested in playing the geek version of that ancient benchmark “I got a bigger one,” quality pretty much goes out the window, doesn’t it?

Up to a point in both games, “bigger is better” is probably true. The problem is, in both areas, there comes a point where bigger isn’t better anymore.

In “I got a bigger one,” eight is better than two. Probably four or even more times better.

But that doesn’t make thirty-two four times better than eight or even as good.

Nor is a eight that fails a lot better than a good reliable seven. But you wouldn’t know that just looking at the numbers, would you?

Video benchmarking numbers are quite the same. There’s no point getting a big one if you can’t keep it up or effectively use it.

Sometimes I wonder what’s the more popular game is: Quake, or benchmarking it.

There Can Only Be One

I’ve railed about this before, so I’ll keep it short. People want a single all-purpose number so they don’t have to think, or
thinking that is somehow more “objective,” not knowing or caring if that number mixes apples, oranges, grapefruit, and a few dozen others, or is about as meaningful as
“42” was in Douglas Adams’ books.

Let’s See How Hooked You Are

As I mentioned the other day, I have a Radeon 8500 card. Sooner or later (probably sooner), we’ll have a GF3 Ti card, too.

Something we always keep in mind before we do something is, “What do we bring to the picture?” That’s the main reason why you don’t see us running 912 benchmarks on a product, plenty of others do just that well before we could. We rather look at the aspects others generally don’t cover.

I personally see no point in running orthodox comparisons; other places have been there, done that.

What I think is something worth exploring is focusing on image quality, and coming up with ways to do that.

So here’s the question, and we’ll give you a vote:

We can:

1. Solely benchmark image quality or
2. Solely benchmark performance.

If you choose “image quality”, I’ll run a few performance benchmarks, but no numbers. I’ll just very briefly describe relative performance in a sentence or two. If you choose “performance,” we’ll drop the idea of image quality.

Choose one or the other. Either/or. DON’T TELL ME “BOTH”. “Both” is a wuss answer and NOT acceptable; do you fill in all the dots on multiple-choice tests, too?

If you somehow manage to ignore the above, I’ll count it as a vote for performance because you’re obviously hooked on those performance numbers. By answering “both,” you also agree to become subject to highly negative observations about your level of reading comprehension and ability to follow simple instructions (hey, it works for shrinkwrap software licensing :))

Drop me a line.

Email Ed