I’m beginning to think this is going to be an educational experience in more ways than one.
I’ve been reading through the comments on video quality, and I think to make a few reassurances and other comments.
What Do We Consider To Be “Quality” There’s no point in having a video card that delivers perfect images in a 3D game at 3 FPS. For us,
a video card is doing a good quality job when it delivers highly accurate and seamless results. So speed does count up to a point.
What’s that point? Good question. It certainly is going to have to be a good deal higher than the minimum 24fps to account for variations. On the other end, I’m going to have to
hear one hellacious argument to think it has to be higher than the refresh rate of the monitor (say, 85 FPS).
A number I’m seeing tossed around a lot for pretty much guaranteed playability is around 60 FPS. Unless I see some very persuasive arguments otherwise, I think 60-65 fps is likely to be the “who cares if it’s more” boundary line.
How Do You Determine Quality?
To us, video quality consists of two elements:
- Accuracy
- Seamless motion consistency
We consider accuracy to be the core basis of quality. If you don’t have an accurate frame, how good can it be?
However, frame accuracy in-and-of-itself isn’t good enough. It’s the difference between looking at a painting and looking at a movie. You can have the greatest photography in the world in each of a movie’s frames, but if the movie projector is herky-jerky, that’s no good either.
What Areas Are You Going To Measure?
From what we’ve seen so far, eventually, we need to look or at least try to look at the following areas:
- Gaming
- DVD reproduction
- Still image reproduction
- Text quality
I am still open to arguments about other areas, though you’re going to need to show why the video card rather than something else (i.e., rendering) makes a real difference.
What Are You Going To Use To Test?
A lot of you sent me this URL: http://www.geocities.com/legion88/. This details the quality test for 3DMark2001 Pro, essentially by taking four frames from
the benchmark run and testing to see how accurately the card portrays the image. I think this is a good start, but since a stationary target like this one is subject to cheating, what I’d rather have in the long run is some program or procedure that would let me do the same thing for any image I’d like.
We’re also going to take a look at the Video2000 program and one or two other items to see what they have to offer, also.
That’s probably going to be the very first installment of what will be a long-running series. To a large degree, this is unexplored territory, so this is going to be more
a reconnaissance mission than anything else. Since there isn’t going to be anything new on the horizon for a while, it’s a good time to take time to find out what works and what doesn’t for future generations of cards.
If you get the sense I’m groping around a bit, I sure am. I’m trying to get feedback as I think this through, while trying to keep this within our financial and human resources (more on that later).
What Are You Going To Test?
Right now, we’re looking at four cards: the ATI Radeon 8500, the Visiontek Ti200, the Matrox G450 and the Kyro II. We think that offers a reasonable sample of what’s available today. Remember, the emphasis is on quality, not speed.
I gave a lot of thought about buying a Voodoo 5 but decided against it because it’s not going to have any direct offspring, and even in its areas of strength, the next generation or two of games/sims, etc. should render it to history.
Again, we’re trying to establish a reasonable methodology for the future. I can’t do that trying 92 cards.
What Are You Going To Test It With?
The computer itself will probably be an AthlonXP running on a Via KT266A board. Tests will be run at stock speed.
I very much doubt we’ll buy a PIV system just to do this, simply because most people don’t have PIVs, and any differences between the two platforms seem to be speed, not quality.
I have two 21″ Sony G500 Trinitron monitors to look at. When I get to DVD, I’ll be using a Lite-On LTD-163.
I will use the latest video drivers available at the time of testing. I don’t plan on revisiting tests I’ve already done just because somebody comes out with new drivers. If they’re announced as making a major difference in quality, then I’ll consider revisiting them.
What Sort of Settings Will You Use For Games?
That’s not decided yet, but you’ll probably see two or at most three settings.
How Are You Going To Judge?
I’m going to try to follow two basic rules:
What Are Your Biases?
I’ve been using a Matrox/Trinitron combo for a long time. That’s going to be the standard by which I’m going to be looking at the others.
The Matrox “bias” is probably a good one to have. It’s a very good 2D card, so I have a high standard by which to judge the others. It’s a terrible 3D card, so there’s no chance I’d tell gamers to buy one.
Using a Trinitron may give a few fits, especially in the text legibility area.
I may “farm out” a few tests in those areas where I think the type of monitor may make a big difference. When I use screen shots, I’ll try to also provide a reference image so people can compare.
One Man Can Just Do So Much
The biggest problem I foresee doing this is not figuring out what to do or how to do it, but to get people to understand that limits have to be set.
That’s what I meant by “the perfect being the enemy of the good.”
First, keep in mind that I’m running these tests for the whole audience, not just you. I’ve already gotten plenty of emails suggesting elaborate solutions for their pet peeves with at least a gentle hint of “My way or no way” (for themselves).
This is not going to happen. Even if we had all the money needed to buy everything you wanted tested, it still wouldn’t happen, and I’ll explain why.
The amount of testing required grows geometrically as you add more items. For instance, if you want a PIV, that all by itself doubles the testing and thus doubles the time to complete it. Add a different monitor, too, and you’ve quadrupled the testing.
If I did literally everything suggested so far, by the time I was finished, it would all be obsolete.
If I literally tested everything each of you liked the way you liked it, I’d probably be dead before I was finished.
Even if I with an army of people managed to do all of this timely, in all likelihood, nobody would read it. Why? Because it would be too long and complicated.
Keep in mind the current environment. People want something simple and quick. The goal is to see if we can come up with a reasonably simple and quick measurement that covers a whole lot of ground at least decently.
We may very well find that isn’t the case. I doubt we’ll be able to measure overall quality with something as simple as FPS, but it shouldn’t take reams of data, either.
That’s probably going to boil down to generalizations, if they can be made. If a number won’t do it, a summary paragraph or two should.
While doing this, I’ve looked around quite a bit at other attempts to measure quality. To a large degree, it seems like a lot of places tried this once or twice, then dropped it like a hot potato.
Don’t know why, but I hope it wasn’t because people whined and moaned that it wasn’t a perfect standard for them. Maybe this is why quality testing never took off.
Remember, something is better than nothing.
Be the first to comment