Overclockers

There has been some discussion recently about how we rate products on Overclockers.com. We typically publish three or more reviews per week currently, from a number of different reviewers. People read our reviews for several reasons, but most people have one aim in mind: is it any good? A vast number of different types of ratings are used across review sites, ranging from simple stars and points rankings (n out of 3, 5, 10 …) to percentages, broken down into 10, 5 or even 1% graduations. Other sites take this a step further and have an overall rating derived from a number of sub-categories on which the product is judged. This article takes a look at the different systems, and sets out exactly how we think about reviews and ratings at Overclockers.com.

How Many Categories Are There?

Percentage systems are good for some things, not so good for others. Sites like Metacritic work well with percentages, as they are collating an average score from a number of sites. If Site A rates something as 3/5 stars, Site B rates it as 4/9 points and Site B rates something as 80%, then percentages are a nice easy way to harmonize these; the average would be 61.5%, for example. Or 6.15/10 stars. Or 1.45 dancing antelopes per purple zebra. You get the point, I’m sure.

However, from the point of view of a reviewer: how precise can you get? We can all understand that a “78%” graphics card is better than a “45%” graphics card, but what about versus an “81%” graphics card? Is the extra three percent because the reviewer got a couple of extra MHz out of it, or because the PCB was an attractive color of purple. Having such a finely grained rating system is only workable if there is an incredibly precise set of metrics that each and every product is assessed against.

How Many Reviewers Are There?

Overclockers.com doesn’t employ paid writers. Only one member of Overclockers.com staff, Matt B. (a.k.a. I.M.O.G.), gets a wage from iNet Interactive (who own this site and the associated forums) and it’s not for writing. We source our reviews from a number of people, such as mdcomp, hokie, splat and mudd have contributed a number. Others write one or two articles. It’s a community site, so standardizing review formats is difficult. Each of the editors has spent a number of hours changing article formatting, language and layout to try and make sure we’re as consistent as possible. There will always be noticeable differences between articles – everyone is different, after all – but we try to present as consistent a format as possible. A finely grained rating system makes this a lot harder, as it introduces too many subjective elements to the rating.

The Rating System We Use

With all these things in mind, we’ve stuck to using three ratings. Below is a picture of each badge, plus a description of what it means. If these were to be compared to a star-type rating system, these would most likely correspond to 1, 2 and 3+ stars.

	Fail The product doesn’t do what it purports to, for one of a number of reasons. The exact reason will be specified in the review. It either fails completely, in which case we do get in contact with the manufacturer in case it was a single dud sample. It might have failed to perform, even at stock. It may be a completely overpriced product. It may be out of date, pointless, or otherwise a complete waste of money. It’s a product we could not recommend to anyone, and something you should avoid.
	Meh The product doesn’t outright fail, but it doesn’t quite do the job properly. This is typically a product with mediocre performance, a poor price/performance ratio or a product that is a little behind the times. Alternatively, it could be a product marketed at overclockers that doesn’t overclock well at all. Better alternatives likely exist at a similar price, making the product difficult to recommend. It won’t set your PC on fire, but it’s just not as good as other products in the same market segment.
	Approved The product performs well at stock and at overclocking, for modding, etc. where relevant. It isn’t necessarily the best of its type, but it performs well enough that we could recommend it with a clear conscience.

Final Thoughts

We want people to read our reviews; our reviewers put a serious amount of work into each and every review they write. Please read what they’ve written – don’t rely on an often quite arbitrary points, percentages, or stars system. One of the most important parts of any given review is where the pro’s and con’s are compared, and this is where you will get a good feeling for whether this is a product you want to own or not.

Thanks go to a number of forum staff for their input into this discussion: Jay_III, muddocktor, splat, hokiealumnus, I.M.O.G., splat, MattNo5ss, Thideras, macklin01 and the rest.

I give this article a hearty 4.3 dancing pink antelopes per watermelon-eating brown cardboard tiger-chewed bucket of sugar!