Windows Showdown: 8 Operating Systems in 6 Benchmarks

Since its debut, Windows Vista has taken nothing but flak from almost every demographic one could think of. Everyone from the casual user looking to browse the web and type up a few reports to the benchmark fanatic obsessed with squeezing all the speed he or she could is likely to complain about Vista being bloated and slow. Windows 7 on the other hand has been hailed as being noticeably better performing, and supposedly as light as XP. And what about XP? How do they really stack up to one another? The examination of these questions follows.

Personally, I’m an avid benchmark junkie, so I could only look from this perspective. I’m unconcerned with how things “feel”, but rather how they score. Hard numbers are what matter to me. They might not matter to many, but they measure speed in its true essence, devoid of any subjectivity. Bearing this in mind, I selected six of the most popular benchmarks used by overclocking enthusiasts, each tending to have unique biases with regards to what part of the system they emphasize.

Editor’s Note: While the author is being modest, Gautam is a world renowned benchmarker, and is an authority on the subject of Windows benchmarks.

The Benchmarks

3DMark03 – Predominantly measures GPU performance
3DMark05 – Predominantly measures CPU and memory performance
3DMark06 – Measures both GPU and CPU/memory, and additionally tests multi-threaded performance
Aquamark3 – Almost exclusively measures CPU and memory performance, with an emphasis to the latter
SuperPi 1M – Measures single-threaded CPU performance and is slightly influenced by memory
wPrime 32M – Measures multi-threaded CPU performance with no influence from memory
Some might be wondering why 3DMark Vantage was omitted. The main reason is that it would be a bit boring. Each operating system appears to score nearly identical in 3DMark Vantage, and any variations are within the margin of error.

System Configuration

I used a setup that I would consider fairly typical for an overclocking enthusiast:

Intel Core i7 965 Extreme at 4 GHz
ASUS P6T Deluxe OC Palm Edition
6GB Corsair Dominator GT 2000C7 at 960MHz CAS 7
2x ATi Radeon HD 4890’s at stock frequency of 850/975

To be perfectly honest, the system configuration will likely have an impact on how the various operating systems compare with each other. Therefore, using one that is modern and high-performing is, in my eyes, the fairest way to compare them.

The Operating Systems

Windows Server 2008 x64
Windows Server 2008 x32
Windows 7 x64
Windows 7 x32
Windows Vista x64
Windows Vista x32
Windows XP x64
Windows XP x32

The operating systems are the usual suspects, all with the latest service packs installed. I added Windows Server 2008, as some people have supposed that it is faster than Vista, which it is based on, and I wished to put that theory to the test. Additionally, I tested both 32-bit and 64-bit variants of each operating system. How they handle the memory subsystem is important when it comes to benchmark performance, as we will see. Lastly, in order to make the tests fairest for the operating systems, I trimmed all eight of them using nLite and vLite. Consequently, I made the running services constant between all of them to rule any out as a factor. My vLite profile is as follows:

Only the important stuff remains, with all the fluff removed.
And nLite (for XP 32 and 64):

The Results

Only one thing predominantly sticks out when viewing the results for 3DMark03—good ol’ XP doesn’t fare too well, while all the others are very close, with Windows 7 being slightly in the lead. Since 3DMark03 is heavily GPU-centric, this dead heat is not too much of a surprise. The benchmark depends mostly on GPU performance and is not heavily influenced by much on the system side, OS included. Still, it certainly shows XP’s obsolescence.

Now is when things start getting interesting. 3DMark05 emphasizes CPU and memory performance, and consequently we can see the operating system having a very noticeable impact on performance. In fact, that the only two operating systems that even perform similarly are Server 2008 and Vista. This does not come as much of a surprise, considering that the two are mostly the same under the hood, and are even more similar after I ensured that the running services and installed components were as close between them as possible. XP once again lags far behind the rest of the pack, but interestingly enough both 7 32 and 7 64 also score considerably lower than Vista and Server 08. 7 and XP being the worst performers certainly flies in the face of conventional beliefs. Another very interesting thing to note is that the 64-bit variants for 7, Vista and Server 08 all perform worse than their 32-bit counterparts. We must bear in mind that this benchmark uses under 1GB of memory, but for this quantity, the 64-bit OSes handle the memory sub-system a bit slower.

The results in 3DMark06 are somewhat similar to those in 05, however, this time around Windows 7 pulls up far ahead, scoring almost evenly with Vista. Also, the hit going from 32-bit to 64-bit in Windows 7 is much smaller than it is going from 32-bit to 64 in Vista and Server 2008. XP is still decisively in last place, but the margin is a bit smaller this time around, thanks to XP scoring better in the CPU test portion of 3DMark06 than the newer OSes.

The results from Aquamark3 are quite similar to 06. Windows 7 once again makes a strong showing, and once again, 64-bit does not seem to hurt 7 very much, but takes a slight toll on Vista. Both versions of XP are far behind, but curiously enough XP 64 is considerably better than XP 32. Server 2008 is similar to Vista, however it’s only fair to point out that run #3 for Server 08 32 was a bit of an outlier, what one would call an unlucky run. The first two runs had it performing on par with Vista.

One important thing to note about Aquamark3 in particular is that there is a heavy dependence on graphics drivers. These results only look this way on ATi GPU’s, like those used in this test. On nVidia GPU’s, XP is actually slightly ahead of the others. You’ll have to take my word on that since nVidia results aren’t included in this roundup, but curiously enough, running ATi in Windows 7 scores about equal to a comparable nVidia setup in XP.

These results are very different indeed from those obtained in the 3D benchmarks, and are almost completely the opposite. XP 64 has a noticeable lead over all the others, and is also the most consistent. Interestingly enough, this is the only benchmark where Server 2008 appears to be considerably faster than Vista. However, just like in the 3D benchmarks, the 64-bit variants of Vista and Server 2008 are slower. In 7 it’s the complete opposite, with 7 64 noticeably outperforming 7 32, further supporting that the 64-bit version of 7 does indeed seem to be optimized in some way that 64-bit Vista is not.

I’ll start out by saying that I tried to work out exactly why XP 64 scored so poorly, but I’m afraid I can’t offer any explanation, so it has to be taken at face value. Otherwise, XP 32 is still ahead of the newer OSes, but by a smaller amount than it is in SuperPi. All OSes in fact are very close to each other, barring XP64. Windows 7 though once again shows some weakness on the 2D side of things, but 32-bit and 64-bit are in a dead heat.

Conclusion

So, who’s the winner? Well, if you’ve scrolled to skip past the graphs, every single benchmark has a unique operating system that does best. Overall, the two most solid performers are Server 2008 32 and Vista 32. Both of these are at the top for the 3D benchmarks, and fare okay in the 2D benchmarks as well. Deserving of flak in every day usage or not, in benchmarks, Vista 32 performs very well. Contrary to popular belief, XP and 7’s supposed “lightness” does not really translate in benchmarks. In fact, the more CPU-centric a benchmark is, the worse 7 tends to do. The only thing XP remains good for are 2D benchmarks, falling far behind the pack in all things 3D. Once again, this article only sets out to show which the fastest operating systems are by the numbers. The fastest choice might not necessarily be the best one for you.

Questions and discussion of this article are on Overclockers Forums, join in!

– Gautam