Windows Showdown: 8 Operating Systems in 6 Benchmarks

Add Your Comments

Since its debut, Windows Vista has taken nothing but flak from almost every demographic one could think of. Everyone from the casual user looking to browse the web and type up a few reports to the benchmark fanatic obsessed with squeezing all the speed he or she could is likely to complain about Vista being bloated and slow. Windows 7 on the other hand has been hailed as being noticeably better performing, and supposedly as light as XP. And what about XP? How do they really stack up to one another? The examination of these questions follows.

Personally, I’m an avid benchmark junkie, so I could only look from this perspective. I’m unconcerned with how things “feel”, but rather how they score. Hard numbers are what matter to me. They might not matter to many, but they measure speed in its true essence, devoid of any subjectivity. Bearing this in mind, I selected six of the most popular benchmarks used by overclocking enthusiasts, each tending to have unique biases with regards to what part of the system they emphasize.

Editor’s Note: While the author is being modest, Gautam is a world renowned benchmarker, and is an authority on the subject of Windows benchmarks.

The Benchmarks

3DMark03 – Predominantly measures GPU performance
3DMark05 – Predominantly measures CPU and memory performance
3DMark06 – Measures both GPU and CPU/memory, and additionally tests multi-threaded performance
Aquamark3 – Almost exclusively measures CPU and memory performance, with an emphasis to the latter
SuperPi 1M – Measures single-threaded CPU performance and is slightly influenced by memory
wPrime 32M – Measures multi-threaded CPU performance with no influence from memory
Some might be wondering why 3DMark Vantage was omitted. The main reason is that it would be a bit boring. Each operating system appears to score nearly identical in 3DMark Vantage, and any variations are within the margin of error.

System Configuration

I used a setup that I would consider fairly typical for an overclocking enthusiast:

Intel Core i7 965 Extreme at 4 GHz
ASUS P6T Deluxe OC Palm Edition
6GB Corsair Dominator GT 2000C7 at 960MHz CAS 7
2x ATi Radeon HD 4890’s at stock frequency of 850/975

To be perfectly honest, the system configuration will likely have an impact on how the various operating systems compare with each other. Therefore, using one that is modern and high-performing is, in my eyes, the fairest way to compare them.

The Operating Systems

Windows Server 2008 x64
Windows Server 2008 x32
Windows 7 x64
Windows 7 x32
Windows Vista x64
Windows Vista x32
Windows XP x64
Windows XP x32

The operating systems are the usual suspects, all with the latest service packs installed. I added Windows Server 2008, as some people have supposed that it is faster than Vista, which it is based on, and I wished to put that theory to the test. Additionally, I tested both 32-bit and 64-bit variants of each operating system. How they handle the memory subsystem is important when it comes to benchmark performance, as we will see. Lastly, in order to make the tests fairest for the operating systems, I trimmed all eight of them using nLite and vLite. Consequently, I made the running services constant between all of them to rule any out as a factor. My vLite profile is as follows:

vlite profile

vlite profile

Only the important stuff remains, with all the fluff removed.
And nLite (for XP 32 and 64):

nlite profile

nlite profile

The Results

3DMark03 Results

3DMark03 Results

Only one thing predominantly sticks out when viewing the results for 3DMark03—good ol’ XP doesn’t fare too well, while all the others are very close, with Windows 7 being slightly in the lead. Since 3DMark03 is heavily GPU-centric, this dead heat is not too much of a surprise. The benchmark depends mostly on GPU performance and is not heavily influenced by much on the system side, OS included. Still, it certainly shows XP’s obsolescence.

3DMark05 Results

3DMark05 Results

Now is when things start getting interesting. 3DMark05 emphasizes CPU and memory performance, and consequently we can see the operating system having a very noticeable impact on performance. In fact, that the only two operating systems that even perform similarly are Server 2008 and Vista. This does not come as much of a surprise, considering that the two are mostly the same under the hood, and are even more similar after I ensured that the running services and installed components were as close between them as possible. XP once again lags far behind the rest of the pack, but interestingly enough both 7 32 and 7 64 also score considerably lower than Vista and Server 08. 7 and XP being the worst performers certainly flies in the face of conventional beliefs. Another very interesting thing to note is that the 64-bit variants for 7, Vista and Server 08 all perform worse than their 32-bit counterparts. We must bear in mind that this benchmark uses under 1GB of memory, but for this quantity, the 64-bit OSes handle the memory sub-system a bit slower.

3DMark06 Results

3DMark06 Results

The results in 3DMark06 are somewhat similar to those in 05, however, this time around Windows 7 pulls up far ahead, scoring almost evenly with Vista. Also, the hit going from 32-bit to 64-bit in Windows 7 is much smaller than it is going from 32-bit to 64 in Vista and Server 2008. XP is still decisively in last place, but the margin is a bit smaller this time around, thanks to XP scoring better in the CPU test portion of 3DMark06 than the newer OSes.

Aquamark3 Results

Aquamark3 Results

The results from Aquamark3 are quite similar to 06. Windows 7 once again makes a strong showing, and once again, 64-bit does not seem to hurt 7 very much, but takes a slight toll on Vista. Both versions of XP are far behind, but curiously enough XP 64 is considerably better than XP 32. Server 2008 is similar to Vista, however it’s only fair to point out that run #3 for Server 08 32 was a bit of an outlier, what one would call an unlucky run. The first two runs had it performing on par with Vista.

One important thing to note about Aquamark3 in particular is that there is a heavy dependence on graphics drivers. These results only look this way on ATi GPU’s, like those used in this test. On nVidia GPU’s, XP is actually slightly ahead of the others. You’ll have to take my word on that since nVidia results aren’t included in this roundup, but curiously enough, running ATi in Windows 7 scores about equal to a comparable nVidia setup in XP.

SuperPi 1M Results

SuperPi 1M Results

These results are very different indeed from those obtained in the 3D benchmarks, and are almost completely the opposite. XP 64 has a noticeable lead over all the others, and is also the most consistent. Interestingly enough, this is the only benchmark where Server 2008 appears to be considerably faster than Vista. However, just like in the 3D benchmarks, the 64-bit variants of Vista and Server 2008 are slower. In 7 it’s the complete opposite, with 7 64 noticeably outperforming 7 32, further supporting that the 64-bit version of 7 does indeed seem to be optimized in some way that 64-bit Vista is not.

wPrimeResults

wPrimeResults

I’ll start out by saying that I tried to work out exactly why XP 64 scored so poorly, but I’m afraid I can’t offer any explanation, so it has to be taken at face value. Otherwise, XP 32 is still ahead of the newer OSes, but by a smaller amount than it is in SuperPi. All OSes in fact are very close to each other, barring XP64. Windows 7 though once again shows some weakness on the 2D side of things, but 32-bit and 64-bit are in a dead heat.

Conclusion

So, who’s the winner? Well, if you’ve scrolled to skip past the graphs, every single benchmark has a unique operating system that does best. Overall, the two most solid performers are Server 2008 32 and Vista 32. Both of these are at the top for the 3D benchmarks, and fare okay in the 2D benchmarks as well. Deserving of flak in every day usage or not, in benchmarks, Vista 32 performs very well. Contrary to popular belief, XP and 7’s supposed “lightness” does not really translate in benchmarks. In fact, the more CPU-centric a benchmark is, the worse 7 tends to do. The only thing XP remains good for are 2D benchmarks, falling far behind the pack in all things 3D. Once again, this article only sets out to show which the fastest operating systems are by the numbers. The fastest choice might not necessarily be the best one for you.

Questions and discussion of this article are on Overclockers Forums, join in!

Gautam

Leave a Reply

Your email address will not be published. Required fields are marked *

Discussion
  1. This is a GREAT article. Especially for overclockers who trim their OS's. Awesome work!
    I would love to have seen a stock vs. stock OS comparison as that is what most people (non serious benchers) run.
    I have a few questions.
    Firstly: how do they perform if you strip them all down to the bare minimum. I understand why you chose to keep them all on an even keel for this article but I think there would be some value in doing the tests again with each OS stripped down as much as possible.
    Secondly: How do Vista and Vista SP1 compare?
    Thirdly: Would it be worth seeing how XP/XPSP1/XPSP2/XPSP3 compare?
    A very interesting article, which also raises some questions :-)
    Firstly: how do they perform if you strip them all down to the bare minimum. I understand why you chose to keep them all on an even keel for this article but I think there would be some value in doing the tests again with each OS stripped down as much as possible.

    He did this, didn't he?
    johan851
    He did this, didn't he?
    I thought the same thing...:chair:
    However I believe he wanted a COMPLETELY stripped down OS as opposed to the version he has. I will hope he clarifies soon.
    Personally, I would have rather seen this run at all stock since thats how 99% of people here run their OS's. :D
    EarthDog
    Personally, I would have rather seen this run at all stock since thats how 99% of people here run their OS's. :D

    Go to google and search for Windows benchmarks - you'll see plenty of places doing stock windows comparisons. ;)
    Taking out all the extra garbage that runs on the OS, and actually evaluating each release on an even scale, performed by someone who knows what they are doing - that hasn't been done anywhere else. (Except maybe on xtremesystems, assuming he might have released his stuff over there too) ;)
    I.M.O.G.
    I was just messing with you bud, and I agree it wouldn't hurt to have someone here doing the stock tests that we can trust. ;)
    I know, all good, never thought otherwise :D.
    And that was my other thought (stock tests here we can TRUST!).
    These guys have A LOT to offer and its wonderful to see them even more involved!!! W00t!
    Thanks guys. :)
    David
    I have a few questions.
    Firstly: how do they perform if you strip them all down to the bare minimum. I understand why you chose to keep them all on an even keel for this article but I think there would be some value in doing the tests again with each OS stripped down as much as possible.
    Secondly: How do Vista and Vista SP1 compare?
    Thirdly: Would it be worth seeing how XP/XPSP1/XPSP2/XPSP3 compare?
    A very interesting article, which also raises some questions :-)

    If I went any lower it would be at the point where they start acting goofy. Let me put it this way, I can already fit all my OSes on CD's (not DVD's) and installation takes about 8 minutes tops.
    Not only that IMHO "stripping" is overrated anyways. I don't think disabling services or removing components changes how an OS scores much anyways. I made them all even though just for the sake of being thorough. Vista without SP1 is wrought with issues especially with a Crossfire setup like I used, so it wouldn't quite be right to try without it. And that along with your 3rd question can also be answered like this. 3DMark06 takes almost 9 minutes to run all by itself. Then multiply that by 3 and you're easily surpassing 30 minutes per operating system per benchmark. I think you know where I'm going with this. :p
    nikhsub1
    Very Nice Guatam! I can't help but wonder how 2008 R2 would fair...??? Isn't R2 based on a different kernel?

    2008 R2 is based on 7. I'm thinking of giving it a whirl as well, but we can probably expect the difference between 2008 R2 and 7 to be similar to that between 2008 and Vista (in other words, negligible)