• Welcome to Overclockers Forums! Join us to reply in threads, receive reduced ads, and to customize your site experience!

A lot more info on Win XP vs Win2k

Overclockers is supported by our readers. When you click a link to make a purchase, we may earn a commission. Learn More.

Itchie

Member
Joined
Aug 15, 2001
Location
Barrie
I got this from the usenet.  Pretty interesting.
-------------------------------------------------------

"I think we did a great job with Win2k's performance. We set a new bar and have had to work like mad to WinXP to that same level of performance.

My team and I know quite a bit about the performance differences between Win2k and WinXP, we directly wrote some of the code that allows XP to perform as it does today and worked with a huge list of other developers to get this product performing well.  We've run a wide variety of performance benchmarks/workloads on a wide spectrum of machines.  We've started to provide information via the web, but haven't completed that effort yet. This is still not the final product, after all. That said, let me give you a sense of where we stand, what our goals were at the onset, and why we chose those goals. Where we stand (128mb or more).

1.  XP is faster than Win2k at some things, but clearly not everything.

2.  Win2k is faster than XP at some things, but clearly not everything.

3.  XP, like Win2k, is faster than Win9x & WinME at just about everything, but some gaming benchmarks.

Some perspective

*** NT4 vs Win2k
Win2k is bigger than NT4. There is more code to execute. Win2k has more UI work to do as well. Win2k generally outperforms NT4 because it manages
memory better, had an improved registry, etc....

Managing memory better is key.  We made fewer mistakes in page replacement than did NT4. So, even though bigger, we were able to offset the costs by being more efficient.  With respect to the registry, the high costs of the registry are often associated with disk i/o's needed to bring in data from the registry.  Win2k did a better job keeping the right registry bytes in memory. In a sense, the registry improvements were just memory mgmt improvements as well.

*** WinXP vs Win2k
WinXP is bigger than Win2k.  There is more code to execute. WinXP has more UI work to do as well. Sounds familiar, doesn't it? To offset the added code and UI work, WinXP needs to manage memory better & improve disk i/o efficiency.  In a sense, we can pay for the costs of a new UI, new services like system restore, and so forth by cutting our disk I/O costs. The disk, by far, is the biggest bottleneck on the system. We focused heavily on reducing disk i/o costs.

   *********************************************
For the most part, the balance equation is this
If (Video/CPU Costs of New UI & New Code)  <   (Savings Gained From Better Mem Mgmt & Disk I/O efficiency)  Then XP will be faster than Win2k    Else, Win2k will be faster.

 **********************************************

****Some of what we've looked at****
   Boot,   (WinXP generally beats Win2k by a wide margin.)
   Resume from Standby,  (WinXP generally beats Win2k by a wide margin.  XP
can resume from standby in <2 seconds on many laptops)
   Resume from hibernate, (WinXP beats Win2k here as well)
   Application Launch (WinXP generally beats Win2k for
cold starts, not so if the app is completely in memory already)
   Content Creation Winstone 2001, (XP and Win2k can both be best here, See
Below)
   Business Winstone 2001, (XP and Win2k can both be best here. (See Below)
   Webmark 2001, (XP is typically winning by a small margin) (See Below)
   Sysmark 2001, (Win2k is currently edging out WinXP, again small margin.)
(See Below)
   PC WorldBench 2000,  (Win2k will beat XP on this, unless you click Best
Performance)(See Below)
   I-bench, (we've not run it in a while.  Win2k should have the edge.)
(See Below.)
   Gaming Benchmarks (I'm gonna skip this now, we can discuss in another message. It is really separate wrt to the central points below)
    OfficeBench, we've never run this.  Results from this are on the anandtech site.  I've spoken to the developer.  To be sure, short running benchmarks that take the disk out of the equation are not sufficient to judge the full performance of a product the size of Windows.  Managing the disk for improved efficiency is the most impressive performance advance in Windows XP. It really is cool and new stuff.  A measure of operating system performance w/o incorporating disk i/o and memory mgmt decisions is incomplete. I'm sure it measures something, but it simply is not possible to make wide sweeping performance claims based on something that runs for 30 secs or less.  A data point?  Yes.  A complete story?  Not even close!

****How do we Run Benchmarks?****
We run the above by taking all OS defaults.  We do NOT turn off services. We do NOT turn off the BIG background bitmap that XP has (which gives Win2k a big edge in some things). We do NOT change any UI settings.  During setup, we clean format the disk and use NTFS.  We take all defaults when setting up the benchmarks and we run them with all defaults as well.  For newer systems, we're generally running 32bpp. Some systems can't support 32bpp, in which case we use 24 or 16 (the max it can do).

======We run workloads one or more times (or just use the apps) and then either WAIT 3 days before timing, or we force the once-every-three-day disk-layout optimizations to occur right then and there.======

That last step sounds odd, doesn't it?  Let me explain.  XP is a dynamic self-tuning OS. One of the most critical things it does is move files around on the disk based upon use.  There is no static list, we build it up dynmaically as you use your system.  Moving files can be a noisy thing, we don't do it often (every 3 days max).  We wait for the system to be idle before we do any idle-time performance work.  Every 3 days we look to see if a new layout for better file placement is warranted.  If it is, we move files around, and keep the files we move contiguous.  This isn't a full defrag so it is usually pretty quick. The code that controls all this is in the Task Scheduler, but not visible as a scheduled task.  If you disable that serivce, you loose this optimization as well as the boot and application launch optimizations. You can trigger the once-every-three-day disk layout optimizations to occur manually by calling the ProcessIdleTasks() api in advapi32.dll.  This can be called from a program or from the cmd line. To call from the cmd line, just run
   rundll32.exe advapi32.dll,ProcessIdleTasks
The cmd line version returns immediately, but the work will proceed in the background.  You need to wait for the dfrgntfs, and wmi* processes to all stop before rebooting.  This can take a few minutes.  Even though the defrag processes are involved, it isn't a full defrag. If you call the API from a program, you'll find it to be synchronous; when the api returns, the work is done.  More information about controlling the dynamic tuning can be found at:
http://www.microsoft.com/hwdev/performance/benchmark.htm

****Rules of thumb for what we measure****

1. Boot

2. Resume from Standby

3. Resume from Hibernate
   The details on these can be found at http://www.microsoft.com/hwdev/fastboot XP should be significantly faster than Win2k across the board.  We did a lot to improve these.

4. Application Launch
   You can find for information at http://www.microsoft.com/windowsxp/pro/techinfo/howitworks/performance/default.asp    XP records previous launches of each application.  It knows the file pages needed and can prefetch them into memory efficiently if they aren't already in memory.  This gives XP the edge when you haven't launched an app recently or some of its pages aren't in memory. For big apps, the difference can be 50%, 3 secs instead of 6.
  If the app is in memory already, Win2k is generally faster.  But here, the times are usually very small (often less than a second to launch an app if no disk i/o is needed).

5. Business Winstone 2001 &

6. Content Creation Winstone 2001
-XP can outperform Win2k on systems with 700Mhz+ processors, 128MB or more of RAM, 16MB of video memory, and 30GB partitions.  You can buy these systems today for ~$700 or so, w/o monitor. -Win2k can outperform WinXP on systems with less capable processors, less capable video, smaller partitions. Those aren't universally true statements. We have some lower end systems, 400MHz, 8MB video, small disks, and such where Win2k is slower than WinXP. We have some higher end systems where Win2k is slightly faster than Win2k. Why does partition size matter?  Well, the truth of the matter is that Win2k's performance degrades as the size of the disk/partition grows.  As the size of the disk increases, so to does the average seek distance for Win2k.  As the seek distance increases, so too does the average seek time for Win2k disk i/os.  XP, because it dynamically moves files based upon usage patterns, doesn't degrade as much (almost no degradation actually) as the size of the partition grows. XP also has a different NTFS format layout than Win2k.  XP also moves files onto the system during setup in such a way that it minimizes the amount of filesystem metadata (MFT entries) required as it runs.

Why does cpu matter?  When taking all defaults, XP executes more code for every window opened, menu pulled down, icon drawn, etc.
Why does video memory matter?  When taking all defaults, XP has the background bitmap. It has more animating, shadowing, and more elaborate icons. The differences between Win2k and WinXP are sometimes small, 1% or so, and sometimes large, 5-10% or so.

7. Webmark 2001
-XP & Win2k are less sensitive to partition size on this workload than the Winstones.  Webmark is a big workload.  It runs for an hour, not just a few seconds, 1 full hour!  We're looking at this workload on 700MHz+ Celerons, Durons and above.  Although, XP is winning across the board on our systems, the margin is so small I doubt anyone would notice the difference.  I also suspect video cards/drivers will allow XP and Win2k to be either slightly slower/faster than the other.

8. Sysmark 2001
- A new benchmark, it really did just started running with consistency on WinXP.  I don't have any general rules of thumb yet.  The two systems we're looking have Win2k edging WinXP.  There are 2 scored parts to this workload. XP wins on one, Win2k on the other. In both cases the margin is small.  As with Webmark, I suspect video cards/drivers will allow one or the other to be slightly faster.
This is a very big workload. It ships on 2 CDs.  it runs for close to 2 hours.

9. PC WorldBench 2000
- Win2k is going to win on this test (and others like it) if you take all defaults with the OS installation, as we do.  PC WorldBench is a good benchmark, I don't mean to be critical of it.  We decided it was OK to lose on this benchmark and others like it.  XP beats Win2k if we turn off two of the default UI settings, "Fade or slide menus into view" and "Show shadows under menus".  If you choose "Adjust for best performance", XP will outperform Win2k by a wide margin. Why?  PC Worldbench runs at a very fast rate.  That's good for evaluating one hardware system against another, but not always so good at evaluating one operating system version vs another. The high rate means the fading in of UI elements is going to have a very demonstrable impact on the overall time it takes to run the workload (and thus the score).  With WinXP, we've added more UI effects.  To pull down a menu, the time it takes to draw a shadow around the box is in the microseconds.  It isn't noticeable by the human eye.  But, if we pull don 1 million menus in a short period of time, that extra shadowing becomes significant. We have looked closely at the operations used within PC WorldBench. It uses real apps to place a load on the operating system and has been immensely helpful in our OS performance efforts.

10. I-Bench
-Like PC WorldBench, we'll probably lose to Win2k on I-Bench.  Win2k is probably slower than NT4 on I-Bench too.  It has sequences where it displays pages, windows, menus at a very high rate. The time it takes to display windows and such is a high percent of the total score. The added costs of WinXP over Win2k are going to show if you take all defaults. To be honest, we've not run I-Bench in quite sometime.  The applications and types of things it does are well represented in many other workloads we have.  If I-Bench were easier to setup/run on a large scale & daily basis (lots of plugins have to be installed), we'd probably still use it regularly.

---- System Restore------
This subject deserves a special note.  The costs of system restore are quite small for the average user.  We leave it on in all the tests we run above. However, I wouldn't leave it on for any development.  As executables, drivers, dlls and such are created, system restore notes their creation and is going to create a new "checkpoint" with them included in it.  That's gonna be costly for people doing development.  I'd suggest turning it off for the drives/directories into which you'll be creating images.

------ A note about XP's size & Tools-----
Many people use task manager and such to measure the size of Windows XP. Task manager does not report sharing between process working sets.  With a dozen processes sharing a page, task manager will report that as 12 pages. That's not at all accurate, it should be 1.  Also, Win2k task manager reports on "Mem Usage", now referenced as "PF Usage" (for pagefile usage). PF Usage is simply the amount of "reserved space" from the paging file.  It doesn't correlate to memory usage well at all. Here is a simple example of why PF Usage isn't too helpfull.  First, if a program allocates 1MB of memory but doesn't reference it, that's a 1MB PF Usage charge, but represents 0 pages of real memory usage. Also, working sets are a tricky thing in Windows NT based systems.  In a nutshell, just because a page is in a process working set doesn't mean the OS is unaware that the page is freely available for use if it needs it.  The mem mgr lets processes hold onto pages for long periods of time if there isn't sufficient memory pssure to warrant taking the pages.  That's why people report increased working set figures for processes as they add memory. It doesn't mean the system needs any more memory to operate, nor does it mean the mem mgr isn't paying close attention to page use (it is).

For most processes, the vast majority of pages touched are touched ONLY during the initialization of the process (during launch).  Absent any memory pressure, those pages can stay in memory forever.  If memory pressure increases, XP knows which pages it can go get and can do so very rapidly. Why isn't task manger better?  We don't want it to be intrusive.  The overhead of figuring out page sharing and such is more costly than we'd like.  It could actually require task manager to touch data structures that may not be in memory, making task manager more obtrusive than we'd like.

**** Upgrades *****
The overwhelming number of upgrade performance issues I've investigated are due to fragmentation.  A full manual defrag after upgrading is recommended. I've seen this too often not to make note of it.  Some of the prior posts on the newsgroup suggest other things may be involved (drivers, bios, ....).I'll be looking more closely at that.


**** Summary *****
No benchmark is perfect.  All are open to some degree of criticism.  All provide insight into the performance of the system.  I won't conclude on WinXP's performance based upon a single benchmark, or even the results of a few.  A full spectrum of workloads is needed to evaluate something as functional as Windows, and even then we have to do more. Beyond benchmarks, we also perform a wide variety of side-by-side performance comparisons. We set up pairs of identical machines and sit two people down side-by-side.  As they go through a series of typical and random operations, we can see visibly if Win9x or Win2k has an edge of WinXP on things.  For the most part, the differences are pretty small between Win2k and WinXP. Of course, the most useful metric for me is the responses we get from customers.

I hope this has been informative.
Mike Fortin

Here are the whitepaper links again.
http://www.microsoft.com/hwdev/performance/benchmark.htm
http://www.microsoft.com/hwdev/fastboot
http://www.microsoft.com/windowsxp/pro/techinfo/howitworks
 
very nice like this little code have to check into it!


rundll32.exe advapi32.dll,ProcessIdleTasks
 
Great post! Thanks! Some info in there that ive not been able to find elsewhere.
AKDUDE
 
Back