- Joined
- Jan 17, 2004
Virtual Memory & Pagefile & Swapfile
How Does It Work?
Virtual memory is created using a special file called a swapfile or paging file.
Whenever the operating system has enough memory, it doesn't usually use virtual memory. But if it runs out of memory, the operating system will page out the least recently used data in the memory to the swapfile in the hard disk. This frees up some memory for your applications. The operating system will continuously do this as more and more data is loaded into the RAM.
However, when any data stored in the swapfile is needed, it is swapped with the least recently used data in the memory. This allows the swapfile to behave like RAM although programs cannot run directly off it. You will also note that because the operating system cannot directly run programs off the swapfile, some programs may not run even with a large swapfile if you have too little RAM.
Swapfile Vs. Paging File
We have all been using the terms swapfile and paging file interchangeably. Even Microsoft invariably refers to the paging file as the swapfile and vice versa. However, the swapfile and paging file are two different entities. Although both are used to create virtual memory, there are subtle differences between the two.
The main difference lies in their names. Swapfiles operate by swapping entire processes from system memory into the swapfile. This immediately frees up memory for other applications to use.
In contrast, paging files function by moving "pages" of a program from system memory into the paging file. These pages are 4KB in size. The entire program does not get swapped wholesale into the paging file.
While swapping occurs when there is heavy demand on the system memory, paging can occur preemptively. This means that the operating system can page out parts of a program when it is minimized or left idle for some time. The memory used by the paged-out portions are not immediately released for use by other applications. Instead, they are kept on standby.
If the paged-out application is reactivated, it can instantly access the paged-out parts (which are still stored in system memory). But if another application requests for the memory space, then the system memory held by the paged-out data is released for its use. As you can see, this is really quite different from the way a swapfile works.
Swapfiles were used in old iterations of Microsoft Windows, prior to Windows 95. From Windows 95 onwards, all Windows versions use only paging files. Therefore, the correct term for the file used to create virtual memory in current operating systems is paging file, not swapfile.
Because both swapfiles and paging files do the same thing - create virtual memory, people will always refer to swapfiles and paging files interchangeably. Let's just keep in mind their innate differences
Why Optimize The Paging File?
Unless your computer is truly loaded with RAM, it will almost always use the paging file. As such, its performance affects the performance of the whole computer.
Now, using a paging file may sound like a really cheap way to run memory intensive programs without the expense of buying more RAM. However, even the fastest hard disk is more than an order of magnitude slower than the slowest RAM.
paging file is only a stopgap solution for the lack of sufficient RAM. As long as you use the paging file, there will always be performance degradation. The ideal solution for insufficient RAM is always more RAM, not more Virtual Memory. But since we can't afford all the RAM we want, a paging file is necessary for us to run today's memory guzzling programs.
As you can tell, more isn't better for the paging file because more paging file space will only give you the ability to run more memory intensive programs at once. It will not speed up your system. But what we can do is to optimize the paging file so that the performance degradation when using it is minimized.
So How Do We Optimize The Paging File?
There have been many theories on how to optimize the paging file. The most important ones are listed below :-
Making the paging file contiguous.
Moving the paging file to the outer tracks of the hard disk.
Creating a huge paging file.
Moving the paging file to a different partition in the same hard disk.
Moving the paging file to a different hard disk.
Creating multiple paging files
Using an optimal cluster size
Moving the paging file to a RAID array
Moving the paging file to a RAM drive
Reducing reliance on virtual memory
We will examine those methods and see what will work and what won't.
Windows 95 And Above
From Windows 95 onwards, Microsoft encouraged the use of its new dynamic virtual memory system. Of course, there is nothing new about the virtual memory part but the keyword in this new technique is dynamic.
The new dynamic virtual memory system no longer relies on a fixed-size swapfile but a paging file that dynamically resizes itself according to need. When the computer runs out of memory, more memory is created by increasing the size of the paging file. Once the virtual memory is freed up, theoretically the paging file diminishes in size.
Microsoft claims that while its dynamic virtual memory system will create a fragmented paging file, it is still faster than Windows 3.1's static virtual memory system. As a bonus, no hard disk space will be tied up in a permanent paging file.
However, this dynamic virtual memory system does have a big disadvantage - it cannot be moved to the outer tracks of the hard disk platters.
Dynamic Paging Files And Data Locality
There are people who assert that when left alone, Windows XP will actually create virtual memory pages in close proximity to frequently-used data in the hard disk, like documents. In other words, they claim that Windows XP monitors disk usage, maintains a database of frequently-used files in the hard disk and uses that information to create the paging file based on spatial locality.
With virtual memory pages created close to frequently-used data, this apparently allows shorter seeks between frequently-used data and the paging file. That is the premise behind their theory of letting Windows XP handle the paging file dynamically. However, I don't think this is true at all.
First of all, while Windows XP does monitor disk usage and maintain a database of frequently-used files, only disk defragmenting utilities use that database. The built-in Disk Defragmenter, as well as third-party disk defragmenting utilities, use this database to rearrange the hard disk so that frequently-used data.
But as far as I'm aware, the paging file does not arrange the location of the pages according to this database. From my observations, Windows XP simply uses the nearest available clusters for the dynamic paging file.
In fact, Microsoft states that if you create multiple paging files, Windows XP will favour the partition that is least active. This completely refutes the theory of virtual memory pages being allocated according to spatial locality. Here is a quote from Microsoft's Knowledge Base.
"By design, Windows uses the paging file on the less frequently accessed partition over the paging file on the more heavily accessed boot partition. An internal algorithm is used to determine which paging file to use for virtual memory management."
In any case, it doesn't make sense for Windows XP to create the paging file based on spatial locality to work files like your documents. Once opened, Windows keeps the working copy in the Temp folder, not your paging file.
In addition, let us remember that Windows pre-emptively pages out pageable parts of an application in system memory. Windows does not directly load data from the hard disk into the paging file. Therefore, creating virtual memory pages close to frequently-used files will not help at all.
How Does It Work?
Virtual memory is created using a special file called a swapfile or paging file.
Whenever the operating system has enough memory, it doesn't usually use virtual memory. But if it runs out of memory, the operating system will page out the least recently used data in the memory to the swapfile in the hard disk. This frees up some memory for your applications. The operating system will continuously do this as more and more data is loaded into the RAM.
However, when any data stored in the swapfile is needed, it is swapped with the least recently used data in the memory. This allows the swapfile to behave like RAM although programs cannot run directly off it. You will also note that because the operating system cannot directly run programs off the swapfile, some programs may not run even with a large swapfile if you have too little RAM.
Swapfile Vs. Paging File
We have all been using the terms swapfile and paging file interchangeably. Even Microsoft invariably refers to the paging file as the swapfile and vice versa. However, the swapfile and paging file are two different entities. Although both are used to create virtual memory, there are subtle differences between the two.
The main difference lies in their names. Swapfiles operate by swapping entire processes from system memory into the swapfile. This immediately frees up memory for other applications to use.
In contrast, paging files function by moving "pages" of a program from system memory into the paging file. These pages are 4KB in size. The entire program does not get swapped wholesale into the paging file.
While swapping occurs when there is heavy demand on the system memory, paging can occur preemptively. This means that the operating system can page out parts of a program when it is minimized or left idle for some time. The memory used by the paged-out portions are not immediately released for use by other applications. Instead, they are kept on standby.
If the paged-out application is reactivated, it can instantly access the paged-out parts (which are still stored in system memory). But if another application requests for the memory space, then the system memory held by the paged-out data is released for its use. As you can see, this is really quite different from the way a swapfile works.
Swapfiles were used in old iterations of Microsoft Windows, prior to Windows 95. From Windows 95 onwards, all Windows versions use only paging files. Therefore, the correct term for the file used to create virtual memory in current operating systems is paging file, not swapfile.
Because both swapfiles and paging files do the same thing - create virtual memory, people will always refer to swapfiles and paging files interchangeably. Let's just keep in mind their innate differences
Why Optimize The Paging File?
Unless your computer is truly loaded with RAM, it will almost always use the paging file. As such, its performance affects the performance of the whole computer.
Now, using a paging file may sound like a really cheap way to run memory intensive programs without the expense of buying more RAM. However, even the fastest hard disk is more than an order of magnitude slower than the slowest RAM.
paging file is only a stopgap solution for the lack of sufficient RAM. As long as you use the paging file, there will always be performance degradation. The ideal solution for insufficient RAM is always more RAM, not more Virtual Memory. But since we can't afford all the RAM we want, a paging file is necessary for us to run today's memory guzzling programs.
As you can tell, more isn't better for the paging file because more paging file space will only give you the ability to run more memory intensive programs at once. It will not speed up your system. But what we can do is to optimize the paging file so that the performance degradation when using it is minimized.
So How Do We Optimize The Paging File?
There have been many theories on how to optimize the paging file. The most important ones are listed below :-
Making the paging file contiguous.
Moving the paging file to the outer tracks of the hard disk.
Creating a huge paging file.
Moving the paging file to a different partition in the same hard disk.
Moving the paging file to a different hard disk.
Creating multiple paging files
Using an optimal cluster size
Moving the paging file to a RAID array
Moving the paging file to a RAM drive
Reducing reliance on virtual memory
We will examine those methods and see what will work and what won't.
Windows 95 And Above
From Windows 95 onwards, Microsoft encouraged the use of its new dynamic virtual memory system. Of course, there is nothing new about the virtual memory part but the keyword in this new technique is dynamic.
The new dynamic virtual memory system no longer relies on a fixed-size swapfile but a paging file that dynamically resizes itself according to need. When the computer runs out of memory, more memory is created by increasing the size of the paging file. Once the virtual memory is freed up, theoretically the paging file diminishes in size.
Microsoft claims that while its dynamic virtual memory system will create a fragmented paging file, it is still faster than Windows 3.1's static virtual memory system. As a bonus, no hard disk space will be tied up in a permanent paging file.
However, this dynamic virtual memory system does have a big disadvantage - it cannot be moved to the outer tracks of the hard disk platters.
Dynamic Paging Files And Data Locality
There are people who assert that when left alone, Windows XP will actually create virtual memory pages in close proximity to frequently-used data in the hard disk, like documents. In other words, they claim that Windows XP monitors disk usage, maintains a database of frequently-used files in the hard disk and uses that information to create the paging file based on spatial locality.
With virtual memory pages created close to frequently-used data, this apparently allows shorter seeks between frequently-used data and the paging file. That is the premise behind their theory of letting Windows XP handle the paging file dynamically. However, I don't think this is true at all.
First of all, while Windows XP does monitor disk usage and maintain a database of frequently-used files, only disk defragmenting utilities use that database. The built-in Disk Defragmenter, as well as third-party disk defragmenting utilities, use this database to rearrange the hard disk so that frequently-used data.
But as far as I'm aware, the paging file does not arrange the location of the pages according to this database. From my observations, Windows XP simply uses the nearest available clusters for the dynamic paging file.
In fact, Microsoft states that if you create multiple paging files, Windows XP will favour the partition that is least active. This completely refutes the theory of virtual memory pages being allocated according to spatial locality. Here is a quote from Microsoft's Knowledge Base.
"By design, Windows uses the paging file on the less frequently accessed partition over the paging file on the more heavily accessed boot partition. An internal algorithm is used to determine which paging file to use for virtual memory management."
In any case, it doesn't make sense for Windows XP to create the paging file based on spatial locality to work files like your documents. Once opened, Windows keeps the working copy in the Temp folder, not your paging file.
In addition, let us remember that Windows pre-emptively pages out pageable parts of an application in system memory. Windows does not directly load data from the hard disk into the paging file. Therefore, creating virtual memory pages close to frequently-used files will not help at all.
Last edited: