Finding and Fixing NT Memory Leaks

Memory leaks have plagued Windows NT since its first release. While diagnosing a slow server problem, I searched the Microsoft Knowledge Base for articles related to memory leaks. The sidebar "Windows NT Memory Leaks," lists Knowledge Base articles about memory leaks and demonstrates how pervasive the problem is. Troublesome areas include the Server Service, remote procedure call (RPC) service, Remote Access Service (RAS), Performance Monitor, LMRepl service (the replication service), Client Services for Novell NetWare, File and Print Services for NetWare, Active Server Pages (ASP), Winlogon, Domain Name System (DNS), NTFS, and print spooling. Service Pack 4 (SP4) corrects most of these problems.

To understand memory leaks, you need to understand how OS services and applications allocate memory. NT’s Memory Manager manages OS and application program memory requests. The Memory Manager assigns and releases memory for OS components and applications and separates the OS address space from the application processes’ address space. The Memory Manager also prevents processes from accidentally or purposefully changing an address space other than their own. For an in-depth discussion of NT memory management, see Mark Russinovich, "Inside Memory Management," parts 1 and 2, August and September 1998.

A thread is the basic unit of execution in NT, and multiple threads make up a process. When a thread starts, the Memory Manager allocates physical memory and paging file space to let the thread load and run. While running, a thread can request additional memory to accomplish its task. The Memory Manager handles thread space on a process basis—when a process has many threads, the process’ allocated memory grows accordingly. The Memory Manager allocates application memory from free pages or the paged pool and OS memory from the paged or nonpaged pool.

The Memory Manager allocates pages from the paged pool when it can temporarily write process data to the paging file, whereas pages in the nonpaged pool never leave memory. NT components such as device drivers use the nonpaged pool to store data structures that are necessary for interrupting processing or symmetric multiprocessing (SMP) routines. These pages must remain in memory because NT doesn’t allow page faults (i.e., removing a page from the process’ working set) while processing device code.

What Is a Memory Leak?
A memory leak results when a memory allocation occurs repeatedly without the corresponding memory release. Over time, memory leaks can cause the system to allocate all available memory (i.e., exhaust physical memory and paging file space). When no more memory is available, NT hangs until the system releases memory.

Memory leaks commonly occur in two scenarios: when a process creates worker threads but doesn’t delete them, and when a thread allocates but doesn’t release memory. In many cases, the effects of a memory leak are subtle—a few pages here, a few pages there—until much later when the offending application or service has run for a long time.

Memory leaks can cause temporary memory shortages in application programs that run for a short time. However, memory leaks in services that run as part of the OS or in production applications (e.g., a database) can cause severe problems. If pages in the paged pool cause a memory leak, system performance slows as the paging file fills up. A nonpaged pool memory leak typically doesn’t affect system performance.

When a paged pool memory leak exists, you see a pop-up error message on the console and an entry in the system event log. A common error message is Your system is running without a properly sized paging file. Please use the System applet’s Virtual Memory option in Control Panel to create a paging file or to increase the size of your paging file. This message appears when you boot NT without a paging file and when the paging file is full.

To increase the size of the paging file, right-click My Computer, select Properties, and go to the Performance tab. The total of all paging files appears in the lower part of the screen. Click Change to adjust the size of the current paging file or add another paging file on another drive. I recommend a starting paging file size of at least twice the amount of physical memory in your system; multiple paging files on different disks can significantly improve performance for systems running large applications.

When the paging file or paged and nonpaged pools don’t contain enough free space, you receive an error message. Two common error messages are The server was unable to allocate the nonpaged pool from the system because the pool was empty and Your system is running low on virtual memory. Please close some applications.

If you have a memory leak, increasing the size of the paging file provides a temporary solution. A larger paging file gives the system a little longer to run before exhausting the space in the extended or secondary paging file.