Inside User-Mode Root Kits
The highest API layer is the Windows API layer in user mode. This layer consists of the base OS API that Microsoft documents in its Platform Software Development Kit (SDK). The Windows API for listing files in the directory consists of FindFirstFile and FindNextFile, both of which are implemented in \windows\system32\kernel32.dll. An application such as Windows Explorer or the command prompt that wants to use the API imports it from kernel32.dll either statically or dynamically.
A DLL is loaded into a process only when an executable running in the process requires APIs exported by the DLL. When an executable imports an API statically, the executable's image contains references to the DLL and the APIs in a section called an import table that the OS loader evaluates during the process's startup. The loader loads each DLL listed in the import table into the process address space and connects the image's references to the referenced functions, as Figure 3 shows.
A dynamic import occurs when an already running process uses the Windows LoadLibrary function to load a DLL and the GetProcAddress function to ask the OS for the address of a function exported by the DLL. You can use the Dependency Walker tool from http://www.dependencywalker.com to view an image's static imports and exports and even run an executable with profiling to capture its dynamic imports. Figure 4 shows how Notepad imports FindFirstFile.
One method used by many root kits to hide malware is to scan a process's import table and replace the references to key system DLL functions with pointers to root kit functions that it implements in its own DLL or that it has written directly into the address space of the process. After a process has been hooked in this manner, known as DLL import hooking, a call it makes to FindNextFile, for instance, routes to the root kit code. The root kit code invokes the original function in Kernel32 but regains control when the function is completed. The root kit then examines the output and strips out data associated with the files and directories the root kit is hiding. To deal with dynamic linking, the root kit must hook GetProcAddress and return to the caller a pointer to the root kit's hook function instead of to the function the caller wants to use. The Vanquish root kit uses this technique, which Figure 5 shows.
Most file-, memory-, and process-related Windows APIs rely on lower level APIs exported by \windows\system32\ntdll.dll, which provides interfaces to the user-mode native API that's exposed by the OS kernel. Task Manager uses the NtQuerySystemInformation API implemented in Ntdll to query active processes, so a root kit that wants to hide a process can hook this function and massage its output to remove malware processes.
The Windows file-listing APIs mentioned earlier use Ntdll's NtQueryDirectoryFile, so manipulating the data returned by NtQueryDirectoryFile affects the data returned by FindFirstFile and FindNextFile. Root kits often hook the higher level functionshooking Ntdll only when necessarybecause it's typically easier and many system utilities use only the higher level functions.
Another user-mode root kit technique is to patch Windows API and native API functions in user mode. The popular Hacker Defender root kit takes this approach. Patching involves writing root kit code into a target process and modifying the first few bytes of subverted functions in that process so that they transfer control to the root kit's code. After the root kit code has gained control, it invokes an unmodified version of the function and manipulates the results before returning them to the caller.
Inside Kernel-Mode Root Kits
More sophisticated root kits intercept APIs in kernel mode at their entry to the OS by using system-call hooking. Ntdll's functions execute a system-call instruction to transition into kernel mode, in which the OS's system-call dispatcher executes. System-call APIs are internally identified by number, so the native API function in user mode indicates to the system-call dispatcher the number of the kernel-mode equivalent function. The dispatcher uses the number as an index to a table in which it stores pointers to system-call functions. In a manner similar to the way in which DLL import hooking works, root kits can replace a pointer in the table with a pointer to the root kit's function, located in a device driver, which behaves like a user-mode hook, manipulating data as it's returned to an application. NT Rootkit relies on system-call hooking. The function-patching method used by Hacker Defender is also possible in kernel mode, though none of the published root kits use patching.
A kernel-mode root kit technology that has become popular in the root kit community is direct kernel object manipulation. This method attacks the kernel's data structures themselves, rather than the APIs that return information about the data structures. To hide a process, for instance, the root kit takes the process off the kernel's list of active processes. Then NtQuerySystemInformation won't report the process because NtQuerySystemInformation relies on the active process list for its information, but the kernel will still schedule threads running within the process. This type of hiding has also been applied to device driver structures but isn't applicable to registry key, registry value, file, or directory cloaking because these objects aren't maintained with simple static structures. The FU root kit cloaks by using direct kernel object manipulation.
A final approach taken by kernel-mode root kits is to implement cloaking in a file-system-filter driver. File-system-filter drivers layer on top of file-system drivers, such as NTFS, and below the system-call API layer so that they can see all file-system activity. All "on-access" virus scanners use file-system filtering to intercept file-open operations and scan files for viruses before they allow the operations to proceed.
NT Rootkit uses a file-system-filter driver to intercept directory queries and remove references to malware files from the output returned by the file-system driver. Thus, anything that performs a directory listing operation, whether it's an application such as Windows Explorer in user mode or even another driver in kernel mode, is subject to NT Rootkit's cloaking.
Hiding in a Process
Perhaps the most advanced form of cloaking is to hijack the memory of an already running process. Malware that does this, such as the Code Red worm that infected Microsoft IIS, doesn't need to actively hide from tools such as Task Manager. However, unless the malware stores itself on disk before a shutdown, it won't survive a reboot. Some malware authors are sure to decide that avoiding detection is worth the risk of losing control of a system at a reboot.