Using the Windbg Debugging Tool

[Author's Note: Last month, I explained how to download and install the Windows 32-bit debugging tools from http://www.microsoft.com/ddk/debugging. You need those tools to perform the tasks and examples I provide this month. Also, you must have symbols installed. (See "Starting the Troubleshooting Process," June 2001.)]

Over the past 2 months, I've been setting up the theory necessary to start troubleshooting IIS problems. This month, I show you how to use the Windbg debugging tool to gather information about your servers.

The Microsoft 32-bit debugging tools that you installed give you several debugging tools, such as Command Debugger (Cdb), Kernel Debugger (KD), and Windows Debugger (Windbg), and support tools. As I go through different scenarios in future articles, I'll point out which tools are most appropriate and refer you to the online Help file that comes with the debugging tool.

Preparing to Use Windbg
To begin using Windbg, you need to create a dump (.dmp) file that you can load and look at. On the machine on which you've installed the debugging tools, open a command window, then switch to the directory in which you installed the debugger (e.g. C:\dbg). Type the command

cscript.exe adplus.vbs hang iis

This command causes the Autodump Plus tool, which is bundled with the debugging tools you downloaded, to produce full-memory dumps for the inetinfo.exe process and all Out-Of-Process (OOP) applications that are running. You'll use the dump file simply as a sample file, but this process is also an example of how to create dump files when your production site is hung. Dump files appear in a subfolder of the debugger folder with a name such as C:\dbg\Normal_Hang_Mode__Date_04-17-2001__Time_12-39-30PM.

To start Windbg, choose Start, Programs, Debugging Tools For Windows, then select Windbg. Because you haven't used this tool before, you need to set up the program so it knows where your symbol files reside. To tell Windbg about the symbol path you're using, choose File, Symbol File Path, then add the fully qualified path to your symbol file root (e.g., C:\winnt\symbols). Now, when you load a dump file or attach Windbg to a running process, Windbg will know where to find symbolic information.

To open the dump file, from Windbg, choose File, Open Crash Dump. Select the dump file you used Autodump Plus to create (e.g., PID-1188__Inetinfo.exe__Date_04-17-2001__Time_12-39-30PM__full.dmp). When a dialog box asking to Save Workspace Information appears, click Yes. You now have two windows within Windbg. The top window is labeled Disassembly and looks like a lot of assembler code. Because I'm not going to explain the process in that much depth, close the top window. Your screen will look like the one that Figure 1 shows.

The top pane shows the output from Windbg. The bottom pane, which is preceded by 0:000>, is where you input commands. This set of numbers is the prompt area. Whenever a command is running, the prompt window is blank. When the command finishes, the numbers reappear. The number to the left of the colon (:) identifies the processor on which the current thread in Windbg was running. The number to the right of the colon indicates what number thread in the dump file is the currently active thread. (For information about the terminology I'm using, see the chapter "Terminology" in the debugger online Help file Introduction to Debugging.) Because you made the dump file manually, Windbg sets the active thread to 0. If the dump file had been generated as the result of a crash, the tool would have set the active thread to the thread that caused the crash.

Examining a Dump File
Now that you've created a dump file and loaded it into Windbg, let's dig into it and see what IIS was doing. Type kb, which stands for Stack Backtrace, in the prompt window, then press Enter. Output similar to that in Figure 2 appears. This output tells you what the active thread was doing at the time the dump was created. The output in Figure 2 shows five columns to the left of the function names. These columns are

• ChildEBP (column 1)
• RetAddr (column 2)
• Args to Child (columns 3 through 5)

(For now, don't worry about what these five columns mean. I'll explain the columns and functions as I use them in specific scenarios in future articles.)

Look at the functions that have been called. The top function (i.e., ntdll!ZwReadFile) is the most recently executed function. This function should be the top item for thread 0 in any dump of the inetinfo.exe process. (Note that other threads in inetinfo.exe and all threads in other processes will most likely have different functions on top.) You should see a stack similar to the one in Figure 2 on any dump file you generate from inetinfo.exe. If you don't see such a stack, you might have received a symbol mismatch error, which means that the symbols weren't lined up properly (i.e., the loaded symbol file doesn't match the module that it's supposed to represent). You might see some variations in thread 0, but the stack should end up with ZwReadFile as the top function.

However, what if you wanted to see what all the threads were doing? You can type the command

~*kb

in the prompt window, where the tilde (~) means run the command on a thread different from the current one, the asterisk (*) means that you want all threads, and Kb is the command that should be run on all threads. Depending on where your symbols reside, this command can take a while to execute. Be patient, and wait for the prompt to return to the prompt window. When the command finishes running, browse through the results. Can you tell what the threads were doing based on the function names you get back? Figure 3 shows one thread that comes up repeatedly in dump files.