Load Testing Exchange 2000: Analysis and ESP

In "Load Testing Exchange 2000," May 2002, InstantDoc ID 24126, I introduce several load-simulation tools for Microsoft Exchange 2000 Server. I explain how load testing Exchange can be a quick and easy way to stress-test new hardware or a network design or to illustrate a proof of concept such as a distributed Web architecture with front-end servers for client connection points and back-end servers for mailbox stores. In that article, I describe how to set up the Microsoft Exchange Load Simulator (LoadSim), which simulates Messaging API (MAPI) clients such as Microsoft Outlook, and how to use Windows 2000's Performance Monitor to capture the results. But as I explain in "Load Testing Exchange 2000," you need to rely on other tools—in particular, the Exchange Stress and Performance (ESP) tool—to test non-MAPI clients and protocols such as POP3, IMAP, SMTP, and HTTP-DAV (which Outlook Web Access—OWA—uses). In fact, you must use ESP rather than LoadSim to test a front-end/back-end architecture, which MAPI doesn't support. After you learn how to analyze your captured LoadSim results, you're ready to take the next step: learning how to use ESP to simulate Internet-protocol clients.

Processing LoadSim Output Files
LoadSim uses several load-generating computers to simulate the effect of hundreds or thousands of Outlook clients. After you run a load simulation against your Exchange mailbox servers, look in the folder in which you installed LoadSim. You should see a file called loadsim.out. (I find that the easiest way to locate the file is to sort the folder contents by date, then look for the newest file. If you can't find the loadsim.out file, run a search for it; I've seen the file's location change depending on whether I run LoadSim manually or through an automated option.) When you select the Archive previous file check box during LoadSim configuration (as I detail in "Load Testing Exchange 2000"), the tool will rename loadsim.out to loadsim.00x (where x is an increasing integer beginning with 1) and generate a new loadsim.out file each time you run a load test. For example, the first time you run a test, LoadSim creates loadsim.out. At the second run, the tool renames that file to loadsim.001 and creates a new loadsim.out file. At the third run, the tool renames loadsim.out to loadsim.002 and creates a new loadsim.out file. (Note that you must clean up these files manually.) Open loadsim.out in a text editor such as Notepad, and you see lines of logged information, similar to the following:

Jan 31 10:34:47: Action 1
Jan 31 10:34:49: Action 2

Notice that LoadSim logs a timestamp for each action. You can use the Lslog utility (lslog.exe), which comes with LoadSim, to process these files and present the results in a table of response times.

To run Lslog, open a command prompt in the LoadSim directory. You can process each client separately to determine the scores and watch for variations (which typically indicate a client-resource bottleneck, such as a lack of memory), or you can merge all your clients' results to get an average score for all clients. The following command merges the results:

c:\loadsim>lslog merge /r cl*.log
> all-cl.log

I first copied each client's loadsim.out file to a central network share, renaming the files clxx.log (where xx is a numeric identifier), then ran the merge command to merge the client logs into one file. (You might not need to copy the output files to merge results to a central location—for example, when you're simulating 500 clients from only one machine.) This example uses the /r option to rebase all the clients to the same time (i.e., Jan 01 00:00)—a useful option when your simulation clients' clocks aren't synchronized.

The next step is to examine the merged file (i.e., all-cl.log) to determine the length of time that the test run covered. During the first few hours of a test run, clients are logging on and Exchange is allocating memory; performance isn't optimized until the steady-state period. The final few hours of a test run involve shutting down and cleaning up pending requests. You want to analyze only the steady-state period, which occurs in the middle of the test run. To view the test run's length, enter the following command:

c:\loadsim>lslog times all-cl.log

Next, truncate the file to eliminate the ramp-up and ramp-down times, isolating the steady-state interval:

c:\loadsim>lslog trunc all-cl.log 2 6 > all-cl.out

In this example, I use hours 2 to 6, for a total of 4 hours, which is the average length of the steady-state interval. (If you run only one simulation client, this step will be your starting point.)

The next step, which analyzes the response times and generates the output table, is the most important. The first command processes the merged and truncated LoadSim output file, then pipes the output to a text file. The second command opens that text file in Notepad.

c:\loadsim>lslog answer all-cl.out 
> all-cl.txt
c:\loadsim>notepad all-cl.txt