Hunt down message data

Users are wonderful: They never lose messages, never forget to send messages, never forget to add someone important to an address list, never send a message to someone who shouldn't receive the message, and never forget to add an attachment. In fact, users are perfect.

However, on the rare occasion when a user makes a mistake, a tool that helps you determine the user error can be indispensable. Short of monitoring every keystroke, you can't be sure exactly what the user did. However, Microsoft Exchange Server 5.5 and earlier include a Message Tracking Center facility, and Exchange 2000 Server includes and enhances this facility. This tool can help you track a message's path between servers, as well as determine when the user sent the message, to whom the user sent the message, and other important pieces of information.

You can also use this data to evaluate your company's email trends—handy information when you need to request additional hardware. After you learn the basics of the Exchange 2000 Message Tracking Center, you can use tracking-log data to hunt down errant messages or investigate message flow across your organization.

Start the Logs Rolling
You can't expect to track a message if it passes invisibly between servers. In an Exchange Server organization, you can search for a message only when you've already configured the Exchange Server machines to generate message-tracking log files for you to interrogate.

Tracking is simple when a message remains on one server, more complicated when the message passes across multiple servers en route to its final destination, and even more complex when the message passes out across the Internet or across another messaging system. Exchange 2000 can't force every email system on the planet to generate and maintain tracking data in a common format and make that data available to any program that might request the data. Therefore, your options are restricted to tracking messages as they pass between servers within one Exchange Server organization.

When you enable tracking, every Exchange Server machine can maintain a set of message-tracking logs. Each server creates a new log daily and names the log according to the date in yyyymmdd format (e.g., 20000725.txt). The logs reside on a network share called server_name.log (in Exchange 2000) or tracking.log (in Exchange Server 5.5, Exchange Server 5.0, and Exchange Server 4.0). Prefixing the name of the Exchange Server system creates the full name of the share. For example, the full name of the share on an Exchange 2000 server named Excserver would read as follows:

\\excserver\excserver.log

Exchange 2000 supports the concept of virtual servers. On a Windows 2000 cluster, an Exchange Virtual Server (EVS) is a collection of the resources (e.g., an IP address, network name, disks, software) that an Exchange Server machine comprises. In a cluster, each EVS generates a set of message-tracking logs you can identify and access exactly as you would logs from standard servers. (For more information about EVSs and clustering Exchange 2000, see Jerry Cochran, "Clustering Exchange 2000, Part 1," page 145.)

When Exchange Server generates messages, it gives each message a unique identifier and records each identifier in the message-tracking logs, thus giving you the capability to trace a message as it makes its way to its final destination. A sample message identifier might read as follows:

BE8B1DCC92D77E4C9CC70E141E3B583B02226F@EXCSERVER.acme.org

The only part of this identifier that makes any sense to a human is the server name (i.e., EXCSERVER.acme.org). To create and control message-tracking logs, Exchange 2000 offers the following set of server properties, which Figure 1, page 140, shows.

Enable subject logging and display. Select the Enable subject logging and display check box to record message-subject information in the message-tracking logs and to display this information when you track a message. Message subjects can contain confidential information, so some installations opt not to collect this data. However, users shouldn't put confidential data into message subjects—truly confidential information should be secured through encryption. Also, the subject field is a good way to isolate a message when the sender generates a lot of traffic. In most cases, recording this data is harmless. By default, Exchange Server disables subject logging.

Enable message tracking. Select the Enable message tracking check box to instruct Exchange 2000 to begin creating and populating daily message-tracking logs. Exchange 2000 will log every message that passes through the transport engine. By default, Exchange Server disables message tracking.

Remove log files. Select the Remove log files check box to instruct the Exchange System Attendant process to clean up old logs shortly after midnight each day. By default, the System Attendant removes files older than 7 days. You can choose to keep files for a longer period; message-tracking logs don't usually occupy enough space to cause problems. However, most users won't wait longer than 7 days to request that you track a truly important message.

Because you can track messages only when servers are recording details in the tracking logs, you need to establish an enterprisewide decision whether to enable logging. The best practice is to enable logging on all servers because it provides a useful facility at very little cost.

Launch the Investigation
After you've enabled the creation of tracking logs, you can manage message tracking through Exchange 2000's Exchange System Manager (ESM) console's Tools node or through the Microsoft Management Console (MMC) Exchange Message Tracking Center snap-in. Both options use the same code: The ESM console comprises a set of MMC snap-ins.

You must provide certain criteria before you can track a message. The more information you have about a message, the better. Searching for a message that a user thinks he or she sent 4 days ago to some group of people is much more difficult than looking for a message that the user knows he or she sent yesterday to specific recipients. Figure 2 shows an example search result for a message that didn't reach a specified recipient. As you can see, the search criteria specified the name of the originator, the recipient that I'm interested in, and the name of the server on which to start the search. A scan of the logs found one message. You can use the Browse buttons next to the From and Sent To text boxes to search Active Directory (AD) for users, contacts, or groups that sent or received the message. By default, the search begins on the server to which you're connected with the ESM console or the MMC Exchange Message Tracking Center snap-in, but you can begin the search on another server (for example, if the person whose message you're tracking has a mailbox on another server). To search the logs of one or more other servers, click the Browse button next to the Server(s) text box and select the servers you want to interrogate.

Even on large servers that handle heavy volumes of email, searches proceed quite rapidly and you'll usually have a result within a couple of minutes. The number of messages that the search finds depends on the quality of your search criteria. To view full details of a message's progress through the routing engine, you can select a message from the list of found messages and click Message History.

Figure 3, page 142, shows the kind of information you can expect to see when you click Message History. Note that the Message Tracking Center fills in the Subject field in the Message History window because I enabled this server to collect information about message subjects. The detailed view also returns the unique Message ID. The Message history list details each step in the message's progress. Following these steps, you can see that Exchange 2000 first submitted the message to the Exchange 2000 Advanced Queuing (AQ) engine, the component that decides how best to route a message. Exchange 2000 then categorized the message and placed it in a queue for the Message Transfer Agent (MTA). The way that a message passes through the server might come as a surprise because Exchange 2000 uses SMTP for routing whenever possible; however, in this example, the user sent the message to some users whose mailboxes still reside on an Exchange Server 5.5 machine. In an Exchange 2000 organization, the MTA provides backward compatibility with earlier Exchange Server versions; the MTA also serves as a route to connectors that aren't upgraded for Exchange 2000, such as the IBM PROFS connector.

The Message Properties window (at right in Figure 3) shows the type of details you can view for any step in a message's progress. To open this window, select a step from the Message history list and click Details. In the example, you can see the message recipients on the Exchange Server 5.5 machine.

Tracking halts if the Message Tracking Center can't open a log, which might happen for a couple of reasons. First, a server's network share might be disabled, or someone might have changed the permissions on the directory. Second, the System Attendant might already have removed the tracking logs from the system. Unless you've extended the time limit, searches further back than the default 7 days will fail because the logs are no longer available.

Tracking Log Format
Exchange Server versions earlier than Exchange 2000 created message-tracking logs that used a strange format; correctly analyzing this format required substantial effort. These older logs held data as simple text, but the layout was unpredictable. Several bugs prevented the logs from correctly capturing and recording all information; these bugs frustrated administrators who wanted to review tracking logs to determine message flow and programmers who needed to write code to analyze files and generate traffic-analysis data, such as the number of messages sent per day.

Microsoft fixed the bugs over time, and the data that Exchange Server 5.5 captures is fairly reliable. Exchange 2000 introduces a new tracking-log format. Despite the format change, you can still track messages between mixed-vintage servers. All fields are now tab-delimited, which makes the data much more consistent and easier to understand. However, the new format means that developers must upgrade commercial and homegrown analysis programs to work with Exchange 2000. The new format provides the following data:

  • The date and time that the user first sent the message
  • The client IP address and network name that generated the message
  • The partner or messaging service (e.g., the Information Store—IS—for local delivery) that received the message
  • The server IP address and server name that generated the message
  • The recipient address (SMTP or X.400)
  • An event identifier that specifies the processing that occurred
  • A unique message identifier that never changes, no matter how many servers the message traveled through
  • The priority of the message (­1 is low, 0 is normal, and 1 is high)
  • The total message size in bytes
  • Whether the message was encrypted (1 means signed, 2 means encrypted, and 0 means clear text)
  • The version of the routing engine running on the server
  • The message's subject (if you've enabled subject logging and display), truncated to 256 bytes, if the subject is longer than that
  • The number of recipients
  • The sender's email address (i.e., the primary address of the originating mailbox), if known; this address can be SMTP, X.400, or a distinguished name (DN), depending on the transport that introduced the message
  • The individual addresses in distribution groups

The easiest way to understand tracking-log data is to copy a log from a test server, then load the log directly into Microsoft Excel, as Figure 4 shows. You can, of course, review the log data from a production system, but these logs can grow to hundreds of megabytes on a large server. Therefore, these logs load slowly into Excel, and finding specific information is difficult.

The Exchange 2000 log format is suitable for loading into a database program such as Microsoft Access, from which you can analyze the data to your heart's desire. However, if you're interested in using the data to analyze a server's email traffic trends, I suggest you reduce the data to its essential figures (e.g., the number of messages sent during a period, the number of addressees, message size). According to comments at conferences and on Internet mailing lists, Perl is a particular favorite among developers who write homegrown code to analyze message tracking log data. Several commercial applications, including Seagate Software's Seagate Crystal Reports and PROMODAG StoreLog, can use log data to generate many different management reports. You can then use the data in these reports to discover the flow and volume of messages within an organization—information that could help justify hardware upgrades when message volume increases significantly.

The Path to Success
Although you won't use message tracking every day, being prepared when you need to trace the path of an important message is a smart idea. Exchange Server's message-tracking logs are also an interesting source of information about email flow on a server, and you can accumulate this data across an organization to get an overview of where messages go inside your company. This information can help you evaluate your deployment and justify hardware requests as your mail volume grows.