In August 1999, Microsoft released an Exchange Server 5.5, post-Service Pack 2 (SP2) hotfix to address a specific Exchange mail-relaying vulnerability. This fix plugged a security hole that let the Internet Mail Service (IMS) relay encapsulated SMTP messages, even if you'd configured the server to prevent relaying.

Many companies, however, haven't secured their servers. In a quick test of 20 sites, I found that between 60 and 70 percent of them either hadn't applied this security patch or hadn't configured their servers to block relays in the manner that the Internet Engineering Task Force (IETF) Request for Comments (RFC) 2505 suggests.

If you haven't completely secured your server against relaying, an outsider can use your server to deliver email that appears to have originated from your server and possibly from one of your mail accounts. In the United States and other countries, people periodically debate whether systems' owners are liable if someone uses their system as a relay. The crux of the debate is whether an owner knew about the problem but chose to do nothing about it. Aside from concern about owners' liability, relaying is also a form of Denial of Service (DoS) attack because it takes system resources away from legitimate operations. Relaying is also bad for your company's business because it can generate ill will among your customers.

Let's look at some ways to protect your system against relaying. The sidebar "Protect Your Server Against Encapsulated SMTP Address Vulnerability" discusses the fix for a specific type of relaying problem.

RFC 2505
In February 1999, the IETF released RFC 2505, "Anti-Spam Recommendations for SMTP MTAs." This RFC explains the problems associated with unsolicited commercial email (UCE, or spam) and specifies the functionality that an SMTP Message Transfer Agent (MTA) needs to reduce UCE's effects. RFC 2505 makes 13 recommendations, two of which are most closely related to relaying:

  • A system must be able to restrict unauthorized use as a mail relay.
  • A system must be able to configure and provide different return codes for different rules (e.g., 451 Temp Fail vs. 550 Fatal Error). Specific return codes let you diagnose configuration problems that are blocking legitimate mail delivery.

If you're using Exchange Server 5.5, you have the tools to implement these recommendations and prevent the unauthorized use of your resources. To maximize your ability to use these capabilities and configure your system's antirelay defenses, you need to be running SP2 or later and the encapsulated relaying hotfix. An alternative to installing the hotfix is to install SP3, which includes the hotfix. In Exchange Server 5.5 before SP2, you could block relaying only by manually editing the Registry.

As Screen 1 shows, two options on the Routing tab of the Internet Mail Service Properties sheet—Do not reroute incoming SMTP mail and Reroute incoming SMTP mail (required for POP3/IMAP4 support)—control mail relaying. The obvious choice seems to be Do not reroute incoming SMTP mail, but this option has several drawbacks, including that it isn't as secure as it might seem. Selecting this option prevents the IMS from relaying messages, but the IMS still accepts the messages and then generates a nondelivery report (NDR) that specifies that the recipient's address doesn't exist on the system. This configuration doesn't meet the second RFC recommendation because it results in an error code that indicates success and generates an incorrect delivery failure report.

If your server is running in the Do not reroute incoming SMTP mail configuration and you have an email account elsewhere on the Internet, you can confirm the flaws in the process I describe above by configuring a POP3 mail client to use your Exchange server as its SMTP host. For example, if you're using Outlook (i.e., a POP3 mail client) and your external address is joe@realdomain.com, on the General tab of Outlook's Internet E-Mail Properties sheet, set the email and reply addresses to joe@realdomain.com. On the Servers tab, set the Outgoing Server to the IP address or domain name of your Exchange server. Compose a message, and try to send it to a friend's address. Exchange will accept the message, but you'll receive an NDR at your external address.

The IMS Help screen states that this behavior is by design, but this feature has two major drawbacks. First, this configuration could allow your system to be the origin of reverse UCE or even a mail flood. Second, the configuration places an unnecessary processing burden on your system.

Reverse UCE. If you set your system to Do not reroute incoming SMTP mail, malicious users can use your system as a launching point to flood another computer system with SMTP messages. Because the messages include your company's address, they look as if you sent them. For a more detailed look at the SMTP protocol dialog between sender and server, see Mark Minasi's Windows NT Magazine article "Untangling Email" (April 1998). Figure 1 shows a typical SMTP dialog.

As the system issues each of these commands, the Exchange server returns a code to the sending system. All the server responses in the dialog start with a numerical code. This code is either 250 OK, which represents success, or another error code, such as 501 Unrecognized parameter. The SMTP protocol specifies that 200-level codes signify success, 400-level codes signify temporary failures, and 500-level codes signify critical failures.

The logic that the IMS uses when determining whether to deliver a message flows as follows:

  1. Accept sender's address.
  2. Accept recipient's address.
  3. Accept a data stream containing message header and body text.
  4. Determine whether the recipient is a member of the local system (i.e., whether the address is in the Global Address List—GAL).
  5. If yes, deliver the message to the local recipient. If no, use the sender's address to generate an NDR.

Because step 5 can result in the system directing a message back to the sending address, a malicious user could engineer a mail flood by supplying a bogus addressee (RCPT TO) with a valid sending address (MAIL FROM)—the target of the flood—and then infinitely generating messages to the Exchange IMS. In any case, the malicious user achieves the objective of getting an unsolicited message to a destination.

Unnecessary burden on system. You can see the second disadvantage of using the Do not reroute incoming SMTP mail setting by reading the SMTP dialog. You're providing the IMS with the following information: the message sender (MAIL FROM), the message recipient (RCPT TO), and the message (DATA). When your POP3 client sends a message, as in this test, it goes through a similar process. In the current configuration (Do not reroute incoming SMTP mail), the IMS accepts the RCPT TO specification and sends the return code 250 OK even if the recipient isn't part of the local system. Next, the IMS lets you execute the DATA command, through which you could supply megabytes and megabytes of message body text. This action can tie up your system resources, even though the system eventually generates an NDR.

A Better Option
If you attempt to perform these operations on most UNIX hosts that you've configured to prevent relaying, you'll receive a 550 Relaying prohibited SMTP error code when you specify the nonlocal recipient. This result is more desirable, meets the RFC recommendation, and eliminates the two flaws I just described. In this configuration, the IMS proceeds through only four steps before preventing a relay.

  1. Accept sender's address.
  2. Accept recipient's address.
  3. Determine whether the recipient is a member of the local system (i.e., whether the address is in the GAL).
  4. If yes, return 250 OK and accept the recipient; if no, return 550 Relaying prohibited (don't proceed to steps 5 and 6).
  5. Accept a data stream containing message header and body text.
  6. Deliver the message to the local recipient.

When the IMS follows this logic, if the sending system doesn't supply a valid recipient, the server responds with 503 Need RCPT TO command and doesn't let the sender execute the DATA command. This action prevents the sender from tying up system resources as it reads and stores a possibly long message. The action also re-lieves the system of the burden of generating and sending an NDR.

Making the Changes
To configure your system in a similar way, you need to select the Reroute incoming SMTP mail option on the Routing tab of the Internet Mail Service Properties sheet, as Screen 2 shows. Although this choice might seem counter to the objective, it provides a more secure system if you configure it properly. After you've selected this option, you must specify all the domains for which your IMS handles incoming mail. For example, assume that you have no internal or external systems that need to use the Exchange server as a relay and that you're not using any POP or Internet Message Access Protocol (IMAP) clients. Your primary domain is realdomain.com, and you host mail for one other organization: virtualdomain.com.

On the Routing tab, add realdomain.com and virtualdomain.com to the Sent to list. When you add these domains, you have three routing options to choose from, as you see in Screen 3.

  • Should be accepted as "inbound" signals that all recipients with this domain name must match a corresponding SMTP address in the GAL.
  • Override relay restrictions. Always "relay" exempts the domain from any restrictions that you set by using Routing Restrictions.
  • The Should be rerouted to this domain option lets you specify the domain where the system will redirect mail. The IMS replaces the original domain name with the value you specify here. For example, if mail comes into joe@dec.com, the system rewrites the address as joe@compaq.com and sends the message on to the compaq.com mail host.

Select Should be accepted as "inbound" for both realdomain.com and virtualdomain.com. This choice assumes that you don't have any complex scenarios in which you're hiding internal hosts and addresses or allowing relaying through to specific domails.

Choosing the routing option is only part of the configuration. You need to set routing restrictions to protect your system from being wide open for relaying. The Microsoft article "XFOR: Restricting Routing in the Internet Mail Service" (http://support.microsoft.com/support/ kb/articles/q196/6/26.asp) describes how to choose a routing restriction option. Click Routing Restrictions on the Routing tab to display the dialog box you see in Screen 4. Enter the IP addresses of systems that you want to let deliver and reroute mail through your server. This dialog lets you control access to Exchange Server's relaying capabilities under several conditions:

  • The Hosts and clients that successfully authenticate option assumes that an additional security mechanism is in place to confirm the identity of the sender or system. For example, a host might need to use an Enhanced SMTP Auth command or NT LAN Manager (NTLM) credentials to validate its right to relay.
  • The Hosts and clients with these IP addresses option lets you grant specific machines or machines on specific subnets the ability to use Exchange as a relay.
  • The Hosts and clients connecting to these internal addresses option lets you let systems relay if they can access a specific network adapter on a multihomed system.
  • The Specify the hosts and clients that can NEVER route mail option lets you prohibit specific hosts from relaying when you've granted a large group, such as a subnet, the ability to relay.

What the Microsoft article and online Help don't spell out is that when you select a routing restriction, you can choose not to enter any IP information. The trick is that you can select the Hosts and clients with these IP addresses check box but not specify any IP addresses. Unless you have a specific need to have your Exchange server relay, don't enter any IP addresses on this page. This selection changes the rules that the IMS uses when evaluating the SMTP protocol. Instead of letting the IMS accept the RCPT TO specification blindly, this selection causes the IMS to check for local delivery before letting it upload a message. If the recipient isn't local, the IMS will return 550 Relaying prohibited.

Confirming the Configuration
To make these changes take effect, stop and restart the IMS. If you want to confirm that your server is rejecting relays but still accepting mail for your local recipients, you can use a Telnet session on the SMTP port. Open Telnet, and connect to your Exchange server on port 25. You can connect quickly by selecting Start, Run and typing

telnet servername 25

where servername is the name of your Exchange server. The Exchange server will respond with a message similar to 220 host.domain.com ESMTP Server (Microsoft Exchange Internet Mail Service 5.5.2650.10) ready. Then enter the following commands. The commands aren't case-sensitive, but the punctuation (e.g., colons, angle brackets—< >) is important, so include all the marks.

  1. Enter
    HELO me

    The server will respond with 250 OK and identify your IP address and possibly your host name.

  2. Enter
    MAIL FROM: someaddress@somedomain.com

    Again, the server will respond with 250 OK.

  3. Enter
    RCPT TO: nobody@afakedomain.com

    The server will respond with 550 Relaying prohibited.

  4. Using a valid address from your GAL, enter
    RCPT TO: thegaladdress@yourdomain

    The IMS will reply with 250 OK when it accepts the address.

  5. To close the session, type
    QUIT

Protect Your Server and Your Reputation
These changes protect your Exchange server against relaying and help protect your reputation. If you need to allow relaying, check out the Microsoft articles "XFOR: New IMS Routing Functionality in Exchange Server 5.0 SP2" (http://support.microsoft.com/ support/kb/articles/q169/6/83.asp" and "XFOR: Restricting Routing in the Internet Mail Service," which I cited earlier. You can make your server safer when you configure the system to allow as few systems as possible to relay through your server. And don't forget to always thoroughly test your configuration!

Corrections to this Article:
  • The author has made minor changes to this article since it appeared in the print newsletter. He has modified steps 3 and 6 in the "A Better Option" section and added an explanation of the change in the paragraph that follows the steps.