Authenticating and encrypting MAPI, POP3, and IMAP clients and messages

If you're like me, you probably send and receive a lot of email every day. But did you ever stop and wonder how many of these messages are secure from outside attacks? In the June 1998 and July 1998 issues, I described how Microsoft's Exchange, Outlook, and Outlook Express email clients use remote procedure calls (RPCs), Simple Mail Transfer Protocol (SMTP), Post Office Protocol (POP) 3, and Internet Message Access Protocol (IMAP) to access mailboxes on an Exchange server (for more information, see "Related Articles in Windows NT Magazine," page 140).

The default installation of each of Microsoft's email clients is not as secure as you might like, and your correspondence might be more vulnerable than necessary. Fortunately, you can take several steps to help secure your email system. In this article, I'll describe the security and privacy features of these Exchange email clients and show you how to protect your account information and encrypt the messages you send.

Vulnerable Assets
In the process of securing Exchange email clients, you must protect two types of assets: logon credentials and message data. When you attempt to open an Exchange mailbox or browse a public folder, your client software provides logon credentials to the Exchange server. The Exchange server uses this information to decide whether you have the authority to access the requested information. The logon credentials are in the form of an account name and password or a secure hash (i.e., an encrypted string derived from a username and password--for information about hash algorithms, see "Related Articles in Windows NT Magazine," page 140). You need to protect the logon credential information; otherwise a hacker might gain access to your internal mail system or other NT resources. Each Exchange email client has configurable options that control how the client presents the logon credential information.

After the Exchange server validates your logon credentials, your email client can start transferring mail messages. The message content might be innocuous or highly confidential. Regardless of the message content, you need to know what level of protection the systems are providing the message when it is in transit between the Exchange email client and server or stored on the Exchange server. Each Exchange client has configurable parameters that let you control this level of protection. To protect the message content, you need to apply some degree of encryption by using algorithms such as Data Encryption Standard (DES), 3DES, or CAST (a proprietary algorithm that Carlisle Adams and Stafford Tavares devised) to the message before transmission.

Data Transmission
When you send an email message from your desktop to the recipient's home server, the message can travel over LAN links, remote access (asynchronous) links, WAN connections, and even across the Internet. Each link along the way has different vulnerabilities and security features. You can't control all the vulnerabilities and configure all the features, and you usually can't predict which path the information will take from the client to the destination server.

You need to examine data transmittal security from several points of view. For example, some security options that you can configure on your client system depend on the message transfer protocol running at the application layer of the Open Systems Interconnection (OSI) stack (e.g., POP3, Messaging API--MAPI, IMAP, and HTTP). As I discussed in my previous articles, you can use any of these protocols to submit a message from your desktop to an Exchange server. Similarly, at the transport layer, you can configure some security features such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), and IP Security (IPSec--for information about IPSec, see "Related Articles in Windows NT Magazine"). Although security at this layer is independent of what happens at the application (Exchange) layer, the Exchange client or server can call the stack and request that the sending and receiving systems negotiate the use of an encrypted connection to transfer the message.

Authenticating MAPI Clients
MAPI is a proprietary Microsoft protocol that email clients such as Outlook and Exchange use to communicate with Exchange Server when you add the Exchange Server Information Service to your messaging profile. An Exchange or Outlook email client using MAPI can access a mailbox on the Exchange server only over a LAN/WAN or Remote Access Service (RAS) connection. When the client tries to establish a new mail session with the Exchange server, the server attempts to validate the user's logon credentials by using NT's challenge/response mechanism (also known as NT LAN Manager--NTLM--authentication).

The server issues a challenge by sending a one-time key to the client. The client responds to this challenge by running the current user's password through a one-way function (OWF), encrypting the result of this function with the one-time key, and sending the result of this encryption to the Exchange server. Exchange Server uses the same one-time key to encrypt the OWF result that the domain controller stores in the Security Accounts Manager (SAM) database. If the client's OWF result matches the server's OWF result, the server knows that the user on the client system is using the same password he or she provided the server when the user last changed his or her password, and the server establishes this user's identity. (For an in-depth description of this challenge/response process, see "Related Articles in Windows NT Magazine.") After the Exchange server confirms the user's identity, it can check whether the user has the appropriate rights on the mailbox and respond accordingly. At no time during this process does the client or server transmit the user's password over the network.

When the MAPI client connects to the Exchange server over a LAN connection, a lot of the validation process is transparent to the user. In essence, the user is already logged on to the NT network, so the network doesn't need to ask the user to enter the appropriate account and password information a second time. Instead, the network reuses an established logon token. However, when the MAPI client connects to the Exchange server over a RAS connection, the RAS server adds a step. The RAS server must authenticate the user before the user can access the network. When you configure a RAS or Dial-Up Networking (DUN) client, you specify the type of authentication that the dial-up client and server mutually understand. You can configure the RAS/DUN client to use the standard dial-up security protocols, including Password Authentication Protocol (PAP), Challenge Handshake Authentication Protocol (CHAP), and a Microsoft version known as MS-CHAP (for more information about these protocols, see "Related Articles in Windows NT Magazine"). Conversely, you can configure the RAS server to support any of these protocols. The essential difference between PAP and CHAP is that PAP passes a clear-text version of the username and password over the network to the server, whereas CHAP uses a challenge/response mechanism (MS-CHAP uses NT's challenge/response). (For further information about RAS authentication, see Microsoft Support Online Article Q136634, "Remote Access Services Authentication Summary," at http://support.microsoft.com/support/kb/articles/q136/6/34.asp.)

You configure the RAS client phonebook entry with the type of authentication you want to use, as Screen 1 shows. The Accept any authentication including clear text option corresponds to PAP, the Accept only encrypted authentication option corresponds to CHAP, and the Accept only Microsoft encrypted authentication option corresponds to MS-CHAP. If you can control both the client and the server, you will want to choose to use the MS-CHAP option to ensure that RAS uses the NT challenge/response for authentication on all stages of the desktop-to-server data path.

Encrypting MAPI Messages in Exchange
Once you have proved your identity to the NT server running Exchange Server, you can use the MAPI transfer protocol to send mail messages. You compose the message in an email client such as Outlook or Exchange. Exchange stores the mail message in your outbox until you send it, and in the inbox of the recipient receiving the message after you send it. The basic Exchange and Outlook email clients don't encrypt mail messages stored on the disk or in transit. If you want to encrypt a message in Exchange, you must use Exchange Server's Advanced Security features, which involve installing the Key Management Server and generating encryption keys for the clients to use.

Before Exchange 5.5 Service Pack 1 (SP1), the Key Management Server was a Certificate Authority (CA) that generated X.509 version 1 certificates and encryption key pairs, publishing the Public key in the Exchange directory. With SP1, the Key Management Server now generates and publishes X.509 version 3 keys and uses the Microsoft Certificate Server to generate the certificates. The Key Management Server also lets you propagate a client-side add-on, that lets the client encrypt a message by clicking an icon in the toolbar or selecting an encryption option from the menu. Several third-party products, including pretty good privacy (PGP) and the US Department of Defense's Defense Messaging Service (DMS), provide the same functionality based on alternative software or hardware key distribution systems.

If you use Exchange Server's Advanced Security, the Exchange client uses the CAST or DES algorithms to encrypt the mail messages you compose on your computer, before they leave the client machine. A message remains encrypted until the recipient decrypts it on the fly by double-clicking the message to read it. The message also remains encrypted while stored in the mailbox. This process is an end-to-end (i.e., writer-to-reader) security measure. (For further information about message encryption, refer to Holly Grabowski's TechEd98 presentation, "Exchange Security," on the TechEd98 CD-ROM or on the Microsoft Web site).

The MAPI transfer protocol uses RPCs, an interprocess communications (IPC) mechanism, to deliver information from the email client to the server. Transferring the message is identical regardless of whether you encrypt the message before sending it. However, the RPC transfer mechanism has two modes: unencrypted and encrypted. An encrypted session requires more overhead, and is not the default. You can configure the client to request the secure mode in the Advanced dialog box for the Exchange Server Information Service on the client, as Screen 2 shows.

Under Encrypt information, you select the When using the network check box or the When using dial-up networking check box if you want the Exchange client to request a secure (encrypted) RPC session with the server; otherwise Exchange transfers the message unencrypted. In addition, if you connect to your network and the Exchange server over RAS, you set the RAS client to encrypt the message by selecting the Require data encryption check box, which Screen 1 shows.

Implementing message encryption using Exchange Advanced Security based on the Key Management Server and message transfer encryption using Secure RPCs both require some processing cycles. You need to evaluate your security needs in light of your environment's hardware.

Finally, both message encryption and message transfer encryption happen at the application layer of the OSI stack. You might want to implement message encryption at a lower level, such as at the transport layer. However, you can't set this type of security in a LAN environment, although the situation might change in NT 5.0 with the introduction of native support for secure TCP/IP in the form of IPSec.

Authenticating POP3 and IMAP Clients
POP3 is a Request for Comments (RFC)-based protocol that Microsoft email clients such as Outlook, Exchange, and Outlook Express and third-party email clients such as Eudora and Pegasus use. You use POP3 to access a mail server over the Internet so you can download your mail messages to your desktop. To use POP3 in Outlook or Exchange, you need to add the Internet Mail Information Service in Exchange. As I've mentioned in previous articles, IMAP is a replacement for POP3.

A POP3 (or IMAP) client uses the POP3 (or IMAP) protocol to connect to the mail server, access the mailbox, download or delete mail, and disconnect. To send mail, a POP3 client uses a different protocol, typically SMTP. If you want your Exchange server to support POP3 clients, you need to configure POP3 support at the site, server, or mailbox level.

When a client tries to access a POP3 server, it can choose from several authentication methods. The most common method (as defined by RFC 1939, the original POP3 specification) is to send a clear-text version of the username and password to the mail server. This method is not secure but is the most common on the Internet. RFC 2222 introduced a more secure method, Simple Authentication and Security Layer (SASL), that some Internet Service Providers (ISPs) have started to support. SASL defines a generic challenge/response mechanism, in which the client and server negotiate to determine which challenge/response protocol to use. Exchange Server and clients can negotiate to use NTLM. Some servers and clients (and Exchange 6.0 over NT 5.0) will negotiate Kerberos ver-sion 5 (for more information about KERBEROS_V5, see "Related Articles in Windows NT Magazine," page 140); and some ISPs, the Microsoft Network (MSN), and Microsoft Commercial Internet System (MCIS)-based systems negotiate Distributed Password Authentication (DPA). In each of these SASL implementations, only a password hash travels across the network after the client uses an OWF algorithm to create the hash. This process is similar to the NTLM challenge/response mechanism described previously.

If you configure the Outlook or Outlook Express email clients for use with the Internet Mail Information Service, you can select the type of authentication you want to use. Screen 3 shows the Information Service Properties dialog box on the client. If you enter account name and password information under Incoming Mail Server, you are using clear-text password authentication and the client will send the username and password over the Internet unencrypted (despite the fact that the software hides this information with asterisks when you view it onscreen). If, instead, you select Log on using Secure Password Authentication, you are using SASL, and your client will attempt to negotiate a challenge/response logon with the server. If the server is running Exchange Server, it will use NTLM. The necessary client-side .dll files for NTLM SASL authentication come with the desktop operating system (OS); the .dll files for DPA come with Internet Explorer (IE) 4.0 and various other applications; and the DLLs for Kerberos will come with NT 5.0.

To authenticate POP3 clients, you must configure the Exchange server to support POP3 and the authentication method you want to use. Screen 4 shows the Exchange server properties dialog box. When you configure POP3 support, you use the Authentication dialog box to select the authentication methods supported. You have three options: Basic, Windows NT Challenge/Response (NTLM), and MCIS Membership System. The first option is RFC 1939 clear-text authentication, the second option is RFC 2222 SASL with NTLM, and the third option is SASL with DPA. To use the MCIS (i.e., DPA) option, you need to have installed the MCIS Membership Broker first. You can also select a version of each of the three authentication options that lets you use that type of authentication over a transport-level encryption with SSL.

Encrypting POP3, IMAP, and SMTP Messages
Several options are available for encrypting a mail message before it leaves the client. You need to install an application-layer program to distribute keys and perform encryption. You might be able to use the same encryption programs I previously described for the MAPI clients if they don't require realtime access to the server, but I haven't tested any of these applications.

TABLE 1: Summary of Exchange Security Options
  Authentication   Transport Encryption
  Basic NTLM DPA NT/RPC SSL
MAPI RPC   X X X  
POP3 X X X   X
IMAP4 X X X   X
SMTP X       X
In the original specification, POP3 and IMAP clients use clear text (i.e., 7-bit ASCII data) to download messages. This method is the default for Outlook clients regardless of whether you use clear-text or SASL authentication. In addition, to transmit messages, POP3 and IMAP clients use SMTP, which also uses clear text for the message header and body. (For further information about the default authentication settings, see "Related Articles in Windows NT Magazine," page 140.) RFC 1848 introduced a new encryption specification, Secure MIME (S/MIME), that allows encrypted message transfer, and Outlook Express supports this standard. To use S/MIME you need to acquire and install an X.509 version 3 certificate for the client from a CA such as the Microsoft Certificate Server available in the NT 4.0 Option Pack.

You can use SSL to implement transport-level encryption. To enable SSL, the POP3 server needs to have obtained and installed a digital certificate. You can obtain this certificate from a local Microsoft Certificate Server or from a third-party such as VeriSign (http://www.verisign.com). To install the certificate, you must have installed Internet Information Server (IIS) on the Exchange server and used the IIS Key Manager to request and process the certificate. (Don't confuse the IIS Key Manager with the Exchange Server Key Management Server).

After you install the certificate, a POP3 client can access the Exchange server over an SSL-encrypted transport session. The email client and the server go through a handshaking at the initiation of the communication. If the email client trusts the Exchange server's certificate, it uses the keys in the certificate to encrypt the transmission. This encryption process is independent of whether any applications running at higher levels have added encryption.

You must configure both the email client and the Exchange server to support SSL. Screen 5 shows the client-side configuration parameters for the POP3 client over SSL. To configure SSL support, go to the Advanced tab for the Internet Mail (POP3) Information Service and select the check box specifying that the server requires SSL. The client will then attempt to connect to the SSL port (TCP port 995) of the server rather than the standard TCP port 110 used for POP3.

Screen 4 shows the Exchange server's configuration parameters to support POP3 over SSL. I previously stated that Exchange Server supports three authentication methods for POP3: Basic, NTLM, and DPA. You can configure each authentication method so that the authentication takes place over a secure communications link, forcing the underlying session to be encrypted using SSL. When you select any of the options in Screen 4 to use SSL, the resulting encryption remains valid for the entire session, not just for the authentication process. Thus, the message transmits over an encrypted transport layer. For this process to succeed, the Exchange service account must be a member of the Administrators group.

Options
You can retrieve your mail from an Exchange server in several ways. Table 1 shows a summary of the built-in security options for each of the mail clients I've discussed. To protect your logon credentials, you use NTLM, DPA, or Basic authentication over SSL option on the Exchange server. You can secure MAPI client message transports by selecting RPC encryption. Likewise, you can secure non-MAPI client transports by using an SSL-encrypted transport. To encrypt your messages, you must install and use either Exchange Server Advanced Security and the Key Management Server, or S/MIME certificates. In a future article, I will discuss the security options for the browser client, using HTTP and S-HTTP to access your mailbox.