IPSec and Authentication Guard Administrative Ports

In "Protect Private Ports with IPSec," April 2002, Instant Doc ID 24273, I discussed IP Security's (IPSec's) components. I showed you how to use filter lists to look for certain types of traffic and assign a filter action to permit, block, or negotiate a secure connection that includes authentication, integrity-checking, and encryption. This month, I discuss the three authentication options—Kerberos, preshared-secret-key, and certificate-based authentication—and show you how to use IPSec to protect your Web server's private (i.e., administrative) ports, such as those for Windows 2000 Server Terminal Services, NetBIOS, and FTP.

If your server is part of an Active Directory (AD) domain and the computers that need to connect to a given private port on your server run Win2K or later and are within the same forest as your server's domain, you can specify Kerberos as the authentication method in your rule—and you've done all you need to do about authentication. Win2K will simply use Kerberos based on the computer accounts AD already contains. If you want to limit access to a subset of computers in your AD forest and those computers have static IP addresses, you can use specific source address ranges in your negotiated security rule. Then, only trusted computers within the allowed subnets can connect to your private ports. This technique is particularly useful in a demilitarized zone (DMZ) in which all the members are frequently members of a forest created specifically for the DMZ. By creating a forest for the DMZ, you can ensure that any computer in the DMZ receives traffic from only those systems or devices that are part of the security design.

However, if your trusted computers aren't limited to a few static IP addresses or at least subnets, source filtering won't work. Kerberos isn't an option if your Web server isn't a member of an AD domain, the other computers aren't part of the same forest, or the other computers run a pre-Win2K OS. Your alternatives are then preshared-secret-key or certificate-based authentication.

Preshared-secret-key authentication is the quickest authentication method to set up because users simply need to type in the same string of characters as the key on each computer. However, the preshared secret key is stored in the clear (i.e., in plain text) in the registry, which makes the key more vulnerable to theft than certificate-based private keys are. If a preshared secret key is compromised, you must change the key on every computer on which you've used it, whereas if a private key is compromised, you can simply revoke its associated certificate. If you administer a few computers, you can overcome this drawback to preshared secret keys by giving each client computer a different preshared secret key. However, you need to create a separate negotiate-security rule on your Web server for each client's key.

Certificate-based authentication doesn't require AD to let you use IPSec, and you get all the benefits of public key infrastructure (PKI). To use certificate-based authentication, you must specify a trusted Certificate Authority (CA) in the IPSec policy, and you must request and install a certificate from that CA on the Web server and on all the other computers that will be communicating with the Web server through administrative ports.

A Practical IPSec Policy
I'll show you how to set up a simple IPSec policy that lets port 80 and port 443 traffic into your Web server without applying IPSec but requires that computers connecting to any other port authenticate with preshared-secret-key authentication and subsequently encrypt any traffic. Such a policy wraps a layer of protection around all the ways into your server other than HTTP and HTTP over Secure Sockets Layer (HTTPS)—which IPSec can't protect because a typical public Web server must accept connections to these ports from anyone on the Internet.

To set up this policy, first open the Microsoft Management Console (MMC) IP Security Policy Management snap-in, right-click IP Security Policies on Local Machine, then select Create IP Security Policy to launch the IP Security Policy Wizard. On the wizard's opening screen, click Next. On the screen, leave the default name—New IP Security Policy—and enter

Limit and encrypt communications to authorized computers only
(except ports 80, 443)

in the Description field. Click Next again. Clear the Activate the default response rule check box, click Next, then click Finish.

In the resulting New IP Security Policy Properties dialog box, which Figure 1 shows, the list of IP Security Rules is empty except for the Default Response rule, the check box for which should be clear. Now, you must create a rule that limits all IP traffic to computers on which you've configured the same preshared secret key. Then, create another rule that permits connections to TCP port 80 and port 443 so that the public can still connect to your Web server. Click Add to launch the Security Rule Wizard.