Learn how to implement and configure PKI in Win2K

In "Security Considerations for Migrating from NT to Win2K, Part 3," July 2001, I discussed what public key infrastructure (PKI) and Encrypting File System (EFS) can do for Windows 2000 system security. In Part 4, I look at the additional security that IP Security (IPSec) can provide for Active Directory (AD) domains and even for individual Win2K workstations that need to connect securely to one another.

IPSec: Seamless Security
Among the security improvements you'll realize after a Win2K migration is the inclusion of IPSec services. IPSec can provide machine authentication, data authentication, data-integrity protection, data confidentiality, and antireplay protection for end-to-end network connections (e.g., client/server). IPSec can also help secure remote access connections when users dial from their client machines to remote servers over any private or public IP-based network that uses Layer 2 Tunneling Protocol (L2TP). However, IPSec isn't recommended for use in VPN applications because of the security problems inherent in transferring the data and typically weak shared keys across public connections, such as the Internet. IPSec is best suited for security connections, such as Web or Win2K Server Terminal Services connections.

L2TP is a combination of Microsoft's PPTP and Cisco Systems' Layer 2 Forwarding (L2F). The Internet Engineering Task Force (IETF) didn't want to have two incompatible and competing technologies, so it mandated that Microsoft and Cisco work together to combine these two technologies into L2TP. L2TP encapsulates Point-to-Point Protocol (PPP) packets into IP packets, then uses IPSec to encrypt the packets to provide another layer of security.

Microsoft and Cisco also worked together to develop the IPSec services in Win2K. With AD, you can use Group Policies to define IPSec policies that you want to implement across the domain. When you implement the Internet Key Exchange (IKE) in Win2K, you can use three authentication methods:

  • Kerberos—provides authentication between computers in one Win2K domain or across multiple trusted Win2K domains. Kerberos is the strongest, most secure authentication method.
  • Certificates—uses Win2K Certificate Services or another third-party trusted certificate source (e.g., Entrust, VeriSign) for authentication.
  • Preshared keys—uses passwords that come in the form of preshared authentication keys to establish secure connections between systems. Preshared keys can't provide data-packet protection. Preshared-key authentication is the most basic, least secure authentication method.

When two IPSec-enabled computers establish a connection with each other, they first generate encryption keys (i.e., session keys) to encrypt the application data they send and receive. Only those two computers know the session keys, which helps protect the application data from malicious network sniffers or packet-capture software. The computers then decide the type of authentication to use for the connection based on the policies in place and the key authentication methods available to each system (i.e., Kerberos, certificates, or preshared keys). During this process, the systems constantly refresh the session keys based on the settings in the IPSec policy.

IPSec is rather effective in several scenarios. These scenarios include

  • establishing user-to-user connections (e.g., two engineers requiring secure IP communications to transfer sensitive data)
  • establishing secure connections to internal network servers (e.g., a user connecting to a secure server that stores sensitive data)
  • locking down inbound connections to servers that have a direct connection to the Internet, which aids in securing initial authentication to the server for secure remote connections from an outside user through a VPN tunnel

Let's look at how you can use IPSec to establish user-to-user connections.

Giving IPSec a Try
Currently, Win2K is the only Microsoft OS that supports IPSec. IPSec is relatively easy to implement, but it can wreak havoc in your network connections if you don't set it up properly. Thus, I recommend that you experiment with IPSec on a test system before you implement it on your networks.

Let's implement IPSec on two test computers (i.e., Computer A and Computer B) that have Win2K Server or Win2K Professional installed and an operational TCP/IP network connection. In this test implementation, let's use the most basic authentication method to secure IP communications: preshared-key authentication. This method lets you quickly set up a test environment because you don't need to set up a domain or AD controller. However, be aware that the keys are in plaintext in the registry and thus pose a potential security risk.

To set up preshared-key authentication for user-to-user connections, you need to use the Microsoft Management Console (MMC) IP Security Policy Management snap-in to add and configure the appropriate IPSec policy on both Computer A and Computer B. (Note that you must add the IP Security Policy Management snap-in to MMC—it doesn't exist by default.) Then, you need to implement the policy. Although this test implementation is rudimentary, it shows you the basics about how to set up IPSec and how IPSec works.

Setting up Computer A. On the Start menu, click Run, type mmc.exe, then click OK. In the Console1 window that appears, click Console, then click Add/Remove Snap-in. In the Add/Remove Snap-in dialog box, click Add, then scroll down the list of available snap-ins until you find IP Security Policy Management. Select this snap-in, then click Add. The Select Computer dialog box, which Figure 1 shows, asks you to select which computer you want the snap-in to manage. Select Local computer, then click Finish. You would select a different option if you were setting up the IPSec policy for an entire domain or for other domains or computers in the network (thereby letting you remotely adjust the policy). Close the Add Standalone Snap-in dialog box, then click OK in the Add/Remove Snap-in dialog box.

The Console1 window now contains the IP Security Policy Management console. In the directory tree in the left pane, you see the IP Security Policies on Local Machine entry under Console Root. Click this entry to go to the window that Figure 2 shows. At this point, you need to select the type of policy you want to use. You can select a custom policy or one of the following predefined policies:

  • Client (Respond Only)—If you select the Client (Respond Only) policy, the IPSec client negotiates IPSec security when a connection to a computer that supports IPSec contacts and requests IPSec communication from the client system. However, the IPSec client doesn't initiate security negotiations on its own.
  • Secure Server (Require Security)—If you select the Secure Server (Require Security) policy, all IP communication to and from the policy target must use IPSec. You probably don't want to use this option unless you plan to enable IPSec across your entire network.
  • Server (Request Security)—The Server (Request Security) policy is a combination of the preceding two policies. The server requests to use IPSec when the server connects to another system. If another system connects to the server, the server requests that the connecting system use IPSec for securing the connection. However, the server allows inbound and outbound traffic if the system the server is connecting to or from doesn't use IPSec.

For this test implementation, let's use the Secure Server (Require Security) policy. Right-click this policy in the right pane, then select Properties. In the window that appears, clear the All ICMP Traffic and check boxes under the IP Filter List heading. Usually, you would configure these IP filters to meet any particular IP filtering needs that you might need to implement, such as a filter to block a particular unwanted IP address from connecting to a server. But for this test implementation, let's not use IP filters, to simplify the overall testing process.

Make sure the All IP Traffic check box is selected. Double-click that filter to open the Edit Rule Properties dialog box, which lets you assign the filter's settings. Click the Authentication Methods tab, then click Edit. Figure 3 shows the Edit Authentication Method Properties dialog box that appears. In this dialog box, select the Use this string to protect the key exchange (preshared key) option. In the text box, enter the characters that you see in the text box in Figure 3. These characters will become the preshared key that the policy encrypts and uses for authentication between systems.

Copy this string of characters into a text file on a disk so that you can use the same characters on Computer B. Click OK, then click OK again to return to the IP Security Policy Management console. Right-click the policy, then select Assign from the menu to activate the policy.

Setting up Computer B. On Computer B, repeat the procedure for Computer A until the window that Figure 2 shows appears. Instead of right-clicking Secure Server (Require Security), right-click Client (Respond Only). Leave the check box selected, then click Edit. Click the Authentication Methods tab, then click Edit. In the Edit Authentication Method Properties dialog box, select the Use this string to protect the key exchange (preshared key) option. In the text box, paste the key you saved in the text file. Click OK, then click OK again to return to the snap-in console. Right-click the policy, then select Assign from the menu to activate the policy.

Implementing the policy. On Computer A, choose Start, Run. Type cmd.exe, then click OK. At the command prompt, type the command

secedit /refreshpolicy machine_policy

then press Enter. This command forces the machine to refresh its security policy and implement the changes you made. (You might want to execute this command on Computer B as well to speed the implementation.) While you're still in the command-prompt window, type ping followed by Computer B's IP address, then press Enter. If you receive a reply from Computer B, IPSec is working. If you see the message Negotiating IP Security but you don't receive a reply from computer B (i.e., the Ping test fails), the security policies might not have propagated yet. Wait a few minutes, then try pinging Computer B again. If the Ping test fails again, either the configuration is incorrect or you didn't enter the same key on both computers.

Another good test is to open a Web browser on Computer A and try connecting to a Web site. The connection should fail. The computer waits for the Web site to respond with the preshared key. Because the Web site doesn't have this key, the attempt to access the site fails.

If you're experiencing problems with an implementation, you can use Win2K's IP Security Monitor to monitor IPSec connections and troubleshoot problems. To run IP Security Monitor, choose Start, Run, type cmd.exe, then click OK. Type ipsecmon.exe at the command prompt, then press Enter. When IP Security Monitor opens, click Options. Change the Refresh Seconds value from 15 to 1, then click OK. Try the Ping test again. IP Security Monitor will now contain output that can help you troubleshoot problems.

Until Next Month, Tinker Away
Implementing IPSec is a great way to further secure your network traffic. I've shown you the basics. Now you can tinker with implementing IPSec on your test systems. You can experiment with configuring the predefined policies or creating custom policies.

Next month, I'll conclude this five-part series by introducing you to the new Win2K Service Pack 2 (SP2). I'll discuss the security fixes that Win2K SP2 includes and give you some tips about how to secure Microsoft Internet Information Services (IIS) 5.0 servers.