Configuring Microsoft's multitalented security product in 6 steps

Internet access presents unique security challenges for both large and small corporations. As your business relies more and more on the Internet, you're increasingly concerned about how your employees use this resource. Employee email and Web surfing, intruder attacks, and bandwidth-hungry applications are straining your gateways to the Web. Microsoft Internet Security and Acceleration (ISA) Server—the successor to Proxy Server 2.0—offers a compelling solution to many of these problems with its dual-purpose firewall and caching functionality that lets you publish services running on internal servers. Completely mastering ISA Server's feature set can be time-consuming, but Microsoft has done a good job of simplifying the basic configuration of a relatively complex and comprehensive product. Independent security laboratory International Computer Security Association (ICSA) recently certified ISA Server—a development that will bring even more attention to this enhanced product. (For more information about ISA Server and Proxy Server, see "Related Articles in Previous Issues," page 72.)

With a little planning and some basic knowledge of the Internet, you can quickly configure ISA Server to control access to your network and reduce your company's bandwidth usage. You can also implement a host of other features, including basic network-intrusion detection, robust logging, reports, and alerts.

Topology Considerations
ISA Server's flexibility permits deployment in a variety of topologies. To streamline your installation, you must clearly identify and document the type of connection and security policy you want. You need to define how and where to restrict corporate access, what your remote or satellite offices' needs are, and whether you want to allow remote user connections. Also, think about how you can benefit from ISA Server's caching functionality. Caching stores previously received objects. In other words, the ISA Server cache—rather than the Internet—satisfies subsequent requests for an object (e.g., a picture, a Web page). Caching benefits you in two ways: First, it greatly reduces an object's repeated download time; second, it reduces your Internet bandwidth usage because the system retrieves the object from the Web only once. ISA Server's caching functionality improves on that of Proxy Server 2.0. To enable basic caching, you needn't worry about a customized setup; however, for more advanced installations, ISA Server provides many controls with which you can tweak its caching functionality's behavior and performance.

You also have two important installation decisions to make: First, which mode of operation do you want to implement? You can install ISA Server in one of three modes: Caching, Firewall, or Integrated, which combines the features of the Caching and Firewall modes.

Second, you need to define the ISA Server system's relationship with other ISA Server computers on the network. ISA Server can act independently of other ISA Server systems (if you install it as a standalone server) or as part of a team (if you install it as part of an array). The standalone installation provides caching and firewall functions that are similar to that of an array installation. However, the standalone installation has a few drawbacks: It doesn't integrate with Active Directory's (AD's) directory services, it limits scalability, and it doesn't provide centralized management capabilities. Arguably more secure, the standalone installation effectively isolates the ISA Server machine from your Microsoft network domain—assuming it isn't a domain member. Even if attackers compromise the ISA Server machine, they wouldn't have direct access to domain account information or your domain design. However, because of the isolation, you need to independently manage each additional ISA Server system's access policies, and you must use external methods to perform client load balancing. Because of these limitations, standalone installations are more suited to smaller deployments.

If you choose to implement an array configuration, avoid installing ISA Server on a domain controller (DC) on the perimeter (i.e., exposed to the Internet). Although locking down a DC for placement on the Internet is possible, I recommend minimizing the exposed applications and services by keeping your DC behind the ISA Server firewall. If your ISA Server system is a member server of your company's primary (trusted) domain, consider applying multiple layers of security and utilizing existing equipment in your network. For example, you could enable packet filtering on your Internet router or install multiple layers of ISA Server systems in a back-to-back perimeter network configuration. This type of compartmentalization can significantly increase your security. For example, you might configure the external ISA Server array for packet filtering and server publishing and configure the internal ISA Server array for authenticating internal users and applying your outbound security access policy. In this example, the external ISA Server array would be a member of an independent perimeter network domain and the internal ISA Server array would be a member of your corporate domain. These layers provide additional security by forcing network traffic that crosses your network to pass multiple independent checkpoints. Alternatively, you can put ISA Server systems in a separate AD forest and establish a one-way trust between that forest and your production domain. This approach gives you the benefits of an array without exposing your critical domain or requiring the additional servers that the compartmentalized model necessitates. If you use this design, keep an eye on your configuration to ensure that services such as application authentication work across this type of domain trust relationship. How you design your configuration depends on how much security you require as well as on your budget.

Installing ISA Server
ISA Server runs on the Windows 2000 Server family, and ISA Server's administration tools run on either Win2K Server or Win2K Professional. If you install ISA Server on a member server or DC, the computer running the administration tools must be a member of the domain. During the setup process, you can specify the location of the ISA Server cache files. At a minimum, you need one partition for the OS's boot files and the ISA Server software, and one NTFS-formatted disk for the cache files. Microsoft recommends installing multiple physical NTFS-formatted disks for best performance.

Before you proceed with the ISA Server installation, prepare your servers for Firewall mode by installing at least two network adapters—one for the Internet and one for your internal network. You can use one network adapter and one dial-up adapter, but the configuration I describe uses two network adapters. When you install ISA Server, the software defaults to a locked-down configuration. No communication can occur across the firewall. Therefore, make sure you're comfortable with your test network's setup before you install ISA Server on your production machines. Thoroughly testing your network configuration before you install ISA Server will save much time and frustration.

First, configure the external adapter with a routable IP address, a default gateway, and primary and secondary DNS addresses. In the external network adapter's Properties dialog box, clear the Client for Microsoft Networks and File and Printer Sharing for Microsoft Networks check boxes, as Figure 1 shows. Clearing the Client for Microsoft Networks check box helps isolate your ISA server from the rest of your external (untrusted) network, including the Internet. Clearing the File and Printer Sharing for Microsoft Networks check box prevents external users from using NetBIOS services to connect to your machine.

Next, configure the internal adapter. Assign a private IP address to this adapter, and leave the default gateway empty. The internal adapter, along with the Local Address Table (LAT), defines how ISA Server communicates with the internal network. You can use a routable IP address for another external interface (e.g., for a screened subnet), but ISA Server will treat this traffic differently from the way it treats traffic destined for the internal network. Much of ISA Server's proxy functionality is restricted to external-internal network traffic, as opposed to external-external network traffic. To increase security, I recommend clearing the Client for Microsoft Networks check box for the internal adapter. Doing so prevents the ISA Server machine from accessing internal shared resources. If you don't disable this client, an intruder that compromises the server can use it as a launching pad against internal servers. ISA Server supports Microsoft's implementation of Network Address Translation (NAT), called SecureNAT. If you plan to use SecureNAT, the private IP address that you assign to the internal adapter will be the default gateway for your client machines or intermediate routers. I don't recommend clearing the File and Print Sharing for Microsoft Networks check box for the internal interface because ISA Server creates a share of the installation files for ISA Server's Firewall mode. (These files reside at \isaservername\mspclnt.)

If you remove or add a required network service or change a network setting such as an IP address after you install ISA Server, the ISA Server services might fail to start. If you find yourself in this situation, try reinstalling ISA Server through the Control Panel Add/Remove Programs applet to solve this problem and preserve your configuration.

ISA Server extends Win2K's RRAS functionality. Install ISA Server without making any changes to Win2K's default RRAS installation. You can use an ISA Server wizard to configure a VPN through RRAS. If you plan to install ISA Server in an array configuration, add your ISA Server system as a member of the domain that will host the ISA Server services.

ISA Server's installation is straightforward. For details about the process, see Sean Daily, "Microsoft's Stellar ISA Server," October 2000. The article also provides a good introduction to the three available modes and describes when to use them. This resource is essential reading as you prepare your AD schema for ISA Server and install the application in a basic configuration. After you finish the installation, remember to install the most recent hotfixes and service packs. Microsoft has already released a few hotfixes for ISA Server. Keeping on top of hotfixes for this kind of perimeter-guarding security product is essential.

ISA Server Management
You accomplish all configuration management through the Microsoft Management Console (MMC) ISA Management snap-in. By default, the ISA Management shortcut resides on the Start menu under Programs, Microsoft ISA Server, ISA Management. ISA Server installs the ISA Management files in \Program Files\Microsoft ISA Server\msisa.msc.

Figure 2 shows the ISA Management snap-in with the Taskpad view enabled. The Taskpad view lets you access special Properties dialog boxes and useful scripts and wizards. The Taskpad view is enabled by default, but you can switch to the Advanced view for a more traditional Windows Explorer­like interface that doesn't provide access to some of the scripts and wizards.

Most changes that you make to the ISA Server configuration don't require you to restart the ISA Server service. ISA Server alerts you when a service restart is necessary to effect a particular change. Sometimes, a significant delay occurs between changing a rule and the rule taking effect. This delay can frustrate your troubleshooting efforts because you'll find yourself wondering whether the ISA Server system has applied a given configuration change. To start, stop, and check settings for the Web Proxy, Firewall, and Scheduled Content Download services, select the Servers and Arrays node from the ISA Management snap-in, select the name of the array, expand the Monitoring node, and select Services.

Six Steps to a Basic Configuration
Configuring ISA Server for basic firewall functionality is fairly straightforward but not intuitive. The following steps, which provide a basic packet-filtering and server-publishing configuration, enable your internal network to access the Internet with functionality similar to that of a traditional firewall using SecureNAT. Deploying ISA Server with SecureNAT doesn't require that you add software to client machines; however, you need to route outbound traffic from your internal clients to the ISA Server system's internal adapter (i.e., set client machines' default gateway to the internal adapter's IP address).

These steps will give you a good foundation on which to build configurations appropriate for your environment. Review the steps carefully and adjust them according to the level of security that you desire.

  1. Configure Web access for internal users. Granting Internet access to internal clients requires the creation of at least two policy rules. The following examples provide wider access than you probably want, but they will give you a good introduction to using ISA Server to configure Web security.


  2. The first example is to create a Site and Content Rule to permit any request, at any time, to any external destination with any content group. To create a Site and Content Rule for the enterprise (i.e., a rule that affects every server and array), in the ISA Management Tree, select Enterprise, Policies, and expand the default Enterprise Policy node. Then, right-click Site and Content Rules. (If you installed ISA Server as a standalone server, the Site and Content Rules folder appears below the server's Access Policy node.) Select New, Rule. Name the rule—for example, Internet Access for Internal Users—then complete the wizard to create a rule that permits access for all client requests to all destinations.

    You can later reconfigure or review an existing rule by double-clicking the rule in the Details pane to display the rule's properties. You can also configure rules to allow or deny access based on a destination (URL or IP address and path), time of day, authenticated user or IP, or content type. Building rules on such a wide variety of elements lets you closely monitor and control what passes through your firewall.

    The second example is to create a Protocol Rule to permit select protocols at any time by anyone. Protocol Rules determine which protocols internal clients use to access the Internet. Under the Enterprise Policy node or the individual server's Access Policy node, right-click the Protocol Rules folder, then select New, Rule. Name the rule—for example, Internet Access for Internal Users—then click Next. Select the Allow check box next to Client requests to use this protocol, click Next, then select the protocols to include in this rule. For each protocol that you want to permit users to access, click Apply Rule. If the wizard doesn't list a protocol that you need, expand the default Enterprise Policy's Policy Elements node and right-click Protocol Definitions to define a new protocol in ISA Server. If you don't want to select individual protocols, you can choose the All IP Traffic option; however, All IP Traffic includes only ISA Server­defined protocols. Therefore, you'll still have to create a Protocol Rule for any protocol that Microsoft hasn't predefined.

  3. Enable intrusion detection and IP routing. ISA Server augments its access controls with network-intrusion-detection features based on technologies from Internet Security Systems. Network-intrusion detection compares individual packets against a database of known attacks to determine whether an attacker is attempting to penetrate your firewall. ISA Server includes packet-level protection from several well-known attacks. You'll find additional application-level intrusion-detection features, including DNS and POP intrusion-detection filters, under ISA Server's Application Filters, which Figure 3 shows. To access these filters, expand the Extensions node (under the individual server node in ISA Management) and open the Application Filters folder. The ISA Server software development kit (SDK) includes samples for creating new custom application filters. However, these samples aren't for the faint of heart—they're coded in Visual C++ (VC++).


  4. ISA Server can also take control of IP routing. When you enable this feature, the software routes all traffic between the external and internal networks based on routing tables that you define outside of ISA Server (e.g., with the Route command). The software supports IP packet filtering, so you can permit or deny traffic routed between external adapters or destined for the ISA Server machine.

    To enable intrusion detection and IP routing, expand the server's Access Policy node, right-click the IP Packet Filters folder, then click Properties. On the General tab, select the Enable intrusion detection and Enable IP routing check boxes. On the Intrusion Detection tab, select the check boxes for each attack that you want the software to detect.

  5. Create IP packet filters. IP packet filters let external requests pass through (or land on) the ISA Server system. To create an IP packet filter, expand the Access Policy node, right-click the IP Packet Filters folder, then select New, Filter. The New IP Packet Filter Wizard will walk you through the steps of naming the filter and defining its characteristics. ISA Server conveniently predefines many popular filters. If the filter you want to define isn't on the list, click Custom on the Filter Type screen and define the protocol, traffic direction, and local and remote ports. When in doubt, use a packet sniffer (e.g., Network Monitor) to identify the characteristics of the packets for which you need to filter.


  6. Configure packet filtering and PPTP passthrough. You also use the IP Packet Filters Properties dialog box to configure other important filtering and routing properties. On the Packet Filter tab, you can configure ISA Server to filter IP fragments or IP options. Intruders sometimes use malformed IP fragments to launch Denial of Service (DoS) attacks. Also, to harm your firewall or internal host destinations, malicious users might use IP packets that contain invalid processing rules as IP options in their headers. You can configure ISA Server to filter these types of packets. However, the capability to filter IP fragments is incompatible with streaming media because of the way streaming media packets are constructed. Therefore, if you intend to stream video or audio packets across the firewall, don't enable filtering of IP fragments.


  7. You configure PPTP passthrough on the PPTP tab. PPTP passthrough enables the Generic Routing Encapsulation (GRE) protocol 47, which lets a PPTP tunnel client behind the firewall connect to an external PPTP server—a capability that Proxy Server 2.0's Winsock Proxy client didn't have. To assist you in troubleshooting any problems that might arise, select the Log packets from allow filters check box. Enabling this logging will significantly increase the size of your logs and the logging processor load but can be very helpful when you define your filter set.

  8. Host services on the ISA Server system. You might need to host services (e.g., Web, FTP, email services) from the ISA Server system or route the services across the ISA Server system to a screened subnet. Figure 4 shows how ISA Server represents the protocol type, local and remote ports, and traffic direction for several common services. Hosting services uses packet filters and IP routing to enable communication to services. Hosting is different from publishing, which step 6 describes.


  9. To host a service on an ISA Server system, install the application on the server and configure the packet filters to allow traffic on the services' specific ports to the ISA Server system's external adapter. Use caution when hosting services on the firewall: Your firewall server might be susceptible to a hosted service's vulnerabilities.

  10. Manually publish a server. ISA Server offers comprehensive options for making services running on your internal servers accessible to the Internet. When you publish a server, remote Internet users can target the ISA Server system's external adapter to access a particular service. Publishing services extends the traditional model of port forwarding and mapping. ISA Server uses the URL and port to redirect the user request to an internal server. To the external user, the ISA server appears to be hosting the service. The benefits of this functionality include stronger security for the publishing server and a reduction in the number of external routable IP addresses that you need. You can publish many services from one external IP address. The process of publishing a server to the Internet consists of a few easy steps.


  11. Expand the server's Publishing node, right-click the Server Publishing Rules folder, and click New, Rule. Name the rule, then enter both the internal server's IP address and the ISA Server system's external IP address. Next, from the list of predefined inbound server protocols, choose the server that you want to publish. These 19 protocols include Telnet, SMTP, DNS, IMAP, POP3, Network News Transfer Protocol (NNTP), remote procedure call (RPC) servers, and SQL. You can also define a new protocol with an inbound direction; the new protocol will automatically appear in the list. Because the server running the protocol resides in the internal network, you don't need to create an IP packet filter for the protocol as you would for hosting a service on the ISA Server system or in a screened subnet. Configure the internal server as a SecureNAT client. (The internal server's default gateway must point to the ISA server or an intermediate router.) No other special configuration of the internal server is required. ISA Server also includes wizards that can help you publish mail and Web servers.

Don't Stop Now
ISA Server is a compelling firewall-and-proxy solution that's impressively integrated with Win2K services. I encourage you to install ISA Server and delve into its other exciting features. Be sure to explore its alerts, reports, logs, Quality of Service (QoS) controls, and cache-monitoring features. Microsoft's SDK for ISA Server lets you extend this foundation .NET server product to special configurations. Regardless of your company's size—from small office to large distributed corporation—you'll appreciate the variety and flexibility that Microsoft has built into its new hybrid security product.

Related Articles in Previous Issues
You can obtain the following articles from Windows 2000 Magazine's Web site at http://www.win2000mag.com.

ZUBAIR AHMAD
"Windows 2000's Network Address Translation," February 2000, InstantDoc ID 7882
"Proxy Server 2.0," October 1998, InstantDoc ID 3848
SEAN DAILY
"Microsoft's Stellar ISA Server," October 2000, InstantDoc ID 15477
"Maximizing Proxy Server Security," October 1999, InstantDoc ID 7197