Ensure a highly available system
For at least the next few years, many Exchange administrators will likely continue to support Exchange Server 5.x installations. Although Exchange 5.5 is seemingly stable and well established, problems can occur if you don't properly set up and maintain this system. To look for potential problems, I've developed the Exchange 5.5 Reality Check. In this article, I cover the first part of the Reality Check, which entails creating sound policies and procedures; monitoring the system's health and performance; updating, documenting, and configuring the system; tweaking performance; and preparing for an eventual Exchange 2000 Server migration. In a future article, I'll address the second part of the Reality Check: Exchange security concerns.
Creating Policies and Procedures
Although hardware and software are important, minimizing downtime must start with creating the necessary policies and procedures. Here are several important policies and procedures that you should implement in your organization:
- Create a policy that says administrators shouldn't aggravate a problem by making random changes if they don't know the problem's cause.
- Create a policy that says administrators should plan ahead for the deployment of service packs, updates, and hotfixes. Applying hotfixes on a whim should be strictly forbidden.
- To avoid restoring mailboxes that users might have accidentally deleted, create a procedure in which administrators don't delete mailboxes right away. Instead, administrators should rename and hide them. Placing a standard character string such as zz_ in front of the display name will cause the mailboxes to appear in the same area in the address list. After a mailbox has been inactive for 30 to 60 days, an administrator can delete it.
- Create a policy that says administrators should follow the principle of least permissions. They should assign only the Exchange and Windows permissions necessary to accomplish the task at hand.
- Create a policy that says unscheduled downtime isn't permitted unless a disaster occurs. Create an accompanying procedure in which administrators must notify the user community 7 to 10 days before any scheduled downtime.
Monitoring the System's Health and Performance
After you create sound policies and procedures, you need to regularly monitor your system's health and performance. If managing the Exchange servers is your responsibility, you should have a checklist of daily tasks you need to accomplish. You might be pleasantly surprised that this checklist can be fairly short. For example, here's my checklist of daily tasks:
- Perform a backup (preferably a full normal backup) of the private and public Information Stores (ISs).
- Scan the System and Application event logs for events that signal problems and events that signal typical operations, such as completion of online maintenance. Web Table 1 (http://www.exchange admin.com, InstantDoc ID 27483) lists the events that are important in Exchange 5.5.
- Check the disk space on all server disks.
- Check the Message Transfer Agent (MTA) and Internet Mail Service (IMS) queues for stalled messages.
- Make sure that your virus scanning software has the latest virus signatures and scanning engine. Review the event logs to confirm that signatures are being downloaded and that scans are being completed.
Two of my daily tasks are checking the queue lengths and checking the amount of free disk space. However, queue lengths can quickly increase and the amount of free disk space can quickly decrease, so checking them only once a day might not be enough. To avoid any problems, you can use Windows 2000's or Windows NT 4.0's Performance Monitor counters to continually monitor queue lengths and free disk space and generate an alert if any selected counters stray outside a preconfigured limit. The article "Keep Tabs on Exchange Server," March 1999, http://www.exchangeadmin .com, InstantDoc ID 5033, describes how to set up Performance Monitor counters.
Figure 1 shows a sample Exchange Alerts Properties dialog box for monitoring an Exchange server named HNLEX01. For each Exchange server you want to monitor, you must decide which queues to monitor. Web Table 2 shows a list of the counters that I recommend monitoring. The threshold values in Web Table 2 might have to be adjusted depending on how high your Exchange server's activity level is.
In the Exchange Alerts Properties dialog box, you need to set the monitoring frequency in the Interval option at the bottom of the dialog box. The default interval for polling the data is 5 seconds, but this interval is too short for this type of data. A value of 60 to 120 seconds will place less burden on your network and Exchange servers.
If you want to monitor free disk space, you must enable the logical disk performance counters. To do so, open the command shell window (click Start, select Run, enter cmd.exe, then click OK) and type
for NT 4.0 or type
Updating, Documenting, and Configuring
You can save yourself a lot of headaches by getting your Exchange and Windows systems up-to-date, then documenting those systems. You can also make a few quick configuration changes that will make your job a little easier.
Updating. If you're using Exchange 5.5, Standard Edition (Exchange 5.5/S), make sure that your private and public ISs don't approach 16GB, which is the maximum IS size for that edition. If your IS sizes are approaching 16GB, strictly enforce mailbox storage limits or consider upgrading to Exchange 5.5, Enterprise Edition (Exchange 5.5/E).
I recommend that you have the latest version of the OS and Exchange service packs installed on your Exchange 5.5 servers. The latest service pack for Exchange 5.5 is Service Pack 4 (SP4). In addition, Microsoft has released several hotfixes since SP4. You can find SP4 and the hotfixes at the Microsoft Download Center (http:// www.microsoft.com/download). For earlier hardware, check to see whether flash upgradable BIOS updates are available for that hardware. If you support many similar servers, standardize on the BIOS revisions for servers and RAID controllers; this standardization can make troubleshooting problems much easier.
You should install Exchange-aware antivirus software (e.g., Symantec AntiVirus/Filtering for Microsoft Exchange, Sybari Software's Antigen for Microsoft Exchange) on your Exchange servers if you haven't already done so. Picking different antivirus solutions for your Exchange server and your desktop machines is a good practice. Having different virus engines and signatures scanning for viruses reduces the risk of having a virus sneak onto your Exchange server.
Documenting. Review your Exchange server configurations and the configuration of all connectors, and record this information. Even making simple screen captures as documentation is better than having no documentation at all. For a good article about documentation, see "Documenting Your Exchange Server 5.5 Systems," December 2001, http://www.exchangeadmin .com, InstantDoc ID 22893. You can also purchase third-party documentation solutions if you want to create more advanced documentation. For example, Ecora Software offers tools for documenting Windows and Exchange systems.
Configuring. If needed, make the following configuration changes. These configurations might seem obvious to many administrators, but I often find them overlooked:
- Set deleted-item retention for at least 15 days for all public and private ISs. For example, to change this configuration for the private IS, open Microsoft Exchange Administrator and navigate to Configuration container, Servers, server- name (where servername is the name of your server). Right-click the Private Information Store object and choose File Properties.
- Item recovery and storage limits are found on the General tab. As Figure 2 shows, set the Deleted item retention time \[days\] option to 15 (or more).
- For each private IS, configure a Prohibit send and receive (K) storage limit, as Figure 2 shows. Even if you set a large storage limit, it will prevent a mailbox from filling up the IS disk.
- Convert all Exchange server file systems to NTFS.
- Confirm that all IMSs have SMTP-relaying restricted. For information about how to restrict SMTP relaying, see the Microsoft article "Controlling SMTP Relaying with Microsoft Exchange" (http://www.microsoft.com/technet/treeview/default.asp ?url=/technet/security/prodtech/mailexch/excrelay.asp).
- If you're performing Exchange 5.5 directory replication between sites, make sure that the schedule isn't set to Always. (You'll find this setting on the Schedule tab of each directory replication connector's Properties dialog box.) The Always setting causes directory replication to occur every 15 minutes, which can cause a lot of replication traffic for organizations that have more than 10 sites.
- If you have a multisite organization, confirm that the DS Site Configuration's tombstone lifetime is set to 30 days, which is the default value. (You'll find this setting on the General tab of the DS Site Configuration Properties dialog box. DS Site Configuration is in Configuration container on each site.) Tombstone lifetime controls how long each directory service holds on to objects deleted from the directory database before permanently purging those deleted objects. In most Exchange environments, changing this default value isn't a good idea. If a site doesn't receive directory updates from another site for longer than the tombstone lifetime, those deleted entries might reappear in the address lists as orphaned objects.
- Disable circular logging in both the directory service and the IS databases on each Exchange server. If circular logging is enabled, you can't use the transaction logs to perform disaster recovery because the transaction logs are purged shortly after all transactions in the logs are committed to the database. In addition, you can't perform incremental or differential backups. To disable circular logging, right-click the Exchange server, then click the Advanced tab. Clear the Directory and Information Store check boxes.
- Implement the Exchange 5.5 SP4 version of the Mailbox Manager to clean out the users' Deleted Items folder. You can find the Mailbox Manager in the \support\mbmngr directory of the Exchange 5.5 SP4 CD-ROM. To learn more about the Mailbox Manager, read "Control Mailbox Size with Mailbox Manager," November 1999, InstantDoc ID 6253. Carefully consider the ramifications of implementing Mailbox Manager, and notify your user community of the Mailbox Manager policies before using it.
- If you have a file-based virus scanner (e.g., Symantec AntiVirus Corporate Edition), make sure that it isn't scanning the Exchange directories.
- Don't run disk defragmentation utilities on any disk that contains an Exchange database when the database is in use. If you run a disk-defragmentation utility on a disk that has an Exchange database, do so only after backing up that database.
- Confirm that the Windows time zone and current time are configured correctly. You can use the Control Panel Date/Time applet to check these settings. If you adjust the time zone and correct the computer's system clock at the same time, the changes might affect delivery recipients, so you should restart the server. To keep the time synchronized, install the timeserv.exe utility from the Microsoft Windows NT Server 4.0 Resource Kit or a third-party time-synchronization tool.
- Increase the size of the System, Application, and Security event logs to at least 10,240KB to log more events. You must use the NT Event Viewer's event-log settings to individually configure each log's maximum size.
- Set the boot.ini timeout to 5 seconds to reduce the time it takes to restart the Exchange server. You can edit this value through the Control Panel System applet.
The IMS has a couple of configuration options that you might want to enable. These configuration options are on the Internet Mail tab of the Internet Mail Service Properties dialog box, which Figure 3 shows.
The first option is in the Attachments (outbound) section. In this section, you specify whether message content is in MIME or UUENCODE format. By default, the MIME option and the Plain text check box are selected. I usually select the HTML check box as well. This option prompts Exchange to transmit the message body as both plain text and HTML. Most mail clients today support HTML messages. (Outlook 97 is an exception.) Some people don't like HTML messages because the HTML formatting slightly increases the message's size and the possibility exists that an HTML message might contain an embedded script that could transmit a worm or virus.
The other option that I recommend selecting is the Clients support S/MIME signatures check box. People are increasingly using digital signatures to confirm the authenticity of email. If you don't enable this option, Exchange will strip the signature from any sent messages that have Secure MIME (S/MIME) digital signatures. However, if any software appends disclaimers or message signatures to a message after the user sends it, Exchange will strip out the digital signature.
After you update, document, and configure your Exchange 5.5 system, you might consider performing a disaster-recovery drill. This drill can help you determine whether you have everything you need (i.e., software, documentation, good backups) to rebuild your Exchange server in a timely fashion.
Chances are good that the next hardware that you purchase for Exchange will probably be for Exchange 2000 servers, so upgrades to the existing hardware your Exchange 5.5 servers use aren't likely. You can, however, improve the performance of those existing servers with little additional investment. Here's how:
- An Exchange server can almost always benefit from additional RAM. A simple test will tell you whether you might need more RAM. On the Exchange server, open Task Manager, select the Performance tab, and under Physical Memory, look at Available Memory. If the available memory is below 10MB, you need more RAM immediately. If it's below 50MB, you should consider adding more memory.
- If your IS transaction logs aren't on separate physical disk drives, you can improve performance by putting them on separate mirrored disks. Nothing else should be on the disks. The mirrored-disk trick is one of the oldest in the book. It not only improves performance but it also might help you recover more data in the event of a disaster.
- For larger Exchange sites, designate only one or two public folder stores per site to be the official public folder servers. Remove the replicas of all public folders from the other Exchange servers in the site. Don't forget to move the System folders, such as the Schedule+ Free/Busy, Offline Address Books, and Organization Forms folders. Using Exchange Administrator, delete the public folder stores from the Exchange servers that you've designated as mailbox or communi- cations servers.
- For servers that handle a lot of inbound and outbound mail or that expand distribution lists (DLs), move the \exchsrvr\mtadata and \exchsrvr\imcdata directories to different disks (physical or logical) and regularly defragment these disks.
- Disable unnecessary services and remove unnecessary connectors to free up additional memory and resources.
Preparing for Migration
Exchange & Outlook Administrator has published many articles about migrating to Exchange 2000. If you're planning to use Exchange 5.5 for another year or two, you might have simply ignored these articles because your migration is so far in the future. However, keep the following recommendations in mind:
- Before migrating to Exchange 2000, make sure that all the servers are running Exchange 5.5 SP4 with all the latest hotfixes. Although Microsoft requires only that at least one server in each site be running Exchange 5.5 SP3, the migration will go more smoothly if all the servers are running Exchange 5.5 SP4.
- If possible, complete your migration to Active Directory (AD) before beginning the Exchange 2000 migration.
- Read the Microsoft articles "XADM: Description of Exchange Server Migration Methods" (http://support .microsoft.com/?kbid=327928) and "HOW TO: Migrate from Exchange Server 5.5 to Exchange 2000 Server" (http://support.microsoft.com/ ?kbid=q316886) for an introduction to migration. These articles will help you learn about your migration options, how to prepare for a migration, and even pitfalls to avoid.
- Get training in AD and Exchange 2000. Good courses include "1572: Implementing and Managing Microsoft Exchange 2000," "1573: Designing Microsoft Exchange 2000 for the Enterprise," "2154: Implementing and Administering Microsoft Windows 2000 Directory Services," and "2355: Upgrading from Microsoft Exchange Server 5.5 to Microsoft Exchange 2000." To obtain information about these courses, go to http:// www.microsoft.com/traincert/training/find/findcourse.asp and perform a search on the course number.
Exchange 5.5 Onward Ho
Many organizations will probably still be running Exchange 5.5 for several more years, which isn't necessarily a bad strategy because Exchange 5.5 is stable and well understood. Many organizations have no downtime with their Exchange 5.5 systems because they have sound policies and procedures, check their system's health and performance, and update, document, and properly configure their systems. By performing a Reality Check, you, too, can enjoy a highly available Exchange 5.5 system.