Proper preparations make a disaster manageable

I have a love-hate relationship with service packs. I love all their bug fixes, but I hate the quirky things they do. This month, I had to rebuild my home office's Domain Name System (DNS), Windows Internet Naming Service (WINS), and Dynamic Host Configuration Protocol (DHCP) server from scratch after Windows NT 4.0 Service Pack 4 (SP4) zapped it. (For information about SP4's fixes to these services, see "Service Pack 4," page 87.) But the experience was good practice, and it reminded me that I've never discussed NT disaster recovery the way I need to.

Upgrading My Network
When I moved into my current house, I wired the house to Category 5 specifications, knowing that one day I'd upgrade my home network from its snail's pace of 10Mbps to 100Mbps Ethernet. The price of 100Base-TX Ethernet boards has fallen rapidly for the past few years, and 10/100Mbps boards have been about the only networking card on the market for the past year. So by the time I moved to my current home, 40 percent of the devices on my network could already handle 100Mbps. For months, I had the necessary cables in place and many of the necessary Ethernet cards, but I didn't upgrade my network to 100Mbps because I couldn't justify paying the price of 100Mbps hubs. However, when I saw eight-port NETGEAR hubs for less than $250 at an Office Depot recently, I walked out the door with the three hubs the store had in stock. (Incidentally, I found the NETGEAR hubs shortly after I applied SP4 to my network's servers.)

Before you start writing to tell me that I could have found cheaper 100Mbps hubs elsewhere, let me say I already know that. I bought the NETGEAR hubs for three reasons. First, I've never purchased a bad piece of equipment from Bay Networks. That doesn't mean that Bay Networks doesn't make any bad boxes, but I haven't tripped over one yet. Second, the NETGEAR switches have many useful LEDs that tell you the connection's speed, network utilization rates, and other similar information. (Minasi's First Law of Datacom Equipment is, "The more lights, the better.") Third, the switches were right there. I was in the store buying other products, and the hubs were on the shelf; I didn't have to wait around for a delivery. (OK, I admit that I need instant gratification.)

Soon after I popped the new hubs into the network—which took me only 5 minutes—I noticed that my DNS, WINS, and DHCP server was communicating at only 10Mbps. (You immediately see the value of all those LEDs.) I scratched my head at the slow speed for about 10 minutes, wondering whether the network cabling was faulty. Eventually, however, the senility clouds parted, and I remembered that the reason the server runs only DNS, WINS, and DHCP is that it's old. The reason for the slow network speed was that the server contained only a 10Mbps card. I popped a standard 3Com XL 100Mbps card into the server, loaded the card's driver, rebooted the server, and—wham!—received a blue screen of death, a serious blue screen of death. NT refused to get past the blue kernel initialization screen, claiming that it couldn't find winserv.dll. Because I swap NICs among my servers all the time, because 3Com is a well-respected firm, and because the 3Com card is popular, I guessed that an SP4 fragility caused the server's failure.

Dealing with a DHCP Disaster
I decided that the fastest way to solve the server problem was to skip the attempted heroics ("If I reboot with ERD Commander and swap the SYSTEM hives around ...") and just reinstall NT on the server. If you've read this column for the past 12 months, you won't be surprised that I have an installation script that lets me run a batch file to automatically rebuild any of my servers' OSs and settings from scratch. The script takes less than half an hour to run. So I started the installation script, left it for half an hour, and returned to find a pristine NT server with DHCP, WINS, and DNS already installed.

The only trouble the new NT system caused me was that it required me to restore the DHCP, WINS, and DNS databases to their precrash states. I had run restoration drills in the past, so I was ready to restore DNS and WINS. But I hadn't found an easy way to restore the DHCP database.

Three different Microsoft articles describe how to rebuild a DHCP server. Unfortunately, the three articles suggest restoring the database in slightly different ways, and not any of the articles' methods have worked for me. However, after a bit of experimenting, I figured out how to put an old DHCP database on a new DHCP server.

Preparing for DHCP Recovery
You won't be able to restore a DHCP database if you don't have access to backups of four DHCP files. Three of the necessary backups hold your DHCP database.

NT 4.0 DHCP databases live in three files: dhcp.mdb, the main database; dhcp.pat, a smaller database that is similar to Access system.mdb files (DHCP uses an Access engine to store its data); and a transaction log, a file with a name such as J5xxxxxx.log (the x's represent numbers and letters that specify the log's sequence number). NT 3.51 DHCP databases live in these three files: dhcp.mdb, system.mdb, and jet.log. You can find these files in \Winnt\system32\dhcp OSs. (For more information about the DHCP database files, see Darren Mar-Elia, "WINS and DHCP Preventive Maintenance," page 61.)

DHCP automatically backs up its database hourly. When you're looking for backups to restore the DHCP database from, dig a bit deeper than the standard database directory to \Winnt\system32\dhcp\backup\jet\new. In that directory, you can find the system's most recent backups of dhcp.mdb, dhcp.pat or system.mdb, and the latest log file.

In addition to these three files, you need a file called dhcpcfg to restore a DHCP database. Your DHCP server stores information about the scopes and configuration options that you choose for DHCP in the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DHCPServer\Configuration Registry key. DHCP backs up that Registry key to dhcpcfg, which resides in the \Winnt\system32\dhcp\backup directory.

I use an AT command to copy my DHCP server's dhcp.mdb, dhcp.pat, dhcpcfg, and log files to another server twice a day. However, if you need to restore a server, you'll probably be able to access the files regardless of whether you copy them to a second machine. Even if you're facing a dead DHCP server, you might be able to get the files. If your system drive is FAT, try booting DOS to read the files; if your system drive is NTFS, try using a second NT system root or a program such as ERD Commander to read the files.

Restoring a DHCP Server
You have the four necessary backup files handy, and you're ready to rebuild your DHCP database. What do you need to do next?

First, get the DHCP Server service running on the new machine. If the machine has never been a DHCP server, installing the service is no sweat. Fire up Control Panel, and select Network, Services, Add, Microsoft DHCP Server.

However, you might run into a problem if your server has previously run DHCP Server. The first time you install DHCP Server, the installation process creates the \Winnt\system32\dhcp and \Winnt\system32\dhcp\backup directories. Even if you wipe DHCP Server off a system, root out the software's Registry entries, and reboot before reinstalling DHCP Server, NT apparently remembers (probably through a Registry key I can't find) that DHCP Server once existed on the system. The software's installation process doesn't re-create the \Winnt\system32\dhcp and \Winnt\system32\dhcp\backup directories. When you reboot after reinstalling the service, DHCP causes a Dr. Watson error in tcpsvcs.exe, and the DHCP Server service won't start. If you wipe out and reinstall DHCP Server, erase the files in \Winnt\system32\dhcp and \Winnt\system32\dhcp\backup but don't erase those two directories. Install the Microsoft DHCP Server service through Control Panel's Network applet, then without rebooting, apply the service pack you want to use. Reboot after you apply the service pack. When the server restarts, go to Control Panel's Services applet, locate Microsoft DHCP Server, and stop the service.

Next, restore your database. Copy the three \Winnt\system32\dhcp\backup\jet\new backup files—either dhcp.mdb, system.mdb, and jet.log or dhcp.mdb, dhcp.pat, and the log file—into \Winnt\system32\dhcp. This action will overwrite files that the DHCP installation creates; when NT provides a warning dialog box, confirm that you want the OS to copy over existing files.

After you copy the three database backup files, restore the DHCP configuration Registry key. Open regedt32 (regedit won't work for this restoration), navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\DHCPServer, and select the Configuration key. With the Configuration key highlighted, select Restore from the Registry drop-down menu. Regedt32 will prompt you for a filename; enter the path to your copy of dhcpcfg. When regedt32 asks whether you want to overwrite the existing Registry keys, click OK.

Start the DHCP Server service, and open DHCP Manager. You'll see that NT has restored your old scopes. Click Scope, Active Leases to display the leases that DHCP Manager knows about; the window will be empty. Click Reconcile, and you'll see a dialog box that shows all the leases that the old DHCP server knew about. Click OK, and DHCP will reintroduce those addresses into its database. When you finish this step, your DHCP server will be fully recovered.

If you want to make sure this procedure works on your network, try it out. Take the four backup files off a working DHCP server, and try restoring them on a test machine that isn't connected to your production network—you don't want a rogue DHCP server on your network. Don't trust anyone else's disaster recovery methods until you've tried the procedures.