Do you know how powerful (and dangerous) Exchange Server's ESEUTIL has become? ESEUTIL is Microsoft's "can't-live-without" utility for Exchange that looks at database files and lets you perform different functions using one of six modes: Defragmentation, Recovery, Integrity, Dump, Repair, and Restore. I had a chance to use ESEUTIL this week when I returned from a business trip to find my Exchange 2000 Server down and the database that holds my mailbox dismounted and inconsistent.
Here's a quick summary of ESEUTIL's six modes. Defragmentation mode (/d) lets you perform offline database compaction and maintenance. However, Exchange 5.5 enabled online maintenance, so there's little need for this mode unless you delete large amounts of data from a database. Recovery mode (/r) is the mode I used to recover my Exchange database, and I'll look at it more closely later in this column. You can use Recovery mode to perform database soft recovery and to bring databases to a consistent state using information contained in the database header and checkpoint file. Integrity mode (/g) is new (since Exchange 5.5 Service Pack 2—SP2) to ESEUTIL and lets you check the integrity of database pages by verifying the page checksum. ESEFILE, another Exchange utility, also performs this function but does so much more rapidly. Dump mode (/m) is also new and provides detailed information about Exchange database files, including .edb, .log, and .chk files. This information is dumped from the header information for each of the files. Repair mode (/p) lets you manually repair a corrupt database by scanning the physical database structure and fixing errors (extremely dangerous if you don't know what you're doing). Last, but not least, is the Restore mode (/c). Available only with Exchange 2000, Restore mode lets you manually perform hard recovery of a database and is useful during disaster recovery operations when you've selected the wrong options. Restore mode reads the contents of the restore.env file and performs hard recovery operations accordingly. Although ESEUTIL is a handy tool, always proceed with caution and under the supervision of Microsoft Product Support Services (PSS).
So, what happened to my Exchange database this week, and how did I recover it? First, the volume on the server that holds my transaction log files ran out of space over the weekend when some unusual replication traffic caused more than 2GB of logs to accumulate. This accumulation wouldn't have caused a problem except that my backup also failed (which would have purged the logs after they were backed up). The server ran out of space for the transaction logs and dismounted the database. When I returned to the office, my inbox wasn't available.
I began the recovery process by moving all the log files to an alternative location and making a backup copy of my database files (both the .edb and the .stm files). I used ESEUTIL with the /mh switch to determine the status of my database (it was inconsistent) and the log files required for recovery (this information is stored in the database header). Recovery proceeded by replaying log files (using the ESEUTIL /r switch) from the alternative location to bring the database to a consistent state, which I verified with the /mh switch after replay was complete. After the database was consistent, I simply deleted the old log files and checkpoint file and mounted the database. After I verified that the database was OK and I hadn't lost any data, I immediately performed a full backup. I also took a number of other steps to ensure that I don't run out of space again and that backup operations are reliable. Overall, I was lucky, and ESEUTIL saved my "bacon."
Obviously, by following some best practices and practicing what I preach, I could have avoided this situation altogether. However, it's nice to know that a tool such as ESEUTIL is available. Take time to learn the power of this tool and how it can aid you in day-to-day Exchange operations. As always, test it in a lab environment first, and use it with care.