At a recent conference, I spoke to a senior engineer for Aelita Software, an Independent Software Vendor (ISV) that produces Windows 2000 and Windows NT administration and migration tools, about a major bug related to Active Directory (AD) backups. During Aelita's development of ERDisk for Active Directory, the company's new AD backup product, engineers discovered that roughly half of all their AD backups resulted in corrupt backup copies. When restored, these backups cause the directory services on the restored domain controller (DC) to be unstartable. Upon further investigation, engineers discovered that the problem wasn't unique to Aelita's software but appeared to be a bug in Win2K's native API for performing AD backups and restores. Thus, this problem affects all software that uses the native API, including Win2K's built-in NT Backup utility and most third-party backup software (e.g., Computer Associates'—CA's—ARCserve, VERITAS Software's VERITAS Backup Exec).
Aelita engineers believe that the bug hasn't received attention because System State restores that include AD are fairly uncommon. Aelita reports that the problem is simple but unpredictable, and thus is difficult to reproduce during testing.
You might not discover this bug until you attempt to use a backup to restore AD on a Win2K DC. At that point, you're too late—when you attempt to restore a backup, the bug prevents the DC from starting and causes the system to display a Directory Service cannot start error message. If you use Ntdsutil with the semantic database analysis option to run the database semantic checker, you receive error 550: Database is inconsistent. With NT Backup, the restores become corrupt even when the Verify option is turned on.
In a conference call with Aelita President and CEO Ratmir Timashev, I confirmed that Aelita believes this problem is a bug in Win2K's base release and Win2K Service Pack 1(SP1). Aelita discovered this bug just before Microsoft released Win2K SP2. Timashev mentioned that during the month before SP2's release, Aelita worked with Microsoft to identify the bug, and Microsoft discussed its intention to include a fix in SP2, which it did.
At the time I heard about this problem, Microsoft hadn't documented it. Although empirical data proves that SP2 resolves the problem (neither Aelita nor I have been able to reproduce the bug under SP2), Microsoft doesn't even mention the bug in the SP2 documentation.
A week after I reported the bug in WinInfo Daily UPDATE, "The AD Backup Bug: Monster in the Closet?" (http://www.wininformant.com/articles/index.cfm?articleid=21351), Microsoft officially acknowledged the problem in the article "Windows 2000 Domain Controllers Restored with System State Backups Made Prior to SP2 May Not Boot" (http://support.microsoft.com/support/kb/articles/q295/9/32.asp). The article describes the bug's symptoms and acknowledges that SP2 includes a fix. The article also sheds some light on why the problem occurs.
To protect customers from this bug, backup software vendors must enable their products to verify an AD backup before the backup is restored. Aelita has already implemented this functionality in ERDisk for Active Directory. You can protect yourself by updating your Win2K DCs with SP2. If you're not planning to upgrade to SP2 immediately, consider installing the bug's hotfix, which is available through Microsoft.