A Trip to the Store with Exchange 2007

Transaction Logs and Replication
One of the more interesting changes that Microsoft made in Exchange 2007 is to reduce the size of transaction logs from 5MB to 1MB. Transaction logs capture details of every change made to databases in an SG; a busy server can generate tens of gigabytes of log data daily. The size change was made to accommodate log shipping, the mechanism introduced in Exchange 2007 to replicate databases to a location on the same server (local continuous replication—LCR) or to another server in a Majority Node Set (MNS) cluster (cluster continuous replication—CCR). Microsoft announced its intention to expand this functionality to accommodate database replication to a standby server in a different data center (standby continuous replication— SCR) in Exchange 2007 SP1. Log shipping is an important competitive step for Microsoft because IBM Lotus Notes, the company's major competitor in the enterprise market, has supported database replication for many years. It's also a feature that customers have demanded because they don't want to buy third-party products, such as Double-Take, to get similar functionality.

You can use log shipping only for SGs that hold a single database, which is an important consideration for system designers when planning a server's database layout. However, an Exchange 2007 Mailbox server can host up to 50 SGs, all of which can be replicated, so a great deal of flexibility exists. CCR can be used only for Mailbox servers, but you can deploy LCR on multirole servers such as those that support the Mailbox, Client Access, and Hub Transport roles. CCR and LCR don't support public folder databases directly, but Microsoft enables replication by supporting an SG that contains the only public folder database in the organization. If you have more than one public folder database, you can use regular public folder replication to ensure that multiple copies of public folder data exist within the organization.

When you enable an SG for database replication, Exchange "seeds" a passive copy of the database by copying the live database to the location where you want to host the copy (on the local server or another server). After seeding is complete, Exchange copies and validates transaction logs as the Store generates them, then replays the data in the logs to update the passive copy. If a failure occurs, you can replace the active copy with the passive copy of the database and continue operations. As storage vendors gain experience with Exchange 2007, it's likely that we'll see broader and deeper support for features such as database seeding and log shipping incorporated into third-party storage management products.

Reducing the size of the transaction log exposes Exchange to less risk of losing data due to incomplete file copies caused by a hardware failure. Clearly, you'll lose less data if Exchange can't copy a 1MB transaction log than if it can't copy a 5MB log. Exchange 2007 also includes a new feature called lost log resilience (LLR), which lets Exchange mount databases after a failure even if some data in transaction logs is missing (because of copy failures) and so can't be used to bring a database up to date. LLR runs on CCR Mailbox servers and is a major change for the Store; all previous Exchange versions require manual intervention by administrators if missing data caused the Store to refuse to mount a database. Typically, administrators have to run Eseutil to patch the database. Any manual operation is prone to error, and there are many horror stories where administrators have run Eseutil with the wrong command switches or attempted to apply the wrong set of transaction logs and caused major problems for a database.

Log shipping doesn't happen free of charge; servers incur a performance penalty to copy and replay logs into the passive copy of the database. Read and write I/O operations occur as Exchange copies transaction logs from the active location to an "inspector" directory, where Exchange checksums each log to ensure its integrity before replaying its content into the passive database. Memory is also consumed because Exchange uses a separate ESE instance to replay the transactions. Overall, early experience indicates that you can expect a 20 percent overhead to accommodate these operations on a server that supports LCR. With CCR, the passive node incurs the performance penalty because it pulls logs from the active node and replays them into its copy of the database.

The transport dumpster, another feature introduced in Exchange 2007, further reduces the risk of losing data by caching copies of messages on Hub Transport servers that go to mailboxes in replicated databases. When a failure occurs, Exchange can resend messages from the cache to affected mailboxes. Exchange suppresses the messages if they already exist in the destination mailbox. The transport dumpster can't cache some transactions, such as draft messages, but between log shipping, LLR, and the transport dumpster, Exchange 2007 is more resilient to hardware failure than any previous release.

A minor but important change associated with the smaller log size is the change in transaction log naming. Exchange 2007 still uses the SG prefix to identify the SG that a log belongs to (e.g., all logs beginning with "E0" belong to the first SG on a server), but Microsoft increased the hexadecimal number used to identify an individual log from six to nine characters, so you end up with names such as E010000aa15.log. Microsoft says you can now create more than 4 billion transaction logs per SG before Exchange is forced to reuse a log name.

Portability
Exchange 2007 databases are portable. In other words, you can take a database from a server and mount it on any other Exchange 2007 server in the same organization—something you couldn't do before. Portability helps with disaster recovery because it lets you quickly transfer databases from a failed server and mount them on another server, providing that you can still access the database files. The ability to access the database files to move them is a good reason why shared storage is still an important component in Exchange disaster-recovery planning: It's obviously much easier to switch databases between servers in a SAN than it is if the databases are on direct attached disks.

After the databases are mounted on the new server, you use the Move-Mailbox cmdlet with the -ConfigurationOnly switch to update the configuration data that's stored in Active Directory (AD) for the mailboxes that belong to the databases you just moved and redirect users to the server where the databases are now mounted. Portability is a welcome feature.

Deleted Items
Microsoft introduced the deleted items cache in Exchange Server 5.5 to let administrators avoid restoring databases to recover items deleted in error. The feature has been around for a while and is well understood; the only thing that has changed in Exchange 2007 is that the new default deleted-items retention period is 14 days (instead of 7 days). The deleted items retention period dictates when the Store permanently removes items from databases, so the change means that Exchange 2007 keeps deleted items for twice as long as Exchange 2003 does. This longer retention period means an increase in database sizes, possibly up to 10 percent depending on user behavior and the flow of message traffic in your organization.

The End of Streaming
Exchange 2000 introduced the streaming database (.stm) file as a companion to the regular Messaging API (MAPI) property database (.edb) file. .Stm files let Exchange store any information that arrived in raw Internet format, such as MIME-encoded messages and attachments, in a database purposely designed to support these formats instead of using MAPI properties in the .edb. Behind this decision was a belief that Internet formats would become much more prevalent than they have and that Exchange could avoid the performance hit of converting MAPI data when IMAP or POP clients such as Outlook Express fetched messages. Since Exchange 2000 appeared, server performance has increased dramatically, so the performance hit for format translation isn't all that important now. Outlook continues to be the most popular Exchange client by far, so MAPI remains the predominant format in use today. Microsoft therefore concluded that the .stm file was no longer needed and omitted it from Exchange 2007.