The Bad Item Conundrum (or, How much data would you like MRS to drop?)

My recent article “Corrupted Items and Mailbox Moves in Exchange 2010” provoked some comments about the number of corrupt items that are met when administrators attempt to move mailboxes as well as some suggestions as to how you might reduce the problem.

A “bad item” to Exchange 2010 is an item that the Mailbox Replication Service (MRS) considers to be corrupt in some form when it attempts to process the item. The most common scenario is when MRS moves a mailbox, but bad items also come into play when MRS imports or exports mailbox data from or to a PST.

The default bad item threshold for MRS is zero. In other words, unless you explicitly tell MRS otherwise, it will attempt to process every single item, and if corruption is met in any item, MRS will halt the operation. You can restart a mailbox move or PST import/export by changing the properties of the MRS request with a higher bad item threshold and then resuming the operation. For example, commands like this might be used to set a higher bad item limit for a mailbox move and then resume it:

Set-MoveRequest -Identity “TRedmond” -BadItemLimit 20

Resume-MoveRequest -Identity “TRedmond”

Unfortunately, you can only amend the bad item threshold using the Exchange Management Shell (EMS) as the Exchange Management Console (EMC) interface dealing with mailbox move requests doesn’t allow access to this property. And of course, EMC doesn’t have any UI at all for dealing with PST imports and exports.

Avoiding the need to intercede with stalled requests is one reason why administrators usually specify a higher bad item threshold for mailbox moves (in particular). This is especially so when you’re moving mailboxes from older versions of Exchange as all of the evidence indicates that there’s a far higher chance that corruption lurks in mailboxes hosted on older servers. One reason why this might be so is that older versions of Exchange usually support older clients: most corruptions are caused by MAPI clients that fail to update properties to the satisfaction of MRS. Another reason is that the Exchange Store has become better and better as the versions have rolled by, meaning that there are fewer corruptions caused by server bugs in Exchange 2010 than previous versions.

It’s also true that Exchange 2010 mailboxes are “cleansed” by MRS every time they are moved. In fact, those of us that remember when it was common practice to take databases offline to run ESEUTIL to rebuild their internal structures (colonic irrigation for databases) could think of an MRS move mailbox operation in similar terms because essentially all of the mailbox structures are rebuilt by MRS when it moves the mailbox from one database to another.

One good piece of advice I heard from another Exchange MVP is to run ISINTEG on databases before attempting to move mailboxes from an Exchange 2003 server. This makes sense because ISINTEG is all about fixing minor logical corruptions within Exchange databases and so might well remove some of the lurking corruptions that MRS will complain about when it moves mailboxes. I don’t think this step is necessary for Exchange 2007 databases and it’s absolutely useless for Exchange 2010 as ISINTEG is replaced by a new set of store “online fix” cmdlets in this release.

Other pieces of good advice about preparing for successful mailbox moves include 

  1. Adjusting the size of the organization's MaxSendSize property with the Set-TransportConfig cmdlet if you know that mailboxes contain very large items. MRS uses MaxSendSize to know the maximum size of an item it can process when moving mailboxes or importing/exporting from/to PSTs.
  2. Review the quota sizes for target databases before you start to move or import mailboxes to ensure that the databases will be able to accept the data you want to move. MRS is unable to dynamically adjust mailbox or database quotas to make things fit. 

The bad item threshold can go as high as 2,147,483,647. This is a nice number but it’s unreasonable. A more practical value is somewhere in the 10 to 100 range. However, the critical point to remember here is that MRS drops bad items. In other words, if MRS meets an item it deems to be corrupt, it won’t process that item and so the item won’t make it into the destination mailbox or PST. Most of the time this doesn’t matter because the item is probably corrupt in such a way that a client can’t access it anyway, but if you run mailbox moves with a bad item threshold of 100, you’re saying that MRS can cheerfully discard up to 100 items while it does its work. I have a nagging feeling that something important might be present in 100 items but maybe not. In any case, the Exchange developers force you to accept the responsibility for your actions once you specify a bad item threshold higher than 50 by requiring you to include the AcceptLargeDataLoss parameter when you create the request.

The bad item conundrum is determining the right threshold. Zero is good from the perspective that MRS will never drop anything, but bad in that you’ll spend a lot of time managing stalled move requests. Ten to twenty is a range that should allow the vast majority of mailbox moves to complete without administrator intervention. Above this we start to get into the realm of increasing data loss. You might view this as a way to help users clear out useless crap from their mailboxes; they might view it as an unjustifiable assault on their most precious possessions. I don’t know your users so I can’t say how they’ll react. But I’m sure you do…

Follow Tony @12Knocksinna

Discuss this Blog Entry 3

on Jul 18, 2012
A good article Tony. I work for a company that specialises in migrating data archived in third party archives, often back into Exchange, but we have to do something similar working with customers to define an acceptable threshold for items that fail to migrate. This is often a small percentage, but it does need careful handling. No customer Ive worked with yet has had any expectation so far that there will not be some percentage of unrecoverable data for some reason. Generally those who have been through an Exchange migration or have had Exchange in for a long time, have experienced occasions with items corrupting. In particular anyone whos dealt with PSTs truly understands this! Our experience has been that businesses and users do accept a small degree of corruption as 'normal', as long as the items in question can be identified somehow for future records.
on Jul 18, 2012
An interesting twist to this is when a migration project has some oversight from a compliance or governance department - particularly in a common scenario that our firm gets involved in: the migration of a legacy email archive at a time of acquisition, merger or litigation - when a lawyer takes the perspective that ANY email might be fundamentally important to a matter they are involved in. This approach is more common in the US and in financial institutions of course, but can be anywhere. The provenance of emails can be highly variable, they might have been originally generated in a long-forgotten desktop mail application, sent out by an iffy MTA over the internet, delivered into a Notes system, migrated to Exchange, archived by a long-defunct archive system, etc. Over their life of a few years some email standards may have evolved also... As claredrew highlights above, whenever email is being moved it pays to set expectations that there could be difficulties - and if recovering the maximum amount of data is vital better make sure you are working with a tool vendor that will adapt their software to help you when that is what it takes.. Barney TransVault
on Jul 13, 2012
Hi Tony, EMC actually does allow you to give a bad limit count in E2K10 SP1 for individual mailbox moves. For more than 1 mailbox move, you will have to use the EMS.

Please or Register to post comments.

What's Tony Redmond's Exchange Unwashed Blog?

On-premises and cloud-based Microsoft Exchange Server and all the associated technology that runs alongside Microsoft's enterprise messaging server.

Contributors

Tony Redmond

Tony Redmond is a senior contributing editor for Windows IT Pro and the author of Microsoft Exchange Server 2010 Inside Out (Microsoft Press) and Microsoft Exchange Server 2013 Inside Out: Mailbox...
Blog Archive

Sponsored Introduction Continue on to (or wait seconds) ×