The term regulatory compliance is in the everyday vocabulary of many individuals responsible for designing, implementing, and managing Exchange email systems. A rush of legislation has come into effect recently, and companies both large and small are seeking ways to make their messaging systems comply with the new directives.

However, the various pieces of legislation don't explicitly define exactly how or when email needs to be retained. This is a boon for lawyers who thrive on interpretation but is less helpful for Exchange systems administrators who are looking for cost-effective, easy-to-implement solutions to prevent them falling foul of the law. In this article, I look at some of the main provisions in the most relevant pieces of compliance legislation and translate them as best I can into practical advice for Exchange system administrators.

Highlights from Key Legislation
Table 1 summarizes some key pieces of legislation that are likely to be of interest to Exchange systems administrators. Some of the main provisions in these pieces of legislation clearly identify areas that many Exchange administrators must address. Let's focus on two of the better-known directives mentioned in Table 1: the Sarbanes-Oxley Act of 2002 (SOX) and the Securities and Exchange Commission (SEC) Rule 240 Section 17a-4 directives.

The much-talked-about SOX is the 600-pound gorilla of compliance regulations. From an email perspective, SOX focuses firmly on company officers, directors, and financial staff and their actions with regard to email related to company audits or accounting reviews. But the responsibility for complying with SOX isn't incumbent only upon company officers. Everyone in the company—especially messaging systems administrators—must comply with the email retention requirements (e.g., by not willfully deleting an email message in contemplation of a federal investigation, by maintaining all relevant email in connection with an audit or review for 5 years). Unfortunately, SOX—and other compliance regulations—provides no clearly defined rules as to which email messages should be retained. Company legal counsel and technologists need to work together to put in place policies that define and retain email that's considered to be relevant to particular regulations.

Penalties for failure to comply with SOX aren't just financial, but can extend to incarceration for up to 20 years. Does this mean that the average Exchange systems administrator risks going to jail if he or she hasn't implemented a system to retain relevant email? Probably not, but in the eyes of the law, ignorance of the scope of regulations is generally little excuse for not meeting them. Certainly, planning for SOX compliance should head the to-do list for Exchange administrators whose companies are regulated by SOX, but ultimately budgets and instructions from senior management will determine company policy. Nevertheless, having at least a strategy for compliance might be sufficient to keep prosecutors at bay, despite what company directions dictate or, more importantly, fail to dictate.

SOX is a US law that applies to any public domestic or foreign company listed on a US stock exchange and to private US companies in the accounting and financial sectors. The regulations also come into play for private companies that are planning an IPO or that might be the target of an acquisition by a SOX-regulated public company. Although SOX applies to companies of all sizes, small private companies are subject to only a subset of SOX regulations. For small private companies, for example, SOX regulations forbid retaliation against whistleblowers who reveal document tampering.

The costs of SOX compliance can be steep. According to widely reported estimates from European companies, large multinational businesses can expect to pay up to $100 million to achieve full compliance with all the law's provisions. The cost of compliance means that some companies are electing to opt out—if they can. Some European businesses are planning to de-list from US stock exchanges and thus exempt themselves from the SOX requirements.

SOX legislation mandates that potentially relevant email be retained for 5 years after a review or audit, while SEC Rule 240 Section 17a-4 (which applies only to the financial services industry) mandates at least 6 years retention of potentially relevant email. However, simply retaining email isn't sufficient. Under both sets of regulations, the implementation must also guarantee the integrity of the retained email and ensure that it can't be deleted or modified.

SEC Rule 240 Section 17a-4, for example, leaves no room for doubt by stating that companies must "preserve records exclusively in non-rewritable, non-erasable format." Furthermore, the retained email must be indexed, searchable, and readily available for inspection. For example, the SEC rule states that compliance systems must "have capacity to readily download indexes and records" and requires that companies "at all times have available, for examination ... facilities for immediate, easily readable projection or production of micrographic media or electronic storage media images and for producing easily readable images." Although the SEC guidelines apply strictly to financial services firms, they define generally accepted best practices for long-term storage of information and should be considered the benchmark for retention solutions.

The potential volume of email that must be retained is significant. According to The Radicati Group, the average number of messages that corporate users send and receive each day is expected to grow by more than 30 percent from 2004 to 2008. And because corporate users are expected to process up to 15.8MB of data per day by 2008 compared with 10MB daily in 2004, storage requirements are predicted to increase by almost 60 percent in the next few years.

Thus the impact of implementing a regulatory compliance system leads to two storage challenges for email administrators: the retention of a large volume of data, and implementation of a search and retrieval system for that data. All in all, Exchange email administrators have their work cut out for them dealing with these challenges.

What do Exchange systems administrators need to do to comply with regulations such as SOX and SEC Rule 240 Section 17a-4? One of the first steps is to identify the employees whose email needs to be retained. Of course, you'll want to make sure all executive officers' email is retained, but you can't stop there. You'll typically need to ensure that executive assistants, corporate lawyers, accounting department staff, and other such users are subject to the same archiving policies.

Exchange journaling is the cornerstone of most compliance solutions, whether they're implemented in-house for small environments or through third-party solutions. Journaling sends to a journal mailbox a copy of all messages sent to or received by mailboxes in the database that's being journaled, giving you a complete email record for the archive. Exchange Server 2003 provides journaling functionality out of the box. (For details about the two forms of Exchange 2003 journaling and how to configure them, see "An Exchange 2003 Journaling Primer," April 2005, InstantDoc ID 45348, and "Exchange 2003 Advanced Journaling," May 2005, InstantDoc ID 45644.)

Many organizations implement a blanket archiving policy for all users. The downside of this approach is the amount of storage it requires. If you want to implement an organization-wide archiving system for your company's 1000 employees, for example, and those employees generate an average of 10MB of email per user per day, you'll need storage for 9.8GB of data every day, or 3.5TB per year. To journal the email for all an organization's users, you need to enable journaling for all databases on which users' mailboxes reside.

A wiser alternative is to classify users into those who are likely to engage in correspondence that's subject to strict archiving regulations and those who are unlikely to do so. Then you can group those users together on dedicated databases and implement journaling and archiving for those databases only, rather than for everyone in the company. This approach reduces the volume of email you need to retain and limits unnecessary journaling activity. You need to carefully determine and segregate all the users who engage in relevant correspondence, however, or your process might not satisfy lawyers that all relevant email is being captured.

Journaling facilitates basic retention requirements. To satisfy integrity requirements, however, you'll need another approach, such as offloading information from journal databases to a write once, read many (WORM) storage system. A drawback to WORM solutions, however, is that they don't allow modifications to records held in error. Other archiving solutions rely on arrays of traditional magnetic media disk drives with extensive mirroring and data duplication facilities. These solutions tightly control access to the archived data and the ability to modify it, thereby satisfying the requirements that the data be nonrewritable and nonerasable. To satisfy the indexing and searchability requirements of compliance legislation, you can use Exchange full-text database indexing or, more conveniently, some form of client-based indexing using Microsoft's Lookout search software or Google-style indexing technology.

Although the WORM and home-grown indexing approaches outlined in the preceding paragraph might be sufficient for small companies, they aren't scalable solutions for large organizations because the volume of data to be retained and managed quickly becomes overwhelming. Large organizations almost certainly will need a third-party archiving or regulatory compliance solution, such as one of those listed in Table 2. Such third-party solutions have myriad benefits, including compression implementations that often yield email volume reductions of up to 80 percent, automated indexing, storage integrity checking, and rules-based retention policies. Some solutions are formally ratified by bodies such as the SEC, but it's your responsibility to ensure that a particular solution meets the specific requirements with which you must comply.

Backup Myths
Many small organizations assume that retaining backup tapes satisfies the requirements for regulatory compliance. In general, however, it doesn't.

Retention of a backup tape simply provides a snapshot of the email stored in the Exchange system at the time of the backup. Email might have been sent to or by a user during the course of the day but subsequently permanently deleted by the sender or recipient (by emptying the Deleted Items folder). Such deleted email messages won't appear in the nightly backup tape, so retaining backups clearly doesn't satisfy regulatory compliance requirements.

Savvy Exchange systems administrators have turned to the Deleted Items Cache (aka the "dumpster"). Enabling the Deleted Items Cache for a period of, say, 30 days means that any items that have been permanently deleted by users will remain in the Deleted Items Cache for 30 days. Therefore, a backup tape would contain all of the email that a given user sent or received (and possibly later deleted). Furthermore, you'd need only one set of backup tapes to retain all email for a period of 30 days. However, you need to be careful about using long Deleted Items Cache retention periods because they increase storage requirements. Setting the Deleted Items Cache to a period of 30 days, for example, can result in Exchange database bloating of about 60 percent.

Clearly, backups have a part to play in retaining email for regulatory purposes. Where such solutions fail, though, is in the requirements for accessibility. Most of the regulatory compliance directives mandate that email archives be maintained in an "easily accessible format" for a certain period of time. It's unlikely that trawling through tens or hundreds of backup tapes to find a particular thread of messages could be deemed "easily accessible."

Even where regulations don't stipulate an easily accessible format for email archives, many organizations are keen to avoid the costs of such a time- and labor-intensive process to locate archived emails and often elect to implement third-party archiving systems instead. In the long run, as an Exchange administrator, your life will be simpler and your organization's exposure to litigation significantly reduced if you can easily access archived information, so you'll probably want to implement an archival system that offers indexing and fast retrieval. Increasingly, even many companies that aren't subject to regulatory compliance legislation choose to implement third-party archiving solutions in an effort to respond quickly to legal discovery demands. If a disgruntled employee has initiated legal action against the organization, for example, you might be required to locate relevant information.

Retaining the context of messages is crucial. When you consider all the places that a copy of a message might persist—in primary and CC recipients' Inboxes and personal folder stores (PSTs) and as backup copies, for example—you'll realize that messages are rarely completely deleted from an organization's archives. Chances are that a message that you believe has been purged from your email environment still exists somewhere. If that message outlines, say, "cutting off a competitor's air supply," the message might be damaging if it exists as a single entity. But if the comment was made in an email thread that's clearly part of a joke, the offending message takes on a completely different context. Thus, it's vitally important to implement a complete archive of relevant communications.

PSTs remain the proverbial thorn in the side of Exchange systems administrators for a variety of reasons. PSTs are difficult to back up in a centralized, controlled fashion, and users can use them to override storage management quotas and hoard information in an unstructured manner. As I've explained, isolated email messages lose context and can be dangerous. An organizational archiving policy must incorporate PST data as well as server-resident data.

Various vendors offer solutions that can discover PST-based email data and archive it into centralized repositories. Table 2 includes basic information about the PST archiving functionality of some third-party products. Such products are definitely worth exploring as part of a comprehensive compliance solution.

Meeting the Call
As Exchange systems administrators, it's not our responsibility to determine the policies our organizations need to implement to meet compliance regulations—that job lies squarely within the domain of the audit and compliance department. However, it is our job to work with the audit and compliance department to implement a technical solution that meets the company's legal obligations. Companies in regulated industries have no choice but to implement compliance infrastructures. When senior management issues the call to "Do it now!," we must be able to respond quickly and effectively.