Admin essentials for your BlackBerry guru
You probably consider your email—and by extension, your Research in Motion (RIM) BlackBerry device—not a luxury but rather a necessity. Many organizations have ever-growing BlackBerry deployments and depend on BlackBerry devices for everyday work and emergency communications. BlackBerry Enterprise Server for Microsoft Exchange (BES) is now a mission-critical 24x7x365 component of IT services.
As an administrator, you might not only be responsible for maintaining efficiently working systems, you might also be the go-to guy or gal for all things BlackBerry, including such related concerns as security auditing, user documentation, and even training. With those items in mind, let's take a look at some tips and tricks that you can use to get a handle on your BlackBerry-related tasks.
First and foremost, you're probably expected to make sure BES does what it's supposed to do: Provide wireless remote access to Exchange Server data. As with any system, to ensure that BES runs smoothly, you need to monitor it. Some items, such as services and connectivity to RIM, need constant monitoring; others, such as pending messages and user statistics, need only periodic monitoring.
For the most part, I've found that BES is a set-it-and-forget-it system. Although I'd never suggest that BES maintenance is never necessary, the solution has evolved to the point at which it's good at self-correcting and keeping itself running efficiently. For example, the BlackBerry Controller service monitors the processes that are responsible for the MAPI interaction with Exchange. If the controller detects that the dispatcher or an agent has stopped processing—or worse, has terminated—the controller restarts them. However, although the Black-Berry Controller service monitors the dispatcher and its agent processes, it doesn't monitor any of the other services (e.g., Router, Policy Agent, Synchronization Agent). My first suggestion for keeping BES running smoothly is to monitor all services to make sure they haven't stopped.
Another suggestion for keeping BES running smoothly is to implement a standard practice of rebooting the BlackBerry server whenever you restart an Exchange server. Occasionally, I see instances in which a Black-Berry server is unable to perform email redirection following an Exchange server reboot. (Admittedly, this problem occurs far less frequently in BES 4.0.) This problem doesn't always affect all the mailboxes that the server supports, but it typically affects many or all mailboxes on a specific Exchange server.
Suppose you have four Exchange servers and one BES server. In this scenario, the users on three of the servers can send and receive email as usual and see no service degradation. Users on the fourth server report that they're unable to send email and that they're receiving messages very slowly, if at all. On the BES server, per-user Last Contact Time statistics indicate that the server and handhelds are communicating, but the pending counts for the users on the same Exchange server are steadily climbing. The Controller service never restarts the agents or the dispatcher, but after a BES reboot, the queues quickly clear and processing returns to normal. As this example illustrates, indicators such as Last Contact Time and Pending to Handheld aren't always great barometers of connectivity problems. Unfortunately, I haven't found a simple way to detect a stopped message flow such as the one I've described. Adding a BES reboot to your standard Exchange-reboot procedures is a good preventative step.
Another best practice is to monitor for failed connections. In essence, BES has two primary types of connections: one to Exchange and the other to the RIM Server Routing Protocol (SRP) network. To detect loss of connectivity to the SRP host, you can scan the BlackBerry server logs and look for events 10000 (\[SRP\] Ping Response not received) or 50000 (\[SRP\] Ping Response received). Every minute, the BlackBerry server sends a connection message to the SRP host to ensure that it's reachable. The BES logs event 50000 when the connection is successful, and it logs event 10000 when it isn't. If you see periodic 10000 events, don't panic! I've noticed that an occasional SRP response failure is typical and can occur because of network congestion. If you see them more frequently, however, your network or Internet connection might be overburdened, the LAN connection might be too slow, or you might have a misconfigured NIC. For another way to test for SRP connectivity problems, check out the Web-exclusive sidebar "Building an SRP Monitor," http://www. windowsitpro.com/ microsoftexchangeoutlook, InstantDoc ID 48403.
You might be aware of the Connection counter that BES adds to the Windows Performance Monitor subsystem, but I don't recommend relying on this counter as a means of detecting connectivity problems. In my experience, the counter indicates only the running or stopped state of the agent processes. To confirm connectivity between Exchange and BES, you again need to turn to the BES logs. As with the SRP connectivity tests, you can search for specific error codes that indicate problems. Two codes that you'll see when the BlackBerry server can't access a mailbox are 20176 and 20539. These event codes indicate that BES can't access a mailbox when scanning for new messages or personal information manager (PIM) changes, respectively. Network-connectivity problems or an offline Exchange Information Store (IS) can cause these events. You might also see them if a mailbox has recently been deleted from the IS, but BES hasn't caught up with the change. So, don't set off any alarms unless you see a large number of these events logged for multiple mailboxes in a short time span.
More About the Logs
You have two options for accessing the BES logs: You can read the text-based BlackBerry debug logs at C:\program files\research in motion \blackberry server\logs, or you can review Windows' Application Event Log. The BlackBerry debug logs are excellent sources of detailed information about what's happening on the BES server; however, the sheer amount of data to review can be overwhelming. For example, on a server containing about 300 user accounts, I typically see log-file sizes between 200MB and 300MB per day.
Some of the data that you'll find in the debug logs also appears as events in Windows' Application event log. Generally, unless you increase the debug-logging level, only the errors and highlights of BES activity appear in the Application event log. Although you'll find far more information in the text-based debug logs, I find the Application event log much easier to review for the errors I've mentioned. BlackBerry logging (in either the text files or the event log) uses a five-digit Event ID number scheme to specify activity and severity. The system logs errors in the 10000s, warnings in the 20000s, informational events in the 30000s, debug data in the 40000s, and any other events in the 50000s and higher.
In terms of review frequency and priority, you should investigate errors as quickly as possible, but you don't typically need to review warnings more than once per day. Warnings have many causes, so you'll need to develop a baseline to determine whether certain warnings are significant in your environment. Most of the warnings I see are related to individual user accounts rather than systemwide problems. Most tend to be transient mailbox-management concerns, so unless you receive a complaint, you can ignore specific-user warnings.
For example, you might receive a warning such as 20301 - BenjaminN @neulan.net - Unable to save configuration settings or statistics. This warning indicates that the BES can't write to the referenced mailbox; in most cases, this warning is logged when a mailbox has exceeded its send-and-receive limit. Unless Ben is complaining, this warning doesn't require immediate administrative action. However, don't blindly ignore all warnings. An example of a warning that might warrant administrator attention is 20000 - SRP Connection Failed. Recurring warnings can warn you of situations that—left unresolved—might affect system performance or cost your organization money. A simple way to capture and track generated error messages and warnings is through the BESAlert Service. As I describe in the Web-exclusive sidebar, "Using BESAlert for Error and Warning History," http://www .windowsitpro.com, InstantDoc ID 48404, this tool is most useful in determining event baselines over time.
Out in the Trenches
So far, I've covered some things you can do directly at the BES server, but another area that I find interesting and challenging is much more visible to the end user: training and documentation. As the BlackBerry solution has evolved, I've realized that I need to develop new FAQs, tip sheets, and how-to guides. To make these documents more useful and interesting for the user, I think it's important to include diagrams and screenshots. Unfortunately, you don't have a Print Screen button (as you have on your PC) for capturing images of the handheld interface. Or do you?
If you've browsed the BlackBerry Web site, you might have come across the BlackBerry Developer pages (http://www.blackberry.com/devel opers). Contained in this section of the Web site is a cool utility called the BlackBerry Java Development Environment (JDE). The JDE, which you can download, has a BlackBerry simulator, which Figure 1 shows. The simulator is a Java application that runs on your PC in its own window and emulates the functionality of a BlackBerry handheld. The simulator's primary purpose is to assist software developers in producing applications for the BlackBerry.
I don't do much Java development, but I've used the BlackBerry simulator to produce screenshots for my documentation. It's also a nice utility to have if you need to present a Black-Berry training class to a group of users. Because the simulator runs in a window, you can project it in giant size in the training room, thereby eliminating the need for a group of full-grown adults to huddle around one BlackBerry.
Installing the simulator is straightforward. First, you need to download and install the Java 2 runtime environment—Java 2 Platform, Standard Edition (J2SE) 1.4.2—from http://java.sun.com/j2se. Next, download and install the JDE from http://www.blackberry.com/developers/down loads/jde. The JDE comes with a simulator for the BlackBerry 7290. After you install the JDE, if you want to simulate other devices or load skins that look more like the BlackBerry that your organization uses, you can follow the BlackBerry Device Simulators link from the Developers page to load more than 20 other device simulators.
My hope is that these tips and tricks will help you become a more productive BlackBerry administrator. BES and the BlackBerry are ever evolving, so your skills and understanding of the solution must also constantly evolve. Pay attention to your system. Dig deeper and understand what BES is doing and how it works, and you'll soon begin coming up with your own tips and tricks.