Recently, one of my consulting clients was notified by the power company that the company needed to replace a transformer over the weekend. As a result, the client was going to lose power to the entire main office building from Friday night until Saturday night. Usually this wouldn’t be too much of an inconvenience, but Saturday is the busiest day of the week for this client. Losing power to the building starting on Friday night equated to a minor disaster. The power company gave four days notice before it cut power to the office building. This client has offices located throughout the United States and the central server is located in the building that was going to lose power. All the remote sites are connected to the central location with VPNs. The company also has a warehouse location that's connected to the main location with a point-to-point T1 connection.

One workaround was to somehow provide Internet access at the client's warehouse location in Los Angeles, move the server and firewall, and reconfigure all the VPN tunnels to connect to the Los Angeles warehouse so they would stay up on Saturday. Unfortunately, on such short notice, obtaining even a DSL line was out of the question. I did some research and discovered that Verizon Wireless has a product called Broadband Access Business Continuity. Here's a link to its product page http://b2b.vzw.com/productsservices/wirelessinternet/broadbandaccessbusinesscontinuity.html.

This product is essentially the same as the company's PDA Evolution Data Optimized (EVDO) product that provides Internet connectivity to PDAs and laptops, except it uses a Mobile Bridge MB8000 to provide wireless Internet access to an entire LAN rather than to a single device. I called Verizon and was able to establish service within two days --just enough time to get Internet access before the client’s office building was going to lose power. It’s even possible to order the bridge with a static IP address (actually the bridge still uses DHCP, but Verizon can configure the bridge with a reservation for a specific IP address). You might need to download the latest firmware to get the static IP address to work. We were able to obtain a static IP address after downloading the firmware version 5.02 00002 for the MB8000 Wireless Bridge.

After the bridge is rebooted, it takes as long as two minutes before the static IP address is usable. First the bridge will hand out a private IP address on the LAN side, then 30 to 120 seconds later after the bridge establishes a connection to Verizon Wireless; it passes the static public IP address to the firewall. You have to configure the bridge for DHCP on the WAN side. You can configure the firewall for the public static IP, but you have to wait a minute or two for the MB8000 to connect with Verizon. At first I thought the static IP address wasn't working, but you have to be patient and wait for the bridge to establish a connection with Verizon before the static IP address is usable.

As a test, I configured and established a test site-to-site VPN tunnel, although the tunnel had to be configured in aggressive mode to establish the connection. The distance between the wireless bridge and the cell tower has a significant affect on the performance of the wireless link. I tested the connection and received roughly 700Kbps download and 100 Kbps upload speeds to the Internet. Not blindingly fast, but certainly better than nothing. Because the test tunnel was successful, I was relatively confident that the VPN tunnels to the other remote locations would work.

On Friday afternoon, I reconfigured the remote firewalls to allow for management on the WAN site, just in case something went wrong establishing the tunnels. On Friday night, I relocated the server and firewall to the warehouse location and reconfigured all the tunnels. Fortunately, all the tunnels came up. The firewall had an available port, so I configured this port for the same IP scheme as the server. This prevented me from having to change the IP address of the server to get it to work at the warehouse location. The users were able to complete their work on Saturday. On Sunday, after their work was completed and I verified that power was restored to the main office, I moved the firewall and server back to their original location, reconfigured the tunnels and verified that everything was up and running. Ironically, on Monday they lost power around lunchtime for a few hours, but fortunately it didn’t affect them too much, because Monday is one of the client’s slower days.

Other than the advantage of obtaining connectivity on such short notice, the biggest advantage of this solution is its flexibility. As long as companies are close to a cell tower that provides Evolution-Data Optimized (EVDO) connectivity, they should have Internet access. The client isn’t tied to a specific physical location, which can provide tremendous advantages in a disaster recovery scenario, especially in earthquake prone Southern California. The client will keep this service active as a backup connection to guard against future power outages, as well as a contingency plan to aid in disaster recovery.

Tip
VMware recently released a new version of its VMware Server (version 1.01) with a release date of 8/14/06. This release addresses the performance problem of using a 64-bit host with Intel Extended Memory 64 Technology (EM64T) processors. If you use this release you no longer have to enter the sched.mem.pshare.enable=FALSE parameter to the guest’s configuration file.