SAN Topology

An old saying about storage claims that data will expand to consume the available storage space, and many applications support this claim. E-commerce, imaging, data warehousing, enterprise resource planning (ERP), and customer relationship management (CRM) are applications that fill storage media quickly and seemingly without end. Data accessibility for these applications needs to be fast, and availability is paramount. Storage Area Networks (SANs) provide high-speed storage pools through a group of connected servers and high-speed workstations. (For more information about the rising need for SANs, see Mark Smith, Editorial, "Storage Area Networks," April 1999.)

Outside the mainframe world, a discrete instance of each crucial application (e.g., ERP) resides on a server (i.e., 10 servers will house 10 applications). This trend arises from the modularity of systems—especially client/server applications—and the history of adding applications after other successful application deployments. System modularity creates server farms and can result in multiple instances of data. If these instances need to relate to each other, you must use replication or synchronization methods to resolve them. Therefore, the monolithic server data becomes painful to organize and manage. SANs help ease this administrative burden.

SANs are networks within networks. The SAN design disassociates server applications from data storage without sacrificing storage access times and lets numerous servers and applications access the data.

SANs minimize the need for servers with discrete, enormous stores of data, and you can balance reliability and availability needs. You can also amortize storage costs over several servers and their applications.

SAN storage farms support many host OSs and data filing systems. The host OS defines how SAN members access a file system. SANs logically appear to Windows NT as locally accessible volumes under FAT or NTFS.

Servers (or high-speed I/O workstations) with connections to a high-speed I/O channel make up SANs. For example, in a SAN, which Figure 1 shows, servers and workstations connect to the hub through a switch. SCSI or fibre channel connects the workstations and servers to the storage. The SAN connection method dictates the SAN design and affects the extensibility and accessibility of the data that the SAN stores. Let's examine the available methods and their features.

The SCSI Method
SCSI is a good connection method for SANs because most servers have a SCSI host bus adapter (HBA). SCSI is a parallel bus (i.e., each bit occupies a separate wire) that has a 25m distance limitation (i.e., length). Wide SCSI can send two bytes (16 bits over 16 wire pairs) per time frame, and Fast SCSI increases the data transfer rate from the standard 5MBps to 10MBps. Ultra SCSI enhances the speed and the number of bytes that it can send concurrently. Ultra 160 and Ultra3 SCSI increase the speed to 160MBps.

SANs use SCSI because of its speed. At 160MBps, an Ultra 160 SCSI can far exceed (at burst speed) a full duplex Ethernet's speed (200 megabits per second), and even comes close to full duplex gigabit Ethernet (theoretically 2000 megabits per second, but closer to 186MBps before packetizing overhead).

SCSI has its limitations. One problem with SCSI is that an electrical interruption, a SCSI reset, occurs when you add or remove devices from the SCSI bus. During the reset, the bus loses pending commands. Although some vendors have made their devices less vulnerable to reset, a post-reset hiatus often occurs while the device sorts out the commands that were pending.

SCSI connections also have practical limitations. Each HBA uses one SCSI ID out of the seven available. As Figure 2 shows, three hosts that each have an internal disk and a SAN connection use up six of the seven IDs, leaving only one SCSI ID available for a storage unit, such as a RAID system. (RAID subsystem disks represent one SCSI ID because of the subsystem cabinet's intelligent controllers. The cabinets can contain several SCSI disks, which they represent as one ID to a host server.) Fast, Wide, and Ultra SCSIs increase the number of SCSI IDs and devices available on a bus to 16, but as speed increases, the maximum distance between devices drops from 25m to 12m, just as it does with Ethernet. SCSI bus extenders and other repeating devices extend distances, but these devices are expensive. Despite SCSI's limitations and problems, it's the least expensive way to provide SANs with multiple-host connectivity.

The High-Fibre Diet
The other SAN connection method, fibre channel, removes many SCSI limitations. To use fibre channel, many IT professionals have the impression that they must string fibre (instead of copper SCSI) between fibre channel devices, but fibre isn't necessary. Fibre merely increases the maximum distance of fibre channel SANs from 25m to 10km.

Fibre channel mimics the SCSI command set but uses a different communication method that handily bypasses SCSI limitations. SAN fibre channel connections come in three varieties: point-to-point, arbitrated loop, and switched topologies. A point-to-point connection has one hard disk connected to a host. An arbitrated loop is similar to a SCSI, but it's faster than SCSI, has a 127-node maximum, and is much easier to cable. Switched topologies are similar to switched Ethernet and Token-Ring networking and have more than 16 million possible nodes.

The most commonly deployed fibre channel SANs use arbitrated loops for a couple of reasons. Fibre Channel Arbitrated Loops (FC-ALs) cost about the same as Ultra 2 SCSI in terms of HBA and hard disk expense. And FC-ALs provide up to 127 nodes and a 10km length, so you can easily scale FC-ALs by adding workstations, servers, or storage devices. FC-AL SANs typically use hubs for bus stability.