Storage Over IP: Death Of The SAN, Or New Beginning? - Industry Trend or Event

Computer Technology Review, April, 2001 by Mike Flannery

In the movie The Perfect Storm, two major weather disturbances collide off the Atlantic Coast, creating a storm of unprecedented proportion. Watching the fishermen succumb to the raging elements made me glad I was safely home and dry.

Weather on the SAN has been choppy at best, with sudden squalls and a constant undertow by those vendors who believe in controlling the marketplace and getting standards neatly aligned behind them. Still, a strong prevailing wind constantly blows in the direction of more open and more productive technology: LAN-free backup becoming server-less then NDMP-standardized, virtualization increasing capacity utilization and vendor-neutral capacity assignment. Calmer seas are in sight--or were.

Another weather system has been expanding over in networking. The Ethernet now reaches gigabit speeds, with 10 gigabits, then Infiniband looming. Network speeds and connectivity surpass internal bus speeds, at least nominally.

The collision of gigabit internetworking and the fiber-SAN will be happening this year, with a flurry of vendor announcements already started or planned. This has the potential to whip up a "Perfect Storm" in storage networks, as past investments and future blueprints get reconsidered: should you access your storage over Ethernet or Fibre Channel? Is the SAN going back to square one? Will the iSCSI gale sink your storage infrastructure, taking your job down in the process?

These coming months will test the seamanship of many a CIO. Granted that this is a sea change, it can be turned to your advantage.

First The Basics

We can look forward to widespread debate and ample confusion about protocols: iSCSI, SoIP, and other varieties will vie for attention. Let us take some time to unpack them.

The most basic level is a block I/O request. This request is issued by the file system or DBMS based on an application need for a file segment. The accepted low-level protocol between the server and its storage is SCSI, a well-known protocol especially adapted to moving large blocks of data. However, SCSI comes with a very weak transport layer. This layer allows addressing only of seven or fifteen devices, depending on the SCSI revision level. The solution is to wrap the SCSI messages in a routable, widely addressable transport layer. The issue: Which transport layer works best, Fibre Channel or IP?

Fibre Channel 101

Let us first dispel a common misconception. Fibre Channel is not synonymous with the use of fiber-optic. You can use multimode or monomode fiber-optic, but also coaxial copper cable. Fibre Channel, as codified by its standard body, the Fibre Channel Industry Association (FCIA) is a transport protocol, or rather two protocols:

Fibre Channel Arbitrated Loops (FC/AL) allow connection of up to 128 devices on a single 1Mbps fibre loop. Addressability is much better than SCSI's fifteen-device limit, but still falls short of any major data center's needs. More importantly, the shared bandwidth discourages any sizeable configuration. For this reason, FC/AL is disappearing as a valid contender.

Switched fabrics make use of the full addressability of the Fibre Channel protocol, enabling connectivity of 224 or 16 million devices in the same network. Moreover, it is designed to allow each device the full benefit of the megabit-per-second pipe.

Broad connectivity and scalable bandwidth have made the success of the Fibre Channel protocol. An additional strength is the efficiency with which it uses the bandwidth, allowing up to 90% utilization or 900Kbps effective data throughput.

Although very economical, Fibre Channel also carries a number of weaknesses, some academic, others more serious. The weakness of its security is not a practical deterrent as most connections occur over very short distances in a physically secure data center, and intrusion into an input/output stream would be very challenging and impractical to exploit. The greater challenge is the physical limitation to a very short distance: ten to thirty kilometers, further limited by FCC regulations preventing transmission beyond private boundaries.

The Ethernet Cometh

What is commonly called the Ethernet protocol (more specifically IEEE 802.3) uses broadcasting within a given domain, and routing across domains. It is used in Internet protocol (IP). Above the IP layer, a choice of two main transport protocols (ISO Layer 4) has been developed: the basic Universal Datagram Protocol (UDP) and the more commonly used and reliable Transmission-Control Protocol (TCP).

Ethernet-based protocols have long reigned over the LAN, and more recently over Wide Area Networks (WANs), for any peer-to-peer computer communications. Ethernet products serve a huge market, and contain most of the bells and whistles that are missing in the Fiber Channel world. Security, such as PGP (Pretty Good Privacy) public key encryption, and elaborate management tools are available especially for TCP/IP.

Most importantly, Ethernet now offers nominal bandwidth at the same gigabit speed as Fibre Channel, with more aggressive plans to increase it over the next few years.