InfiniBand: Where It's Been, Where It's Going - Industry Trend or Event

Computer Technology Review, Feb, 2001 by Christine Chudnow

IDC defines the InfiniBand market as the development and installation of InfiniBand technology in servers, switches and other devices. The first InfiniBand products are due to ship by mid-2001, primarily to beta test sites. 2002 should see an increase in InfiniBand products in data center environments, and by 2003 analysts are forecasting that 35% of shipped servers will be InfiniBand-enabled, with that percentage reaching 80% in 2004. That represents about four million servers out of six million in a market worth about $2 billion. And that, folks, is a lot of hardware.

InfiniBand's developers are engineering the channel-based, switched fabric as an interconnect architecture for servers that provides great scalability, availability and performance along with a common specification. It provides an intra-system interconnect that can link together numerous systems, including storage area networks, individual systems, servers and clusters and backend databases. Developers are defining InfiniBand products around the rapid growth of the Internet and its convergence of data and telecommunications, including voice, data, video and storage. In addition, it is an important technology for server clustering as well as I/O for remote storage and networking.

The architecture will provide a range of performance for entry-level servers through high-end data-center machines using interoperable links with aggregate bandwidths of 500MB, 2GB, and 6GB with a 2.5 Gbit wire signaling rate. It supports both copper and fiber-optic cabling.

In spite of these numbers, no one is seriously suggesting that InfiniBand will immediately replace legacy I/O, leave alone more recent I/O approaches such as PCI-X. InfiniBand is a sophisticated and powerful architecture, but it requires a new investment in hardware and software.

InfiniBand will shine in enterprise environments. Current clustering technology is limited in both bridging and design, which often is not meant to support clustered environments. For example, Ethernet-based clusters do not prioritize traffic across complex distributed networks, and developers must work their clustering software around these limitations. InfiniBand can create high-speed system clusters and SANs, both of which require fast communications between nodes and a solid infrastructure for data movement. However, these advantages require building hardware, system software, and management environments to control complex systems.

Benefits And Challenges

The InfiniBand Trade Association (ITA) has presented an InfiniBand architecture value proposition, but according to consultant Rawn Shah, there are challenges as well. A top one is the high cost of implementing new hardware and software in a network. As time goes on and manufacturing ramps up the cost will fall, but through the next two years InfiniBand may be an expensive proposition. It can also be a complicated proposition, for it requires its own silicon chips, its own hardware devices and its own software management programs. Even if the customer expands slowly, the installation may take some time to stabilize. However, by early 2003 InfiniBand costs will probably drop. This will make it a natural for MAN and WAN installations, as InfiniBand devices use Ipv6 for remote connections.

Another challenge is that developers must implement InfiniBand's system software in the host platform's operating systems, and include embedded logic within the fabric's enclosures and devices. Switches and routers may be InfiniBand-specific, or they may be upgraded from existing devices. This new software could cause performance problems on these devices due to both InfiniBand's and Ipv6's added processing needs.

Shah also cites only minor support in the ITA's 1.0 specification for embedded devices, though many industry experts foresee a plethora of such devices and application appliances working over networks. Many of these are consumer devices that do not fit InfiniBand's profile, but a number of high-end consumer and professional electronics equipment do fit the bill. Other high-end devices could also use InfiniBand to interconnect, including industrial equipment, specialized server appliances and professional digital video equipment.

InfiniBand Trade Association

InfiniBand specifications and development are largely centered on the Association's activities. Founded in 1999, the ITA has seven steering companies: Compaq, Dell, Hewlett-Packard, IBM, Intel, Microsoft and Sun. These companies oversee the overall direction of the ITA's activities and elects Sponsoring Members, including Agilent, Brocade, EMC, 3Com, Adaptec, Cisco Systems, Fujitsu-Siemens, Hitachi, Lucent Technologies, NEC and Nortel. Additional 200 companies complete the ITA. Special workgroups consist of Marketing, Technical, Applications, ElectroMechanical, Link, Management, Software, and Compliance and Interoperability.

The ITA began with three main objectives. First was to develop a specification for a channel based, switched fabric that could scale to meet the growing demands of data centers, offered flexible connectivity, was OS-independent and could interoperate from the entry level to the enterprise. Second, the architecture was to draw on existing technology in switched fabric implementations, in order to ensure a smooth transition from legacy I/O. The third objective was to balance the need to drive the technology forward quickly, while at the same time involve the industry throughout the development process.