Xtending PCI - Technology Information

ENT, May 24, 2000 by Mark McFadden

Lightning fast processors and quickly rotating disk drives ought to make for superior servers -- and often do. But no matter how fast they are, performance will still suffer if the connection between them remains a bottleneck. The venerable PCI interface is a potential logjam for advanced storage.

As new devices get attached to servers, the strain on the underlying bus becomes obvious. Soon, new storage interfaces, the Ultra36O bus and the 2-GBps Fibre Channel bus, will start to tie together high-performance storage. Attaching those interfaces to the ancient PCI architecture could quickly overwhelm the once enormous 533 MBps throughput that a PCI system delivers.

Computer I/O is headed toward a new, channel-based, 6-GBps architecture called Infiniband. Until then, can anything be done to extend the lifetime of PCI for new, high-performance storage requirements?

The PCI Steering Committee, the group charged with managing and updating the PCI standard, responded with PCI-X, a new bus interconnect strategy. Originally developed by IBM, Hewlett-Packard, and Compaq, PCI-X increases the speed of the underlying connection bus and the number of available high-speed slots.

Under the older PCI specification, the server ran at 33 MHz, and one slot could be boosted to twice that speed. With the PCI-X specification, one slot can run at 133 MHz, and the rest can run at 66 MHz or 100 MHz -- giving the entire system a potential throughput in excess of 1 GBps.

It might seem as if performance is the most important feature of PCI-X, but for storage system architects two other issues loom large: backward compatibility and vendor acceptance.

PCI-X slots are thoughtfully designed to be backward compatible with all PCI cards, including 133-MHz and 66-MHz cards. The interfaces will deliver this backward compatibility by selecting the lowest common denominator of the PCI-X bus and the attached card. One of the advantages of providing backward compatibility is that it simplifies the process of developing drivers for new SCSI, Fibre Channel, or other adapters.

More complicated is the issue of industry acceptance. At a recent PCI-X Forum, IBM's director of server architecture and technology gave a spirited talk called "Processor MHz Gets the Glory[ldots] But I/O Bandwidth Wins the Game." At the forum, familiar names like Adaptec, Mylex, and Qlogic all committed to PCI-X-based products this year. IBM and Compaq are also committed to supporting PCI-X in servers later this year.

Still, PCI-X has been slow to reach critical mass because it requires significant reworking of both the electronics and architecture of motherboards and adapters. For many of us, the additional speed will be neither necessary nor cost-effective. PCI-X, therefore, is unlikely to show up in workstations or low-end machines because of the increased cost of PCI-X motherboards.

PCI-X only makes sense in situations where data can overwhelm a server's ability to handle it. Even with the improvement in bandwidth, new bus architectures -- primarily for high-speed storage, networking, and graphics -- will eventually overwhelm even the new PCI-X standard. This will require an even bigger paradigm shift away from the slot and memory-mapped strategy of the past and toward the channel-based, "computing brick" architecture of Infiniband. Even IBM admits that PCI-X has a limited shelf life.

Having a technology with such a short adoption window may lead to unexpected results. It's happened before. Veterans of the storage marketplace remember that the IEEE 1394 standard, once thought to be the holy grail for disk drive interfaces, eventually transformed itself into a market leading connection strategy for consumer electronics devices. It seems to me that PCI-X could also become a technology that solves an unexpected -- and different -- problem.

Storage architects that need very high performance interfaces for SCSI, RAID, or Fibre Channel solutions should investigate the PCI-X products as they emerge later this year. I'm convinced, however, that PCI-X is a short-term fix for a significant problem. Organizations with significant needs are going to need an architecture that moves beyond the new standard. Instead of a better bottleneck, we eventually will need a whole new approach.

Mark McFadden is a consultant and is communications director for the Commercial Internet eXchange (Washington).