Switch-fabric requirements for next generation storage directors - Storage Networking

Computer Technology Review, May, 2003 by Ofer Iny

Currently, storage equipment vendors are faced with a problem: products of ever-increasing complexity requiring a long and expensive product development cycle. Typical figures are two to three years of development requiring massive teams of R&D and incurring tens to hundreds of millions of U.S. dollars of R&D investment. The end products, however, suffer from a short life span, typically three years and sometimes less. This results in a short period of sale where revenues can be accumulated in order to justify the massive investment in R&D. This forces a cycle of equipment development, introduction to market, and retirement that leaves very little room for profits.

A look at voice networks, a more mature and profitable section of the communications industry, presents a different picture. Voice-products life cycles are seven to twelve years, and sometimes more. The products are a-priory built to scale, which enables the networks to evolve gradually, meeting the requirements to higher port count, higher port rate and more comprehensive services.

Reasoning Behind the Problem

The reason for the problem is that current director architectures are inherently constrained in their growth potential, just like current data-centric architectures. Specifically, they embody "hard" limitations on the number of ports, on the ports rates, and the services that they provide.

If we look deeper at the products' architectures we can discern that often the limiting factor is the switch fabric that lies at the heart of storage directors. Stripping away all the upper level protocol processing, the switch fabric is responsible for the "raw" data switching from the input port to the output port.

The architectural paradigm of most switch fabrics that exist or are being designed today is such that they inherently limit the scope of the products' future growth. As an example, most of the current switch design can accommodate only the "port-rate of the day" and the "port-count of the day." A system design that starts today around an "aggressive" 320 FC 1/2G fabric, will hit the market in early 2007. Even a moderate five-year life cycle would require that system to be attractive in 2012. Will the number of ports be sufficient in 2012? Will the port rate? It is likely, looking at the storage market predicted growth, that neither the number of ports nor the rate indicated today, will be sufficient in 2012.

As a second example, consider the service scheme offered by the fabric. Any design that fulfills the current director service scheme, which is hardwired within the switch-fabric and does not reside only on the interfaces line cards, will force the end-customer to replace this centralized element (e.g., switching card) upon the introduction of line cards that require new scheduling disciplines and inter-mixing of other types of line cards. Even a farsighted equipment vendor, when designing a product to be released in two to three years, can hardly anticipate the environment its product will need in order to operate in six to ten years.

To grasp the implausibility of such foresight, we can examine the evolution of directors. A few years ago, a storage network built with a fabric supporting up to 32 ports of 1Gbps FC ports was considered advanced. Such a fabric supported a single, or at best two levels, service scheme. Today, within less then seven years, the latest directors offer up to 320 of 1Gbps and 2Gbps FC ports and are offering at least a 4level class-based service scheme. There is also talk of a 10Gbps FC ports and convergence with data networks in the future. This evolution spans at least three generations of equipment architecture at the leading equipment vendors.

A new switch-fabric architecture is needed, one that is built to scale and, hence, to last.

Requirements for Switch-Fabric Solution

In order to last, the switch fabric must have the following properties:

Port Count Scalability: Since a switch-fabric solution is the "heart" of directors, it needs to scale in the number of ports it supports. The architecture must enable the director switching systems to start with few ports, grow to tens, hundreds and thousands.

Without port count scalability, the only way to increase the port count without replacing the equipment is to cascade directors with proprietary Inter Switch Links (ISLs). The problems with simply using ISLs are numerous. Networks constructed out of several directors interconnected with ISLs always exhibit blocking behavior that is traffic pattern dependent. Also, cascading directors increase the delay through the network, since more hops may be required for a data packet traversing the network. For storage networks, latency has a big impact on performance, just as bad as blocking. Last, this involves separate devices where each of them has to be maintained, operated and managed separately.

The end user should be able to start with any desired port count and progressively increase the director capacity (as they acquire more customers). Directors should be able to grow beyond a single physical chassis without any impact to performance. In addition, the extension of the system port count should entail merely the insertion of additional line cards and fabric cards without disrupting the service of existing ports (live upgrade).