The challenges of testing SATA and SAS: part 3: testing wide SAS

Computer Technology Review, March, 2004 by Dale Smith

One of the first obstacles to testing SAS devices is getting an analyzer connected into the point-to-point bus. The similarities between SAS and FC have made it possible to borrow from Fibre Channel techniques, but making sure the analyzer can adapt may still require special cables.

SAS specifies four different types of cables and connectors. There are single-link, dual-link and four-link cables designed for use inside an enclosure. There is also a four-link connector specified for external use, such as connecting a server to an external RAID box. Since SAS analyzers today use the single-link connector, connecting an analyzer to a dual-link or four-link bus requires special adapter cables. Fortunately, these are the same cables used in standard SAS storage applications so they are starting to become available from cable manufacturers.

[FIGURE 1 OMITTED]

The single-link connector (Fig. 1) is the same as the SATA connector and is manufactured by many different companies. The dual-link connector (Fig. 2) is new and physically similar to the SATA/SAS single-link connector, but adds additional pins on the unused opposite side, providing for a redundant set of connections between the drive and host in case the primary physical connection goes bad.

SCSI marketing departments have always maintained that a significant differentiator between ATA and SCSI drives is reliability, and the redundancy of the new dual-link connector for SAS is further proof of this difference going forward. The internal four-link connector (Fig. 3) is just like a very-wide SATA connector. RAID applications inside the box will use the internal four-link connector. The external four-link connector is based on the same MicroGiga connector used by 4x Infiniband, but uses jackscrews instead of the quick-release latches that Infiniband cables use. External four-link SAS applications include expanders and RAID boxes.

Testing "What If" Scenarios

SAS provides an unusual level of flexibility when it comes to some of its handshaking using primitives. Table 1 shows some areas where SAS state machines are more flexible than their FC equivalents.

The flexibility of SAS primitives makes it difficult to verify that a SAS device is robust enough to support the primitives, no matter when they occur. Typically, those designing SAS silicon will make or buy the simulation code they need to verify the RTL (Register Transfer Level) code, ensuring that the different scenarios have been verified.

[FIGURE 2 OMITTED]

One company that develops and sells SAS protocol verification tools is Perfectus (Santa Clara, CA). Their verification engine generates the packets and primitives in software to test all the possible timing and sequence possibilities. The Perfectus SAS verification tests are also translated to the Data Transit PacketMaker hardware for verification of the final device--not just a simulation. Figure 4 shows the sequence for verifying RTL prior to finalizing the hardware, and later verifying the final hardware with the same tests.

[FIGURE 3 OMITTED]

Perfectus tests are categorized according to the different protocol layers of PHY, Link, Transport, and Command. They give the developer the ability to adjust a variety of test 'knobs' which vary the timings and occurrences of SAS Errors, OOB, Primitives, and Frames within legal specifications, or beyond specifications if the user chooses. The test case generation is coupled with a protocol checker, which analyzes a device's response to the generated traffic to determine if the device followed protocol. The protocol checker is implemented in software and verifies both the simulated results of the pre-silicon design, and the actual results of the final hardware as received by the PacketMaker.

[FIGURE 4 OMITTED]

Table 2 shows the tests performed in each protocol category by the combination of the Perfectus verification engine and the Data Transit PacketMaker.

[FIGURE 5 OMITTED]

Obstacles to Analyzing Wide SAS

Wide SAS uses multiple links in order to handle multiple commands simultaneously. In this multi-link environment, each link carries the data for a different command. The links cannot be aggregated to achieve higher bandwidth for a single command, but multi-lane performance gains can be achieved at the application layer when a host adapter breaks a data request into multiple commands, using a different link for each command. Thus, the data can be sent down multiple lanes simultaneously even though commands can only be issued on one lane at a time.

Certain features of wide SAS make it impossible to analyze each link as a distinct SAS bus using a single-link SAS analyzer. On a wide SAS bus, a command will be issued on whichever link is available, rather than waiting for a specific link, which would add more latency.

One of the difficulties of analyzing commands on a wide SAS bus is that the connection on one link can be closed before the data transfer for that command is complete, such as when the drive empties its buffer and closes the connection in order to free-up the link while it refills its buffer.