On the performance and use of dense servers

IBM Journal of Research and Development, Sep-Nov 2003 by Felter, Wesley M, Keller, Tom W, Kistler, Michael D, Lefurgy, Charles, Et al

Using a single disk image of the system software that runs on the blades effectively eliminates two problems that plague disk-based, clustered server systems-the difficulty of ensuring that all of the nodes in the cluster run the same software and the time and effort required to perform a separate software install on each node. However, there are environments in which different blades may require different versions of the system software; this can occur, for example, because an installation has incrementally upgraded the blades from an older to a newer level of a set of software packages. Although there are good reasons to do such upgrades one enclosure or one MetaServer at a time, it is quite easy to support multiple system images with a single MetaServer. The name of the /dsa directory that contains the system images for the blades is only a convention; by changing the DHCP configuration file and the file that maps individual blades to their associated /var file systems, a system administrator may have different blades use different system images. Once this is done, the administrator can make any alterations required to the request distribution mechanism used by the cluster and restart any blades that arc to run the new software; the major cost of this is the disk space consumed by the additional system images.

Swapping

Much of our work with Linux-DSA has been done without enabling swapping by running entirely within memory. However, this is impractical for certain workloads. Normally, if we do use swapping with a Linux-DSA system, we set up the MetaServer to act as the swap server as well by running the EBD server on it with a separate disk partition for the swap space of each blade. However, because of the limited capacity of the disk on the Ziatech board, we used a separate Linux-based swap server for our performance evaluation.

Blade monitoring

Although Linux-DSA supports the standard sar and sadc programs, it also provides a special performance-monitoring feature that collects performance and utilization data from the blade and sends it, using user data protocol (UDP) packets, to a system-monitoring application. From an architectural perspective, the system-monitoring application should run on the MetaServer for the blades, but in practice we run it on a different machine because the MetaServer is too limited to handle this function in addition to its other responsibilities. This mechanism acts as a distributed replacement for the sar and sadc and allows the system-monitoring application to collect information about the individual blades and combine it into an overall view of the resource utilization of the blades as a whole. The mechanism is also used to provide the monitoring data used by the request-distribution mechanism described in the section on power-aware request distribution.

Other system software enhancements

Beyond Linux-DSA and the MetaServer infrastructure that it requires, we have also implemented the following:

* A mechanism for distributing requests coming from the network across the blades in such a way as to reduce overall power consumption.