12 questions for Phil Schwan

Computer Technology Review, Dec, 2004

Phil Schwan, CEO of Cluster File Systems (CFS) and developer of the Lustre Open Source Cluster File System spoke with CTR about the state of his company and the adoption and positioning of the Lustre File System--a high-performance cluster file system developed for managing very large volumes, with very high bandwidths. Many of the world's largest supercomputers have been using Lustre in production for more than a year.

What is the DNA behind Lustre?

The Lustre project was started in 1999 by CFS's founder, Dr. Peter Braam, when we were together at Carnegie-Mellon University. Peter began working on an answer to the Department of Energy's Scalable/Global/Secure file system challenge. The goal was to solve the next ten years of cluster file system problems, for systems of tens of thousands of nodes and many hundreds of petabytes.

Lustre's architecture was developed to address these massive storage needs with the expectation that if we could solve the hardest problems of the biggest clusters, then scaling down to the more typical systems would be possible. Working with the early-adopter customers and partners, we've had systems in production for about a year and a half, handling HPC workloads for government and private industries.

Would you give us an overview of the Lustre architecture?

There are three types of nodes in a Lustre installation. Metadata servers manage the namespace. The names of the files are stored here, but the actual file data is split up and distributed over one or more Object Storage Servers. Today's largest Lustre installations can have several hundred of these commodity servers. You can scale the bandwidth and capacity of your clusters very linearly by adding more, because they operate completely in parallel. Many shared file systems have a single pool of disk shared directly between the entire cluster. That's an approach that really doesn't scale, so we give each server its own pool of disk to manage on its own.

The third type of node is the client node. These speak a Lustre object protocol to the metadata and object servers, and provide a standard file system interface to applications. Today, those native clients run only on Linux, but you could use Samba or NFS to re-export, so that you have access to your Lustre file system from whatever other OS you may happen to have.

How is Lustre sold and distributed today and what is the licensing model?

We saw that there was no other really good open source cluster file system for the Linux community, so Cluster File Systems was founded to create, distribute, and support Lustre as an open source product. Today, CFS distributes Lustre in two ways. For the first year of any new release of Lustre, like our recent Lustre 1.4 release, the software is available to our paying customers and our commercial partners under the GNU General Public License (GPL). After a year, often sooner, we make that new version available on the Web to anyone, under the GPL.

We stay in business by licensing and supporting the Lustre software directly and through partners such HP, Cray, Sun, Linux Networx, Verari and similar companies. Directly, we've fostered some exceptional relationships around very high-end deployments for which there are very few choices other than Lustre. For example, the largest supercomputer in the world. Lawrence Livermore's IBM-based BlueGene/L system, will use Lustre. We're also working very closely with Cray on the new Red Storm supercomputer, which will be installed in 2005 at Sandia National Laboratories.

HPC was clearly the birthplace of Lustre, but how far off are you from being adopted in commercial enterprise environments?

We already have some strong acceptance within the industries that resemble our engineering and scientific customers. These are industries such as oil and gas, pharmaceuticals, digital media, manufacturing and automotive, and I expect this part of our business to continue growing rapidly. Transaction processing environments, such as those found on Wall Street or in the insurance industry, are still a few years off, as they tend to be very conservative and require additional features. While these are on our roadmap, they're not finished yet.

Your partners like HP, Sun, and others sell in both the HPC and commercial markets. Can we assume that while they are pursuing the engineering and scientific markets today, they will move more into the enterprise with Lustre at some point downstream?

Yes, that's true of many of our partners. It doesn't do them any good to push Lustre where it's not yet ready. We are, however, having many discussions with partners and enterprise customers about where they want Lustre to be in the future.

How far down the development path have you been able to go? How far along is Lustre today?

Lustre is ready for prime time. It is extremely usable in production for a large number of applications on both large and small clusters. Organizations in many industries are using it very successfully with the help of CFS and our partners. We've had the fortune of being able to develop a very healthy, respectable customer base without the need of a great marketing effort. While there is still a lot more to accomplish, and our roadmap is long, Lustre is a reality today--and our customers are proving that.