Virtual storage and real confusion: a big disconnect between what vendors offer and what users want - Industry Overview

Computer Technology Review, Nov, 2002 by Christine Taylor Chudnow

In its broadest sense, storage virtualization is nothing new. It has deep roots in computer virtual memory and networking technologies like host-based logical volume managers. But with the growth in storage networking, virtualization became quite a bit more complicated. It may be limited to single servers and arrays or expanded to domains and even full SANs. It may affect disk-based arrays or tape libraries, and virtualization engines can be based within the SAN or outside of it. Virtualization also differs according to file or block-level virtualization procedures: Although all computer data rests on blocks, virtualizing files adds different management elements.

Sorting the Blocks

All computer data is basically made up of blocks, units of information that move quickly between servers and their storage. Many applications make sense out of blocks, creating files with added definitions and file headers, which in turn requires more sophisticated processing at the server and storage levels. Virtualizing file-based data requires file systems that can interpret the packets and store file metadata, while virtualizing block-based data works closer to the raw disk levels. Block-level storage virtualization distributes blocks of data to virtual storage pools across multiple storage devices, which helps to manage space and control devices at the hardware level. File-level virtualization manages the stored objects--the files--which may be scattered around different physical storage devices.

SANs are ideal for sharing block data over open systems networks such as Fibre Channel or iSCSI. Block-level virtualization generally refers to aggregating storage devices in a storage area network, increasing storage space to applications while increasing flexibility and ease of management. Block-level virtualization technologies often include data-protection technologies such as replication, snapshots, and local and remote mirroring. Since the SAN offloads these computer-intensive operations from the local network, this results in better storage consolidation and improved storage resource management (SRM). Physically consolidating storage helps control over-provisioning and improves ROI, while simplified SRM allows storage administrators to efficiently manage shared block storage as a single volume. By separating storage management and allocation from the physical hardware and specific application servers, storage administrators can manage and control escalating storage costs.

Although SANs are by nature shared storage environments, virtualization does not automatically follow. Even in a storage area network, there are technical limitations to servers sharing their storage devices over Fibre Channel or IP. Servers still communicate with their storage devices using SCSI protocols and must meet certain SCSI requirements. For example, servers do not want to be confronted by a large pool of amorphous storage. They expect to see specific targets with addresses containing a target ID and logical unit number (LUN). In addition, some hosts will grab any LUN they can see, regardless of what the storage administrator meant to do. Such hosts may also compete for the same device, unwittingly over-writing each other's data. Because of this, storage administrators commonly zone their Fibre Channel switches to block other servers from seeing the same devices. A related approach is called LUN masking, which renders certain LUNs invisible to other servers even in the same zone. This keeps the stora ge intact and secure, but whittles away at the SAN's reason for existence: the ability to share storage among application servers.

Virtualization

To get around these limitations, block virtualization presents virtual layers that appear between the servers and physical storage devices. Actual blocks may be stored across different storage devices, while the storage administrators create virtual devices by virtually partitioning a single disk, or aggregating multiple disks to widen the storage pool. The servers no longer see (and try to grab) specific physical targets, but instead "discover" logical volumes for their exclusive use. The servers send their storage directly to the virtual volumes, happily thinking they are their direct-attached property. In fact, these logical volumes are highly flexible.

For example, Fujitsu Softek's Virtualization application, which is built on a DataCore engine, builds a transparent layer between the application server and storage devices. The virtualized layer shows the application server a set of devices optimized for its needs. Meanwhile the virtualization engine maps the virtual devices to actual physical devices. As is common with these types of virtualization schemes, the engine also uses advanced caching, local and remote mirroring, and snapshot capabilities to optimize and protect the virtualized storage.

File-level virtualization requires global distributed file systems, or what passes for global file systems: proxy-like file systems that push data through a centralized server. These file systems process file requests from different operating systems and translate them into common commands for the target storage device. File virtualization (which can work in DAS and NAS as well as SAN), provides a virtual file system layer between files and block level storage. This virtual file system allows the administrator to manage files on virtual volumes, while in fact the files are scattered across different physical devices. Note that simply using a distributed file system does not equal virtualization--for example, NAS appliances and clusters often use such file systems to allow different applications or operating systems to access the same files. But if the global file system additionally allows the storage administrator to create and administer a storage pool, it is a virtualization technology.