Streamline data to support the ILM infrastructure

Computer Technology Review, Jan, 2005 by Todd Viegut

Information Lifecycle Management (ILM) has become increasingly important as new regulations continue to mandate retention of more and more data for longer periods of time. But that's not the only reason that efficient ILM infrastructure components are needed.

The "online, all the time" business environment stemming from the Internet and the global business market--where it's always "dbh" somewhere--means that 24X7 data access now requires massive of amounts of data be available and immediately accessible--and not just fixed-content, but "live" data that is actively being accessed.

The problem then becomes, how can you afford to keep that much data online-with its need for massive amounts of primary and archival storage, enormous network capacity to move even quiescent data around and very high speed access to "live" data.

And finally, how do you backup--or even more difficult--restore these massive amounts of data, should that become necessary?

In The Beginning

To gain a clear perspective, let's go back to the roots of today's ILM movement, review current approaches and discuss what's still needed to make today's products ILM-ready.

Managing information from "cradle to grave" is not a new concept--it's been around for over a quarter of a century. HSM (hierarchical storage management) and SMS (systems managed storage) software have addressed the issue of managing the file-level storage and migration of data. They moved data from primary to secondary storage (i.e., disk to tape), based primarily on the age of the files.

Evolving data management needs over the ensuing decades brought new solutions (or at least new acronyms) in the form of SRM. DLM and ILM.

During the 1990s, as shared storage and storage networks began to supplant much direct-attached storage, SRM (storage resource management) tools became popular. SRM, like HSM, can function as a data management tool, but it deals with network-based (NAS and SAN) storage rather than traditional DAS with network-based backup.

SRM provides storage provisioning and information about disk utilization and access. Depending on the implementation, it may be no more than the moral equivalent of a network management system (NMS) for storage. Or SRM may support data storage, backup/restore, as well as tools for scaling the storage infrastructure and, in some cases, virtualization capabilities.

Managing Data Over Time

Data Lifecycle Management (DLM) tools automate the process of dealing with data--how it's handled, where it resides, when it's moved, and how long it's retained--based on policies determined by the implementing enterprise.

DLM tools may be used to manage the tiered storage environment. Generally speaking, the newest (and most frequently accessed) data is stored closer to the point of most frequent access and on the most expensive storage equipment. Older or otherwise less important data is stored on less expensive, slower and more remote storage media.

DLM vs. ILM

DLM and ILM are sometimes used interchangeably, but most storage professionals differentiate them based on the way they handle the data. Data management tools are an important building block in an ILM infrastructure.

DLM products generally deal with data from outside the file, looking at externally visible file attributes, whereas ILM deals with the information contained within the file. ILM is able to access and use the content both in the conduct of business and to make decisions about the value of the data.

Classification, automation and evaluation of data are the critical tasks. While other tools, perhaps SRM, are used to classify the data, DLM deals with day-to-day data management and automates data movement. The ILM process goes the next step by augmenting these functions with policy-based management of the information itself, according to its value to the enterprise.

Why ILM?

Because we need a better way to manage the massive amounts of data that enterprises generate, work with on a daily basis and need to retain for business and legal reasons. This need goes beyond the solutions developed for managing data and the storage media where it resides--straining the capabilities of traditional SRM, HSM and DLM products.

What is ILM? Well, one thing that it's not is a product or even a product suite. It's better described as a process that requires both business planning and technology infrastructures to support its realization.

What exactly must the technology infrastructure provide? A means for efficiently storing, continuously protecting and efficiently moving information, while managing content and restoring the data, as needed, throughout its life-cycle.

On the technology infrastructure side, ILM product building blocks will span tiered storage solutions, content storage, continuous data protection and DLM tools. But for any of them to become effective components of the ILM technology foundation, they'll need to master the data explosion by efficiently reducing the unnecessary data being stored every day.

We can call this data "unnecessary," because it's redundant--and we're not talking about backup copies, which then, in turn, replicate the redundancy of the original. That goes a long way toward explaining the "data explosion" that everyone is talking about.