Pervasive data volume reduction: a high-value alternative to aging tape-based backup technologies; A paradigm shift in backup technology can cut lifecycle costs by 50% or more - Storage Networking

Computer Technology Review, Dec, 2003 by Janae Lee

Open systems tape backup processes, originally developed back in the late eighties, were designed to make optimum use of low-cost serial media; data and process efficiency took a second priority to the goal of keeping steady, continuous streams of data flowing to the tape. As a result, the backup process, even with modern enhancements, stores and manages ten times the amount of primary data being protected just in active (non-shelved) backup data volumes. While this data-intensive process worked well when data volumes were relatively small, as volumes continue to increase in the enterprise, this old-style backup process simply cannot keep up.

The problem has become particularly acute now that collaboration has become a favored business practice. With a large and growing number of staff members sharing information by copying it, communicating it, and storing their own versions of it, data volumes have truly exploded. The result: tape backup systems become increasingly overloaded, even as the number of tapes required to house archives soars along with associated backup costs. At the same time, backup windows need to be extended, forcing companies to choose between comprehensive data protection, and decreased access to network resources while backups are running during business hours.

And the situation is likely to get worse as new applications that increase data volumes become increasingly popular. As an example, consider that content management applications, fast becoming a business-critical need for companies striving to deal with exploding content volumes, are often based on XML technology--a technology whose metadata typically expands data by a factor of two. In fact, in an Avamar survey of 100 companies, there are on average at least two copies of all files stored on disk, and when data replication associated with backup processes is factored in, this number is multiplied by anywhere from 10 to 100. In other words, every single file created in the enterprise can result in hundreds of copies of that file stored on backup tape.

While many companies have worked to control the magnitude of their tape storage issue by judicious retention policy management, the value of these programs is fast becoming moot because of regulatory requirements that virtually force companies to save all data forever, just to protect against possible downstream litigation or compliance issues. Against this backdrop, tape backup vendors are working furiously to come up with fixes. But all suffer from the same basic flaw: they're based on outdated serial access technology and rely on high-volume data management tapes that are thinner than a human hair and prone to distortion and, therefore, data corruption. And, they're expensive.

Instead of band-aids, what the backup industry really needs is a totally different approach, a re-engineering similar in scope to the great ERP transformation. Instead of tape, the industry cries out for a disk-based alternative. Instead of complex and time-consuming retention management practices and data compression algorithms, companies crave cost-effective, automated techniques for reducing data volumes. What they don't need are systems that increase overhead, require large investments in new software applications and massive amounts of disk space, or that demand extensive manual intervention to laboriously identify and mark every piece of data to indicate whether it should be archived forever or not. A massive content management system for all structured and unstructured data across the entire enterprise is simply not a cost-effective or practical solution--and is not likely to be so for the foreseeable future.

Ironically, re-engineering storage actually requires far fewer infrastructure changes, far less cost, and creates far less network impact than other options currently available and proposed--if the re-engineering effort is based on a pervasive data volume reduction (PDVR) paradigm that dramatically reduces data volumes. Equally critical is that the data reduction is applied at the data source, not downstream, on volumes of aggregated data. After all, destroying a snowball is far easier at the top of a mountain, then it is after it rolls down to the base.

With PDVR at the source, each piece of new data is automatically tagged with an identifier, these tags efficiently indexed, and the data backed up. Then, whenever data with these same tags are seen in the future, the backup system immediately knows not to back this data up. The result: the volume of data that has to be backed up in order to completely protect resources and ensure full recovery and business survivability can be reduced by a factor of 10 to 100.

While some hardware-centric data storage systems already rely on incremental change tracking techniques that reduce data volumes, these systems only work on single files, or file systems, as they change over time--and only if the system itself contains unique knowledge of how those files and file systems were initially constructed. What's different about pervasive data volume reduction is that it is pervasive: the technique can be applied heterogeneously, across all files, file systems, data types, and servers--even across a global enterprise.