Putting information storage in its place: we're generating digital data at a record clip. Is there a digital closet big enough to store it all? - Tech Trends from Deloitte & Touche

Computer Technology Review, Jan, 2002 by Naresh Lakhanpal

This article is the first in a continuing series from Deloitte & Touche.

From digital photos to e-commerce transactions, our reliance on digitized images, numbers, and words is straining the data storage and management capacity of home PCs and enterprise servers alike. In 1999 alone, we generated 1.5 exabytes (equal to 1 billion gigabytes) of information. According to University of California, Berkeley researchers, we can expect that amount to double every year for the foreseeable future.

This dramatic jump in material to be stored doesn't mean that as a society we're becoming more industrious. Rather, it represents a change in the way we record information, according to professor Peter Lyman, coauthor of a UC Berkeley study released last year that analyzed the world's digital and non-digital data volume. "A lot of that data is literally data--scientific data, for example, as opposed to text or multimedia," he says. "Many things that have been invisible are now being monitored by instruments. For example, economic transactions at the point of sale go into databases now, weather satellites collect data, and inventory changes are monitored by data trails."

While bandwidth-clogging rich media, such as photographs and video, may not represent a huge portion of today's hard disk content, Forrester Research predicts that by 2005, 57 percent of U.S. households will use some form of personal rich media once a month, and video e-mail will replace text messages as the primary means of online communications.

Once we've looked at this information, it has to be either deleted from our hard drives or archived. Much of it is stored, a fact that delivers both an opportunity and a challenge to the information storage industry.

Consolidation Ahead

In the enterprise, digitized information has traditionally been archived on hard disks that are attached to file servers. As PC storage capabilities improve, people are keeping more data on their own PCs. Although disk storage costs have been dropping steadily--today it costs approximately 30 cents per megabyte, half of what it was two years ago and twice as much as it is expected to cost in 2003--hard disk capacity is being tested.

But the real strain isn't a hardware or software problem, it is a manpower problem. According to Don Swatik, vice president, global alliances for EMC, a leading storage vendor, "If you go back five-plus years, the cost of physical storage was 80 percent of the cost of owning the storage. The cost of people to run it, floor space, the power to run it, etc., was maybe 20 percent. Today, it's the reverse. Each time you add another server you have to add more resources to manage the incremental storage. You have to train new storage administrators and get them on board quick enough to keep up with your company's growth demands."

The remedy? Swatik and other industry experts predict that within the next three years, the racks of servers at corporate data centers will be replaced by information plants--storehouses that hold not only archived data, but also data that needs to be accessed on a daily basis. "At the enterprise network level, you're going to have slowdowns in information movement. You have so many touch points with your customer base--your CRM systems, for example--that unless you're taking that information and putting it into storage areas, you won't be able to react to your customers' needs," says Jeff Balentine, a partner at Deloitte & Touche.

Indeed, the hallmark of the information storage plants will be their ability to give users quick access to data. "From the user's standpoint, it will feel like local storage. You'll have the same response time and the same capabilities," says Swatik, who adds that even home computer users will be linked to information storage plants. The information plants now on the drawing board would house storage devices employing technologies that are still being perfected by the storage industry, like data placement optimization. Using data placement optimization schemes, new data--which is typically referenced more frequently than older data--is placed on the disk drive in a more easily accessible location.

Information storage plants won't win everyone over without a hitch. Outsourcing storage brings with it concerns about privacy and security. Advanced security technologies like biometric interfaces, which verify the user's identity by reading his or her retina or fingerprint, might help storage plants enjoy widespread adoption.

Network bandwidth isn't expected to be a roadblock for large enterprises as long as plans to add optic-fiber relays stay on course. But for smaller companies and home offices, it could be a problem. "I don't believe you'll drive fiber to the home or use wireless to get to the home. It's too expensive to put fiber in the ground to the house, and wireless has reliability issues. Adequate spectrum isn't available to the carriers," says Balentine. "We'll use the existing infrastructure and see new compression technologies come into the marketplace. Eventually that will take data from point A to point B."