NEW APPLICATIONS Break Through Storage Boundaries - News Briefs

Computer Technology Review, Oct, 1999 by Fred Moore

It's terra incognita on the Internet

Storage is no longer a specialty, it has become the primary business for most every IT organization. In he past when we thought of large storage repositories, we thought of government agencies and scientific-based businesses. Today, we can think of large storage repositories in terms of any business by looking at the rapidly expanding universe of storage intensive applications. The mass storage market has taken on a much broader scope and includes highly interactive and real-time applications, not just the large archival data repositories of the past decade.

Mass storage today means more than just capacity storage with low access rates; it now includes data demanding all the storage properties such as response time, a high level of concurrent access, bandwidth to transfer large files or blocks, rapid recovery, and high-availability. The technological advances of doubling clock speeds every 18 months coupled with the capacity of storage devices increasing at 60% per year and storage prices falling at 35% or more per year have proven to be the ingredients to create a virtually unlimited demand for storage.

Visionary data storage companies are now looking far into the future to more fully understand the impact of exponential storage growth on IT architectures. Long-range server and storage requirements are being projected by examining new applications that are based on intensive computing that require large amounts of contiguous data movement. Clustered, SMP (Symmetrical Multi-Processor) and NUMA (Non-Uniform Memory Architecture) servers will gain momentum as these applications proliferate. The need for significant increases in effective data-transfer capability will exceed all previous developments in this area, an area that has lagged in improvement relative to storage capacity growth in particular.

Databases now represent an estimated 65-70% of all data on disk subsystems across enterprise, midrange, and distributed computing platforms. As these databases scale beyond l0TB in size, the need for timely information delivery accelerates and data mining applications must be contentdriven, rather than index-driven, requiring fast parallel readout. Application requirements for video-on-demand, HDTV, 3-D Video, video mail, and visual computing are pushing the envelope on server capacity and transfer rate. Multimedia applications using video, voice, and text will drive throughput to the 150MB/sec levels but have relatively low I/O per second demands.

At the other end of the application spectrum, data warehouse and OLTP applications push I/O requirements beyond 25,000 I/Os per second while transferring smaller data blocks. Projections for 100TB applications requiring at least l00Gbit (10GB) per second transfer rates by the year 2010 are now on the long-range planning horizon. Let's examine a few of the newer storage intensive applications that are moving the limits of mass storage and how they may impact our everyday lives.

GIS (Geographic Information Systems) are becoming vital tools for scientists. Viewed as the new horizon in Earth exploration, GIS is a digital mapping technology used for demographics, tracking systems, dispatch, and exploration. As many as 10,000 commercial remote sensing systems in nearly 100 countries may be in place as the decade ends. GIS software can search a database, extract various forms of information, and overlay those forms on digital maps performing spatial analysis.

GPS (Global Positioning System), a satellite-based navigation system, has improved the capability of GIS for new applications such as disease surveillance and discovery. Public health is now using GIS to monitor the spread of diseases, including malaria, polio, Lyme disease, and dengue fever, around the world. Exposures to pesticides, lead poisoning, chemical releases, and pollutants are being addressed by the increased capabilities of GIS applications. Timeliness of data is critical in GIS, as old information is bad information. GIS is possibly the leading driver of the concept of a central data repository. Significant amounts of data from a wide variety of input sources define the need for a single and consistent data source or information utility that many users can access. GIS may become one of the best examples of an ideal SAN application, given the large amounts of data and the connection of heterogeneous servers that collect the data. Storage requirements for GIS applications are enormous, vary by the application, types of data collected, and often include two, three, or four-dimensional images. Estimates of ITB to 2TB of digital content generated per day are common for GIS applications [Of course, the data needs to be backed up and made available in case the primary copy is inaccessible generating even more storage requirements].

The emergence of Electronic Medicine as a new discipline arises from the awareness that continuing advancements to the medical knowledge base on traditional paper-based methods is impossible. Studies suggest that 8590% of all healthcare information is stored on paper or film. The remainder is stored on film or on digital computer-based storage. A typical radiological x-ray takes 12MB of storage. If a hospital performs 200 x-rays per bed per year, a five hundred-bed hospital will generate 100,000 x-rays per year, resulting in l.2TB of storage. Backing up this data doubles the storage requirement. If a person is critically injured, access to the x-ray is required in seconds. A discharged patient x-rays may seldom, if ever, be accessed again, but a lifelong archive of the data is still required. X-rays represent only one of many medical applications driving storage demand; others include CT-scans, digital echocardiograms, and lab reports, brain scans, and document management.