Storage challenging business creativity: hard choices ahead for management and government compliance - Storage Management

Computer Technology Review, Nov, 2003 by Fred Moore

Given the mounting storage management challenges faced by businesses today, some of the long-standing beliefs about storage management are being questioned. In some cases, the new viewpoint is the opposite of conventional thinking. Here are a few examples of the inverse thinking some businesses are considering as their management challenges escalate.

[ILLUSTRATION OMITTED]

Question 1: Is it better to manage storage or just add more hardware?

Managing storage has become far more complicated than simply making sure enough capacity is available to meet demand. Disk and tape technologies from multiple vendors, a variety of switching devices, network management, SAN, NAS, DAS, implementing acceptable backup/recovery and high availability schemes, and meeting the SLAs of key applications leave the cost and complexity of complete and effective storage management out of reach for most non-mainframe businesses.

Throughout the mid-1990s, the easiest way to manage storage subsystems and data was usually accomplished by simply adding more storage. This straightforward strategy worked well for many small and medium-sized businesses for many years. As the growing management gap between storage growth and the number of storage administrators continued to diverge and put business applications at risk, this strategy began to fall out of favor and storage management became a stated mission or mantra for most IT organizations by the late 1990s.

Today, data is growing faster than the deployment of storage management tools and the supply of trained people to manage storage doesn't keep pace with demand. As a result, a great deal of valuable business data is not effectively managed meaning that backup, recovery, performance and capacity are dealt with in a reactive manner, if at all. By 2007, it is expected that the average (non-mainframe) storage administrator will be able to manage about 15 terabytes of storage while the amount of data to be managed will exceed 50 terabytes.

Storage management can be simplified by adding NAS for certain applications, SANs for others, adding virtualization software, unifying block and file storage systems and undertaking consolidation efforts. Overall, these actions have not kept up with the 50-70% annual growth rates of the digital storage pool, and they are costly in terms of people, processes, and expensive to implement given the tough economic landscape.

Possibly the most significant and long-awaited technological solution will arrive in the form of storage management functions and intelligence that move away from the server and into the storage subsystem or network fabric. This concept enables more of a "black box" proactive approach to storage management moving away from labor-intensive and reactive host-server-based approaches. Though under various stages of development, the intelligent fabric is not here yet and until it arrives, businesses are, out of necessity, considering other options.

One option that is gaining more consideration in some areas, rather surprisingly, is not to manage storage. Why try to invest all that money, time and energy to deal with such complexity? After all, if storage management costs have become so much higher than declining hardware expenses, why not return to the mid-1990's philosophy of just adding more hardware? Is this a sustainable or viable solution given that the resources and infrastructure to effectively manage storage and data are falling further behind the demand curve?

Bottom-line: Storage deployment is falling behind storage growth. With the management gap widening, management expenses growing, and hardware prices falling 30-40% annually, for the near future, businesses are reconsidering simply adding more hardware as the less-expensive management strategy.

Question 2: Is compliance with regulatory agencies worth the expense?

Increasing regulatory pressure to comply with federal mandates for e-mail, medical, insurance, legal, financial and government classified data are quickly forcing many businesses to examine any potential weak points in their long-term storage systems. New applications and a variety of legal and business requirements are driving the need for many businesses to re-examine or create their archival policies. One of the most visible examples of the emphasis on the increasingly critical value of archival data lies with the HIPAA (Health Insurance Portability and Accounting Act) requirements. Not only does HIPAA require health providers to preserve data for a yet-to-be-determined time period, but the failure to protect critical patient data carries with it penalties presently ranging up to--or exceeding--$25,000 per violation.

The threat of the fines and other forms of non-compliance are encouraging storage administrators to examine the increasing amount of archival data that would be required to be kept indefinitely for future reference. For example, the PACS application (Picture Archiving and Communications System) that captures and stores radiology information and other types of medical images is a primary component of the HIPAA requirement. Data used to be retained for one year, and then three years, then seven years; now infinite retention seems inevitable for some applications. Some health care businesses are talking about retaining digital records for the patients' lifetime plus seven years! This could be over 100 years, in some cases. Given today's legal, economic and political climate, the value of archival data changes as it ages rather than just declining as it ages. This partial list of regulations is becoming increasingly important to the storage administrator's strategy and includes: