New Strategies In Performance Storage Solutions - Technology Information

Computer Technology Review, April, 2001 by Martin Bock

It is a very exciting time for the disk storage systems market. With the growth of video data content, music files, high resolution photographs, etc., there appears to be a never-ending demand for more and more disk storage solutions. The Internet has created vast depositories of information all over the world which contain every imaginable form of data. Once we talked mainly of gigabytes, then terabytes, and now it is petabytes. Where will it end? It is difficult to see any slow down at this time, so we will soon be talking about exabytes and zetabytes. The re-purposing of data into digital formats for Internet consumption is one of many significant factors in this exploding demand for disk storage solutions.

Complementary product enhancements have helped pave the way for the dissemination of these large files. Host processors, of course, are continually improving their handling of large video, music, and picture files. The delivery transport has improved with the availability of DSL, cable modems, and more affordable T1 lines. Performance fibre networks to support large databases are improving and becoming more cost effective, and, disk technology continues to advance in speed, storage capacities, and performance user interfaces.

Changing Storage Requirements

With these new tools available, it is time for a new look at the architecture of current storage products today and how they could be enhanced to meet the challenging growing needs of the video, music, download program files and picture storage. The challenge for storage solutions is to handle vast amounts of large block files as efficiently as possible. Even heavily compressed music and picture files can be multiple megabytes each, and video files can be gigabytes. The demand today is for even greater picture and video quality which will drive the file sizes up by a minimum of four times.

The requirements for large block files on the surface sounds a lot like any storage application, but there is a twist to the requirements that cannot be overlooked. These requirements include:

* Faster throughput

* Guaranteed performance (real-time)

* Higher storage capacities

* Scaleable configurations which build throughput as well as capacities

* Redundant/protected data

* Lower costs per gigabyte

Faster throughput means the ability to deliver large files in the least amount of time. This is a bandwidth issue as well as an I/O issue, since video, music, pictures/photographs, and program downloads are contiguous transfer applications. Moving a large file typically clogs the path over the delivery channel. The key is therefore to move the file through as fast as possible so that the path is clear for the next transfer request.

Guaranteed performance is actually an extension of faster throughput. For the video or music creation application, guaranteed performance means that the content can be reviewed without interruption. For the information/content provider, it means that the provider can get maximum utilization of the system hardware. For example, if a supplier is paid by the amount of data he or she delivers, losing throughput capability by 25% in the event of a drive failure means that his revenue stream will also be cut by 25% until that failed drive is replaced. This is an extremely important issue for Internet product providers.

Video, pictures, and music files are storage hogs, with video being the guiltiest culprit. A standard length, medium resolution music file is easily 2MB. A photograph of average resolution is 14MB. A video file stored in low resolution MPEG1 format is over 1GB, while a DVD quality MPEG2 file averages 3GB.

Scaling storage systems to meet higher throughput as well as capacity requirements means that parallel access must be easily facilitated by the storage system. Piling up more capacity behind the same delivery path is no help, once the data path is maxed out. Parallel (multiple) data paths to the storage system allow not only higher I/O access, but higher throughput capability, and can insure higher guaranteed performance.

Redundant capability in storage solutions for large block applications is extremely important. You can reload a text file in seconds, but to reload a single DVD quality video from a tape backup system requires 10-15 minutes. For the poor uncompressed video user, that backup requires six hours for an average length, standard definition video. If the database is a collection of hundreds of videos, the restore time is not an attractive activity to think about.

Lower cost storage is a term that seems to be filler material in this article, but actually, if storage costs are not minimized for large file applications, it can become economically problematic. The best example I remember is a request from a customer at a trade show approximately four years ago. This gentleman had a collection of videos/films of old television shows that he wanted to preserve and have available via computer monitor. He had approximately 1200 episodes that he wanted to digitize and store on disk. Additionally, he needed the resulting digital content to be quality equal to that of the original videos, because he needed this digital content to be the new master, since the videos and film were deteriorating. If each video was a 30 minute episode and it was stored in standard uncompressed form (Dl format) the required disk storage would be approximately 45TB. Four years ago, with the prevailing disk storage capacities at 9GB and a cost of $1,000 per disk (Cheetah Variety) the customers' raw dis k costs would be approximately $5 million. He decided to wait. Another very popular example is the video-on-demand fiasco that peaked for the first time approximately seven years ago. This application was impacted greatly by the economics of disk stored digital video.