Sizes of rich media files are changing the rules: managing them is the trick

Computer Technology Review, Oct, 2002 by Fred Moore

Fueled by the widespread use of computers and the falling prices for digital storage, the world is generating 1.5 to 2 exabytes (1x[10.sup.18]) of original digital content each year, and this number is growing. Five exabytes would equal all of the spoken words by humans in the history of the world. The ever-lower cost to generate digital content and the continually lower costs to store and retrieve data make the growth of digital content an event of infinite duration. Most data used to come from governments, businesses, newspapers, and publishing houses. Today, individuals create the bulk of the world's original content. Estimates from the landmark University of California at Berkeley study (www.sims. berkeley.edu/research/projects/how-much-info/inde.html) on digital information indicate that as much as 93% of information produced each year is stored digitally. Other estimates show that paper usage grows as much as 40% when email is implemented. Who would have ever thought that email would be the biggest driv er of paper consumption?

The long-term growth opportunities for content- and rich-media-management solutions abound. Rich media consists of voice, text, graphic images, audio, HDTV, 3-D graphics, and movies. New and emerging digital applications will continue to fuel many years of explosive growth for storage as terabyte-plus data-warehouses, VCR to HDTV quality movies, the coming digital cinema, electronic voice and video-mail, digital security systems, digital photography and the "Second Internet or Hypemet" all will drive major increases in data rates and capacity. Approximately one terabyte is now required for a digital theater complex. A single movie production may require over 800 terabytes to complete. The worldwide commercial video storage market including television, movies, commercial production, and video distribution could reach Over 700 petabytes (700x1015th) by 2006. A robust personal-video market also exists for recording movies and video at home. This market, like the data-storage market, is expected to exceed 900 petabytes by 2006. For these markets, backup, replication or mirroring techniques will double or triple the total storage demand, pushing the rich-content market into the exabytes. In addition, the consumer electronics and mobile markets are just beginning to enter advanced-growth phases on a worldwide basis. Rich media and digital content requires much larger multimedia objects (voice, text, video, games, and graphics) and has pushed storage management requirements far beyond the capabilities of available management tools only adding to the upcoming storage disruption.

In addition to digital information, a variety of non-digital information applications remain, including analog audio, analog video, many types historical scientific data, books, maps, old music, photographs, and medical images. As these applications become digital, the amount of rich content further increases. Paper is estimated to generate over 200 terabytes per year, X-rays between 15 and 20 terabytes per year, and music downloads Onto CDs anywhere upwards of 50 terabytes per year. College campuses in the United States have just reported that about 75% of their campus bandwidth is now consumed from downloading music. Camcorders are estimated to record over 300 terabytes of data per year on a variety of magnetic tape formats. The rate of adoption for a new consumer and rich media technology has been shrinking. The time frame it took to achieve 5% penetration of the consumer market:

* Color TV = 12 years

* VCR = 7 years

* CD players = 4 years

* Camcorders = 4 years

What we don't know is which mechanism(s) will be the preferred method of delivery of rich media to the appropriate markets.

Estimates now show that over 50% of the world's digital data sits within the well-publicized last-mile boundary and is generated by individuals, not large companies or governments. The last-mile problem refers to the performance problems associated with transmitting information to and from individuals either at home or to those who don't have high-speed fiber broadband access Cable modem and DSL are the two most popular choices for providing higher speed Internet access and each have an availability index of about 95%, well below standard telephone connections. Presently, cable modem often represents the fastest, cheapest and most reliable solution for most applications.

Various considerations exist for all three options. Performance bottlenecks slow cable modem and DSL speeds to as low as i megabit per second, far below their speed limits. Cable and DSL are still much faster than the 56 kilobits per second of a traditional phone-line modem in use on most of the last-mile market. The top speeds of each technology are seldom realized. Cable performance suffers when bandwidth is shared with neighbors and becomes congested. Unlike cable, DSL uses a private line requires that you reside within two to three miles from the provider's central office and performance is very distance sensitive. Satellites are the most expensive solution and have far fewer subscribers than either DSL or cable. Availability levels are unpredictable and are affected by weather and a variety other obstacles causing signal interference. Satellite has the advantage that it can serve geographically remote areas not covered by DSL or cable. Satellite services normally provide about 150 channels while large c able systems typically provide 75-100 channels. With all of the tradeoffs now apparent, no clear winner for solving the last mile problem has emerged and the stakes are getting very high based on the market potential. Though the early leaders have been cable and DSL, a wireless or satellite solution is the most likely longrange solution. Until a clear winner emerges, all methods will co-exist to meet demand.