Information Overload Weighs Down Web

0 Comments | Insight on the News, Feb 28, 2000 | by William Glanz

While the World Wide Web continues to expand exponentially, users struggle to find ways to gather pertinent information quickly and efficiently.

The World Wide Web hit a milestone in January when the number of pages floating in cyberspace passed the 1 billion mark. But quantity is undermining quality. Consumers and researchers more often than not are overwhelmed by the sheer amount of data available online, says Vinton Cerf, known as the father of the Internet.

In 1974, Cerf helped design the protocol to handle network transmissions that became known as "www." While he continues to be amazed at the Web's rapid growth, he admits online research remains problematic. "If librarians were worried that they would be out of a job because all information was moving to the Web, just the opposite is true" says Cerf.

The Web had 320 million pages in December 1997; it had grown to 820 million pages by February 1999, and to more than 1 billion one year later, according to a study by NEC Research Institute in Princeton, N.J., and Inktomi Corp., a Foster City, Calif., company that makes software to run search engines. (By contrast, the Library of Congress has 18 million published works.) Despite the huge number of Web pages, many rarely are seen -- most Net surfers revisit familiar sites rather than negotiate the overwhelming vastness of cyberspace.

"Customers look at 12 to 24 sites regularly and bookmark about a dozen," says Stuart Gibbel, vice president of New York-based Internet marketing research firm Cyber Dialogue. "What it means is branding still matters. As much as people love choice, they don't want to look through millions of sites."

Even though people congregate at a limited number of Web pages, the proliferation of pages will continue, predicts Mike May, digital commerce analyst at New York Internet research firm Jupiter Communications. That's because there's no limit to what the Web can handle. But more pages won't necessarily mean more hits, or finds, when surfers perform a search. That's because search engines -- Websites used to find other sites -- each cover only a portion of the Web, explains Danny Sullivan, editor of search Enginewatch.com, a Connecticut-based online publication covering search engines.

The biggest search engine -- alltheweb.com, run by Oslo, Norway-based Fast Search and Transfer ASA -- encompasses 300 million indexed pages. Alta Vista, based in Palo Alto, Calif., has indexed 250 million pages. (Yahoo Inc., the most popular search directory, based in Santa Clara, Calif., doesn't release the number of indexed pages available at www.yahoo.com.)

"The biggest search engines realize -- it's not enough to be big," says Sullivan. "You need relevancy, and the search engines definitely are getting better in terms of relevancy." Relevancy is most determined by the quality of a search engine's indexing.

Speed is another factor that influences search habits. Keynote Systems Inc., a Website-performance measurement firm in San Mateo, Calif., measures the length of time it takes 306,000 computers at 32 commercial locations to download 40 separate Websites. The company estimates download times are 37 percent faster than a year ago. In January, for instance, it took an average of 5.78 seconds to download a page, compared to an average of 9.27 seconds a year ago.

Customers are getting faster speeds because Internet service providers are increasing the number of telephone and fiber optic lines carrying data. But not everyone is benefiting from the changes. Keynote Systems' weekly measurements don't include pages downloaded via standard dial-up modems -- the kind found in most households. Cable modems and digital subscriber lines transfer data up to 100 times faster than 56-kilobit modems -- an increasingly old-fashioned way of accessing the Internet, and one wags have nicknamed the World Wide Wait.