The Metrics System

Brandweek, Nov 13, 2000 by Hassan Fattah

Advertisers and e-businesses alike are questioning the effectiveness of online ad traffic measurement. But is there a better way?

The ritual starts about the middle of every month or fiscal quarter. From New York to Silicon Valley, printers and mail servers at Web sites and public relations firms churn out press releases citing each site's traffic figures in a chorus of superlatives: "The Street.com makes the list of Media Metrix's top gainers." "Media Metrix and Nielsen NetRatings confirm Monster.com is the most powerful careers Web site around." "IVillage garners 12 percent reach in September."

Thanks to the metrics developed by Media Metrix, Nielsen NetRatings (which is part of VNU, Adweek's parent company) and @Plan, the top three Web traffic measurement firms, it may seem like there's always a good figure for Web publishers to talk up. Some numbers stress individual users, others celebrate audience "reach," while still others highlight a site's so-called "gross usage minutes." And the variety of online metrics continues to grow as each site struggles to better define its model and its audience.

BUT DO METRICS MEASURE ANYTHING?

All of this hoopla is more than an exercise in chest beating. Indeed, big money-in the shape of advertising revenue and ad rates--rides on those numbers. Wall Street wants to see continued growth to support its investments. Partners and joint venturers want proof they've consummated with the right team. And most important, perhaps, advertisers want guidelines on where to spend money and where their ad dollars are being spent. Like the Nielsen ratings on TV, and Arbitron on radio, the various Web metrics have been an important-if not the only-indicator of a Web site's relative health, growth and placement potential.

But the numbers seem to be telling a different story these days. After more than a year of extravagant online spending, scores of advertisers who've used the metrics are looking over the results and wondering what went wrong. For many publishers, Web advertising has proven less effective than it was supposed to be. In some cases it's proved to be completely ineffective. And a vociferous band of media planners has begun asking a very fundamental question: Do these metrics mean anything?

"When we've used these research tools to predict what sites to buy, they just haven't worked," says Brian Monahan, president of Inrhythm Marketing and a co-founder of online agency Exile on Seventh. "The research tools don't predict success ... they're measuring a site's traffic in the aggregate, but that's not what we buy. We buy blocks of impressions."

SO WHO ARE ONLINE ADS TALKING TO?

Everyone's used to the traditional measures--reach, clickthrough and overall traffic. But those say little about the advertiser's audience, media planners argue. While Media Metrix and Nielsen NetRatings have built their businesses by hyping things like overall traffic, time spent on a given site and clickthrough rates, these things may say very little about the user's lifestyle or their psychographics. In short, they say nothing about the people advertisers want to speak to.

"They're trying to fit a square peg into a round hole," says Tom Hespos, Internet strategist at Mezzina Brown & Partners. "Everyone's been stuck to the cickthrough model, and that doesn't work anymore."

The dissatisfaction with site traffic measurement doesn't stop with advertisers, either. Many beleaguered Web sites are grumbling that they're not getting credit where it's due. The well publicized case of Britannica.com, which saw its independently reported traffic figures come in far below traffic measured off its servers, underscored the fickle nature of Web traffic numbers. As Britannica's managers see it, neither Media Metrix nor Nielsen NetRatings accounted for the gobs of international traffic to its site, though both measurement firms have disputed that assertion. Still, major variations in the numbers from each service--sometimes amounting to 40 percent or more--only intensify questions about the metrics' efficacy or truthfulness.

"I don't tend to put a lot of credence in the numbers themselves," stresses Martin McClanan, CEO of Red Envelope Gifts Online, an online retailer and frequent online advertiser. The numbers may give short shrift to sections within a site that might attract an avid group of users, and tend to only speak to a run of site audience. "For us, the key thing they tell you is the relative comparisons," says McClanan. "They can't be used like the Nielsen ratings on TV."

NO STANDARD METHODOLOGY

And that's at the heart of the controversy. With no standard measure, planners must be especially careful when using the Web metrics. Media planners and even the research firms say planners must understand the strengths and weaknesses of the research to use it right. And at the heart of that are the techniques each firm uses. Much like Nielsen Media Research's TV ratings, all three firms rely on polls of users to make their measurements. But the way they build their panels and cull their figures has a huge effect on the results.