Method To Their Madness - Industry Trend or Event

Industry Standard, The, August 7, 2000

New York University's Stern School of Business professor Joel Steckel criticizes e-commerce forecasting methods. But Forrester Research senior analyst James McQuivey responds that the Net's rapid growth calls for unique techniques.

The Net needs established market-research methodology.

Why do the e-commerce forecasts of research vendors vary so much? Some of the variance is a result of the different research methodologies used. Internet researchers rely on several techniques. Expert judgment is one approach to Internet forecasting. Another approach, customer surveys are likely to overestimate; research has shown that people are notoriously bad at predicting their own future behavior.

A more traditional market-research approach is time-series analysis, which extrapolates historical data to predict the future. However, forecasting the adoption of online shopping is difficult since historical data is sparse. Another method is causal forecasting, in which market data is used to produce a model.

Unsatisfied with available forecasts from the usual researchers, conducted my own study to forecast online shopping using a standard time series-based marketing methodology applied to U.S. Census data and survey data from Cyber Dialogue.

I found that Internet access is reaching a plateau in the U.S., which will slow the rate of e-commerce growth. Online consumer purchasing in the U.S. will grow about one-third over the next year or two and 20 percent for a year or two after that. This amounts to a 150 percent to 200 percent increase in online buyers within five years. While this is in line with Forrester Research's projected growth for the number of households participating in e-commerce, it's not consistent with estimates from the Gartner Group and other research firms.

Unfortunately, research companies generally do not expose their precise assumptions and specific methods to public scrutiny. They argue that competitive advantage depends on their methodologies being proprietary. But more openness would enable the media, investors and potential customers to evaluate whether their forecasts are produced in a reasonable manner.

Pioneering research firms made Internet estimates possible

The only real problem with Steckel's analysis is that it has arrived too late to help Wal-Mart decide how to deal with that upstart Jeff Bezos. By applying time-honored equations to the online shopping market, Steckel has accomplished what every good social scientist hopes for in a market forecast methodological rigor. Only with this kind of analysis can the growth of online retail be verified.

But analysis like his is impossible to conduct until several years worth of data reveal trends that can be relied on for forecasting. Without those data points, the equations produce nothing - high lighting why firms like Forrester Research, IDC, Jupiter Communications and others are necessary.

This is the domain of the strategy analyst - part statistician, part strategy bloodhound, part gossip. We lurk between equations, identifying assumptions and creating models to fill in the gaps that data can't provide, and then stake our reputations on the results we build.

I first forecasted the growth of online retail in the fall of 1998 after speaking with 100 online retailers and spending a month analyzing spreadsheets. The only thing more exciting than the fact that the model still largely stands two years later (although it has been enhanced to reflect new data points), is that to build the model, we crystallized insights into the market that are nearly identical to the takeaways Steckel offers at the conclusion of his research.

That means Forrester clients had a two year strategic lead over those who would have waited until the "numbers were in" to perform the more rigorous statistical analysis. That's a lead worth paying for - as long as the analysis is meticulous and the analysts produce high-quality insights. And that's the challenge all analyst types fade as they turn their attention to the next big quantifiable: the wireless Web.