Reality check, or the alchemy of long-range industrial forecasting

Business Economics, July, 1999 by Andrew C. Gross, Renaud de Maricourt, D. Steven White

Forecasting is such an irresistible endeavor: Who would not like to know the future? All private sector firms, government agencies, and nonprofit institutes like to predict demand or market size for their goods or services. The idea of planning is to construct plausible scenarios. Yet the road is full of peril, pitfalls and setbacks. Improved models are constantly touted, but accuracy of past projections is ignored. This paper reviews what happened to selected long-range industry, forecasts in four highly industrialized nations: the United States, the United Kingdom, France, and Italy. We look not only at accuracy of forecasts, but lessons learned as well.

Sources, Methods, and Accuracy

Assessing the size and nature of national - and beyond that, regional and global markets has become simpler, yet more complex in the past forty years. Compared to earlier decades when information gaps existed, we now have information overload. More sophisticated techniques, faster computers and vast databases exist, vying for attention. Yet we still struggle to identify proper sources and relevant methods; more important, we wonder about underlying assumptions, biases, and inaccuracies.

In the 1960s and 1970s, key sources consisted of national censuses, trade association reports and the annual publications of international agencies such as the IMF, the OECD, and the UN. Another set consisted of corporate annual reports. They continue to serve as foundations to this date. But in the past twenty years, we have seen major growth in private databanks, some with their own original estimates, but many more with recasting of already published data.

Major players are ABI-Inform, BPI-Wilson, Bloomberg, DataStar, Disclosure, EIU, Lexis-Nexis, Predicasts/Information Access, RDS, and others. Today, one can find not dozens, but thousands of industry, product, and market sites on the World Wide Web. To locate data, many search engines now assist in expanding or narrowing the scope of investigation.

The major methods of forecasting have been known for decades, but sophisticated techniques have come to the forefront more recently. The two major categories are still qualitative and quantitative, both with subcategories. In the former, we speak of sales force estimates, users' expectations, expert panels, and Delphi rounds. In the latter, the range includes trend fitting, simple moving averages, exponential smoothing, multiple regression, Box-Jenkins, econometric models and many others. A search in Books in Print (US) conducted in April 1999 yielded a total of nearly 300 book titles on the topic of"business forecasting," and a similar search on the Web gave thousands of citations even when the topic was narrowed by year, country, or industry.

Before one is ready to forecast, however, it is imperative to consider the nature of the underlying data. In the 1960s, in a classic book on the accuracy of economic observations, Oscar Morgenstern estimated that the range of error on current statistics was about 10 percent for industrialized and 20 percent for emerging nations. Put differently, Morgenstern, along with Darmstadter, Kendrick, Kuznets, Russell, and other pioneers during the 1950-70 period, found that economics is a one- or at best a two-digit science. In a recently published book (1995), Duncan and Gross argue that, [TABULAR DATA FOR TABLE 1 OMITTED] while the situation improved, problems remain. These include poorly trained survey workers, ambiguous questions, deliberate misinformation, simple nonresponse, and revisions in primary data collection and lost details, transcript errors, differing definitions, and conflicting classification with regard to both secondary and tertiary information (this last category refers to abstracts, digests, databases, etc.).

If the underlying statistics for past and present are often suspect, how can one make accurate forecasts? The simple answer is: One cannot. But we try with various methods and various degrees of success, and we go beyond macroeconomic indicators to project the future of industries, products, and markets. But even at this point, humility is in order.

Toward a Conceptual Framework

In our view, the past, present and future must be linked intimately in what we call consensus, combination, or composite estimation (we use the terms interchangeably). In this framework, (and this is the key premise), we try to simulate the real world, wherein the diverse plans of many individuals and organizations are converted into mutually consistent events. This is easier for the past than for the present or the future. At its simplest, the method calls for averaging everyone's estimate of output. At its most complex, the framework calls for a sophisticated model with thousands of inputs, hundreds of time series, dozens of equations, and constant checking, all of it coupled with intuition and judgment.

Past Forecasts and Their Accuracy

Several examples in this section were taken from the 1960-80 records of Predicasts, an economic and market research firm with which the lead author of this paper was formerly affiliated. In 1960, Predicasts collected time series data, especially forecasts, from hundreds of public and private sources, classified them according to the SIC code and published each data bit (with source attribution) on a single line. Just a few years later, the staff began to develop its own consensus forecasts based on the underlying data. Then in the 1970s, industry studies were undertaken, first for the United States and later on a global basis; over fifteen years, many titles, especially on durable goods, appeared.