Evaluating the 1995 BLS projections - US Bureau of Labor Statistics labor, industry, and occupational projections for 1984-1995 - Cover Story

Monthly Labor Review, Sept, 1997 by Neal H. Rosenthal

Projections of the labor force and industry and occupational projections to 1995 were relatively accurate; differences that did occur are identified and analyzed to inform users and to improve future projections

Introduction

The Bureau of Labor Statistics prepares projections of the labor force, industry employment, and occupational employment every other year. Because of the uncertainty inherent in making these projections, the Bureau evaluates the results of past projections each time a target year is reached, to gauge how well the projections tracked against actual change. The evaluations provide users of BLS projections with information to enhance their understanding of the problems faced in developing accurate projections and to assess the manner of using projections in the future. Among the many users of projections are those in the fields of career guidance, education planning, and public policy formulation. In these fields, numerous decisions are made based on differences in projected labor force growth rates by race, age, sex, and Hispanic origin and on comparisons of growth rates among industries and occupations. State employment security agencies incorporate date from BLS projections into the models they use to develop industry and occupational projections for their State. Business officials in the private sector utilize the projections in personnel planning and marketing research. Academic researchers employ the projections as background information to study a wide variety of topics dealing with the labor market. Knowledge of the accuracy of BLS projections affects whether an individual or agency will rely on the projections in the future and, if so, how the projections will subsequently be used.

The Bureau also considers evaluation to be an important stage of its projections program. through evaluation, the Bureau can identify strengths and weaknesses in procedures used to prepare the projections. Such identification leads to changes in those procedures for use to the development of later projections.

Many causes of inaccuracies in the projections are identified in studies comparing actual with projected change in the labor force, industry employment, and occupational employment. Some of the broad assumptions made in the models used to develop the projections, such as levels of defense expenditures, are very sensitive to political forces and, therefore, subject to great uncertainty over a 10-year period. Other broad assumptions, such as the strength of the world economy, also are subject to considerable variability and are very important to levels of exports and imports that have a great impact on employment. In addition, a multitude of judgments stemming from the analyses conducted by BLS staff in determining specific effects of technological change are subject to significant error. For example, in assessing the impact of word-processing equipment on the employment of typists, it seemed clear in 1984 that technology would have a negative effect. That assessment turned out to be correct, but the projected decreases in the utilization of typists made by BLS analysts in gauging future staffing patterns of industries fell short of actual decreases.

Evaluation measures

The general procedure the Office of Employment Projections uses to evaluate projections is to compare projected with actual data. Such comparisons can be presented in a variety of statistical measures, such as percent change or numerical growth. One of the problems with these measures is that it is difficult to determine the quality of the projections, because there are no established criteria to categorize them as good or bad. For example, if actual data show that employment in an occupation increased by 20 percent, and the Bureau projected a growth rate of, say, 10 percent (or 30 percent or 40 percent, for that matter), it is not clear whether the projected rate is good or bad. Consequently, users must establish their own standards of quality, based on the uses they make of the projections. In general, if their decisions would not differ if the projections were perfect, then the error is not significant. For instance, referring back to the foregoing numbers, if a user would have made the same decisions if the projected level were perfect (20 percent) as he or she in fact made with the 30-percent projection of growth, then the 30-percent projection may not be accurate, but it can be assumed to be accurate enough for that user's needs.

Because standards are not available, several measures that compare projected and actual employment arc presented in the evaluations of labor force, industry employment, and occupational employment projections that follow. These include a simple comparison of the direction of change to see whether employment in the occupations that were projected to grow or decline actually did change in the direction projected. Numerical change also is presented, but the use of this measure to determine general accuracy can distort an evaluation, because the quality of projections having the same numerical error are different for occupations having different employment levels in the base year of the projection. For this reason, the measure given the most attention in the evaluations is the absolute percent error--the numerical error (positive or negative) divided by actual employment in the target year of the projection. Another measured used to evaluate the projections is a comparison of the distribution of the projected change with the distribution of the actual change. This measure is important for identifying whether any particular projection errors are related to judgment errors tied to a specific industry or occupation or to general projections of total employment.