Physician workforce data: when the best is not good enough

Health Services Research, Oct, 2004 by Phillip R. Kletke

In this issue of HSR, the article by Rittenhouse et al. (2004) calls into question two different sources of information about physicians' decisions to leave clinical practice: data from the Physician Masterfile of the American Medical Association (AMA), and physicians' self-reported intentions to leave patient care. The authors conclude that neither source of data provided reliable information on withdrawals from clinical practice. In particular, they find that the AMA Masterfile had a sensitivity of only 9% in detecting physicians who left clinical practice during the previous three years.

One could first respond to this paper by arguing that these findings are neither new nor surprising. The low sensitivity of the Masterfile in recording changes in practice status is attributable to the long reporting lags, which on average can run as long as several years. Thus, the Masterfile cannot be expected to accurately register changes of individual physicians within a time frame shorter than the report lags.

A second, much more appropriate and compelling response is that Rittenhouse et al. point to a major shortcoming in physician workforce data. Reporting lags severely limit what we know about the current size and the geographic and specialty distributions of the physician population. Because of these reporting lags, workforce analyses overestimate both current and future physician supply. These inadequacies must be resolved so that policy analysts can properly assess the adequacy of the physician workforce to meet the nation's demand for physician services.

REPORTING LAGS IN THE MASTERFILE

Researchers have long been aware of reporting lags in the Masterfile--and other inaccuracies as well (e.g., see Kletke et al. 2000). The Masterfile is continuously updated with information from a variety of data sources, including medical schools and graduate medical education training programs, hospitals, state licensing agencies, medical societies, specialty certification boards, and an ongoing survey of the entire allopathic physician population that collects data on physicians' professional activities. All of these updates are associated with some reporting lag. The length of reporting lags varies considerably with the type of information on which the update is based.

The AMA updates the Masterfile by creating data records for new physicians. The reporting lags for this type of update are relatively short. The AMA tracks medical students training in U.S. medical schools and posts their information onto the Masterfile at the time of their graduation. New international medical graduates (IMGs) are posted onto the Masterfile when they enter residency training. These updates are made with information from the Annual Survey of Graduate Medical Education Programs, which collects data on virtually all physicians in residency and fellowship training programs. (1)

On the other hand, lags are much longer for updating changes in physician practices (e.g., whether physicians provide direct patient care, whether they are still professionally active). Much of the information used to update Masterfile data on physician practices comes from an ongoing survey of the entire physician population. Physicians are sent a questionnaire every three to four years. Thus, for the Masterfile's data on physicians' practices, lags of two years are average for individual physicians who respond to the survey and can be much longer for physicians who fail to respond to the questionnaire.

Despite these reporting lags (and other inaccuracies), researchers continue to analyze Masterfile data simply because the AMA Masterfile is the most complete and authoritative source of information on the nation's supply of allopathic physicians (MDs). Similarly, the Masterfile of the American Osteopathic Association (AOA) is the most complete and authoritative source of data on the nation's supply of osteopathic physicians (DOs). Virtually all analyses of the current and future size and composition of the U.S. physician workforce are based, either directly or indirectly, on data from the AMA and AOA Masterfiles.

THE IMPACT OF REPORTING LAGS ON PHYSICIAN WORKFORCE ANALYSES

Rittenhouse et al. examined Masterfile data to analyze physicians' departures from clinical practice, which can happen in two primary ways--retirement, or switching professional activities to nonpatient care. (2) Let's focus our attention on physician retirements. How do reporting lags for retirements affect physician workforce analyses? (3)

First, because of lags in the posting of physician retirements, the Masterfile data overestimate the current supply of active physicians. Furthermore, the delayed reporting of physician retirements on the Masterfile is reflected in the projection estimates as well, because projection models use separation rates based on retirement and mortality data from the AMA Masterfile. Finally, the impact of reporting lags on physician supply estimates is expected to increase significantly with the aging of the physician workforce--that is, with the increasing number of physicians entering the older age categories. (4)