Measuring the quality of diabetes care using administrative data: is there bias? - Managing Chronic Illness in Managed Care Settings

Health Services Research, Dec, 2003 by Nancy L. Keating, Mary Beth Landrum, Bruce E. Landon, John Z. Ayanian, Catherine Borbas, Edward Guadagnoli

In recent years, efforts to measure and improve the quality of care have expanded to include care for chronic medical conditions, such as diabetes mellitus, that are primarily treated in outpatient settings. In addition to providing information for purchasers of health care (Epstein 1995; Iglehart 1996; Marshall et al. 2000; National Committee for Quality Assurance 2000) and internal efforts to improve quality (Petitti et al. 2000; Kiefe et al. 2001), quality data are increasingly being included in publicly released performance reports (Epstein 2000; Jencks et al. 2000; Bost 2001) and as criteria for determining compensation for providers by medical groups and health plans (Schlackman 1993; Pedersen et al. 2000; Kowalczyk 2001a; Sussman et al. 2001). Because the results of quality measurements may have widespread consequences, it is important that the data collected be accurate and unbiased.

Quality measures for diabetes care are collected primarily from two sources: medical records data and administrative data (which are used primarily for billing purposes). Although some efforts incorporate data from both of these sources (National Committee for Quality Assurance 2000; Diabetes Quality Improvement Project 2001), others may use administrative data exclusively to measure quality of care (Jencks et al. 2000; Kowalczyk 2001b) because these data are typically more readily available and less expensive to collect than data from medical records (Iezzoni 1997).

It is important to know whether quality assessments that rely on administrative data alone fail to detect compliance with accepted standards and whether this problem varies systematically for certain groups of patients, particularly based on age, sex, race, or socioeconomic status. This could be the case, for example, if certain groups of patients are more frequently cared for at clinics that lack sophisticated systems that might facilitate more complete billing. The accuracy of quality assessments is crucial to determine whether disparities in quality of care persist (Fiscella et al. 2000). Moreover, if quality of care is systematically underdetected for some groups of patients when administrative data alone are used, then quality profiles may be unfairly biased against providers who more frequently care for such patients.

In this study, we first compared assessments of the quality of care for diabetes that result from using either administrative data only, medical record data only, or the combination of both sources. We next assessed whether quality measurement using administrative data without medical records data is likely to systematically underdetect quality indicators for certain groups of patients based on age, sex, race, or income. Finally, because we were also concerned that there may be systematic differences in administrative processing of claims among clinics (for example, some clinics may submit less complete administrative data to the health plans), we explored whether such differences could be explained by the clinic where a patient receives care.

METHODS

Study Population

We collaborated with three health plans in Minnesota to examine quality of care for patients with diabetes. Using administrative data, we identified all patients aged 18 years or older with type 1 or 2 diabetes mellitus as defined by having two or more encounters listing an ICD-9-CM code for diabetes mellitus (250.xx), diabetic polyneuropathy (357.xx), diabetic retinopathy (362.0-362.0x), or diabetic cataract (366.41) during the 18-month period from July 1, 1997, through December 31, 1998. We based our identification on primary and secondary diagnoses for outpatient encounters and on primary diagnosis for inpatient encounters. We excluded any patients with a code for end-stage renal disease, patients with a break in enrollment of more than 30 days, and women who were pregnant during the 18-month period because they may have had gestational diabetes and because their standard of care may differ from other patients. The sample included 1,335 patients from the three health plans (n = 792 from Plan A, n = 250 from Plan B, and n = 293 from Plan C). The study protocol was approved by the Harvard Medical School Committee on Human Studies and by participating health plans.

Data Collection

In addition to administrative data supplied by the health plans, trained abstractors collected data from the medical records of the physician providing most or all of the diabetes care for each patient in the sample. We used administrative encounter data to select the physician with whom the patient had the most outpatient visits with a diagnosis code for diabetes during the 18month period. For cases in which a patient had the same number of such visits to two or more physicians, we selected the physician with the greatest number of total (diabetes and nondiabetes) visits. If more than one physician provided the same number of encounters, we selected the primary care physician, or if no primary care physician provided care, we selected the physician most likely to provide diabetes care based on their speciality (for example, if a patient saw an oncologist and an endocrinologist, we selected the endocrinologist). Medical records were not abstracted for 183 patients because they could not be located or patients did not consent; those whose records were abstracted were more often from Plan B (p<.001) and more often older (p<.001). We limited analyses to the 1,152 patients for whom we had both administrative data and medical records data. As part of a larger study, we also surveyed patients in the sample to collect information about their demographic characteristics and experiences with care. The survey response rate was 65.5 percent.