Are quality improvement messages registering?

Health Services Research, April, 2005 by Ethan A. Halm, Albert L. Siu

More often than not, commentaries highlight the findings of studies with important new positive findings. Instead, we highlight three papers that we believe are important and have largely negative or null findings. We are all familiar with efforts, particularly in the last decade or so that have been undertaken to improve the quality and efficiency of medical care. Changing the behavior of physicians and other health care workers is at the crux of many of these efforts. Some progress has been made in improving quality, but three papers in this issue deal with various aspects of these efforts, and they highlight the difficulty of communicating and bringing about some of the changes and improvements that many would agree need to occur.

The papers report on three commonly used approaches for changing clinical practice. Clinical pathways and computerized decision support have been advocated to improve adherence with practice guidelines, and Quality Improvement Organizations (QIOs) have been charged with assisting providers in quality improvement (QI) activities. Tierney et al. (2005) report on a randomized trial of just-in-time, tailored guideline-based suggestions via an electronic medical record for clinicians for improving outpatient management of asthma or chronic obstructive pulmonary disease. They found that the intervention had no effect on adherence with recommendations, medication compliance, quality of life, satisfaction, or emergency room/hospital visits.

Clinical pathways, structured care plans that note the essential elements of care by hospital day, have been widely used by hospitals in efforts to improve the quality and efficiency of care. In a second paper, Dy et al. (2005) report that only seven of 26 pathways used at a large academic medical center had the desired impact. They report on a qualitative analysis to describe characteristics that differentiate effective from ineffective pathways. They found that many of these care plans were unused. Even among the few that did appear to have an impact, their effectiveness may not have been related to their actual use.

Bradley et al. (2005) in another paper surveyed hospital quality management directors about their interactions with QIOs, formerly known as Professional Review Organizations (PROs), that are contracted by the Centers for Medicare & Medicaid Services (CMS) to promote quality of care in the Medicare program. They found that hospital quality management directors had largely favorable views of QIOs, but the findings also indicate that the interactions may have been fairly limited and superficial. A fifth of the sample could not name anyone at the QIO involved in a major initiative. About a quarter of the sample could recall no contact with the QIO on the initiative, and half of those who could recall no contact also desired no contact!

Taken together, these papers indicate that computerized suggestions to improve care are often ignored, that care plans to improve hospital care are often ineffective and/or go unused, and that outside technical assistance to improve quality is unwanted by many. Other studies have documented benefits of clinical pathways, of computerized reminders, and of Medicare's quality improvement efforts (Campbell et al. 1998; Grol et al. 2003; Jencks et al. 2003). Indeed, a few of the authors' prior studies, using the same techniques, were effective (McDonald et al. 1984). There are several possible explanations for these disappointing findings. Perhaps these three studies differed because of methodological approaches or perhaps design weaknesses. However, we believe they were well conducted and employed appropriate health services research techniques (including randomized trials, qualitative analysis, and survey research), and fatal methodological flaws are not likely to explain differences from previous reports. Perhaps, the particular interventions were not useful in actual clinical situations or were otherwise poorly designed or inadequately implemented. This would be consistent with the finding that clinical pathways seemed to be more effective for procedures with less complex patients Dy et al. (2005). The interventions, however, appear to be a reasonable representation of the state of the art. If anything, they were likely to have been more systematically and faithfully implemented because they were part of a study.

Perhaps the novelty of these interventions simply wore off. It is worth noting that in their previous studies, Tierney and colleagues found that the same computer-based reminder system improved physician adherence to preventative care recommendations (McDonald et al. 1984). Similarly, Dy et al. (2005) indicate that the first critical pathways at their institution were more effective. We have also noted a waning in efficacy over time with similar QI interventions (Horowitz and Chassin 2002; Halm et al. 2004). It is possible the first time a new technology or management tool is deployed, the newness garners the precious attention from the target physicians. Early projects may also be those aimed at "low hanging fruit," the easiest, highest yield problems to solve.