Community Health Improvement Approaches: Accounting for the Relative Lack of Impact

Health Services Research, August, 2000

The articles in this issue speak to the broad field of health services research. They range from the opening article, which evaluates the impact of community health improvement interventions, to the concluding article, which identifies a more restrictive set of criteria for determining psychiatric emergencies. In between, readers will learn about the effect of capitated and resource-based individual physician payment on costs of care; the use of home care as the most cost-effective post-hospital treatment modality for selected groups of patients; the cost of workplace-related physical assaults and the identification of subgroups for targeting prevention efforts; the use of randomized clinical trials in continuous quality improvement research; and the development of new and creative measures for determining the "managedness" and covered benefits of health plans. Given the limited space available, we choose to highlight the opening study, by Wagner, Wickizer, Cheadle, et al., which represents the first to be pu blished in our section on "Community Health Improvement Research," supported by a grant from the W. K. Kellogg Foundation.

Using a randomized design together with matched controls in four communities, Wagner et al. find little evidence that positive changes in health outcomes result from targeted interventions. Their results are consistent with other recent evaluations of community health improvement initiatives (cf. Luepker, Murray, Jacobs, et al. 1994; Carleton, Lasater, Assaf, et al. 1995; Green and Kreuter 1993; and Susser 1995). Why is this? Is the community health coalition approach to health improvement simply an ineffective model for addressing difficult community health problems?

First, Wagner and colleagues note the usual "suspects" for lack of results, namely, (1) an insufficient period of time to observe results-in this case, four years; (2) lack of statistical power; and (3) measurement error associated with self-reported behaviors. But it is reasonable to expect that some positive changes might have occurred by the end of the fourth year in some of the targeted health behaviors, for example, teenage pregnancy and injury prevention. As for level of statistical power, the authors note that the differences between the experimental and control communities were both positive and negative suggesting that they did not miss "small, but unintended effects." Measurement error did not appear to differ between the experimental and control communities, so this alone cannot account for lack of differences. A fourth possibility, however, might be considered: that the 11 control communities--the original applicants--may have gone ahead anyway to implement various initiatives designed to address the problems of interest even though they did not receive funding from the Henry J. Kaiser Family Foundation. It is important to recognize that one is dealing with a highly motivated self-selected group of applicants who, despite the lack of funding or technical assistance provided by the Foundation, nonetheless might have obtained other sources of support for their efforts.

The above aside, the authors draw on their own experience and that of others (cf. Green and Kreuter 1993; McKinley 1996; Murray 1995; and Dusenbury and Falco 1995) to suggest three major underlying "hypotheses" for the lack of results. First, they suggest that the interventions were too weak to affect individual behavior. Second, they suggest that the interventions were too limited to reach broad segments of the population at risk--the intervention was "underexposed." Third, they suggest that some specific intervention components, such as parenting classes, are relatively untested interventions to be used as part of an overall strategy. The net conclusion appears to be that stronger, bigger, better interventions are needed.

We suggest two additional considerations: (1) the need for more refined "theories of action" (Patton 1978) regarding how community health interventions ought to improve outcomes for a specific problem or condition, and (2) the need for mid-level theories and measurement of the organizational behavior of coalitions to obtain a fuller understanding of the implementation issues involved.

The Kaiser Family Foundation Community Health Promotion Grants Program was based on a "trickle-down" macro theory of action. The logic model of cause and effect might be diagrammed as follows:

Activated community [right arrow] (produces) interventions with broad

population exposure [right arrow] (leading to) changes in community

norms and environment [right arrow] (resulting in) changes in individual behavior.

This model was assumed to apply "across the board" to five different conditions: substance abuse, teenage pregnancy, cancer care, cardiovascular health, and senior-related injuries. We suggest that an additional reason for lack of impact may have been the use of such a broad global model. Approaches for improving community health may need to be more contingent on and "tailored" to the specific problem addressed. For example, one might even begin with the assumption that each of the five conditions has a different logic model or causal path. For example, some of the conditions, such as cardiovascular disease, cancer, and, to a lesser extent, substance abuse, have strong biological and genetic components relative to teenage pregnancy or, obviously, household injuries. These conditions are also associated with different age groups and differ in the extent to which the problem is under the influence of the individual versus the external environment. Even though it is efficient, parsimonious, and elegant to const ruct large-scale, overarching frameworks or models on which to base and evaluate community health improvement initiatives, the idiosyncrasies, nuances, and complexities of the problems addressed simply appear to elude or defy such "logical" models.