What happens when hospital-based skilled nursing facilities close? A propensity score analysis

Health Services Research, Dec, 2005 by Chapin White, Susanne Seagrave

We include the universe of acute care hospitals in the U.S. that provided services to Medicare beneficiaries throughout the period from 1997 to 2001 and that hosted an HBSNF in 1997. The analysis excludes hospitals that never hosted an HBSNF and also excludes hospitals that opened an HBSNF after 1997. (As Dalton and Howard [2002] have shown, HBSNF openings after 1997 were quite rare; excluding newly opened HBSNFs does not greatly limit the analysis.) Using a propensity score approach, we group hospitals into five strata according to the predicted probability of closing their HBSNF by 2001. Within each propensity score group, we calculate a difference-in-differences estimate of the effect of HBSNF closure. The difference-in-differences estimate equals the change from 1997 to 2001 among those hospitals that closed their SNF minus the change among hospitals that kept their SNF open. All hospital-level analyses are weighted by the number of fee-for-service Medicare hospital admissions in 1997.

Propensity-score methods have been proposed for use in nonexperimental research designs focused on measuring the causal effects of a binary "treatment" (Rosenbaum and Rubin 1984). Examples of such treatments include coronary artery bypass surgery, substance abuse counseling, or, in our case, closure of an HBSNF (Mojtabai and Graff Zivin 2003). Observations are assigned a propensity score, which reflects the likelihood of receiving the treatment as predicted using observed characteristics. Observations are then grouped based on these propensity scores, typically into five groups, and treatment effects are measured within groups.

Propensity-score methods are similar to standard OLS, in that both are potentially subject to bias because of unobserved factors. Propensity-score methods have several advantages over standard OLS, though. First, grouping observations based on the predicted likelihood of treatment forces the researcher to confirm that there is some degree of "overlap" between treatment and nontreatment observations, that is, that there are at least some treatment and nontreatment observations with similar observed characteristics. Second, measuring treatment effects within propensity groups allows treatment effects to vary across groups in a flexible way. Third, identifying characteristics associated with receiving the treatment can be of interest in itself.

We conduct a hospital-level difference-in-differences analysis, with hospitals grouped by propensity score. Within groups, we compare hospitals that closed their HBSNFs (the treatment group) with those that did not. We chose this approach for the following reasons. First, a difference-in-differences approach allows for underlying changes over time in patterns of care and outcomes that may be unrelated to HBSNF closures. For example, home health care utilization and spending dropped substantially over the period we examine. The difference-in-differences approach allows us to identify whether HBSNF closures were associated with any differential change in home health utilization. Second, HBSNFs draw their patients almost exclusively from their host hospital's discharges. Whether a patient uses an HBSNF is, therefore, determined largely by whether the admitting hospital hosts an SNF. A hospital-level comparison of closers versus nonclosers, therefore, offers a sharp contrast in practice patterns. Third, we chose not to conduct a simple individual-level analysis comparing HBSNF users to nonusers because HBSNF use is likely to be strongly related to an individual's clinical characteristics, both observed and unobserved. We hypothesize that patient-level selection concerns are mitigated in the hospital-level analysis. Fourth, by measuring changes over time at the hospital level, we control for stable hospital-level characteristics, which include local practice patterns, hospital quality, and patient populations. Fifth, the propensity score approach allows us to group similar hospitals and, within these groups, compare closers to nonclosers. Compared with a simple pooled difference-in-differences model, the propensity score approach is more robust to variation among hospitals in observed characteristics. The propensity score approach also allows identification of differential effects of HBSNF closure across different types of hospitals.