Exploratory factor analyses of the CAHPS® Hospital pilot survey responses across and within medical, Surgical, and obstetric services

Health Services Research, Dec, 2005 by A. James O'Malley, Alan M. Zaslavsky, Ron D. Hays, Kimberly A. Hepner, San Keller, Paul D. Cleary

The Consumer Assessments of Healthcare Providers and Systems (CAHPS[R]) Hospital survey was designed to enable patients, physicians, and payers to compare quality among hospitals and to facilitate quality improvement in hospitals. Efficient reporting of information about the quality of care and service at hospitals requires identifying the important dimensions of hospital care and reliably evaluating hospitals' performance in each dimension. To do so, surveys that assess the experiences of patients recently discharged from acute care hospitals are analyzed to characterize the dimensions which best summarize variation in patient responses. Such analyses are often performed using factor analyses; we argue that it is important that these analyses also be conducted at the hospital level and show that different results can be obtained from hospital- and patient-level analyses.

Initially, items for the CAHPS Hospital survey instrument were designed to address the Institute of Medicine's (IOM) domains of patient-centered care: respect for patient's values; preferences and expressed needs; coordination and integration of care; information, communication, and education; physical comfort; emotional support; involvement of family and friends; transition and continuity; and access to care (Goldstein et al. 2005). The cognitive testing phase of survey development eliminated many items and some domains (Levine, Fowler, and Brown 2005) for concepts that were too complicated, abstract, or subjective to support the development of unambiguous, easily understood items. This necessitated an exploratory approach to the factor analysis in order to identify composite items. The pilot survey included 33 questions to report on hospital care quality and four questions that elicited global ratings of care. The survey also included two open-ended questions concerning the hospital stay, 16 screener items that allowed respondents to skip the subsequent report items if they were ineligible to answer them, and 11 items on patient characteristics (to support case-mix adjustment models).

The 33 report questions were designed to collect information that is important to patients and discriminates among hospitals (Goldstein et al. 2005). Reports of CAHPS data typically present summaries or composites that average responses within groups of items determined by content relationships and/or empirical associations (AHCPR 1999; Hays et al. 1999; Zaslavsky et al. 2000; Bender and Garfinkel 2001; Marshall et al. 2001; Hargraves, Hays, and Cleary 2003). If several items are strongly associated and substantively similar, they can be combined to reduce the number of scores one must examine to understand variations in quality. Summary measures facilitate interpretation and use of data by consumers, clinicians, and others interested in monitoring and making decisions about health care (Hibbard et al. 2002; Hibbard, Stockard, and Tusler 2003). Factor analysis can be used to identify groups of empirically related items that are the product of the same latent variable.

One can assess patterns of associations at the individual level (identifying items that are scored similarly by patients) or hospital level (identifying items on which hospitals have similar scores). Each type of association can be informative for different purposes (Zaslavsky et al. 2000). To understand individual variations within hospitals (e.g., do men and women report different health care experiences?) patient-level associations are of greatest interest. On the other hand, to assess the relative performance of different hospitals, hospital-level associations are more relevant.

Comparisons of individual- and hospital-level analyses can also help address methodological issues in surveys. Correlations at the individual level might reflect individual patients' response tendencies (e.g., acquiescence bias), the common effects of patient characteristics on several kinds of experiences (e.g., cultural background), or response patterns related to the way the questions are presented or organized (e.g., context effects). A hospital-level analysis that removes the component of correlation because of patterns of patient responses that have nothing to do with quality of care might better reflect associations among aspects of hospital quality. Furthermore, the patient-level correlation analysis is confounded by different nonresponse patterns for the different items (because of skip instructions), while hospital-level mean scores can be calculated and correlated, even for pairs of items that are answered by nonoverlapping sets of patients.

We estimated hospital-level associations by fitting a two-level multivariate model to the hospital mean scores, in which these scores were modeled as estimates, subject to error because of variation of individual patients, of the long-term population means for the corresponding hospitals.

METHODS

Data

Following the removal of 333 respondents with undetermined service or hospital affiliation, 19,720 patients in 132 hospitals were available for this analysis (Levine, Fowler, and Brown 2005). Two atypical hospitals with very few respondents were excluded, leaving 130 hospitals and 19,683 survey responses. We classified patients by the service (medical, surgical, or obstetric) in which they were treated using their diagnostic-related group (DRG) codes. Of the 130 hospitals that provided both surgery and medical services (7,904 and 7,183 survey responses, respectively), only 102 provided obstetrics care (4,596 responses). Thus, there were a total of 362 hospital-service units. The number of surgical, obstetric, and medical respondents per hospital had skewed distributions, with means (SDs) of 61 (52), 45 (42), and 55 (45), respectively.