Beneficiary Survey-Based Feedback on New Medicare Informational Materials

Health Care Financing Review, Fall, 2001 by Lauren McCormack, A., Steven Garfinkel, A., Judith Hibbard, H., Kerry Kilpatrick, E., William D. Kalsbeek

Self-reported understanding of how to read the bar and star charts was relatively high in that over 80 percent of respondents indicated that they could tell which health plans were rated the best from reading the charts. However, this result may be somewhat overestimated due to social desirability bias, in which respondents give the answer they believe an interviewer wants to hear. About 95 percent of beneficiaries correctly reported that the information in the survey report told them at least a little about how quality of care differs across health plans. Just over 90 percent of survey respondents reported that the plans were rated on performance measures they care about. There was no strong preference for presenting the information using the star chart versus the bar graphs in either population. Nearly all beneficiaries said they kept a copy of the CAHPS[R] survey report to share with others or refer to later.

Multivariate Analysis

Perceived Usefulness of the Materials

We asked control and treatment group members to rate the usefulness of the informational materials they received about the Medicare program during the last 6 months. Because most control and treatment group members received at least some information, we were able to make comparisons between all of the groups. However, because not all beneficiaries received information, we restricted our analysis to those who did receive the materials. (2) This included 81 percent of new and 63 percent of experienced beneficiary respondents. (3) Substantive findings did not differ when the models were run with and without those who did not receive any information during the last 6 months.

Using logistic regression analysis, we found that beneficiaries in all three treatment groups were significantly more likely to find the information they received useful compared with control group members who only received information outside the study (Table 4). This finding was further supported by predicted probabilities which showed that 61 percent of experienced beneficiary control group members found the materials useful in contrast to between 71 and 75 percent of treatment group members (Table 5). The pattern was similar for new beneficiaries, except a higher proportion of control group members (72 percent) found the information they received useful.

Among experienced beneficiaries, those who were age 75 or over, had higher incomes, and had 3 or more physician visits in the last 3 months all found the information less useful (Table 4). Those who reported being exposed to quality of care plan performance information were more likely to find the materials useful. Among new beneficiaries, male respondents and those with individually-purchased supplemental insurance, i.e., medigap, were more likely to find the materials useful than those with a regular source of medical care who found the materials less useful. Education level was not a significant variable in either the new or experienced beneficiary models.

Four variables were significant across the new and experienced beneficiary logit models. The greater a beneficiaries' knowledge of the Medicare program was (as measured by a 15-item knowledge index developed previously as part of the study [McCormack et al., forthcoming 2002]), the more likely they were to find the materials useful. Beneficiary knowledge was the factor that had the greatest effect on the probability of finding the materials useful as indicated by the odds ratio (not shown). For a 1-percent increase in beneficiary knowledge, the weighted odds of finding the materials useful was 6 percent for experienced beneficiaries and 32 percent for new beneficiaries.