Short lists, long-term memory, and vice-versa; designing survey questions about behavior

American Demographics, Feb, 1996 by Arlene Fink

Surveys can gather information about the demographics of your customer base, their knowledge, and their attitudes. But if you need to know whether your customers or potential customers subscribe to a national daily newspaper, you need to survey them about their behavior. Questions about behavior typically take forms like the ones below:

* Which of the following magazines do you read at least once a month?

* How often do you exercise?

* Did you vote in the last election?

* How frequently do you go to church?

* In the past 3 years, how often did you apply for federal grants?

All questions about behavior concern time, duration, or frequency. The preceding questions specify time periods: at least once a month, how often, the last election, and within the past 3 years.

When writing questions that specify frequency, choose time periods that meet the survey's needs and that make sense to the respondent. You can obtain reliable information about events and activities that occurred years ago if they are important. People remember births, deaths, marriage, divorce, buying their first house, and so on. They also remember what they were doing at the time of great historical events like wars and assassinations, and during natural disasters like fires and floods. For most other events, do not expect people to remember past one year's time. You can use yearly periods, however, primarily for summary information:

* About how much money did you spend on vacations away from home in the past year?

* In the past 12 months, how often did you go for bicycle rides of 5 or more miles?

With this in mind, avoid asking respondents for specific information over a long period of time. The reliability of your respondents' answers may suffer, as in the following example:

Poor question: In the past year, which of the following items of children's clothing did you buy from Outdoors Clothing Company?

Unless respondents have bought very few items of clothes for their children, or buy exclusively from Outdoors Clothing Company, they might very easily forget.

Better question: In the past 3 months, which of the following items of children's clothes did you buy from Outdoors Clothing Company?

Likewise, very short periods of time can also adversely affect the accuracy or validity of an answer. Respondents may feel the need to compensate for the inadequate time span, as below:

Poor question: In the past week, how often did you buy coffee, tea, or bottled water?

The respondent may not have purchased any of the items during the past week. Nevertheless, because they do purchase and use them regularly, they may overreport by indicating a purchase in the past week that really occurred the week before.

Better question: In the past 3 weeks, how often did you buy coffee, tea, or bottled water?

Using a List to Help Respondents Remember Their Actions

Because questions about behavior have a time element, you are dependent on respondents' ability to recall. To jog the memory, use lists of possible items, brands, or activities.

A list is useful for reminding respondents of events they may have forgotten. To be maximally helpful, the list should be as inclusive as possible.

An inclusive list can go on for many pages, however, which might confuse and bore the respondent. One way to get around the long-list problem is to divide a question into its component parts.

Dividing the Question: How to Avoid Lone Lists and Still Get the Behaviors you Need

By presenting a list to your respondents, you restrict them to the categories included in the survey, and this may result in a loss of information. To avoid losing information, you can add an "other" category in which respondents may write in items not on the list. By adding this option, you are including an open question (one which asks respondents for their answers in their own words). You must be prepared to interpret and catalogue these "other" answers.

This question is about your leisure activities. Since last January, did you do any of these activities? Check yes or no for each.

Yes (1) No (2) Go to a movie 1 2 Eat out for pleasure 1 2 Window shop 1 2 Go to the theater 1 2 Read for pleasure 1 2 Go for a hike 1 2

1. Since last January, have you participated in any of the following activities? Answer yes or no for each.

Yes (1) No (2) Shopping for pleasure 1 2 yes, answer Question 3 Religious groups 1 2 answer Question 5 2. Since last January, did you play any of the following sports? Answer yes or no for each.

Yes (1) No (2) Basketball 1 2 Baseball 1 2 Football 1 2