When asking respondents to select items from a list, make sure the answer choices do not overlap.
Consider the following question:
Where have you seen or heard information about how to improve your credit score? Please check all that apply.
__ Website
__ Magazine
__ Television
__ Radio
__ Newspaper
__ News
__ Other (Please describe: _______________________)
The problem with this question is that “News” overlaps with other items on the list. You can receive news from websites, magazines, television, radio, and newspapers. Some respondents who saw a news story in a magazine, for example, might check magazine but not news, or vice versa. These respondents often believe they have already given a sufficient answer to the question by selecting just one item. As a result, the data you get back from questions with overlapping categories is flawed.
Below is an improved version of the question. Removing “News” eliminates overlapping categories.
Improved Question:
Where have you seen or heard information about how to improve your credit score? Please check all that apply.
__ Website
__ Magazine
__ Television
__ Radio
__ Newspaper
__ Another person (friends, coworkers, or family members)
__ Other (Please describe _______________________)
I also wondered whether the original list included all possible answer choices. Some respondents might have heard about how to improve their credit score from other people, so I added, “Another person (friends, coworkers, or family members)” as an answer choice.
The challenge for questionnaire writers is to come up with a comprehensive list of answer choices that do not overlap. This is why you should do qualitative research before writing the questionnaire and pretest the questionnaire before finalizing how questions are constructed and phrased to make sure you get it right.
