Chapter 8: Data Collection Methods: Survey Research
While posing clear and understandable questions in your survey is certainly important, so is providing respondents with unambiguous response options. Response options are the potential answers that you provide to the people taking your survey. Generally, respondents will be asked to choose a single (or best) response to each question you pose, though certainly it makes sense in some cases to instruct respondents to choose multiple response options. One caution to keep in mind when accepting multiple responses to a single question, however, is that doing so may add complexity when it comes to tallying and analyzing your survey results.
Offering response options assumes that your questions will be closed-ended questions. In a quantitative written survey, which is the type of survey we have been discussing here, chances are good that most, if not all, your questions will be closed-ended. This means that you, the researcher, will provide respondents with a limited set of options for their responses. To write an effective closed-ended question, there are a couple of guidelines worth following. First, be sure that your response options are mutually exclusive. For example, look at the age categories depicted in Examples 1 & 2.
Mutually Exclusive Example 1
How old are you?
- 59 or older
Mutually Exclusive Example 2
How old are you?
- 60 or older
What do you notice in Example #1? If I am 39 years old, do I choose option 2 or option 3? In other words, the options are not mutually exclusive. If you look at Example #2, you will see that the options are now mutually exclusive. Another thing to remember is to keep the span of numbers the same for each category. For example, with the exception of the last category, all other categories should represent the same number of years. In Example #2, all choices represent a span of 10 years.
Surveys need not be limited to closed-ended questions. Sometimes survey researchers include open- ended questions in their survey instruments as a way to gather additional details from respondents. An open-ended question does not include response options. Rather, respondents are asked to reply to the question in their own way, using their own words. These questions are generally used to find out more about a survey participant’s experiences or feelings about whatever they are being asked to report in the survey. If, for example, a survey includes closed-ended questions asking respondents to report on their level of physical activity on a weekly basis, an open-ended question could ask respondents what physical activities they participate in. While responses to such questions may also be captured using a closed- ended format, allowing participants to share some of their responses in their own words can make the experience of completing the survey more satisfying to respondents and can also reveal new motivations or explanations that had not occurred to the researcher.
Other things to avoid when it comes to response options include fence-sitting and floating. Fence- sitters are respondents who choose neutral response options, even if they have an opinion. This can occur if respondents are given, e.g., five rank-ordered response options, such as strongly agree, agree, no opinion, disagree, and strongly disagree. Some people will be drawn to respond “no opinion” even if they have an opinion, particularly if their true opinion is the non-socially desirable opinion. Floaters, on the other hand, are those that choose a substantive answer to a question when really they do not understand the question or do not have an opinion. If a respondent is only given four rank-ordered response options, such as strongly agree, agree, disagree, and strongly disagree, those who have no opinion have no choice but to select a response that suggests they have an opinion.
As you can see, floating is the flip side of fence-sitting. Thus, the solution to one problem is often the cause of the other. How you decide which approach to take depends on the goals of your research. Sometimes researchers actually want to learn something about people who claim to have no opinion. In this case, allowing for fence-sitting would be necessary. Other times researchers feel confident all respondents will be familiar with every topic in their survey. In this case, perhaps it is acceptable to force respondents to choose an opinion. There is no always-correct solution to either problem. Table 8.2 provides examples of the various types of research questions, including their content, structure and wording.
Table 8.2 Survey question examples: Content, Structure and Wording
|Open Ended Question||Closed Ended Version||Type of Closed Ended Question|
|1. What do you like most about your job?||Rate the following statement: I like my job 1 strongly agree2 agree
||Rating Scale – Likert|
|2. What is your income?||How much did you earn in 2018?
What was your income for 2018?
|3. What do you think of the Vancouver Police Department?||How would you rate the Vancouver Police Department on the following dimensions? Fair______ unfair
Respectful ______ disrespectful Knowledgeable ______ Lacking Knowledge
Question Wording Examples
|Question||Critique||Type of issue|
|1. Agree or Disagree: Hookers on the streets are a threat to public safety||The use of the term “hookers” is inflammatory and indicates to the respondent what the “expected” response should be.||Loaded terms|
|2. Agree or Disagree: I support the legalization of street drugs and their taxation||This question asks two questions (legalization and taxation). Respondents who feel differently about these issues will have difficulty answering the question. It also is ambiguous – what is a street drug, and what is meant by legalization and taxation? Not everyone knows what legalization is, and taxation may be applied in many ways and used in different ways.||Double Barreled Ambiguous language|
|3. Agree or Disagree: I believe that the VPD should increase the number of NCOs by increasing the number of Cpls.||This question assumes you know what VPD, NCO and Cpls stand for. It also asks two questions. You may believe the number of NCOs should increase but not by increasing the number of Cpls.||Use of Acronyms Double Barreled|
|4. Agree or Disagree: Canada has good immigration policies||This question could be answered by anyone, but does not indicate whether they have any knowledge of the topic. This might be a good question after asking a series of question to determine that the person has knowledge first. It is also somewhat ambiguous – what does “good” mean in this context?||Ambiguous language|