50 Collecting Primary Data
Surveys and Interviews
While surveys tend to be analyzed both quantitatively and qualitatively, their method is unique and popular enough to warrant a description of their own. Bhattacherjee (2012) defines survey research as a “method involving the use of standardized questionnaires or interviews to collect data about people and their preferences, thoughts, and behaviors in a systematic manner” (p. 73). It is a remarkable method for understanding large populations while situating individual preferences, thoughts, and behaviors as the unit of analysis. (For a full list of the positives and methods on how to construct surveys, check out Bhattacherjee).
Survey research is generally split into two categories: questionnaire and interview. Questionnaire surveys consist of a list of questions constructed to create standardized responses. Surveys most commonly come in three forms: mail surveys, group-administered, and online. Mail surveys are a more uncommon breed these days (besides the census), and is send out to the addresses of the target samples. Group administered surveys involve getting people together at a common place and time to complete the questionnaire (e.g., students in a classroom). Online surveys, typically recruit samples over email or via ads on the internet, allow individuals the fill out the form on the web. Common online survey tools include Qualtrics, Google survey, SurveyMonkey, QuestionPro, and Yesinsights (full citations available in Resources section). The potential limitations needs to be weighed against the strengths when deciding to use online survey (see Sue & Ritter, 2012 for a thorough discussion).
Interview surveys are directly administered by the researcher to participants, using any configuration of open and closed-ended questions. As a result, how you construct the survey questions can dramatically affect response rate and quality. If your study wishes to provide a more descriptive and holistic account of a few participants (ie. interpretive/qualitative), then a smaller amount of subjects will be required, and more open-ended questions would be included. However, if you aim at being able to generalize your results to the larger population, the questionnaire would utilize more closed-ended questions.
Survey research generally also has many significant biases including sampling and non-response biases (explored earlier). In addition, social desirability bias –the tendency not to disclose “anti-social” information about oneself and others –is common in primary data collection. Similarly, recall bias –the bias that results when answers to certain questions are hard to recall (e.g., “describe the last lunch you ate in preschool”), is a form of measurement error, which undermines data quality. As a researcher, it is important that you think through the different biases and errors that can creep into your research and devise strategies to mitigate them. The following table outlines some common problems in primary research and offers some suggestions for overcoming them.
| Table 8.2 - Common Problems with Primary Data Collection | |
|---|---|
| Problem | Description |
| Interviewing Flaws | Interviewing is a complicated and sensitive task. It requires the researcher to be patient and careful about not imposing their view, but direct enough to ensure that the interviewee stays on topic. Interviews without much skill or preparation can botch the value of primary data collection, resulting in a superficial conversation of little value to research. To correct this, carefully prepare for your interviews and ensure you have the energy to conduct them. The following article repeats the important aspects of interviewing to keep in mind: Brinkmann, S., & Ebooks Corporation. (2013). Qualitative interviewing. Oxford University Press. https://doi.org/10.1093/acprof:osobl/9780199861392.001.0001 |
| Information is Incomplete | Besides inaccuracies, there are two major kinds of incomplete information you can experience in your data: with respect to your research question or within the story of the participant. Either by accident or intentionally, the story of your participant can reveal itself as full of important holes. On the other hand, the interview can be brilliant, but move away from your research question, providing data that cannot directly answer your research question. This is always somewhat the case in qualitative research. Our lives are messy, and our reflection of it is bound to have gaps, fragmentary and irrelevant information. All of our information is thus in all respects, ‘incomplete.’ Do not be discouraged by gaps in the information, use what context you can, and be sensitive to the issues that are missed.
Sometimes the gap in your data can become the subject of your exploration. In Alexander Wilson’s (2021) research on Uber, he came to realize that a key thing ignored by media articles representing Uber’s arrival was the condition of Uber and taxi workers. As a consequence, this key silence in the discourse became an important issue for him to cover, where he asked: why was it ignored? If they are occasionally referred to, what might be missing about the Uber labour story? How does Uber stand to benefit from concealing their work situation from public discourse? Or is it because the public is uninterested? |
| Information is Inaccurate | As opposed to merely lacking the data, inaccurate information can trick the researcher into believing it’s accuracy and then thematizing it into your argument. Do not be embarrassed should this be the case. Social research requires much trust between all parties in order to gain accurate information. Because the nature of the research often involves investigating the lives of others as an outsider, the researcher is put in the uncomfortable position of being almost completely reliant on the information of their trusted confidante. There are, however, some methods to determine the validity of your primary method.
The researcher can cross-reference the information with other literature, generally available information, and interviews. Firstly, the literature review is supposed to help hone your understanding of the subject matter, making you able to spot inconsistencies or inaccuracies with regard to your subject matter. If you encounter new information, then do not be afraid to return to the literature in search of information that is similar to what you have found in your research. As discussed in chapter 4, the literature is iterative, and should be used to pay attention to works related to yours. If no work is, do not quickly assume that it is because it has found something everyone missed, what it might have found is an outright lie. To determine this, you will need to go beyond the literature. If you are investigating the labour conditions in Starbucks, then there are likely ample public documents available to determine some of the background information on the labour conditions. You can use this information to determine whether there is at least some alignment between the report of your participant and the reports of others. How labourers feel about those conditions, however, will be subjective and is not to be dismissed by concerns of ‘accuracy.’ But subjective is not to say that those feelings are not shared. The key task of a primary data collector is to determine trends and patterns that emerge in the messy reality of everyday life. This means cross-referencing the information you hear from one participant with as many others as possible. Doing this will allow you to sort through the statements that attain a common sense, a common understanding, and be wary of (without ignoring) those statements that cannot be confirmed by others. |
| Questionnaires Neglect the key issues | It may be that your instruments are flawed at addressing the major issues of the topic. This is not necessarily because of a flaw in their design (i.e., that the questions are not reliable), but may be because the intentions (the information your striving afterwards) has changed as you’ve begun interviewing and getting data. You may have designed your questions to initially focus upon how immigrants struggle in the Canadian workplace but then find in the interview process that it is not so much immigrants who struggle as those who have speech impediments/difference (accent, syntax, diction). Your research may therefore pivot to capturing that key issue.
If your questionnaires do change throughout your research journey, it is helpful to include multiple copies of your questionnaires. Note the key changes between the drafts of your questionnaires, and state which participants had which questionnaires. If you have to revise the questionnaire entirely, then you should also include a detailed argument about this transition in your methods. |
| Interviews or Informants are biased/bored | Suppose your informant has no interest in your research. Their eyes glaze over as you ask the questions, they offer evasive or abrupt answers, and they seem all but ready to sprint out of the room. I think the best way to address this is to just respond to them honestly: “am I boring you?” And to remind them that this research is not coercive, they are welcome to leave. Bored informants are unlikely to provide valuable information anyways.
But of course your questionnaires should not be designed to be boring (though this is sometimes unavoidably the result). Try to make them as quick as possible, and if information can be gotten through another avenue, try not to burden your informants with unnecessary questions. For surveys, try to provide a mix of short answer questions that are engaging and quick MC answers that can be quickly fulfilled. Do not proceed with a lengthy list of long-answer questions. Bias will also pose itself as a significant problem in your research. Bias, the desire of your informant to provide you information which positively or negatively frames the topic, exists in all social research. Since your research topic involves the participant (hence their knowledge of it), it is likely that your participant will feel some level of gratitude or resentment for the thing you discuss. It is not your task as a researcher to disregard, ignore, or take for granted this bias, but to make sense of it. You will want to think about why this participant wishes to present you this almost pastoral account of climate movements, and why the next is entirely cynical of them. The bias of each account will help you to unravel their motivations alongside their experiential description of the topic, providing you (the social researcher) a better sense of not just how they are involved in this given issue, but why |
| Assessment fatigue | Do not over-tire yourself! Research is hard work (we know), and it is important to take regular rests. As argued in the self-care chapter, this rest will not only improve your well-being, but also the quality of your work. Researchers who attempt to do all their interviews in one day will find themselves getting sloppy on the third go, beginning to lose motivation as the exhaustion increases. It is vital, particularly for interview research, that you show more or as much interest in the subject as they do. If they get the sense that you are not even interested in this topic, they will begin to seriously doubt their involvement. In addition, taking breaks in between assessments will help you to reflect on what was working and what was not, providing you an interim chance to refine your tools for future investigations.
But assessment fatigue does not only apply to you. It also applies to your participants, who may simply get exhausted in a 1-hour interview. Make sure they are motivated and rested before beginning your research, and try to keep all your work concise and clear. This clarity should extend as much to the value of your work as to the clarity of information. The more motivated your participant is in the outcomes of your research, the more likely they will attempt to articulate and energetically defend their viewpoint on it. |
References
Bhattacherjee, A. (2012). Social Science Research: Principles, Methods, and Practices https://scholarcommons.usf.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1002&context=oa_textbooks
Sue, V. M., & Ritter, L. A. (2012). Conducting online surveys. Sage.
A research method involving sample data collected via questionnaires or interviews in a systematic manner.
the tendency not to disclose “anti-social” information about oneself and others –is common in primary data collection. Similarly, recall bias –the bias that results when answers to certain questions are hard to recall
the bias that results when answers to certain questions are hard to recall
