Drafting the Methods Section
While the substance of the methods section will differ by genre and method chosen, basic components can be derived across genres. Methods sections in the social sciences tend to have five sub-components. They must:
- summarize the method used while arguing the value and limitations of the method for your data/context;
- discuss the data of your study (participants, artifacts, academic literature)
- present the and measures used;
- outline the of the data (i.e. how it was collected and ensured of its integrity), and
- Analysis: discuss how you analyzed your data.
Splitting these sections into components, however, should not give you the impression that the methods section is merely a list. The methods section is also an argument (Johnson, 2018), meaning that it aims to convince your reader of the value of your method through a narrative that briefly applies your method to the context of your research. In addition, the method must find a way to align with the research question stated in the introduction. Your methodology should build upon the introduction, justifying that the approach you utilized to investigate the problem is suitable (see Table 7.3.1 for common justifications for some established methods). We will refer back to this key point as we overview each step of the methods section.
A simple summary of your method is a good way to begin the method’s section. The length and depth of this summary depends entirely on the method chosen and the audience it is presented to. If the method is commonly known and used within your field, an overly intricate summary of the method is not necessary; a couple sentences overview will work fine. If the method is not commonly used or entirely new, however, more argument will be required for your audience to understand your choice of method. For instance, grounded theory is a commonly used paradigm in many qualitative social science journals. It is therefore unnecessary to sketch the method’s history in detail. Rather, a simple definition, such as grounded theory being an inductive approach which only designates codes after data collection (and how you enacted in your project) will suffice. On the other hand, newer methods such as linked data (methods based on web technologies such as HTTP, RDF, SPARQL, and URIs to enable semantic connections between various databases) would require a more extensive discussion. In addition, while it is good to offer your own summary of the method, the definition used by another researcher (particularly methodological researchers) is a simple way to align your research with a legitimate approach in your field.
|Table 7.1 - Justifications for Some Common Methods
|Strong for exploratory research (where limited existing work are available), as it is able to be open to new themes and codes that the researcher may not have had in mind before combing through their data
Streamlines and integrates data collection and analysis; flexible to research contexts; produces ‘thick descriptions’ (Charmaz, 2003)
|Able to establish the frequency and meaning of particular words, phrases and themes in a larger corpus; it also helps to establish relationships and patterns between them.
Offers objective, systematic and quantitative description
|Strong method for unpacking the depth of a participants understanding of a situation; explores issues in great detail
May allow respondents to feel more comfortable to share information based on rapport established in the research process
|Strong for community research, online research, and other research that requires careful observation of the interactions between participants
Able to capture the behavior of a participant in their environment as opposed to in the staged interview or experimental setting
|Suitable for understanding a discussion between participants over a specific theme (such as Uber, beard products, or memes)
Appropriate for understanding underlying meanings in sociohistorical contexts and for revealing how language and discourse shape reality
|Systematic Literature Review
|Takes a comprehensive approach to reviewing the literature, drawing on research from multiple theoretical, methodological and disciplinary concerns
Identifies biases and gaps in the literature, and nuances such as whether generalizations can be made across populations, subgroups, settings etc.
|Inexpensive method for aggregating large quantities of data. High level of representability
Flexible across contexts and in design (e.g. online, paper etc.) and can be adapted for anonymity; can overcome interviewer effect.
|Suitable for analyzing social change, life course and understanding the interrelationships between later outcomes
Enables us to make causal inferences through controlling unobserved heterogeneity (Laurie, 2020)
|Quantitative (e.g.,, regression or chi-square analyses)
|Strong method for establishing relationship between various variables; reliable for determining variables that impact our research topic
It also helps us to identify outliers and anomalies
The key point to keep in mind for summarizing your method is to outline its theory insofar as it explains your procedure, that is, discuss the method’s intentions with respect to how you applied it. We will touch on this again in the procedure section.
As you provide an overview of your method, you must also justify it. Justification of your method must appeal to the method’s ethical, practical and factual utility for the project. Ethical justifications are those that argue that the method is best for reducing the harm of research on your participants (and the communities they inhabit). It must also make the point that your method aligns with the principles of your institution’s ethical values.
Factual issues have to do with the value of the data your method is able to gather. It must consider whether the method is appropriate to your research question and topic. If, for instance, the research question is about “Malaysian immigrants’ conception of justice in comparison to American immigrants,” then you may argue that ‘in-depth interviews’ are the only approach deep enough to unravel a person’s “conception of justice.” Arguments for the factual benefits of a method frequently highlight its novelty for studying a particular method. For instance, there could be a lack of discourse analyses of Uber’s advertising materials (the previous research being content analyses). One could then argue that a discourse analysis approach not only has merits in its own right, but it also may be able to discover data which other approaches miss. Finally, you should outline the strengths of the methods in relation to all aspects of your research process (e.g., alignment with your theory, personal values, practicality etc.).
Practical justifications highlight why the method is suitable given logistics, administrative and everyday concerns. For example, if you are interested in studying how Malaysian migrants’ prepare for their transit to America, an ethnographic approach might be tempting. But practically (financially and time-wise), you might not be able to visit Malaysia to observe their preparations. Hence, you might decide that surveys or interviews (while less desirable) are more practical in that instance.
Again, we emphasize the importance of highlighting the limitations of your method. Every method has weaknesses, it is vital that show an awareness of them. It is important, however, not to have the weaknesses outweigh your method’s benefits. Your reader should be able to understand why you choose the methods, so you need to explain how, in spite of the limitation, your method is the most suitable choice. Hence, you need to justify why you decided to choose the methods over others. To help in the weighing of the costs and benefits of different methodological approaches, we have provided below a list of the potential limitations of some of the more common methods used in the social sciences.
|Table 7.2 - Potential Limitations for Common Methods
|Lacks a theoretical base to drive the analysis; requires considerable skills from researchers
Reliability and validity might be questionable due to the lack of standard rules to follow; researcher-driven.
|Can lack theoretical base; simplistic and reductive
subject to increased error, particularly when relational analysis is used to attain a higher level of interpretation (see Elo et al, 2014)
|Not generalizable to the wider population
May be prone to bias: interviewer-effect is always present
|It may take time to establish trust in order to generate honest data
Too little data may lead to false assumptions about behaviour patterns, while large quantities of data may not be processed effectively (Baral et al, n.d, p.2.)
|There are multiple methods for doing discourse analysis, making replicability difficult
It focuses primarily on language which often does not tell the entire story; it might need to be supplemented with another method
|Systematic Literature Review
|The results are only as reliable as the method the original authors uses to evaluate the effect of each study i.e., the results are dependent on the study design, interpretation and analysis of the primary authors.
There is some subjectivity in deciding how to pool studies e.g., determining how to treat discordant studies; subjected to methodological flaws
|Various errors might undermine validity and reliability (e.g., measurement, coverage, non-response sampling errors)
In appropriate for getting in depth understandings
|Selective panel attrition can be problematic
Panel conditioning: interviews from previous waves might influence subsequent waves
|Quantitative (e.g. regression or chi-square analyses)
|There may be variables other than the ones in the study which influence the response variable
It does not allow us to identify cause and effect; correlation does not imply causation.
Once you have provided an overview of your method and its value for your context, the next step is to discuss the data or population that the method will be used upon. Summarizing the data or population of your method often requires answering (Johnson, 2017): (a) how many participants/cases compose your data? (b) what are some of the common or key attributes of your data? And c) how did you select your data/participants?
|Table 7.3 - Johnson's (2017) Three Questions About Participants
|Questions About Participants
|What this Means?
|How many participants/cases compose your data?
|Simple answer of the size of your corpus
|What are some of the common or key attributes?
|Discuss the relevant demographics/variables of your population (if race/ethnicity, gender, age are relevant, list them here).
|How did you select your data/participants?
|Discuss the sampling method you used
|Source: Johnson, M. (2017). "Writing a Methods Section" In Allen, M. (2017). The SAGE Encyclopedia of Communication Research Methods (Vols. 1-4). Thousand Oaks, CA: SAGE Publications, Inc doi: 10.4135/9781483381411
The first question means answering the size (e.g., n = 88) of your corpus. Commentary on the size of your sample should also consider the total population that you are attempting to comment on. For instance, if you are writing about “student experiences with online open book examinations,” then it is important to consider how many students participated in the research compared to the students that took the examination (i.e., identify the characteristics of the sample versus the target populations). Answering the common of this case study would mean considering the descriptives (usually sample size, mean, distribution etc.) or characteristics (e.g. gender, age, class, ethnicity) of participants/unit of observation. Building from our previous example, we should ask pertinent questions about the participants that took this exam: what was the average score of the exam? What grade level is this course? Which discipline was this course in, and which discipline do most students in this specific class come from? Finally, we need to also discuss how we recruited these students. Outline your by discussing how you advertised the study, whether a stipend or incentive was offered, and how students finally agreed to join in the study (mainly regarding whether consent forms were required). For online texts or any other data that is not a participant/population, the same questions can be answered. The amount of the texts, components of the texts (their genre and author mainly), and the data collection process is all relevant to your method.
- Have I discussed how I advertised the study?
- Have I stated if a stipend was used?
- Have I stated if a consent form was used?
- How were participants identified?
- Where were they recruited?
While the above questions must be asked for all methods, the key concerns will differ depending on whether your research is primary, secondary, content analysis or theoretical research. is research that collects and derives its own raw data by sampling participants/cases, such as selecting and interviewing participants. uses data from other primary research projects, such as a systematic literature review of other research. uses other papers and articles, sometimes even other data (like secondary research), but with the purpose of advancing a new argument in the field. Its method does not have to be as explicit as the other two methods of research. The following table summarizes the distinct tasks of the three approaches when discussing methodology. As theoretical research often uses articles and other to make its argument, its methodological concerns are similar to content analysis. For that reason, we have grouped them together in the following table.
|Table 7.4 - Contrasting Primary/Secondary/Theoretical Research Methods
Discuss who the target sample was (how many, characteristics). This includes all the points that are discussed above.
|Check Method of Original Research
Check the methodology of the original research and justify its association with your project.
Discuss how you sample your artifacts (e.g., articles, blog posts, social media posts). This includes discussing the search term criteria that you have used to find these artifacts.
Argue the value of selecting and interviewing your target sample with respect to your study.
|Alter Method to fit your project
State if you have removed or changed anything about the methodologies of the studies you have sourced data from.
|Author and Genre
Discuss the author and audience. Briefly summarize them so you can elaborate further throughout your paper.
State your recruitment method and also disclose if a stipend or any other material was used to recruit participants.
If you are focusing on only a particular subpopulation, provide an overview of the sample, but discuss your subsample in detail. Justify why the focus on this subpopulation
|Size of Corpus
Discuss how many articles you found initially and how many you ended up upon. Discuss this in relation to the total number of articles that might have met your search criteria (if you found them all then this means your corpus was exhaustive).
It is important to note that merely listing these components of your data/participants does not satisfy the requirements of a good methods section. Remember that you are still making an argument. While discussing the attributes relevant to your study, you are still arguing why you selected these participants/cases for your study, why you are listing that demographic/characteristic(s) as important to your study, and why you chose this sampling method over the other options. Allow yourself to be guided by the argument and the relevant things to include in your sections will follow. The following box provides a checklist of questions to help you to evaluate whether your methods section has successfully addressed all the key questions regarding your ‘data’.
- Have I stated how many participants/data was in my study?
- Including in comparison to the relevant groups that they occupy?
- Have I discussed all the key attributes of my data?
- Have I discussed the relevant demographics of the populations and groups I am researching?
- If it is textual data, what is important to know about the author and genre of this text?
- Have I discussed the sampling method of my study?
- Have I argued the value and limitations of my sampling method?
Instruments and Measures
Research instruments are the tools that you use to collect, analyze and measure data in your research. These include literature reviews, survey questionnaires, interviews, focus groups, ethnography etc. In your methods section, you must describe how these serve to collect data, for example, was the survey composed of only open-ended questions, how many questions were included in the survey etc. Likewise, if you are using interviews, you need to discuss what kind of interviews were conducted (semi-structured, unstructured, structured)? How were the questions organized (e.g. by theme, no order etc.). Again, if you are doing a meta-analysis (literature review), you need to detail how it was done, what search engines, databases and search criteria were used etc. To reiterate, description of your instruments requires a an outline of how you used the particular measure to collect data.
As you will remember from your methods classes, when we talk about measurement in social research, we are referring to the process of (i.e., what concrete observations are being checked to empirically indicate a concept?). To do this, you must first identify the key variables or issues in your research question and explain how your research instrument captures it. For example, say your research question is “Do international students engage in economic transnationalism while studying on campus?” You will want to indicate how you will determine (a) who is an international student? and (b) what is economic transnationalism? It might seem silly that you need to operationalize “international student” but practically, you must have a system to determine who is an international student. It would be hard to walk around campus and determine who is an international student without having some criteria e.g. a student without Canadian citizenship. But again, how would you determine which student has Canadian citizenship and which ones do not? Operationalization requires that you specify precisely how you determine who is an international student (maybe you simply let participants self-declare their status on the survey or at the beginning of the interview). Likewise, “economic transnationalism” is a concept that needs to be measured (operationalized). You need to specify how in your research you will determine that economic transnationalism was taking place. Maybe you determined that if participants engaged in at least one economic activity (such as remitting money to origin countries, conducting business in origin countries, investing in origin countries etc), then economic transnationalism has taken place. The key to measurement is making it clear (and justifying) to your readers how your key concepts and variables are determined in the study. This requires that you consider how other researchers have measured these variables. If your measures are different from how other researchers measure the same concepts/variables, you need to justify why. On the other hand, the use of existing measures assures measurement consistency and contributes to the reliability of your study.
Measurement in quantitative studies can get complex by recoding, and the creation of composites such as indexes and scales. You must discuss how you recode variables and how the new variables allow you to better measure concepts. Likewise, you need to justify how the scales and indexes that you are using improve your measurements. If you are using established indexes and scales, it is important that you justify why they are applicable in your research. Again, if you are creating new measures, you need to identify why existing ones were inadequate for your research goals. As mentioned before, the methodology is not merely checking boxes, it requires justification of your choices, and engagement with an argument and or the literature (see the following example).
Whenever feasible, we adopted measures already used in the literature to facilitate comparisons. Two aspects of friendship are of interest: the nationality of friends and the strength of the friendships. We asked respondents to rank list their top ten friends and identify the nationality of each friend. This is a modification of a friendship grid used by Hendrickson et al. (2011) to study international students in Hawaii. Hendrickson et al. (2011) used social network analysis to map, for each student, a list of all people with whom they interact, recording if they are from their own country or another country, and a self-determined measure of the strength of each relationship on a scale of 1–10. We felt this was a valuable measurement tool; however, we also felt that asking participants to record an exhaustive list of every person with whom they socialize was an overly onerous task, and not necessary for our purpose.
We therefore modified this grid by asking respondents to rank order the top 10 friends with whom they socialize the most and to identify their friends’ national origin (host-national, co-national or other international). In line with Jindal-Snape and Rienties’s observation that non-university “local-community” can play an important role in supporting international students (2016, p. 6), we do not limit respondents to listing friends who attend the university, which is in keeping with other studies.
Also, like other studies, we only permit participants to list a maximum of ten friends…Respondents ranked their closest friends from 1–10, based on the amount of time spent socializing; friend 1 is the friend with whom they socialize the most. As a result, in our study we operationally define an increase in time spent together as indicative of closeness. According to Hall (2019), the closeness of a friendship is appropriately measured by the time spent together; transitioning from an acquaintance to a friend and to a close friend occurs with increased time spent together. Furthermore, since there are arguably cultural differences in the meanings of close friends as opposed to acquaintances (Gudykunst et al., 1985; Hendrickson et al., 2011; Maeda & Ritchie, 2003), we employ ranked time spent together as a measure of closeness as opposed to a self-determined scale of the strength of the friendship in an effort to control cultural bias.
We created three dummy variables: host-national (Canadian), co-national (from the same country as the respondent), other international (from another country but not the host or the respondent’s home country). From the dummy variables, we constructed measures to determine the proportion of respondents’ friends (all ten friends) that are from the host-nation, from their own country, or from a different country. For each of these three ratios we created additional variables to measure whether or not the designated friendship (host, other, or co-national) was a close or distant friend: we segmented the friendship spectrum by creating measures to identify the nationality of respondents’ closest three friends (friends 1–3), middle friends (friends 4–7) and the last three friends (8–10). The segmented measures of friendship facilitate an examination of the effect of different types of friends on the international student satisfaction measures at different levels of the friendship strength spectrum.
Source: Walsworth, S., Somerville, K., & Robinson, O. (2021). The importance of weak friendships for international student satisfaction: Empirical evidence from Canada. International Journal of Intercultural Relations, 80, 134-146.
The procedure is a step-by-step description of what you did to collect and analyze your data (see Figure 7.6.1 and Box 7.6.3). It should be the largest part of your methods section (Johnson, 2017), and it will incorporate explanation of how you collected your data, interpreted your data, and organized your data in your final-write up. As pointed out in the introduction, it is in this part of the methods section that the imitation function of methods – helping others to find our findings – is completed (Johnson, 2017). After providing a walkthrough of each step of your research process, it then discusses how you analyzed that data. The procedure clearly divides and orders the steps of your research chronologically, seeking to present a summative narrative of your research from recruitment/sampling of data to analysis. Documenting your procedure allows another researcher to attempt the exact same process as you with the expectation that they should be able to find (roughly) the same results. It is through this cross-reference system that the method can likewise become generalizable. If other researchers can replicate your procedure across contexts with effective results, then the method proves itself intuitive and effective to be useful for further research. It is an important kind of tedium!
- Did I discuss how I recruited my participants and/or collect my data? Did I mention and outline the type of sampling method I used?
- Did I discuss how I ensured ethical fairness in gathering my data?
- Did I discuss how many participants I reached out to, and if any problems occurred in gathering my data?
- Did I discuss how I solved problems in recruiting my participants?
- Have I discussed the method for collecting data in my study (interviews, surveys, census data)?
- Have I discussed procedures for cleaning, recoding, and otherwise ensuring the integrity of my data?
The final aim of your methods section will be to discuss how you processed and analyzed the data collected. This will differ significantly depending on the method chosen, but there are a few standard things to do depending on whether your paper is qualitative or quantitative. The following table highlights:
|Table 7.5 - Steps in Describing Your Analytical Strategy
|Describe how you processed data for analysis (e.g., how you cleaned the data set, how you treated missing and extreme values, did you transform or recode variables for analysis?)
|Describe how you processed the data for analysis (e.g. transcription verbatim or for general ideas) and the coding procedures identified.
|Identify what software was used to conduct analyses (e.g. Excel, SPSS, STATA, Python etc)
|Identify what software if any was used to facilitate analysis (e.g. NViVo, AtlasTi etc)
|Describe what statistical test was used (e.g. regression, ANOVA etc.)
|Describe what kind of analytical strategy was used (e.g. grounded theory, thematic analysis, content analysis, discourse analysis etc.).
The next table provides a demonstration of the analysis portion of Alexander Wilson’s (2021) methodology in his undergraduate honours thesis.
|Table 7.6 - Breakdown of Alexander's Analysis
|“After the articles and government reports were collected, I uploaded them into NVivo where they were read with attention to the context of the discourse.”
|Open Reading into Coding
|“After an open reading, I divided the issues present in the media into Beckert and Dewey’s (2017, p. 14) two concepts, "externalities and hopes for the future," while paying attention to the time and narrator framing Uber's legitimacy or illegitimacy.”
|Example of coding
|“The externalities voiced in the literature, such as lower wages and safety concerns, were compared alongside the hopes expressed by Uber representatives and finally taken up by the government in legislative decisions.”
|Argument for discourse analysis to analyze this data
|“I used an interpretive method (discourse analysis) to allow room for me to reason and justify my understanding of the context and influence of Uber's frame on the broader debate”
|Source: Wilson, A. (2021). Driver’s of Dissidence: A Discourse Analysis of Vancouver’s Road to Ride-Hailing. Undergraduate Thesis. (p. 13).
The above table shows that honours student, Alexander Wilson (2021) began by discussing the instrument used to organize the analysis (NVivo) and then moves onto the steps taken to conduct a discourse analysis (which is overviewed earlier in the paper). It discusses how the media data was analyzed, through an open reading to a codification of the issues according to their being “hopes or externalities” of Uber’s service. The codification is situated within the overall method of discourse analysis, which seeks to interpret the meaning of speech/text with respect to the larger discussion that it is contributing to. For Alexander, this larger discussion leads to a change in legislation, so it is his goal to interpret the significance of Uber’s promises and externalities in the media discourse with respect to the final legislative conclusions.
- Have I summarized the method?
- Will this summary make sense to someone doing similar research?
- Have I adequately highlighted the elements of this method which are especially relevant for my research?
- Have I adequately highlighted elements of this method which are relevant for my argument regarding its value for my research?
- Have I argued the value of my approach?
- Have I argued that this methodological approach will be effective for gathering data in an ethical manner?
- What other approaches can I compare my method with? What are the tradeoffs?
- Have I summarized the data of my study?
- Have I clarified the type of my data (participants, text, articles etc.)?
- Have I stated common attributes of that data?
- Have I disclosed important ethical concerns regarding interaction with that data or population?
- Have I outlined the instruments and materials used in my study?
- Have I discussed the design and procedure of my study?
- Have I outlined how I analyzed the data?
- Did I adequately discuss the key steps of my analysis?
- Did I state how many iterations of my analysis were conducted (how many readings, how many calculations)?
Johnson, M. (2017). “Writing a Methods Section” In Allen, M. (2017). The SAGE Encyclopedia of Communication Research Methods (Vols. 1-4). Thousand Oaks, CA: SAGE Publications, Inc doi: 10.4135/9781483381411
Walsworth, S., Somerville, K., & Robinson, O. (2021). The importance of weak friendships for international student satisfaction: Empirical evidence from Canada. International Journal of Intercultural Relations, 80, 134-146.
Gudykunst, W. B., Yang, S., & Nishida, T. (1985). A Cross-Cultural Test of Uncertainty Reduction Theory: Comparisons of acquaintances, friends, and dating relationships in japan, korea, and the united states. Human Communication Research, 11(3), 407-454. https://doi.org/10.1111/j.1468-2958.1985.tb00054.x
Maeda, E., & Ritchie, L. D. (2003). The concept of shinyuu in japan: A replication of and comparison to cole and Bradac’s study on U.S. friendship. Journal of Social and Personal Relationships, 20(5), 579-598. https://doi.org/10.1177/02654075030205001
Hall, J. A. (2019). How many hours does it take to make a friend? Journal of Social and Personal Relationships, 36(4), 1278-1296. https://doi.org/10.1177/0265407518761225
The tools used to obtain data (e.g., questionnaires, interviews etc.).
The activities that a researcher takes to collect data.
the descriptives (usually sample size, mean, distribution etc.) or characteristics (e.g. gender, age, class, ethnicity) of participants/unit of observation
how you advertised the study, whether a stipend or incentive was offered, and how students finally agreed to join in the study
Research that collects and derives its own raw data by sampling participants or cases, such as selecting and interviewing respondents.
Uses data from other primary research projects, such as a systematic literature review of other research.
Uses other papers and articles, sometimes even other data (like secondary research), but with the purpose of advancing a new argument in the field.
things created by humans e.g., books, graffiti, advertisements, photographs, blogs etc.
The process of defining how one is going to measure a phenomenon that is not directly measurable.