Drafting the Methods Section

While the substance of the methods section will differ by genre and method chosen, basic components can be derived across genres. Methods sections in the social sciences tend to have five sub-components. They must:

1. summarize the method used while arguing the value and limitations of the method for your data/context;
2. discuss the data of your study (participants, artifacts, academic literature)
3. present the instruments and measures used;
4. outline the procedure of the data (i.e. how it was collected and ensured of its integrity), and
5. Analysis: discuss how you analyzed your data.

Splitting these sections into components, however, should not give you the impression that the methods section is merely a list. The methods section is also an argument (Johnson, 2018), meaning that it aims to convince your reader of the value of your method through a narrative that briefly applies your method to the context of your research. In addition, the method must find a way to align with the research question stated in the introduction. Your methodology should build upon the introduction, justifying that the approach you utilized to investigate the problem is suitable (see Table 7.3.1 for common justifications for some established methods). We will refer back to this key point as we overview each step of the methods section.

A simple summary of your method is a good way to begin the method’s section. The length and depth of this summary depends entirely on the method chosen and the audience it is presented to. If the method is commonly known and used within your field, an overly intricate summary of the method is not necessary; a couple sentences overview will work fine. If the method is not commonly used or entirely new, however, more argument will be required for your audience to understand your choice of method. For instance, grounded theory is a commonly used paradigm in many qualitative social science journals. It is therefore unnecessary to sketch the method’s history in detail. Rather, a simple definition, such as grounded theory being an inductive approach which only designates codes after data collection (and how you enacted in your project) will suffice. On the other hand, newer methods such as linked data (methods based on web technologies such as HTTP, RDF, SPARQL, and URIs to enable semantic connections between various databases) would require a more extensive discussion. In addition, while it is good to offer your own summary of the method, the definition used by another researcher (particularly methodological researchers) is a simple way to align your research with a legitimate approach in your field.

The key point to keep in mind for summarizing your method is to outline its theory insofar as it explains your procedure, that is, discuss the method’s intentions with respect to how you applied it. We will touch on this again in the procedure section.

As you provide an overview of your method, you must also justify it. Justification of your method must appeal to the method’s ethical, practical and factual utility for the project. Ethical justifications are those that argue that the method is best for reducing the harm of research on your participants (and the communities they inhabit). It must also make the point that your method aligns with the principles of your institution’s ethical values.

Factual issues have to do with the value of the data your method is able to gather. It must consider whether the method is appropriate to your research question and topic. If, for instance, the research question is about “Malaysian immigrants’ conception of justice in comparison to American immigrants,” then you may argue that ‘in-depth interviews’ are the only approach deep enough to unravel a person’s “conception of justice.” Arguments for the factual benefits of a method frequently highlight its novelty for studying a particular method. For instance, there could be a lack of discourse analyses of Uber’s advertising materials (the previous research being content analyses). One could then argue that a discourse analysis approach not only has merits in its own right, but it also may be able to discover data which other approaches miss. Finally, you should outline the strengths of the methods in relation to all aspects of your research process (e.g., alignment with your theory, personal values, practicality etc.).

Practical justifications highlight why the method is suitable given logistics, administrative and everyday concerns. For example, if you are interested in studying how Malaysian migrants’ prepare for their transit to America, an ethnographic approach might be tempting. But practically (financially and time-wise), you might not be able to visit Malaysia to observe their preparations. Hence, you might decide that surveys or interviews (while less desirable) are more practical in that instance.

Again, we emphasize the importance of highlighting the limitations of your method. Every method has weaknesses, it is vital that show an awareness of them. It is important, however, not to have the weaknesses outweigh your method’s benefits. Your reader should be able to understand why you choose the methods, so you need to explain how, in spite of the limitation, your method is the most suitable choice. Hence, you need to justify why you decided to choose the methods over others. To help in the weighing of the costs and benefits of different methodological approaches, we have provided below a list of the potential limitations of some of the more common methods used in the social sciences.

Once you have provided an overview of your method and its value for your context, the next step is to discuss the data or population that the method will be used upon. Summarizing the data or population of your method often requires answering (Johnson, 2017): (a) how many participants/cases compose your data? (b) what are some of the common or key attributes of your data? And c) how did you select your data/participants?

The first question means answering the size (e.g., n = 88) of your corpus. Commentary on the size of your sample should also consider the total population that you are attempting to comment on. For instance, if you are writing about “student experiences with online open book examinations,” then it is important to consider how many students participated in the research compared to the students that took the examination (i.e., identify the characteristics of the sample versus the target populations). Answering the common attributes of this case study would mean considering the descriptives (usually sample size, mean, distribution etc.) or characteristics (e.g. gender, age, class, ethnicity) of participants/unit of observation. Building from our previous example, we should ask pertinent questions about the participants that took this exam: what was the average score of the exam? What grade level is this course? Which discipline was this course in, and which discipline do most students in this specific class come from? Finally, we need to also discuss how we recruited these students. Outline your recruitment process by discussing how you advertised the study, whether a stipend or incentive was offered, and how students finally agreed to join in the study (mainly regarding whether consent forms were required). For online texts or any other data that is not a participant/population, the same questions can be answered. The amount of the texts, components of the texts (their genre and author mainly), and the data collection process is all relevant to your method.

While the above questions must be asked for all methods, the key concerns will differ depending on whether your research is primary, secondary, content analysis or theoretical research. Primary research is research that collects and derives its own raw data by sampling participants/cases, such as selecting and interviewing participants. Secondary research uses data from other primary research projects, such as a systematic literature review of other research. Theoretical research uses other papers and articles, sometimes even other data (like secondary research), but with the purpose of advancing a new argument in the field. Its method does not have to be as explicit as the other two methods of research. The following table summarizes the distinct tasks of the three approaches when discussing methodology. As theoretical research often uses articles and other social artifacts to make its argument, its methodological concerns are similar to content analysis. For that reason, we have grouped them together in the following table.

It is important to note that merely listing these components of your data/participants does not satisfy the requirements of a good methods section. Remember that you are still making an argument. While discussing the attributes relevant to your study, you are still arguing why you selected these participants/cases for your study, why you are listing that demographic/characteristic(s) as important to your study, and why you chose this sampling method over the other options. Allow yourself to be guided by the argument and the relevant things to include in your sections will follow. The following box provides a checklist of questions to help you to evaluate whether your methods section has successfully addressed all the key questions regarding your ‘data’.

Instruments and Measures

Research instruments are the tools that you use to collect, analyze and measure data in your research. These include literature reviews, survey questionnaires, interviews, focus groups, ethnography etc. In your methods section, you must describe how these serve to collect data, for example, was the survey composed of only open-ended questions, how many questions were included in the survey etc. Likewise, if you are using interviews, you need to discuss what kind of interviews were conducted (semi-structured, unstructured, structured)? How were the questions organized (e.g. by theme, no order etc.). Again, if you are doing a meta-analysis (literature review), you need to detail how it was done, what search engines, databases and search criteria were used etc. To reiterate, description of your instruments requires a an outline of how you used the particular measure to collect data.

As you will remember from your methods classes, when we talk about measurement in social research, we are referring to the process of operationalization (i.e., what concrete observations are being checked to empirically indicate a concept?). To do this, you must first identify the key variables or issues in your research question and explain how your research instrument captures it. For example, say your research question is “Do international students engage in economic transnationalism while studying on campus?” You will want to indicate how you will determine (a) who is an international student? and (b) what is economic transnationalism? It might seem silly that you need to operationalize “international student” but practically, you must have a system to determine who is an international student. It would be hard to walk around campus and determine who is an international student without having some criteria e.g. a student without Canadian citizenship. But again, how would you determine which student has Canadian citizenship and which ones do not? Operationalization requires that you specify precisely how you determine who is an international student (maybe you simply let participants self-declare their status on the survey or at the beginning of the interview). Likewise, “economic transnationalism” is a concept that needs to be measured (operationalized). You need to specify how in your research you will determine that economic transnationalism was taking place. Maybe you determined that if participants engaged in at least one economic activity (such as remitting money to origin countries, conducting business in origin countries, investing in origin countries etc), then economic transnationalism has taken place. The key to measurement is making it clear (and justifying) to your readers how your key concepts and variables are determined in the study. This requires that you consider how other researchers have measured these variables. If your measures are different from how other researchers measure the same concepts/variables, you need to justify why. On the other hand, the use of existing measures assures measurement consistency and contributes to the reliability of your study.

Measurement in quantitative studies can get complex by recoding, and the creation of composites such as indexes and scales. You must discuss how you recode variables and how the new variables allow you to better measure concepts. Likewise, you need to justify how the scales and indexes that you are using improve your measurements. If you are using established indexes and scales, it is important that you justify why they are applicable in your research. Again, if you are creating new measures, you need to identify why existing ones were inadequate for your research goals. As mentioned before, the methodology is not merely checking boxes, it requires justification of your choices, and engagement with an argument and or the literature (see the following example).

Source: Walsworth, S., Somerville, K., & Robinson, O. (2021). The importance of weak friendships for international student satisfaction: Empirical evidence from Canada. International Journal of Intercultural Relations, 80, 134-146.

Procedure

The procedure is a step-by-step description of what you did to collect and analyze your data (see Figure 7.6.1 and Box 7.6.3). It should be the largest part of your methods section (Johnson, 2017), and it will incorporate explanation of how you collected your data, interpreted your data, and organized your data in your final-write up. As pointed out in the introduction, it is in this part of the methods section that the imitation function of methods – helping others to find our findings – is completed (Johnson, 2017). After providing a walkthrough of each step of your research process, it then discusses how you analyzed that data. The procedure clearly divides and orders the steps of your research chronologically, seeking to present a summative narrative of your research from recruitment/sampling of data to analysis. Documenting your procedure allows another researcher to attempt the exact same process as you with the expectation that they should be able to find (roughly) the same results. It is through this cross-reference system that the method can likewise become generalizable. If other researchers can replicate your procedure across contexts with effective results, then the method proves itself intuitive and effective to be useful for further research. It is an important kind of tedium!

Analysis

The final aim of your methods section will be to discuss how you processed and analyzed the data collected. This will differ significantly depending on the method chosen, but there are a few standard things to do depending on whether your paper is qualitative or quantitative. The following table highlights:

The next table provides a demonstration of the analysis portion of Alexander Wilson’s (2021) methodology in his undergraduate honours thesis.

The above table shows that honours student, Alexander Wilson (2021) began by discussing the instrument used to organize the analysis (NVivo) and then moves onto the steps taken to conduct a discourse analysis (which is overviewed earlier in the paper). It discusses how the media data was analyzed, through an open reading to a codification of the issues according to their being “hopes or externalities” of Uber’s service. The codification is situated within the overall method of discourse analysis, which seeks to interpret the meaning of speech/text with respect to the larger discussion that it is contributing to. For Alexander, this larger discussion leads to a change in legislation, so it is his goal to interpret the significance of Uber’s promises and externalities in the media discourse with respect to the final legislative conclusions.

References

Johnson, M. (2017). “Writing a Methods Section” In Allen, M. (2017). The SAGE Encyclopedia of Communication Research Methods (Vols. 1-4). Thousand Oaks, CA: SAGE Publications, Inc doi: 10.4135/9781483381411

Walsworth, S., Somerville, K., & Robinson, O. (2021). The importance of weak friendships for international student satisfaction: Empirical evidence from Canada. International Journal of Intercultural Relations, 80, 134-146.

Gudykunst, W. B., Yang, S., & Nishida, T. (1985). A Cross-Cultural Test of Uncertainty Reduction Theory: Comparisons of acquaintances, friends, and dating relationships in japan, korea, and the united states. Human Communication Research, 11(3), 407-454. https://doi.org/10.1111/j.1468-2958.1985.tb00054.x

Maeda, E., & Ritchie, L. D. (2003). The concept of shinyuu in japan: A replication of and comparison to cole and Bradac’s study on U.S. friendship. Journal of Social and Personal Relationships, 20(5), 579-598. https://doi.org/10.1177/02654075030205001

Hall, J. A. (2019). How many hours does it take to make a friend? Journal of Social and Personal Relationships, 36(4), 1278-1296. https://doi.org/10.1177/0265407518761225