5 Evaluating the Question
Now that you have chosen a topic that is worthy of pursuing further, how do you translate it into a good research question? Methodological researcher, Alan Bryman (2007), states that the value of a good research question lies in its capacity to “militate against undisciplined data analysis and collection” (p. 6). Adapting material from the open resource video by Steely Library NKU (2014), we have outlined four main ‘preventive’ mechanisms for you to consider when constructing a resilient research question. A good research question will be:
- interesting to you and relevant to your field (your potential audience),
- operational enough to adequately interpret the phenomena being studied,
- such that you can find evidence which corresponds to your question, and
- feasible enough to achieve steps 1, 2, and 3.
We elaborate on each of these next.
How Interested Are You in the Topic?
Interest and relevance are important evaluation criteria for any research question. You must ask yourself: is this research question interesting to me? Does it arouse my desire to find an answer to the question? Does the lack of an answer cause me anxiety? Do I believe that things will be different for me if I could just answer this pressing question? The many ways in which we become attracted to the unforeseen answer of a question is the fundamental consideration of research. This is what we attempted to unpack in the prior section. However, this is a good time to ask yourself what issue you are genuinely interested in. Even if the topic seems grandiose, allow yourself to indulge in the possibility of studying it, and discuss it with someone. It might be possible to refine it to something do-able for your undergrad work.
Operationalizing the Research Question
The second aspect of the research question involves defining its wording enough to capture the subtle intricacies of the phenomena you are investigating. This means having a methodological and theoretical framework that can adequately explain the appearance of certain types of evidence. For instance, say that I come to the research question: is aggression between baristas and customers class conflict? To come to an operational account of this question, I would need to set up a framework in which this question can account for the complexity of “aggression between baristas and customers”. This means defining your key variables in such a way that there are specific and measurable indicators for each.
For example, how would you measure aggression (maybe, physical and verbal confrontations, perceptions of offensive interactions etc.), and how do you measure class conflict? This second variable is more tricky: are you measuring class in terms of status, if yes, is it real or perceived status? Are you measuring class using proxies such as income, education or other indicators? Third, is determining how this equates to class conflict: would you only be investigating aggression between customers and baristas who have different classes? This could have serious implications because you would be deliberately excluding non-class related conflicts (e.g., what if both the barista and customers are both lower class, you would have to exclude those aggressive encounters because of similar social class). Thus, to make a question more operational, sketch your colloquial concepts a little further in order to make it able to rule out the possibility of attributing a concept to something it is not (like assuming any skirmish with a barista entails ‘class conflict’). More importantly, you need to consider the practicality of the question. In the above example, how would you collect data from irate customers and baristas? They would probably be unwilling to talk to you after an encounter. Surveys and interviews are unlikely to elicit good response rates. Observation might elicit better results but in addition to ethical questions, you have to have precise indicators of what observations would be meaningful to the research question. Operationalizing the research question would therefore mean defining your variables but also the means of collection (i.e. are you examining aggression and class conflict at coffee shops in Vancouver, or in Lifetime movies, in novels etc.?). Think about the entire context that would make the research question do-able. This leads to the third criteria: is the research question manageable?
Is This Manageable?
As discussed above, measuring class conflict and aggression between customers using surveys and interviews might not be optimal. The research question needs more defined parameters to make it manageable (i.e., defining an appropriate setting and method). This involves figuring out whether you can find evidence at all. In the above example, are you likely to find sufficient evidence by visiting coffee shops in Vancouver, reading novels or watching movies? How many visits would give you the evidence you need, how many movies would you need to watch or how many novels would you need to read? These are important questions to ask in order to determine if your research question is practical and manageable.
Consider another example: say your research question was “what doubts does Jeff Bezos have when he first wakes up in the morning?” While a fascinating question, this question is not one that can be sought after unless you can manage an intimate interview with Bezos (and would likely produce a more journalistic article than an academic paper). To consider what evidence can be sought after, think through which methods are manageable to achieve your question and whether you have the means to find data through those methods.
Manageable problems imply certain limits that need to be set on what separates a feasible project from an unfeasible one. This will depend on your aptitude and the time you can allocate to your research. In the establishment of a research question, your time and resources will be most affected by how broad and narrow the scope of your question is. All questions imply a certain framing of what is and is not associated with your question: this is the scope of your question, which can otherwise be defined as the information relevant to answering your research question.
For instance, if I was looking for gender inequality in cafe hiring practices, the colour of the patio chairs at cafes will likely not factor into my analysis. Rather, my core concerns will be in defining the “range” of my core concepts so that they can be evaluated by inquiring what “gender inequality” means when situated in cafe hiring practices. When considering the amount of data pertinent to your topic (and whether you can handle the heat), this has to do with how broad or narrow your topic is. For instance, a topic like “gender inequality” on its own will be far too broad. A quick search in the library database will produce a panoply of different accounts of gender inequality, with little that could connect to an entirely new and unique angle. On the other hand, if the gargantuan amount of data about gender equality scares you into making your topic excessively narrow by adding a bunch of qualifying factors, such as “does gender inequality, the differential treatment of people based solely on gender, exist amongst Fijiean scuba divers living in Russia?” then you may find that the data is too scarce (or non-existent) to be developed into any type of supported argument. Consequently, deciding what is a manageable question involves striking a balance between a question with a scope that provides both meaningful and organizable data, avoiding both hubris and false humility. For an extended discussion of broad and narrow scopes, check out this video from the UBC library.
Schedule
On a final note, ask yourself before you begin the brainstorm about your schedule and what you can handle before taking on research. The answer you give will of course not be perfect, and the timeline of your research will always be unpredictable, but hopefully this questioning will at least allow you to consider the amount of complexity you are willing and able to take on before you dive into a question that is of importance to you. In Chapter 2, we discuss scheduling and the construction of Gantt charts to help you visualize the research process and manage your timelines.
I discovered the topic I wanted to research for my Ph.D. by pure chance. After a year into my Ph.D, I was not sure what I wanted to research. I certainly did not want to spend four years researching the topic that I originally entered the program intending to study. My interests had changed and I wanted a new topic. I spent the summer brainstorming topics but never arriving at one that I was deeply interested in. That summer, I went to the Olympic games in London 2012. The main event for the evening we went was the Women’s 4x100m relay final. The entire stadium was on its feet. It was going to be a big showdown between Jamaica and the United States (who not only won but broke the world record). I wanted to jump and shout like any Jamaican/Caribbean person would at sporting events, but I was not in the Caribbean. I was in the Queen’s country. Instinctively, I look at the anxious faces around me to determine if my jumping about and screaming would upset the people in the nearby seats. It was then that I noticed many Caribbean flags in my area: Trinidad, Barbados, Antigua, Grenada, Jamaica etc. Something struck me as strange. Why were there so many Caribbean people in my area? Was it by chance or by design? The ticketing process was done by bidding for each geographic region of origin (in my case, I bidded in the Caribbean region), but did they still put Caribbean people together? It was a curious question. I had no way to find out. But, I suddenly felt comfortable to shout and jump about. I was with my people. In that instant, I knew I wanted to study something about Caribbean unity. I didn’t know what shape the project would take, but I knew the general area. When I refined it, I ended up studying Migration, Social Identities and Regionalism in the Caribbean. I had never studied migration or identities before, but that chance discovery led to a deep interest in Caribbean identities and migration.
References
Bryman, A. (2007). The research question in social research: What is its role? International Journal of Social Research Methodology, 10(1), 5-20. https://doi.org/10.1080/13645570600655282
NKU Library. (2012). “Developing a Research Question.” Youtube. https://www.youtube.com/watch?v=LWLYCYeCFak.
Robinson, O. (2015). Negotiating identities in CARICOM: How CARICOM nationals experience intra-regional migration and regionalism. (Ph.D. dissertation, University of Saskatchewan).
UBC Library. (n.d.). “Broad & Narrow Topics.” YouTube. Broad & Narrow Topics – YouTube
The process of defining how one is going to measure a phenomenon that is not directly measurable.
The extent to which a phenomena will be studied and the parameters within which the study is being conducted.
