Testing – new part
6 Test
The Basic Process of Scientific Research
Learning Outcomes
After reading this chapter, you will be able to:
1. describe the principles of the scientific method;2. differentiate hypotheses from predictions
3. describe why operational definitions are important 4. understand why we need peer review before publishing our research findings
Psychologists are not the only people who seek to understand human behaviour and solve social problems. Philosophers, religious leaders, and politicians, among others, also strive to provide explanations for human behaviour. However, psychologists believe that research is the best tool for understanding human beings and their relationships with others. Rather than accepting the claim that people do or do not have free will, a psychologist would collect data to empirically test whether or not people are able to actively control their own behaviour. Rather than accepting an argument that creating or abandoning a new centre for mental health will improve the lives of individuals in the inner city, a psychologist would empirically assess the effects of receiving mental health treatment on quality of life. The statements made by psychologists are empirical, which means they are based on systematic collection and analysis of data.
The scientific method can be simplified into a series of steps. Basically, ideas (in the form of theories and hypotheses) are tested against the real world (in the form of empirical observations). Then, those empirical observations lead to more ideas that are tested against the real world, and so on. In this sense, the scientific process is circular. As psychologists learn more about something, that knowledge generates further questions that can be turned into hypotheses. As our knowledge is expanded, we may have to change a theory to account for it.
Figure X
The Scientific Method as an Ongoing Process
Note. A scientific approach to knowledge acquisition starts with proposing a research question followed by a testable and tentative hypothesis. Then data will be gathered to verify or potentially falsify the hypothesis.
Scientific_Method_3-1-768×576.jpg
[Alt text: This diagram shows the ongoing process of the scientific method, including making observations, thinking of interesting questions, formulating hypotheses, developing testable predictions, gathering data to test predictions, and developing general theories.]
A theory is a well-developed set of ideas that proposes an explanation for observed phenomena. Theories are repeatedly checked against the world, but they tend to be too complex to be tested all at once; instead, researchers create hypotheses to test specific aspects of a theory. A hypothesis is a testable prediction about how the world will behave, and it is often worded as an if-then statement (e.g., if I study all night, then I will get a passing grade on the test). The hypothesis is extremely important as it bridges the gap between the realm of ideas and the real world. Psychological researchers may form their hypotheses based on both deductive and inductive processes. In the scientific context, deduction refers to testing theories against empirical observations. Imagine you want to deductively test the general idea that “playing video games will make people happy.” You observe your friend Bob, who plays video games every day. You want to see whether he always seems happy during these gaming sessions. In this case, you are applying a general theory specifically to Bob. Induction, on the other hand, refers to using empirical observations to formulate theories. Let’s say several of your friends all mention feeling happy while playing video games. From these specific observations, you may inductively propose a hypothesis like, “People tend to feel happy when playing video games.” In this case, you are forming a theory based on the specific instances you’ve encountered.
imgKG.3
Figure X
Deductive and Inductive Processes
Note. Scientific research involves both deductive and inductive processes. For example, case studies are closely associated with inductive processes as researchers gather massive amounts of observations and seek interesting patterns in the data. On the other hand, the experimental approach places great emphasis on deductive reasoning, where researchers design experiments to test their hypotheses.
[Alt text: ]
In addition to requiring that science be empirical, the scientific method demands that the procedures used be objective, that is to say, free from the personal bias or emotions of the researcher. The scientific method prescribes the process for researchers to collect and analyze data, draw conclusions, and share their findings. By following these rules, other researchers can understand exactly how the data was collected and analyzed. They can draw their own conclusions and not rely solely on the interpretations of the original research. This promotes transparency and allows for diverse perspectives in scientific research. Some new research also aims to replicate previous findings. They can repeat, add to, modify or even falsify earlier findings. In this way, scientific knowledge grows as researchers report their findings, and other researchers add to or modify the procedures through continuous research and sharing of ideas.
The goal of research in psychology is to explain and predict relationships within a certain area of study. A good theory should offer a simple explanation of a phenomenon (that is, be parsimonious) and suggest ideas for future research. Most importantly, a good theory must be falsifiable, which means that the hypotheses generated from this theory are capable of being shown to be incorrect. Recall from the introductory chapter that Sigmund Freud had lots of interesting ideas to explain human behaviours. However, a major criticism of Freud’s work is that many of his ideas were not falsifiable; for example, it is impossible to imagine how we could measure the three elements of personality described in Freud’s theories — the id, the ego, and the superego. How could we verify or falsify their existence?
A good hypothesis is a testable prediction about relationships between clearly defined variables. Suppose that we want to test this idea: sleep is important for memory. Can you think of three different testable hypotheses? For example:
People who get more than seven hours of sleep will get a higher score on the ABC Memory Test than people who get less than seven hours.
There is a positive association between the amount of time students sleep and their grade point average.
People suffering from insomnia show an increased ABC Memory Test score when they are successfully treated for insomnia.
As you read through these examples, you may have noticed that we use very specific methods to measure sleep and memory. Psychologists use the term “operational definition” to describe exactly how a variable is being measured. In contrast to the abstract concepts, the operational definitions are very specific. For example, we could measure sleep in many different ways: self-reported number of hours of sleep according to brain waves measured in a sleep lab, the number of hours of sleep reported by a fitness tracking device worn by participants, and so on. If we were measuring temperature, we would need to define what we mean by temperature: degrees Fahrenheit, degrees Celsius, or simply our best guess. Having clear operational definitions is crucial because. If a variable is not precisely defined, others may misunderstand the data collected, making it hard for future researchers to replicate the study.
Here are some operational definitions (OD) of variables that have been used in psychological research:
Concept = “aggression”
OD1: number of presses of a button that administers shock to another student
OD2: number of seconds taken to honk the horn at the car ahead after a stoplight turns green
Concept = “interpersonal attraction”
OD1: number of inches that an individual places his or her chair away from another person
OD2: number of millimeters of pupil dilation when one person looks at another
Concept = “employee satisfaction”
OD1: number of days per month an employee shows up to work on time
OD2: rating of job satisfaction from 1 (not at all satisfied) to 10 (extremely satisfied)
Concept = “depression”
OD1 = number of negative words used in a creative story
OD2 = number of appointments made with a psychotherapist
When psychologists complete a research project, they generally want to share their findings with other scientists. The American Psychological Association (APA, 2020) publishes a manual detailing how to write a paper for submission to peer-reviewed, scientific journals. The Online Writing Lab (OWL) at Purdue University can walk you through the APA writing guidelines.
Peer review is an important part of publishing research findings in many scientific disciplines. A peer-reviewed journal article is read by several other scientists (generally anonymously) with expertise in the subject matter. These peer reviewers provide feedback to both the author and the journal editor regarding the quality of the draft. Peer reviewers look for a strong rationale for the research being described, a clear description of how the research was conducted, and evidence that the research was conducted in an ethical manner. They also look for flaws in the study’s design, methods, and statistical analyses. They check that the conclusions drawn by the authors seem reasonable given the observations made during the research. Peer reviewers also comment on how valuable the research is in advancing the discipline’s knowledge. This helps prevent unnecessary duplication of research findings in the scientific literature and, to some extent, ensures that each research article provides new information. Ultimately, the journal editor will compile all of the peer reviewer feedback and determine whether the article will be published in its current state (a rare occurrence), published with revisions, or not accepted for publication.
Peer review provides some degree of quality control for psychological research. Poorly conceived or executed studies can be weeded out, well-designed research can be improved, and ideally, studies can be described clearly enough to allow other scientists to replicate them, which helps to maintain reliability.
So why would we want to replicate a study? Imagine that our version of the Bobo doll study is done exactly the same as the original, only using a different set of participants and researchers. We use the same operational definitions, manipulations, measurements, and procedures, and our groups are equivalent in terms of their baseline levels of aggression. In our replication however, we receive completely different results and the children do not imitate aggressive behaviours any more than they would at the level of chance. If our experimental manipulation is exactly the same, then the difference in results must be attributable to something else that is different between our study and the original, which might include the researchers, participants, and location. If on the other hand, we were able to replicate the results of the original experiment using different researchers and participants at a different location, then this would provide support for the idea that the results were due to the manipulation and not to any of these other variables. The more we can replicate a result with different samples, the more reliable it is.
In recent years, there has been increasing concern about a “replication crisis” that has affected a number of scientific fields, including psychology. One study found that only about 62% of social science studies reviewed were replicable, and even then their effect sizes were reduced by half (Cramerer et al, 2018). In fact, even a famous Nobel Prize-winning scientist has recently retracted a published paper because she had difficulty replicating her results (BBC, 2020). These kinds of outcomes have prompted some scientists to begin to work together and more openly. One example of this more collaborative approach is the Psychological Science Accelerator, a network of over 500 laboratories, representing 82 countries. This network allows researchers to pre-register their study designs, which minimizes any cherry-picking that might happen along the way to boost results. Cherry-picking is a biased approach where researchers selectively report data that supports a researcher’s hypothesis, while ignoring any findings that do not support it. The network also facilitates data collection across multiple labs, allowing for the use of large, diverse samples and more wide-spread sharing of results. Hopefully with a more collaborative approach, we can develop a better process for replicating and checking the quality of research. If you’d like to learn more about the Psychological Sciences Accelerator, you can check out their website.
VIDEO: The Scientific Method By Crash Course Biology #2
https://youtu.be/xOLcZMw0hd4?si=OQlYGs2DjXWg61qC