Chapter 6: Data Collection Strategies
6.1 Experiments
An experiment is a method of data collection designed to test hypotheses under controlled conditions (often in a laboratory), with the goal to eliminate threats to internal validity. Most commonly a quantitative research method, experiments are used more often by psychologists than sociologists, but understanding what experiments are and how they are conducted is useful for all social scientists, whether they actually plan to use this methodology or simply aim to understand findings based on experimental designs.
An experiment is a method of data collection designed to test hypotheses under controlled conditions, with the goal to eliminate threats to internal validity. There are different experiment designs. In the classic experiment, the effect of a stimulus is tested by comparing two groups: one that is exposed to the stimulus (the experimental group) and another that does not receive the stimulus (the control group). The control group, often called the comparison group, is treated equally to the experimental group in all respects, except it does not receive the independent variable. The purpose of the control group is to control for rival plausible explanations.
Most experiments take place in a lab or some other controlled environment. In an experiment, the effects of an independent variable upon a dependent variable are tested. Because the researcher’s interest lies in the effects of an independent variable, the researcher must measure participants on the dependent variable before (a pre-test) and after (post-test) the independent variable (or stimulus) is administered. In this type of experiment researchers employ random assignation (often referred to as random assignment), which means that one group is the equivalent of the other. Random assignation is more fully explored in the following section “Random Assignation”.
It is important to note that social sciences research usually takes place in a natural setting, where the researcher will utilize a quasi-experimental design, rather than an experimental design. Similar to an experiment, the independent variable in a quasi-experiment is manipulated. A quasi-experimental design is discussed in more detail in section 6.3 Quasi-experimental research.
Students in research methods classes often use the term “experiment” to describe all kinds of empirical research projects, but in social scientific research the term has a unique meaning and should not be used to describe all research methodologies. In general, designs considered to be “true experiments” contain three key features:
- Independent and dependent variables.
- Pretesting and post-testing.
- Experimental and control groups.
Pretesting and post-testing are both important steps in a classic experiment. Here are a couple of hypothetical examples.
Example 1
In a study of PTSD, 100 police officer participants from the Winnipeg police department were randomly assigned to either an experiment or control group. All of the police officer participants, from both the experiment and the control group were given the exact same pre-test to assess their levels of PTSD. No significant differences in reported levels of symptoms related to PTSD were found between the experimental and control groups during the pre-test. Participants in the experimental group were then asked to watch a video on scenic travel routes in Manitoba. Both groups then underwent a post-test to re-measure their reported level of symptoms related to PTSD. Upon measuring the scores from the post-test, the researchers discovered that those who had received the experimental stimulus (the video on the car accident) reported greater symptoms of PTSD than those in the control group.
As you can see from Example 1, the dependent variable is reported levels of PTSD symptoms (measured through the pre- and post-test) and the independent variable is visual exposure to trauma (video). Ask yourself: Is the reported level of PTSD symptoms dependent upon visual exposure to trauma (as depicted through the video)? Table 6.1 depicts the design of the study from example 1, above.
Table 6.1 True Experiment Design
| Pretest | Treatment | Posttest |
| 01 | XE | 02 |
| 01 | XC | 02 |
Where:
- X stands for the treatment
- E stands for the experimental group (e.g., car accident video)
- C stands for the control or comparison group (e.g., scenic byways of Manitoba video)
- O stands for time, subscripts stand for time: 1=time one; 2=time two.
Example 2
In one portion of a multifaceted study on depression, all participants were randomly assigned to either an experimental or a control group. All participants were given a pre-test to assess their levels of depression. No significant differences in depression were found between the experimental and control groups during the pre- test. Participants in the experimental group were then asked to read an article suggesting that prejudice against their same racial group is severe and pervasive. Upon measuring depression scores during the post-test period, the researchers discovered that those who had received the experimental stimulus (the article citing the prejudice against their same racial group) reported greater depression than those in the control group (McCoy & Major, 2003).
Now it is your turn. See if you can fill in Table 6.2, based upon what you read in Example 2.
Table 6.2 True Experiment Design
| Pretest | Treatment | Posttest |
Where:
- X stands for the treatment.
- E stands for the experimental group (e.g., ).
- C stands for the control or comparison group (e.g., ).
- O stands for time, subscript stands for ( ).
- The dependent variable is ).
- The independent variable is ).
Answer for Table 6.2, a true experiment design
| Pretest | Treatment | Posttest |
| 01 | XE | 02 |
| 01 | XC | 02 |
Where:
- X stands for treatment.
- E stands for the experimental group (e.g., article on severe prejudice within group).
- C stands for the control or comparison group (e.g., article on severe prejudice outside group).
- O stands for time, 1 and 2 subscripts stand for time: 1=timeone;2=timetwo.
- The dependent variable is depression.
- The independent variable is feelings that prejudice is a significant issue within your racial group.
