Chapter 6: Data Collection Strategies
6.4 Internal Validity
In the preceding sections we reviewed three types of research: cross-sectional, correlational, and observational. It is important to note that when it comes to internal validity, they are not considered equal. You will recall in Chapter 20, Validity, we briefly discussed internal validity. To recap, internal validity is the extent to which the study design supports the conclusion that changes to the independent variable were responsible for the observed changes in the dependent variable.
Of the three types of research (experimental, non-experimental, and quasi-experimental), experimental research usually has the highest internal validity. This is because it addresses directional and third variable problems through manipulation and controlling for extraneous variables through random assignment (Crump et al., 2017). As Crump et al claim, if the average score on the dependent variable changes across conditions, it is likely that these changes are the result of the independent variable. On the other hand, correlational research is said to have the lowest internal validity. This is because if the average score on the dependent variable changes across conditions, it could be because of the independent variable. However, there could be other reasons, e.g., the direction of causality is reversed, or there is a third variable causing the differences in both the independent and dependent variables (Crump et al., 2017).
Quasi experimental research is considered in the middle of the two other types of research when it comes to internal validity. This is because the independent variable is manipulated in quasi-experimental research; however, the lack of random assignment and experimental control can create other problems. Quasi-experimental research is the most common methodological approach utilized in social sciences research.
Example
Suppose a researcher finds two similar fire halls in which to conduct a study on Post Traumatic Stress Disorder (PTSD). As part of her research, the researcher creates a PTSD awareness program and implements the program at one of the two fire halls. At the end of the in-field portion of her study she finds lower levels of PTSD in firefighters at the “treatment fire hall,” than in the “control fire hall (no program).” As she did not choose which fire hall received the program based upon the number of firefighters with PTSD in each fire hall, we can see that she did not have a directional problem with her study design. However, because she did not randomly assign firefighters to one of the two fire halls, it could mean that the firefighters at the treatment fire hall differed somehow from the firefighters at the control fire hall. In other words, it is the difference in the firefighters themselves (or something about their jobs, their superiors, etc.) that was responsible for the lower levels of PTSD at the treatment fire hall, not the PTSD awareness program that was applied.
