{"id":49,"date":"2018-03-28T19:58:49","date_gmt":"2018-03-28T23:58:49","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/chapter\/5-4-practical-considerations\/"},"modified":"2018-03-28T20:21:10","modified_gmt":"2018-03-29T00:21:10","slug":"5-4-practical-considerations","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/chapter\/5-4-practical-considerations\/","title":{"raw":"5.4 Practical Considerations","rendered":"5.4 Practical Considerations"},"content":{"raw":"\n<div class=\"bcc-box bcc-highlight\">\n<h3>Learning Objectives<\/h3>\n<ol class=\"c28 lst-kix_list_84-0 start\" start=\"1\">\n<li class=\"c7 c23 c36\"><span class=\"c13 c1\">Describe several strategies for recruiting participants for an experiment.<\/span><\/li>\n<li class=\"c7 c23 c36\">Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.<\/li>\n<li class=\"c7 c23 c36\"><span class=\"c13 c1\">Explain why it is important to standardize the procedure of an experiment and several ways to do this.<\/span><\/li>\n<li class=\"c33 c23 c36\"><span class=\"c1 c13\">Explain what pilot testing is and why it is important.<\/span><\/li>\n<\/ol>\n<\/div>\n<p class=\"c4\"><span class=\"c1\">The information presented so far in this chapter is enough to design a basic experiment. When it comes time to conduct that experiment, however, several additional practical issues arise. In this section, we consider some of these issues and how to deal with them. Much of this information applies to nonexperimental studies as well as experimental ones.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Recruiting Participants<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">Of course, <\/span><span class=\"c1\">at the start of any research project, <\/span><span class=\"c1\">you should be thinking about how you will obtain your participants. Unless you have access to people with schizophrenia or incarcerated juvenile offenders, for example, then there is no point designing a study that focuses on these populations. But even if you plan to use a <\/span><span class=\"c1\">convenience <\/span><sup><a id=\"cmnt_ref8\" href=\"#cmnt8\" name=\"cmnt_ref8\"><\/a><\/sup><sup><a id=\"cmnt_ref9\" href=\"#cmnt9\" name=\"cmnt_ref9\"><\/a><\/sup><span class=\"c1\">sample, you will have to recruit participants for your study.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">There are several approaches to recruiting participants. One is to use participants from a formal&nbsp;<\/span><strong><span class=\"c35 c1\">subject&nbsp;pool<\/span><\/strong><span class=\"c1\">\u2014an established group of people who have agreed to be contacted about participating in research studies. For example, at many colleges and universities, there is a subject pool consisting of students enrolled in introductory psychology courses who must participate in a certain number of studies to meet a course requirement. Researchers post descriptions of their studies and students sign up to participate, usually via an online system. Participants who are not in subject pools can also be recruited by posting or publishing advertisements or making personal appeals to groups that represent the population of interest. For example, a researcher interested in studying older adults could arrange to speak at a meeting of the residents at a retirement community to explain the study and ask for volunteers.<\/span><\/p>\n[caption id=\"\" align=\"aligncenter\" width=\"314\"]<img class=\"\" src=\"https:\/\/pressbooks.bccampus.ca\/researchmethodsinpsychology\/wp-content\/uploads\/sites\/63\/2016\/10\/study.png#fixme#fixme\" alt=\"image\" width=\"314\" height=\"431\"> \"Study\" Retrieved from http:\/\/imgs.xkcd.com\/comics\/study.png (CC-BY-NC 2.5)[\/caption]\n<div class=\"bcc-box bcc-info\">\n<h4><b>The Volunteer Subject<\/b><\/h4>\n<p>Even if the participants in a study receive compensation in the form of course credit, a small amount of money, or a chance at being treated for a psychological problem, they are still essentially volunteers. This is worth considering because people who volunteer to participate in psychological research have been shown to differ in predictable ways from those who do not volunteer. Specifically, there is good evidence that on average, volunteers have the following characteristics compared with non-volunteers (Rosenthal &amp; Rosnow, 1976)[footnote]Rosenthal, R., &amp; Rosnow, R. L. (1976). <em>The volunteer subject<\/em>. New York, NY: Wiley.[\/footnote]:<\/p>\n<ul class=\"c28 lst-kix_list_85-0 start\">\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are more interested in the topic of the research.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are more educated.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They have a greater need for approval.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They have higher intelligence quotients (IQs).<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are more sociable.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are higher in social class.<\/span><\/li>\n<\/ul>\n<p class=\"c33\"><span class=\"c5 c1\">This difference can be an issue of external validity if there is a reason to believe that participants with these characteristics are likely to behave differently than the general population. For example, in testing different methods of persuading people, a rational argument might work better on volunteers than it does on the general population because of their generally higher educational level and IQ.<\/span><\/p>\n<\/div>\n<p><span class=\"c1\">In many field experiments, the task is not recruiting participants but selecting them. For example, researchers Nicolas Gu\u00e9guen and Marie-Agn\u00e8s de Gail conducted a field experiment on the effect of being smiled at on helping, in which the participants were shoppers at a supermarket. A confederate walking down a stairway gazed directly at a shopper walking up the stairway and either smiled or did not smile. Shortly afterward, the shopper encountered another confederate, who dropped some computer diskettes on the ground. The dependent variable was whether or not the shopper stopped to help pick up the diskettes (Gu\u00e9guen &amp; de Gail, 2003)<\/span><span class=\"c22\">[footnote]Gu\u00e9guen, N., &amp; de Gail, Marie-Agn\u00e8s. (2003). The effect of smiling on helping behavior: Smiling and good Samaritan behavior. <em>Communication Reports, 16<\/em>, 133\u2013140.[\/footnote]. <\/span><span class=\"c1\">Notice that these participants were not \u201crecruited,\u201d but the researchers still had to select them from among all the shoppers taking the stairs that day. It is extremely important that this kind of selection be done according to a well-defined set of rules that are established before the data collection begins and can be explained clearly afterward. In this case, with each trip down the stairs, the confederate was instructed to gaze at the first person he encountered who appeared to be between the ages of 20 and 50. Only if the person gazed back did he or she become a participant in the study. The point of having a well-defined selection rule is to avoid bias in the selection of participants. For example, if the confederate was free to choose which shoppers he would gaze at, he might choose friendly-looking shoppers when he was set to smile and unfriendly-looking ones when he was not set to smile. As we will see shortly, such biases can be entirely unintentional.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c2 c1\">Treatment and Control Conditions<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a&nbsp;<\/span><strong><span class=\"c35 c1\">treatment<\/span><\/strong><span class=\"c1\">&nbsp;is any intervention meant to change people\u2019s behavior for the better. This<\/span><span class=\"c1\">&nbsp;intervention<\/span><span class=\"c1\">&nbsp;includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a&nbsp;<\/span><strong><span class=\"c35 c1\">treatment&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which they receive the treatment, or a <\/span><strong><span class=\"c35 c1\">control&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition\u2014for example, they are less depressed, learn faster, conserve more, express less prejudice\u2014then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a <\/span><strong><span class=\"c35 c1\">randomized&nbsp;clinical&nbsp;trial<\/span><\/strong><span class=\"c1\">.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">There are different types of control conditions. In a&nbsp;<\/span><strong><span class=\"c35 c1\">no-treatment&nbsp;control&nbsp;condition<\/span><\/strong><span class=\"c1\">, participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A&nbsp;<\/span><strong><span class=\"c35 c1\">placebo<\/span><\/strong><span class=\"c1\">&nbsp;is a simulated treatment that lacks any active ingredient or element that should make it effective, and a&nbsp;<\/span><strong><span class=\"c35 c1\">placebo&nbsp;effect<\/span><\/strong><span class=\"c1\">&nbsp;is a positive effect of such a treatment. Many folk remedies that seem to work\u2014such as eating chicken soup for a cold or placing soap under the bed sheets to stop nighttime leg cramps\u2014are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people\u2019s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, &amp; Benedetti, 2008)<\/span>[footnote]Price, D. D., Finniss, D. G., &amp; Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. <em>Annual Review of Psychology, 59<\/em>, 565\u2013590.[\/footnote].<\/p>\n<p><span class=\"c1\">Placebo effects are interesting in their own right (see&nbsp;<\/span><span class=\"c22\">Note \"The Powerful Placebo\"<\/span><span class=\"c1\">), but they also pose a serious problem for researchers who want to determine whether a treatment works.&nbsp;<\/span><span class=\"c22\">Figure 5.2<\/span><span class=\"c1\">&nbsp;shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in&nbsp;<\/span><span class=\"c22\">Figure 5.2<\/span><span class=\"c1\">) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.<\/span><\/p>\n[caption id=\"attachment_382\" align=\"aligncenter\" width=\"900\"]<a href=\"http:\/\/opentextbc.ca\/researchmethods\/wp-content\/uploads\/sites\/37\/2015\/09\/6.2.png\"><img class=\"wp-image-382 size-full\" src=\"https:\/\/pressbooks.bccampus.ca\/researchmethodsinpsychology\/wp-content\/uploads\/sites\/63\/2016\/10\/6.2.png#fixme#fixme\" alt=\"Figure 6.2 Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions\" width=\"900\" height=\"436\"><\/a> Figure 5.2 Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions[\/caption]\n<p class=\"c4\"><span class=\"c1\">Fortunately, there are several solutions to this problem. One is to include a <\/span><strong><span class=\"c35 c1\">placebo&nbsp;control&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment\u2019s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a \u201csugar pill\u201d). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one\u2019s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants\u2019 expectations. This<\/span><span class=\"c1\">&nbsp;difference<\/span><span class=\"c1\">&nbsp;is what is shown by a comparison of the two outer bars in&nbsp;<\/span><span class=\"c22\">Figure 5.4<\/span><span class=\"c1\">.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition\u2014even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a <\/span><strong><span class=\"c35 c1\">wait-list control&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This <\/span><span class=\"c1\">disclosure <\/span><span class=\"c1\">allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply \u201cDoes it work?\u201d but \u201cDoes it work better than what is already available?<\/span><\/p>\n<div class=\"bcc-box bcc-info\">\n<h4 class=\"c7\"><span class=\"c5 c60 c67 c1 c97\">The Powerful Placebo<\/span><\/h4>\n<p class=\"c4\"><span class=\"c5 c1\">Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro &amp; Shapiro, 1999)<\/span><span class=\"c22 c5\">[footnote]Shapiro, A. K., &amp; Shapiro, E. (1999). <em>The powerful placebo: From ancient priest to modern physician<\/em>. Baltimore, MD: Johns Hopkins University Press.[\/footnote].<\/span><span class=\"c5 c1\"> There is even evidence that placebo surgery\u2014also called \u201csham surgery\u201d\u2014can be as effective as actual surgery.<\/span><\/p>\n<p class=\"c4\"><span class=\"c5 c1\">Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002)<\/span><span class=\"c22 c5\">[footnote]Moseley, J. B., O\u2019Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., \u2026 Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. <em>The New England Journal of Medicine, 347<\/em>, 81\u201388.[\/footnote]. <\/span><span class=\"c5 c1\">The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, \u201cThis study provides strong evidence that arthroscopic lavage with or without d\u00e9bridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function\u201d (p. 85).<\/span><\/p>\n<\/div>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Standardizing the Procedure<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">It is surprisingly easy to introduce extraneous variables during the procedure. For example, the same experimenter might give clear instructions to one participant but vague instructions to another. Or one experimenter might greet participants warmly while another barely makes eye contact with them. To the extent that such variables affect participants\u2019 behavior, they add noise to the data and make the effect of the independent variable more difficult to detect. If they vary systematically across conditions, they become confounding variables and provide alternative explanations for the results. For example, if participants in a treatment group are tested by a warm and friendly experimenter and participants in a control group are tested by a cold and unfriendly one, then what appears to be an effect of the treatment might actually be an effect of experimenter demeanor. When there are multiple experimenters, the possibility of introducing extraneous variables is even greater, but is often necessary for practical reasons.<\/span><\/p>\n<div class=\"bcc-box bcc-info\">\n<h4 class=\"c7\"><span class=\"c5 c60 c67 c1 c97\">Experimenter\u2019s Sex as an Extraneous Variable<\/span><\/h4>\n<p class=\"c33 c70\"><span class=\"c5 c1\">It is well known that whether research participants are male or female can affect the results of a study. But what about whether the&nbsp;<\/span><span class=\"c5 c8 c1\">experimenter<\/span><span class=\"c5 c1\">&nbsp;is male or female? There is plenty of evidence that this matters too. Male and female experimenters have slightly different ways of interacting with their participants, and of course, participants also respond differently to male and female experimenters (Rosenthal, 1976)<\/span>[footnote]Rosenthal, R. (1976). <em>Experimenter effects in behavioral research<\/em> (enlarged ed.). New York, NY: Wiley.[\/footnote].<\/p>\n<p class=\"c33 c70\"><span class=\"c5 c1\">For example, in a recent study on pain perception, participants immersed their hands in icy water for as long as they could (Ibolya, Brake, &amp; Voss, 2004)<\/span><span class=\"c29 c22 c5\">[footnote]Ibolya, K., Brake, A., &amp; Voss, U. (2004). The effect of experimenter characteristics on pain reports in women and men. <em>Pain, 112<\/em>, 142\u2013147.[\/footnote].<\/span><span class=\"c5 c1\"> Male participants tolerated the pain longer when the experimenter was a woman, and female participants tolerated it longer when the experimenter was a man.<\/span><\/p>\n<\/div>\n<p class=\"c4\"><span class=\"c1\">Researcher Robert Rosenthal has spent much of his career showing that this kind of unintended variation in the procedure does, in fact, affect participants\u2019 behavior. Furthermore, one important source of such variation is the experimenter\u2019s expectations about how participants \u201cshould\u201d behave in the experiment. This outcome is referred to as an&nbsp;<\/span><strong><span class=\"c35 c1\">experimenter&nbsp;expectancy&nbsp;effect<\/span><\/strong><span class=\"c1\">&nbsp;(Rosenthal, 1976)<\/span><span class=\"c29 c22\">[footnote]Rosenthal, R. (1976). <em>Experimenter effects in behavioral research<\/em> (enlarged ed.). New York, NY: Wiley.[\/footnote]. <\/span><span class=\"c1\">For example, if an experimenter expects participants in a treatment group to perform better on a task than participants in a control group, then he or she might unintentionally give the treatment group participants clearer instructions or more encouragement or allow them more time to complete the task. In a striking example, Rosenthal and Kermit Fode had several students in a laboratory course in psychology train rats to run through a maze. Although the rats were genetically similar, some of the students were told that they were working with \u201cmaze-bright\u201d rats that had been bred to be good learners, and other students were told that they were working with \u201cmaze-dull\u201d rats that had been bred to be poor learners. Sure enough, over five days of training, the \u201cmaze-bright\u201d rats made more correct responses, made the correct response more quickly, and improved more steadily than the \u201cmaze-dull\u201d rats (Rosenthal &amp; Fode, 1963)<\/span><span class=\"c22\">[footnote]Rosenthal, R., &amp; Fode, K. (1963). The effect of experimenter bias on performance of the albino rat. <em>Behavioral Science, 8<\/em>, 183-189.[\/footnote].<\/span><span class=\"c1\"> Clearly, it had to have been the students\u2019 expectations about how the rats would perform that made the difference. But how? Some clues come from data gathered at the end of the study, which showed that students who expected their rats to learn quickly felt more positively about their animals and reported behaving toward them in a more friendly manner (e.g., handling them more).<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">The way to minimize unintended variation in the procedure is to standardize it as much as possible so that it is carried out in the same way for all participants regardless of the condition they are in. Here are several ways to do this:<\/span><\/p>\n<ul class=\"c28 lst-kix_list_86-0 start\">\n<li class=\"c4 c56 c36\"><span class=\"c1 c15\">Create a written protocol that specifies everything that the experimenters are to do and say from the time they greet participants to the time they dismiss them.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Create standard instructions that participants read themselves or that are read to them word for word by the experimenter.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Automate the rest of the procedure as much as possible by using software packages for this purpose or even simple computer slide shows.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Anticipate participants\u2019 questions and either raise and answer them in the instructions or develop standard answers for them.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Train multiple experimenters on the protocol together and have them practice on each other.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Be sure that each experimenter tests participants in all conditions.<\/span><\/li>\n<\/ul>\n<p class=\"c4\"><span class=\"c1\">Another good practice is to arrange for the experimenters to be \u201cblind\u201d to the research question or to the condition in which each participant is tested. The idea is to minimize experimenter expectancy effects by minimizing the experimenters\u2019 expectations. For example, in a drug study in which each participant receives the drug or a placebo, it is often the case that neither the participants nor the experimenter who interacts with the participants knows which condition he or she has been assigned to complete. Because both the participants and the experimenters are blind to the condition, this technique is referred to as a&nbsp;<\/span><strong><span class=\"c35 c1\">double-blind&nbsp;study<\/span><\/strong><span class=\"c1\">. (A single-blind study is one in which only the participant is blind to the condition.) Of course, there are many times this blinding is not possible. For example, if you are both the investigator and the only experimenter, it is not possible for you to remain blind to the research question. Also, in many studies, the experimenter&nbsp;<\/span><span class=\"c8 c1\">must<\/span><span class=\"c1\">&nbsp;know the condition because he or she must carry out the procedure in a different way in the different conditions.<\/span><\/p>\n[caption id=\"\" align=\"aligncenter\" width=\"703\"]<img class=\"\" src=\"https:\/\/pressbooks.bccampus.ca\/researchmethodsinpsychology\/wp-content\/uploads\/sites\/63\/2016\/10\/placebo_blocker.png#fixme#fixme\" alt=\"image\" width=\"703\" height=\"282\"> \"Placebo Blocker\" retrieved from http:\/\/imgs.xkcd.com\/comics\/placebo_blocker.png (CC-BY-NC 2.5)[\/caption]\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Record Keeping<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">It is essential to keep good records when you conduct an experiment. As discussed earlier, it is typical for experimenters to generate a written sequence of conditions before the study begins and then to test each new participant in the next condition in the sequence. As you test them, it is a good idea to add to this list basic demographic information; the date, time, and place of testing; and the name of the experimenter who did the testing. It is also a good idea to have a place for the experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up. This kind of information can be useful later if you decide to analy<\/span><span class=\"c1\">z<\/span><span class=\"c1\">e sex differences or effects of different experimenters, or if a question arises about a particular participant or testing session.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Since participants' identities should be kept as confidential (or anonymous) as possible, their names and other identifying information should not be included with their data. In order to identify individual participants, it can, therefore, be useful to assign an identification number to each participant as you test them. Simply numbering them consecutively beginning with 1 is usually sufficient. This number can then also be written on any response sheets or questionnaires that participants generate, making it easier to keep them together.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Manipulation Check<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">In many experiments, the independent variable is a construct that can only be manipulated indirectly. For example, a researcher might try to manipulate participants\u2019 stress levels indirectly by telling some of them that they have five minutes to prepare a short speech that they will then have to give to an audience of other participants. In such situations, researchers often include a <\/span><strong><span class=\"c35 c1\">manipulation check<\/span><\/strong><span class=\"c1\">&nbsp;in their procedure. A&nbsp;manipulation&nbsp;check&nbsp;is a separate measure of the construct the researcher is trying to manipulate. The purpose of a manipulation check is to confirm that the independent variable was, in fact, successfully manipulated. For example, researchers trying to manipulate participants\u2019 stress levels might give them a paper-and-pencil stress questionnaire or take their blood pressure\u2014perhaps right after the manipulation or at the end of the procedure\u2014to verify that they successfully manipulated this variable. <\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Manipulation checks are particularly important when the results of an experiment turn out null. In cases where the results show&nbsp;no significant effect of the manipulation of the independent variable on the dependent variable, a manipulation check can help the experimenter determine whether the null result is due to a real absence of an effect of the independent variable on the dependent variable or if it is due to a problem with the manipulation of the independent variable. Imagine, for example, that you exposed participants to happy or sad movie music\u2014intending to put them in happy or sad moods\u2014but you found that this had no effect on the number of happy or sad childhood events they recalled. This could be because being in a happy or sad mood has no effect on memories for childhood events. But it could also be that the music was ineffective at putting participants in happy or sad moods. A manipulation check\u2014in this case, a measure of participants\u2019 moods\u2014would help resolve this uncertainty. If it showed that you had successfully manipulated participants\u2019 moods, then it would appear that there is indeed no effect of mood on memory for childhood events. But if it showed that you did not successfully manipulate participants\u2019 moods, then it would appear that you need a more effective manipulation to answer your research question. <\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Manipulation checks are usually done at the end of the procedure to be sure that the effect of the manipulation lasted throughout the entire procedure and to avoid calling unnecessary attention to the manipulation (to avoid a demand characteristic). However, researchers are wise to include a manipulation check in a pilot test of their experiment so that they&nbsp;avoid spending a lot of time and resources on an experiment that is doomed to fail and instead spend that time and energy finding a better manipulation of the independent variable.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Pilot Testing<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">It is always a good idea to conduct a&nbsp;<\/span><strong><span class=\"c35 c1\">pilot&nbsp;test<\/span><\/strong><span class=\"c1\">&nbsp;of your experiment. A pilot test is a small-scale study conducted to make sure that a new procedure works as planned. In a pilot test, you can recruit participants formally (e.g., from an established participant pool) or you can recruit them informally from among family, friends, classmates, and so on. The number of participants can be small, but it should be enough to give you confidence that your procedure works as planned. There are several important questions that you can answer by conducting a pilot test:<\/span><\/p>\n<ul class=\"c28 lst-kix_list_87-0 start\">\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Do participants understand the instructions?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Do participants become bored or frustrated?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Is an indirect manipulation effective? (You will need to include a manipulation check.)<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Can participants guess the research question or hypothesis (are there demand characteristics)?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">How long does the procedure take?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Are computer programs or other automated procedures working properly?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Are data being recorded correctly?<\/span><\/li>\n<\/ul>\n<p class=\"c4\"><span class=\"c1\">Of course, to answer some of these questions you will need to observe participants carefully during the procedure and talk with them about it afterward. Participants are often hesitant to criticize a study in front of the researcher, so be sure they understand that their participation is part of a pilot test and you are genuinely interested in feedback that will help you improve the procedure. If the procedure works as planned, then you can proceed with the actual study. If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed.<\/span><\/p>\n<div class=\"bcc-box bcc-success\">\n<h3>Key Takeaways<\/h3>\n<ul class=\"c28 lst-kix_list_88-0 start\">\n<li class=\"c7 c23 c36\"><span class=\"c66 c60 c1\">There are several effective methods you can use to recruit research participants for your experiment, including through formal subject pools, advertisements, and personal appeals. Field experiments require well-defined participant selection procedures.<\/span><\/li>\n<li class=\"c7 c23 c36\">Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a wait-list control condition. Experimental treatments can also be compared with the best available alternative.<\/li>\n<li class=\"c7 c23 c36\"><span class=\"c66 c60 c1\">It is important to standardize experimental procedures to minimize extraneous variables, including experimenter expectancy effects.<\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c66 c60 c1\">It is important to conduct one or more small-scale pilot tests of an experiment to be sure that the procedure works as planned.<\/span><\/li>\n<\/ul>\n<\/div>\n<div class=\"bcc-box bcc-info\">\n<h3>Exercises<\/h3>\n<ol class=\"c28 lst-kix_list_89-0 start\" start=\"1\">\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">Practice: List two ways that you might recruit participants from each of the following populations:<\/span>\n<ol>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">elderly adults <\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">unemployed people <\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">regular exercisers <\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">math majors<\/span><\/li>\n<\/ol>\n<\/li>\n<li>Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.<\/li>\n<li class=\"c33 c23 c36\"><span class=\"c10 c1\">Discussion: Imagine a study in which you will visually present participants with a list of 20 words, one at a time, wait for a short time, and then ask them to recall as many of the words as they can. In the stressed condition, they are told that they might also be chosen to give a short speech in front of a small audience. In the unstressed condition, they are not told that they might have to give a speech. What are several specific things that you could do to standardize the procedure?<\/span><\/li>\n<\/ol>\n<\/div>\n\n","rendered":"<div class=\"bcc-box bcc-highlight\">\n<h3>Learning Objectives<\/h3>\n<ol class=\"c28 lst-kix_list_84-0 start\" start=\"1\">\n<li class=\"c7 c23 c36\"><span class=\"c13 c1\">Describe several strategies for recruiting participants for an experiment.<\/span><\/li>\n<li class=\"c7 c23 c36\">Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.<\/li>\n<li class=\"c7 c23 c36\"><span class=\"c13 c1\">Explain why it is important to standardize the procedure of an experiment and several ways to do this.<\/span><\/li>\n<li class=\"c33 c23 c36\"><span class=\"c1 c13\">Explain what pilot testing is and why it is important.<\/span><\/li>\n<\/ol>\n<\/div>\n<p class=\"c4\"><span class=\"c1\">The information presented so far in this chapter is enough to design a basic experiment. When it comes time to conduct that experiment, however, several additional practical issues arise. In this section, we consider some of these issues and how to deal with them. Much of this information applies to nonexperimental studies as well as experimental ones.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Recruiting Participants<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">Of course, <\/span><span class=\"c1\">at the start of any research project, <\/span><span class=\"c1\">you should be thinking about how you will obtain your participants. Unless you have access to people with schizophrenia or incarcerated juvenile offenders, for example, then there is no point designing a study that focuses on these populations. But even if you plan to use a <\/span><span class=\"c1\">convenience <\/span><sup><a id=\"cmnt_ref8\" href=\"#cmnt8\" name=\"cmnt_ref8\"><\/a><\/sup><sup><a id=\"cmnt_ref9\" href=\"#cmnt9\" name=\"cmnt_ref9\"><\/a><\/sup><span class=\"c1\">sample, you will have to recruit participants for your study.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">There are several approaches to recruiting participants. One is to use participants from a formal&nbsp;<\/span><strong><span class=\"c35 c1\">subject&nbsp;pool<\/span><\/strong><span class=\"c1\">\u2014an established group of people who have agreed to be contacted about participating in research studies. For example, at many colleges and universities, there is a subject pool consisting of students enrolled in introductory psychology courses who must participate in a certain number of studies to meet a course requirement. Researchers post descriptions of their studies and students sign up to participate, usually via an online system. Participants who are not in subject pools can also be recruited by posting or publishing advertisements or making personal appeals to groups that represent the population of interest. For example, a researcher interested in studying older adults could arrange to speak at a meeting of the residents at a retirement community to explain the study and ask for volunteers.<\/span><\/p>\n<figure style=\"width: 314px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"\" src=\"https:\/\/pressbooks.bccampus.ca\/researchmethodsinpsychology\/wp-content\/uploads\/sites\/63\/2016\/10\/study.png#fixme#fixme\" alt=\"image\" width=\"314\" height=\"431\" \/><figcaption class=\"wp-caption-text\">&#8220;Study&#8221; Retrieved from http:\/\/imgs.xkcd.com\/comics\/study.png (CC-BY-NC 2.5)<\/figcaption><\/figure>\n<div class=\"bcc-box bcc-info\">\n<h4><b>The Volunteer Subject<\/b><\/h4>\n<p>Even if the participants in a study receive compensation in the form of course credit, a small amount of money, or a chance at being treated for a psychological problem, they are still essentially volunteers. This is worth considering because people who volunteer to participate in psychological research have been shown to differ in predictable ways from those who do not volunteer. Specifically, there is good evidence that on average, volunteers have the following characteristics compared with non-volunteers (Rosenthal &amp; Rosnow, 1976)<a class=\"footnote\" title=\"Rosenthal, R., &amp; Rosnow, R. L. (1976). The volunteer subject. New York, NY: Wiley.\" id=\"return-footnote-49-1\" href=\"#footnote-49-1\" aria-label=\"Footnote 1\"><sup class=\"footnote\">[1]<\/sup><\/a>:<\/p>\n<ul class=\"c28 lst-kix_list_85-0 start\">\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are more interested in the topic of the research.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are more educated.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They have a greater need for approval.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They have higher intelligence quotients (IQs).<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are more sociable.<\/span><\/li>\n<li class=\"c7 c31\"><span class=\"c15 c5 c1\">They are higher in social class.<\/span><\/li>\n<\/ul>\n<p class=\"c33\"><span class=\"c5 c1\">This difference can be an issue of external validity if there is a reason to believe that participants with these characteristics are likely to behave differently than the general population. For example, in testing different methods of persuading people, a rational argument might work better on volunteers than it does on the general population because of their generally higher educational level and IQ.<\/span><\/p>\n<\/div>\n<p><span class=\"c1\">In many field experiments, the task is not recruiting participants but selecting them. For example, researchers Nicolas Gu\u00e9guen and Marie-Agn\u00e8s de Gail conducted a field experiment on the effect of being smiled at on helping, in which the participants were shoppers at a supermarket. A confederate walking down a stairway gazed directly at a shopper walking up the stairway and either smiled or did not smile. Shortly afterward, the shopper encountered another confederate, who dropped some computer diskettes on the ground. The dependent variable was whether or not the shopper stopped to help pick up the diskettes (Gu\u00e9guen &amp; de Gail, 2003)<\/span><span class=\"c22\"><a class=\"footnote\" title=\"Gu\u00e9guen, N., &amp; de Gail, Marie-Agn\u00e8s. (2003). The effect of smiling on helping behavior: Smiling and good Samaritan behavior. Communication Reports, 16, 133\u2013140.\" id=\"return-footnote-49-2\" href=\"#footnote-49-2\" aria-label=\"Footnote 2\"><sup class=\"footnote\">[2]<\/sup><\/a>. <\/span><span class=\"c1\">Notice that these participants were not \u201crecruited,\u201d but the researchers still had to select them from among all the shoppers taking the stairs that day. It is extremely important that this kind of selection be done according to a well-defined set of rules that are established before the data collection begins and can be explained clearly afterward. In this case, with each trip down the stairs, the confederate was instructed to gaze at the first person he encountered who appeared to be between the ages of 20 and 50. Only if the person gazed back did he or she become a participant in the study. The point of having a well-defined selection rule is to avoid bias in the selection of participants. For example, if the confederate was free to choose which shoppers he would gaze at, he might choose friendly-looking shoppers when he was set to smile and unfriendly-looking ones when he was not set to smile. As we will see shortly, such biases can be entirely unintentional.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c2 c1\">Treatment and Control Conditions<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a&nbsp;<\/span><strong><span class=\"c35 c1\">treatment<\/span><\/strong><span class=\"c1\">&nbsp;is any intervention meant to change people\u2019s behavior for the better. This<\/span><span class=\"c1\">&nbsp;intervention<\/span><span class=\"c1\">&nbsp;includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a&nbsp;<\/span><strong><span class=\"c35 c1\">treatment&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which they receive the treatment, or a <\/span><strong><span class=\"c35 c1\">control&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition\u2014for example, they are less depressed, learn faster, conserve more, express less prejudice\u2014then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a <\/span><strong><span class=\"c35 c1\">randomized&nbsp;clinical&nbsp;trial<\/span><\/strong><span class=\"c1\">.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">There are different types of control conditions. In a&nbsp;<\/span><strong><span class=\"c35 c1\">no-treatment&nbsp;control&nbsp;condition<\/span><\/strong><span class=\"c1\">, participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A&nbsp;<\/span><strong><span class=\"c35 c1\">placebo<\/span><\/strong><span class=\"c1\">&nbsp;is a simulated treatment that lacks any active ingredient or element that should make it effective, and a&nbsp;<\/span><strong><span class=\"c35 c1\">placebo&nbsp;effect<\/span><\/strong><span class=\"c1\">&nbsp;is a positive effect of such a treatment. Many folk remedies that seem to work\u2014such as eating chicken soup for a cold or placing soap under the bed sheets to stop nighttime leg cramps\u2014are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people\u2019s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, &amp; Benedetti, 2008)<\/span><a class=\"footnote\" title=\"Price, D. D., Finniss, D. G., &amp; Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59, 565\u2013590.\" id=\"return-footnote-49-3\" href=\"#footnote-49-3\" aria-label=\"Footnote 3\"><sup class=\"footnote\">[3]<\/sup><\/a>.<\/p>\n<p><span class=\"c1\">Placebo effects are interesting in their own right (see&nbsp;<\/span><span class=\"c22\">Note &#8220;The Powerful Placebo&#8221;<\/span><span class=\"c1\">), but they also pose a serious problem for researchers who want to determine whether a treatment works.&nbsp;<\/span><span class=\"c22\">Figure 5.2<\/span><span class=\"c1\">&nbsp;shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in&nbsp;<\/span><span class=\"c22\">Figure 5.2<\/span><span class=\"c1\">) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.<\/span><\/p>\n<figure id=\"attachment_382\" aria-describedby=\"caption-attachment-382\" style=\"width: 900px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/opentextbc.ca\/researchmethods\/wp-content\/uploads\/sites\/37\/2015\/09\/6.2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-382 size-full\" src=\"https:\/\/pressbooks.bccampus.ca\/researchmethodsinpsychology\/wp-content\/uploads\/sites\/63\/2016\/10\/6.2.png#fixme#fixme\" alt=\"Figure 6.2 Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions\" width=\"900\" height=\"436\" \/><\/a><figcaption id=\"caption-attachment-382\" class=\"wp-caption-text\">Figure 5.2 Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions<\/figcaption><\/figure>\n<p class=\"c4\"><span class=\"c1\">Fortunately, there are several solutions to this problem. One is to include a <\/span><strong><span class=\"c35 c1\">placebo&nbsp;control&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment\u2019s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a \u201csugar pill\u201d). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one\u2019s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants\u2019 expectations. This<\/span><span class=\"c1\">&nbsp;difference<\/span><span class=\"c1\">&nbsp;is what is shown by a comparison of the two outer bars in&nbsp;<\/span><span class=\"c22\">Figure 5.4<\/span><span class=\"c1\">.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition\u2014even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a <\/span><strong><span class=\"c35 c1\">wait-list control&nbsp;condition<\/span><\/strong><span class=\"c1\">, in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This <\/span><span class=\"c1\">disclosure <\/span><span class=\"c1\">allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply \u201cDoes it work?\u201d but \u201cDoes it work better than what is already available?<\/span><\/p>\n<div class=\"bcc-box bcc-info\">\n<h4 class=\"c7\"><span class=\"c5 c60 c67 c1 c97\">The Powerful Placebo<\/span><\/h4>\n<p class=\"c4\"><span class=\"c5 c1\">Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro &amp; Shapiro, 1999)<\/span><span class=\"c22 c5\"><a class=\"footnote\" title=\"Shapiro, A. K., &amp; Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician. Baltimore, MD: Johns Hopkins University Press.\" id=\"return-footnote-49-4\" href=\"#footnote-49-4\" aria-label=\"Footnote 4\"><sup class=\"footnote\">[4]<\/sup><\/a>.<\/span><span class=\"c5 c1\"> There is even evidence that placebo surgery\u2014also called \u201csham surgery\u201d\u2014can be as effective as actual surgery.<\/span><\/p>\n<p class=\"c4\"><span class=\"c5 c1\">Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002)<\/span><span class=\"c22 c5\"><a class=\"footnote\" title=\"Moseley, J. B., O\u2019Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., \u2026 Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347, 81\u201388.\" id=\"return-footnote-49-5\" href=\"#footnote-49-5\" aria-label=\"Footnote 5\"><sup class=\"footnote\">[5]<\/sup><\/a>. <\/span><span class=\"c5 c1\">The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, \u201cThis study provides strong evidence that arthroscopic lavage with or without d\u00e9bridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function\u201d (p. 85).<\/span><\/p>\n<\/div>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Standardizing the Procedure<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">It is surprisingly easy to introduce extraneous variables during the procedure. For example, the same experimenter might give clear instructions to one participant but vague instructions to another. Or one experimenter might greet participants warmly while another barely makes eye contact with them. To the extent that such variables affect participants\u2019 behavior, they add noise to the data and make the effect of the independent variable more difficult to detect. If they vary systematically across conditions, they become confounding variables and provide alternative explanations for the results. For example, if participants in a treatment group are tested by a warm and friendly experimenter and participants in a control group are tested by a cold and unfriendly one, then what appears to be an effect of the treatment might actually be an effect of experimenter demeanor. When there are multiple experimenters, the possibility of introducing extraneous variables is even greater, but is often necessary for practical reasons.<\/span><\/p>\n<div class=\"bcc-box bcc-info\">\n<h4 class=\"c7\"><span class=\"c5 c60 c67 c1 c97\">Experimenter\u2019s Sex as an Extraneous Variable<\/span><\/h4>\n<p class=\"c33 c70\"><span class=\"c5 c1\">It is well known that whether research participants are male or female can affect the results of a study. But what about whether the&nbsp;<\/span><span class=\"c5 c8 c1\">experimenter<\/span><span class=\"c5 c1\">&nbsp;is male or female? There is plenty of evidence that this matters too. Male and female experimenters have slightly different ways of interacting with their participants, and of course, participants also respond differently to male and female experimenters (Rosenthal, 1976)<\/span><a class=\"footnote\" title=\"Rosenthal, R. (1976). Experimenter effects in behavioral research (enlarged ed.). New York, NY: Wiley.\" id=\"return-footnote-49-6\" href=\"#footnote-49-6\" aria-label=\"Footnote 6\"><sup class=\"footnote\">[6]<\/sup><\/a>.<\/p>\n<p class=\"c33 c70\"><span class=\"c5 c1\">For example, in a recent study on pain perception, participants immersed their hands in icy water for as long as they could (Ibolya, Brake, &amp; Voss, 2004)<\/span><span class=\"c29 c22 c5\"><a class=\"footnote\" title=\"Ibolya, K., Brake, A., &amp; Voss, U. (2004). The effect of experimenter characteristics on pain reports in women and men. Pain, 112, 142\u2013147.\" id=\"return-footnote-49-7\" href=\"#footnote-49-7\" aria-label=\"Footnote 7\"><sup class=\"footnote\">[7]<\/sup><\/a>.<\/span><span class=\"c5 c1\"> Male participants tolerated the pain longer when the experimenter was a woman, and female participants tolerated it longer when the experimenter was a man.<\/span><\/p>\n<\/div>\n<p class=\"c4\"><span class=\"c1\">Researcher Robert Rosenthal has spent much of his career showing that this kind of unintended variation in the procedure does, in fact, affect participants\u2019 behavior. Furthermore, one important source of such variation is the experimenter\u2019s expectations about how participants \u201cshould\u201d behave in the experiment. This outcome is referred to as an&nbsp;<\/span><strong><span class=\"c35 c1\">experimenter&nbsp;expectancy&nbsp;effect<\/span><\/strong><span class=\"c1\">&nbsp;(Rosenthal, 1976)<\/span><span class=\"c29 c22\"><a class=\"footnote\" title=\"Rosenthal, R. (1976). Experimenter effects in behavioral research (enlarged ed.). New York, NY: Wiley.\" id=\"return-footnote-49-8\" href=\"#footnote-49-8\" aria-label=\"Footnote 8\"><sup class=\"footnote\">[8]<\/sup><\/a>. <\/span><span class=\"c1\">For example, if an experimenter expects participants in a treatment group to perform better on a task than participants in a control group, then he or she might unintentionally give the treatment group participants clearer instructions or more encouragement or allow them more time to complete the task. In a striking example, Rosenthal and Kermit Fode had several students in a laboratory course in psychology train rats to run through a maze. Although the rats were genetically similar, some of the students were told that they were working with \u201cmaze-bright\u201d rats that had been bred to be good learners, and other students were told that they were working with \u201cmaze-dull\u201d rats that had been bred to be poor learners. Sure enough, over five days of training, the \u201cmaze-bright\u201d rats made more correct responses, made the correct response more quickly, and improved more steadily than the \u201cmaze-dull\u201d rats (Rosenthal &amp; Fode, 1963)<\/span><span class=\"c22\"><a class=\"footnote\" title=\"Rosenthal, R., &amp; Fode, K. (1963). The effect of experimenter bias on performance of the albino rat. Behavioral Science, 8, 183-189.\" id=\"return-footnote-49-9\" href=\"#footnote-49-9\" aria-label=\"Footnote 9\"><sup class=\"footnote\">[9]<\/sup><\/a>.<\/span><span class=\"c1\"> Clearly, it had to have been the students\u2019 expectations about how the rats would perform that made the difference. But how? Some clues come from data gathered at the end of the study, which showed that students who expected their rats to learn quickly felt more positively about their animals and reported behaving toward them in a more friendly manner (e.g., handling them more).<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">The way to minimize unintended variation in the procedure is to standardize it as much as possible so that it is carried out in the same way for all participants regardless of the condition they are in. Here are several ways to do this:<\/span><\/p>\n<ul class=\"c28 lst-kix_list_86-0 start\">\n<li class=\"c4 c56 c36\"><span class=\"c1 c15\">Create a written protocol that specifies everything that the experimenters are to do and say from the time they greet participants to the time they dismiss them.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Create standard instructions that participants read themselves or that are read to them word for word by the experimenter.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Automate the rest of the procedure as much as possible by using software packages for this purpose or even simple computer slide shows.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Anticipate participants\u2019 questions and either raise and answer them in the instructions or develop standard answers for them.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Train multiple experimenters on the protocol together and have them practice on each other.<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Be sure that each experimenter tests participants in all conditions.<\/span><\/li>\n<\/ul>\n<p class=\"c4\"><span class=\"c1\">Another good practice is to arrange for the experimenters to be \u201cblind\u201d to the research question or to the condition in which each participant is tested. The idea is to minimize experimenter expectancy effects by minimizing the experimenters\u2019 expectations. For example, in a drug study in which each participant receives the drug or a placebo, it is often the case that neither the participants nor the experimenter who interacts with the participants knows which condition he or she has been assigned to complete. Because both the participants and the experimenters are blind to the condition, this technique is referred to as a&nbsp;<\/span><strong><span class=\"c35 c1\">double-blind&nbsp;study<\/span><\/strong><span class=\"c1\">. (A single-blind study is one in which only the participant is blind to the condition.) Of course, there are many times this blinding is not possible. For example, if you are both the investigator and the only experimenter, it is not possible for you to remain blind to the research question. Also, in many studies, the experimenter&nbsp;<\/span><span class=\"c8 c1\">must<\/span><span class=\"c1\">&nbsp;know the condition because he or she must carry out the procedure in a different way in the different conditions.<\/span><\/p>\n<figure style=\"width: 703px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"\" src=\"https:\/\/pressbooks.bccampus.ca\/researchmethodsinpsychology\/wp-content\/uploads\/sites\/63\/2016\/10\/placebo_blocker.png#fixme#fixme\" alt=\"image\" width=\"703\" height=\"282\" \/><figcaption class=\"wp-caption-text\">&#8220;Placebo Blocker&#8221; retrieved from http:\/\/imgs.xkcd.com\/comics\/placebo_blocker.png (CC-BY-NC 2.5)<\/figcaption><\/figure>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Record Keeping<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">It is essential to keep good records when you conduct an experiment. As discussed earlier, it is typical for experimenters to generate a written sequence of conditions before the study begins and then to test each new participant in the next condition in the sequence. As you test them, it is a good idea to add to this list basic demographic information; the date, time, and place of testing; and the name of the experimenter who did the testing. It is also a good idea to have a place for the experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up. This kind of information can be useful later if you decide to analy<\/span><span class=\"c1\">z<\/span><span class=\"c1\">e sex differences or effects of different experimenters, or if a question arises about a particular participant or testing session.<\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Since participants&#8217; identities should be kept as confidential (or anonymous) as possible, their names and other identifying information should not be included with their data. In order to identify individual participants, it can, therefore, be useful to assign an identification number to each participant as you test them. Simply numbering them consecutively beginning with 1 is usually sufficient. This number can then also be written on any response sheets or questionnaires that participants generate, making it easier to keep them together.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Manipulation Check<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">In many experiments, the independent variable is a construct that can only be manipulated indirectly. For example, a researcher might try to manipulate participants\u2019 stress levels indirectly by telling some of them that they have five minutes to prepare a short speech that they will then have to give to an audience of other participants. In such situations, researchers often include a <\/span><strong><span class=\"c35 c1\">manipulation check<\/span><\/strong><span class=\"c1\">&nbsp;in their procedure. A&nbsp;manipulation&nbsp;check&nbsp;is a separate measure of the construct the researcher is trying to manipulate. The purpose of a manipulation check is to confirm that the independent variable was, in fact, successfully manipulated. For example, researchers trying to manipulate participants\u2019 stress levels might give them a paper-and-pencil stress questionnaire or take their blood pressure\u2014perhaps right after the manipulation or at the end of the procedure\u2014to verify that they successfully manipulated this variable. <\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Manipulation checks are particularly important when the results of an experiment turn out null. In cases where the results show&nbsp;no significant effect of the manipulation of the independent variable on the dependent variable, a manipulation check can help the experimenter determine whether the null result is due to a real absence of an effect of the independent variable on the dependent variable or if it is due to a problem with the manipulation of the independent variable. Imagine, for example, that you exposed participants to happy or sad movie music\u2014intending to put them in happy or sad moods\u2014but you found that this had no effect on the number of happy or sad childhood events they recalled. This could be because being in a happy or sad mood has no effect on memories for childhood events. But it could also be that the music was ineffective at putting participants in happy or sad moods. A manipulation check\u2014in this case, a measure of participants\u2019 moods\u2014would help resolve this uncertainty. If it showed that you had successfully manipulated participants\u2019 moods, then it would appear that there is indeed no effect of mood on memory for childhood events. But if it showed that you did not successfully manipulate participants\u2019 moods, then it would appear that you need a more effective manipulation to answer your research question. <\/span><\/p>\n<p class=\"c4\"><span class=\"c1\">Manipulation checks are usually done at the end of the procedure to be sure that the effect of the manipulation lasted throughout the entire procedure and to avoid calling unnecessary attention to the manipulation (to avoid a demand characteristic). However, researchers are wise to include a manipulation check in a pilot test of their experiment so that they&nbsp;avoid spending a lot of time and resources on an experiment that is doomed to fail and instead spend that time and energy finding a better manipulation of the independent variable.<\/span><\/p>\n<h2 class=\"c4\" style=\"text-align: left\"><strong><span class=\"c18 c1\">Pilot Testing<\/span><\/strong><\/h2>\n<p class=\"c4\"><span class=\"c1\">It is always a good idea to conduct a&nbsp;<\/span><strong><span class=\"c35 c1\">pilot&nbsp;test<\/span><\/strong><span class=\"c1\">&nbsp;of your experiment. A pilot test is a small-scale study conducted to make sure that a new procedure works as planned. In a pilot test, you can recruit participants formally (e.g., from an established participant pool) or you can recruit them informally from among family, friends, classmates, and so on. The number of participants can be small, but it should be enough to give you confidence that your procedure works as planned. There are several important questions that you can answer by conducting a pilot test:<\/span><\/p>\n<ul class=\"c28 lst-kix_list_87-0 start\">\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Do participants understand the instructions?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Do participants become bored or frustrated?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Is an indirect manipulation effective? (You will need to include a manipulation check.)<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Can participants guess the research question or hypothesis (are there demand characteristics)?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">How long does the procedure take?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Are computer programs or other automated procedures working properly?<\/span><\/li>\n<li class=\"c4 c56 c36\"><span class=\"c15 c1\">Are data being recorded correctly?<\/span><\/li>\n<\/ul>\n<p class=\"c4\"><span class=\"c1\">Of course, to answer some of these questions you will need to observe participants carefully during the procedure and talk with them about it afterward. Participants are often hesitant to criticize a study in front of the researcher, so be sure they understand that their participation is part of a pilot test and you are genuinely interested in feedback that will help you improve the procedure. If the procedure works as planned, then you can proceed with the actual study. If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed.<\/span><\/p>\n<div class=\"bcc-box bcc-success\">\n<h3>Key Takeaways<\/h3>\n<ul class=\"c28 lst-kix_list_88-0 start\">\n<li class=\"c7 c23 c36\"><span class=\"c66 c60 c1\">There are several effective methods you can use to recruit research participants for your experiment, including through formal subject pools, advertisements, and personal appeals. Field experiments require well-defined participant selection procedures.<\/span><\/li>\n<li class=\"c7 c23 c36\">Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a wait-list control condition. Experimental treatments can also be compared with the best available alternative.<\/li>\n<li class=\"c7 c23 c36\"><span class=\"c66 c60 c1\">It is important to standardize experimental procedures to minimize extraneous variables, including experimenter expectancy effects.<\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c66 c60 c1\">It is important to conduct one or more small-scale pilot tests of an experiment to be sure that the procedure works as planned.<\/span><\/li>\n<\/ul>\n<\/div>\n<div class=\"bcc-box bcc-info\">\n<h3>Exercises<\/h3>\n<ol class=\"c28 lst-kix_list_89-0 start\" start=\"1\">\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">Practice: List two ways that you might recruit participants from each of the following populations:<\/span>\n<ol>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">elderly adults <\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">unemployed people <\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">regular exercisers <\/span><\/li>\n<li class=\"c7 c23 c36\"><span class=\"c10 c1\">math majors<\/span><\/li>\n<\/ol>\n<\/li>\n<li>Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.<\/li>\n<li class=\"c33 c23 c36\"><span class=\"c10 c1\">Discussion: Imagine a study in which you will visually present participants with a list of 20 words, one at a time, wait for a short time, and then ask them to recall as many of the words as they can. In the stressed condition, they are told that they might also be chosen to give a short speech in front of a small audience. In the unstressed condition, they are not told that they might have to give a speech. What are several specific things that you could do to standardize the procedure?<\/span><\/li>\n<\/ol>\n<\/div>\n<hr class=\"before-footnotes clear\" \/><div class=\"footnotes\"><ol><li id=\"footnote-49-1\">Rosenthal, R., &amp; Rosnow, R. L. (1976). <em>The volunteer subject<\/em>. New York, NY: Wiley. <a href=\"#return-footnote-49-1\" class=\"return-footnote\" aria-label=\"Return to footnote 1\">&crarr;<\/a><\/li><li id=\"footnote-49-2\">Gu\u00e9guen, N., &amp; de Gail, Marie-Agn\u00e8s. (2003). The effect of smiling on helping behavior: Smiling and good Samaritan behavior. <em>Communication Reports, 16<\/em>, 133\u2013140. <a href=\"#return-footnote-49-2\" class=\"return-footnote\" aria-label=\"Return to footnote 2\">&crarr;<\/a><\/li><li id=\"footnote-49-3\">Price, D. D., Finniss, D. G., &amp; Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. <em>Annual Review of Psychology, 59<\/em>, 565\u2013590. <a href=\"#return-footnote-49-3\" class=\"return-footnote\" aria-label=\"Return to footnote 3\">&crarr;<\/a><\/li><li id=\"footnote-49-4\">Shapiro, A. K., &amp; Shapiro, E. (1999). <em>The powerful placebo: From ancient priest to modern physician<\/em>. Baltimore, MD: Johns Hopkins University Press. <a href=\"#return-footnote-49-4\" class=\"return-footnote\" aria-label=\"Return to footnote 4\">&crarr;<\/a><\/li><li id=\"footnote-49-5\">Moseley, J. B., O\u2019Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., \u2026 Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. <em>The New England Journal of Medicine, 347<\/em>, 81\u201388. <a href=\"#return-footnote-49-5\" class=\"return-footnote\" aria-label=\"Return to footnote 5\">&crarr;<\/a><\/li><li id=\"footnote-49-6\">Rosenthal, R. (1976). <em>Experimenter effects in behavioral research<\/em> (enlarged ed.). New York, NY: Wiley. <a href=\"#return-footnote-49-6\" class=\"return-footnote\" aria-label=\"Return to footnote 6\">&crarr;<\/a><\/li><li id=\"footnote-49-7\">Ibolya, K., Brake, A., &amp; Voss, U. (2004). The effect of experimenter characteristics on pain reports in women and men. <em>Pain, 112<\/em>, 142\u2013147. <a href=\"#return-footnote-49-7\" class=\"return-footnote\" aria-label=\"Return to footnote 7\">&crarr;<\/a><\/li><li id=\"footnote-49-8\">Rosenthal, R. (1976). <em>Experimenter effects in behavioral research<\/em> (enlarged ed.). New York, NY: Wiley. <a href=\"#return-footnote-49-8\" class=\"return-footnote\" aria-label=\"Return to footnote 8\">&crarr;<\/a><\/li><li id=\"footnote-49-9\">Rosenthal, R., &amp; Fode, K. (1963). The effect of experimenter bias on performance of the albino rat. <em>Behavioral Science, 8<\/em>, 183-189. <a href=\"#return-footnote-49-9\" class=\"return-footnote\" aria-label=\"Return to footnote 9\">&crarr;<\/a><\/li><\/ol><\/div>","protected":false},"author":64,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[48],"contributor":[],"license":[],"class_list":["post-49","chapter","type-chapter","status-publish","hentry","chapter-type-numberless"],"part":45,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/chapters\/49","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/wp\/v2\/users\/64"}],"version-history":[{"count":1,"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/chapters\/49\/revisions"}],"predecessor-version":[{"id":111,"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/chapters\/49\/revisions\/111"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/parts\/45"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/chapters\/49\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/wp\/v2\/media?parent=49"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/pressbooks\/v2\/chapter-type?post=49"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/wp\/v2\/contributor?post=49"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/rmip3amed\/wp-json\/wp\/v2\/license?post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}