{"id":232,"date":"2025-10-03T17:41:47","date_gmt":"2025-10-03T21:41:47","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/?post_type=chapter&#038;p=232"},"modified":"2026-05-26T14:19:51","modified_gmt":"2026-05-26T18:19:51","slug":"8-2-introduction","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/chapter\/8-2-introduction\/","title":{"raw":"8.2 Introduction","rendered":"8.2 Introduction"},"content":{"raw":"Interventions are created and implemented for specific reasons and to achieve specific objectives. Confirming whether implemented interventions are effective is important for decision-makers. Evaluators have a key responsibility in determining whether the intervention\u2019s objectives were achieved. However, this question is often not easy to answer. Evaluators must ask themselves: which research strategy can be implemented to respond to this question with confidence?\r\n\r\nThe production of results is often a complex process that has implications beyond achieving the intervention objectives. It\u2019s particularly complex because interventions often produce effects other than those they were originally designed to achieve; some of these additional effects are beneficial, others not; some will affect people beyond the target population; and some effects may be unexpected.\r\n\r\nAnalyzing the results of an intervention can address different questions such as:\r\n<ul>\r\n \t<li>What are the results of the intervention?<\/li>\r\n \t<li>Did the intervention make a difference? If yes, in what way, for whom?<\/li>\r\n \t<li>Are the effects observed attributable to the intervention?<\/li>\r\n<\/ul>\r\nIn the evaluation literature, people distinguish the effects from the impacts (longer-term). In this chapter, effects will represent the chain of results, that is a continuum from outputs to outcomes to impacts (Mayne, 2015).\r\n\r\nSome evaluators specialize in effect analysis which is niche that requires specialized expertise (Shadish et al., 2002). Historically, effect analysis has been closely associated with experiments (experimental and quasi-experimental research designs). In the last 25 years, the field around the evaluation of effects has expanded. Contribution analysis was created specifically to analyze the effects of complex interventions. There is a wide range of designs to investigate effects, including for interventions that are less suited to experimental methods. Effect analysis is an evaluation question that now covers a rich and diverse set of research designs.\r\n\r\nThis chapter provides an overview of the foundational principles of effect analysis and a description of the diversity of approaches used to evaluate effects. The structure of the chapter was designed to help the reader understand the logic of inquiry behind effect analysis. It starts by defining effects and causal relationships. It then reviews experimental and quasi-experimental research. This review will help the reader understand the strengths and limitations of such approaches. As we will see, some interventions are not good candidates for these research designs. Contribution analysis, specifically designed for complex interventions, will be presented. These sections should help the reader understand how some designs can be a good fit or not, according to the evaluation contexts. Impact analysis will then be presented. Impact analysis is an umbrella term that includes a large range of approaches aiming at evaluating effects. Impact analysis, as presented by Stern and colleagues (Stern, 2015; Stern et al., 2012), includes experimental and quasi-experimental research and contribution analysis. Presenting impact analysis after having discussed more traditional designs should help the reader understand how these different approaches differ and what they bring to evaluating the effects of interventions. In this chapter, the integration of planetary health dimensions into effect and impact analysis is also explored. Finally, this chapter will also touch on outcome harvesting. Outcome harvesting is not an approach focusing on assessing causal relations, but rather an approach for evaluating interventions\u2019 effects and social change in unpredictable and changing contexts. Given that we will likely experience more unstable and fast-changing contexts, outcome harvesting may become more prominent in the upcoming years.","rendered":"<p>Interventions are created and implemented for specific reasons and to achieve specific objectives. Confirming whether implemented interventions are effective is important for decision-makers. Evaluators have a key responsibility in determining whether the intervention\u2019s objectives were achieved. However, this question is often not easy to answer. Evaluators must ask themselves: which research strategy can be implemented to respond to this question with confidence?<\/p>\n<p>The production of results is often a complex process that has implications beyond achieving the intervention objectives. It\u2019s particularly complex because interventions often produce effects other than those they were originally designed to achieve; some of these additional effects are beneficial, others not; some will affect people beyond the target population; and some effects may be unexpected.<\/p>\n<p>Analyzing the results of an intervention can address different questions such as:<\/p>\n<ul>\n<li>What are the results of the intervention?<\/li>\n<li>Did the intervention make a difference? If yes, in what way, for whom?<\/li>\n<li>Are the effects observed attributable to the intervention?<\/li>\n<\/ul>\n<p>In the evaluation literature, people distinguish the effects from the impacts (longer-term). In this chapter, effects will represent the chain of results, that is a continuum from outputs to outcomes to impacts (Mayne, 2015).<\/p>\n<p>Some evaluators specialize in effect analysis which is niche that requires specialized expertise (Shadish et al., 2002). Historically, effect analysis has been closely associated with experiments (experimental and quasi-experimental research designs). In the last 25 years, the field around the evaluation of effects has expanded. Contribution analysis was created specifically to analyze the effects of complex interventions. There is a wide range of designs to investigate effects, including for interventions that are less suited to experimental methods. Effect analysis is an evaluation question that now covers a rich and diverse set of research designs.<\/p>\n<p>This chapter provides an overview of the foundational principles of effect analysis and a description of the diversity of approaches used to evaluate effects. The structure of the chapter was designed to help the reader understand the logic of inquiry behind effect analysis. It starts by defining effects and causal relationships. It then reviews experimental and quasi-experimental research. This review will help the reader understand the strengths and limitations of such approaches. As we will see, some interventions are not good candidates for these research designs. Contribution analysis, specifically designed for complex interventions, will be presented. These sections should help the reader understand how some designs can be a good fit or not, according to the evaluation contexts. Impact analysis will then be presented. Impact analysis is an umbrella term that includes a large range of approaches aiming at evaluating effects. Impact analysis, as presented by Stern and colleagues (Stern, 2015; Stern et al., 2012), includes experimental and quasi-experimental research and contribution analysis. Presenting impact analysis after having discussed more traditional designs should help the reader understand how these different approaches differ and what they bring to evaluating the effects of interventions. In this chapter, the integration of planetary health dimensions into effect and impact analysis is also explored. Finally, this chapter will also touch on outcome harvesting. Outcome harvesting is not an approach focusing on assessing causal relations, but rather an approach for evaluating interventions\u2019 effects and social change in unpredictable and changing contexts. Given that we will likely experience more unstable and fast-changing contexts, outcome harvesting may become more prominent in the upcoming years.<\/p>\n","protected":false},"author":20,"menu_order":2,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[49],"contributor":[],"license":[],"class_list":["post-232","chapter","type-chapter","status-publish","hentry","chapter-type-numberless"],"part":228,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/232","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/users\/20"}],"version-history":[{"count":2,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/232\/revisions"}],"predecessor-version":[{"id":639,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/232\/revisions\/639"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/parts\/228"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/232\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/media?parent=232"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapter-type?post=232"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/contributor?post=232"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/license?post=232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}