{"id":153,"date":"2025-10-03T14:26:52","date_gmt":"2025-10-03T18:26:52","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/?post_type=chapter&p=153"},"modified":"2026-05-26T13:28:46","modified_gmt":"2026-05-26T17:28:46","slug":"evaluative-questions","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/chapter\/evaluative-questions\/","title":{"raw":"4.6 Evaluative questions","rendered":"4.6 Evaluative questions"},"content":{"raw":"Evaluation is at the intersection of research and policy making\/decision-making. A distinction is generally made between evaluative research and monitoring (Brousselle et al., 2011b). Evaluative research analyzes causal relations, specifically the links between the components of the intervention, including the results. Monitoring compares components and aspects of the intervention to some objectives or targets and to record progress. Monitoring provides information on the resources dedicated to the intervention and its structure, the activities (also called process), and the results.\r\n\r\nSeveral questions exist in evaluation. Although authors present and regroup evaluation questions in different ways, most models include the same core questions. The typology and definitions used in this book are inspired and adapted from what was suggested in L'\u00c9valuation: Concepts et M\u00e9thodes<\/em> (Brousselle et al., 2011b). Planetary health considerations have been introduced in each one of them (see Figure 4.1 and the following chapters).\r\n\r\nFigure 4.1<\/strong> Evaluation Questions<\/em>\r\n\r\n\"Circular\r\n

Evaluative research questions<\/h3>\r\nEvaluative research includes questioning in five categories (see Figure 2):\r\n
    \r\n \t
  1. \u00a0Relevance: needs assessment\r\n
      \r\n \t
    • What is needed to meet the needs of the community (ex-ante)? Is the intervention fitting with the needs and commitments of the community (ex-post)?<\/li>\r\n \t
    • Are the objectives of the intervention pertinent particularly when considering the socio-ecological context? Are the right causes targeted? Is the right population targeted?<\/li>\r\n \t
    • Are the right partners involved, or should other partners be involved?<\/li>\r\n<\/ul>\r\n<\/li>\r\n \t
    • \u00a0Coherence: logic analysis\r\n
        \r\n \t
      • Is the program intervention plausible, given what is known from theory and practice to achieve the intended effects (direct logic analysis)<\/li>\r\n \t
      • What are plausible interventions or alternatives to achieve the intended effects?<\/li>\r\n<\/ul>\r\n<\/li>\r\n \t
      • Effectiveness: effect analysis, contribution analysis and impact evaluation\r\n
          \r\n \t
        • What are the results of the intervention? Did the intervention make a difference and for whom?<\/li>\r\n \t
        • Are the observed effects attributable to the intervention?<\/li>\r\n \t
        • What are the impacts of the intervention on planetary health?<\/li>\r\n<\/ul>\r\n<\/li>\r\n \t
        • Efficiency: economic evaluation\r\n
            \r\n \t
          • Is the intervention efficient?<\/li>\r\n<\/ul>\r\n<\/li>\r\n \t
          • Implementation: implementation analysis<\/li>\r\n<\/ol>\r\n
              \r\n \t
            • \r\n
                \r\n \t
              • How does the context influence the implementation of the intervention and the production of the results?<\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\nFigure 4.2<\/strong> Evaluative Research Questions<\/em>\r\n\r\n[caption id=\"attachment_563\" align=\"alignnone\" width=\"1024\"]\"Flow Source: Adapted from Champagne, F., Contandriopoulos, A., Brousselle, A., Hartz, Z., & Denis, J. L. (2011c). L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes. In A. Brousselle, F. Champagne, A.-P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes (2nd edition ed., pp. 35-56). Presses de l\u2019Universit\u00e9 de Montr\u00e9al: 62.[\/caption]\r\n

                Monitoring questions<\/h3>\r\nMonitoring is used to assess the fidelity of the intervention in comparison to the original plans (which includes the costs), the reach, the quality of the intervention, whether the goals were achieved, and the impacts on planetary health. Table 4.1 summarizes the different options for monitoring the intervention.\r\n\r\n\r\n\r\n\r\n\r\n\r\n
                <\/td>\r\nFidelity\r\n(includes costs)<\/td>\r\nReach<\/td>\r\nQuality<\/td>\r\nGoal Achievement<\/td>\r\nImpacts on planetary health<\/td>\r\n<\/tr>\r\n
                Resources<\/td>\r\nX<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
                Process<\/td>\r\nX<\/td>\r\nX<\/td>\r\nX<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
                Results<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/td>\r\nX<\/td>\r\nX<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nTable 4.1: Possible Focus for Monitoring\r\n<\/em><\/sub>\r\n\r\nSource: Adapted from Champagne, F., Hartz, Z., Brousselle, A., & Contandriopoulos, A. P. (2011d). L\u2019appr\u00e9ciation normative. In A. Brousselle, F. Champagne, A. P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation: concepts et m\u00e9thodes. Deuxi\u00e8me \u00e9dition mise \u00e0 jour.<\/em> Les Presses de l\u2019Universit\u00e9 de Montr\u00e9al: 98.<\/sub>\r\n\r\nNote that monitoring questions focus on the intervention\u2019s components, whereas evaluative research questions examine the relationships or links between those components (see Figures 4.2 and 4.3).\r\n\r\nFigure 4.3<\/strong> Focus of monitoring<\/em>\r\n\r\n[caption id=\"attachment_564\" align=\"alignnone\" width=\"1024\"]\"Flow Source: Adapted from Champagne, F., Contandriopoulos, A., Brousselle, A., Hartz, Z., & Denis, J. L. (2011c). L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes. In A. Brousselle, F. Champagne, A.-P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes (2nd edition ed., pp. 35-56). Presses de l\u2019Universit\u00e9 de Montr\u00e9al: 62.[\/caption]\r\n\r\nBe aware that assessing the effects in a monitoring exercise is different from an effect analysis. An effect analysis<\/em> will analyze the causal relationship between the intervention and the observed effects. In monitoring, no causal relationship is assessed; effects are monitored but could potentially be due to another element than the intervention, such as another competing intervention or a change in the characteristics of the context.\r\n\r\nUnderstanding the distinction between evaluative research and monitoring is important. However, in real life evaluators will often mix the use of the two approaches as they are different but complementary ways of analyzing an intervention. For example, in implementation analysis, the first stage may include an assessment of the program\u2019s implementation (fidelity and reach) before explaining the implementation with a process analysis (Patton, 1997; Scheirer, 1994). The distinction between evaluative research and monitoring is mostly a heuristic approach to learning evaluation.","rendered":"

                Evaluation is at the intersection of research and policy making\/decision-making. A distinction is generally made between evaluative research and monitoring (Brousselle et al., 2011b). Evaluative research analyzes causal relations, specifically the links between the components of the intervention, including the results. Monitoring compares components and aspects of the intervention to some objectives or targets and to record progress. Monitoring provides information on the resources dedicated to the intervention and its structure, the activities (also called process), and the results.<\/p>\n

                Several questions exist in evaluation. Although authors present and regroup evaluation questions in different ways, most models include the same core questions. The typology and definitions used in this book are inspired and adapted from what was suggested in L’\u00c9valuation: Concepts et M\u00e9thodes<\/em> (Brousselle et al., 2011b). Planetary health considerations have been introduced in each one of them (see Figure 4.1 and the following chapters).<\/p>\n

                Figure 4.1<\/strong> Evaluation Questions<\/em><\/p>\n

                \"Circular<\/p>\n

                Evaluative research questions<\/h3>\n

                Evaluative research includes questioning in five categories (see Figure 2):<\/p>\n

                  \n
                1. \u00a0Relevance: needs assessment\n
                    \n
                  • What is needed to meet the needs of the community (ex-ante)? Is the intervention fitting with the needs and commitments of the community (ex-post)?<\/li>\n
                  • Are the objectives of the intervention pertinent particularly when considering the socio-ecological context? Are the right causes targeted? Is the right population targeted?<\/li>\n
                  • Are the right partners involved, or should other partners be involved?<\/li>\n<\/ul>\n<\/li>\n
                  • \u00a0Coherence: logic analysis\n
                      \n
                    • Is the program intervention plausible, given what is known from theory and practice to achieve the intended effects (direct logic analysis)<\/li>\n
                    • What are plausible interventions or alternatives to achieve the intended effects?<\/li>\n<\/ul>\n<\/li>\n
                    • Effectiveness: effect analysis, contribution analysis and impact evaluation\n
                        \n
                      • What are the results of the intervention? Did the intervention make a difference and for whom?<\/li>\n
                      • Are the observed effects attributable to the intervention?<\/li>\n
                      • What are the impacts of the intervention on planetary health?<\/li>\n<\/ul>\n<\/li>\n
                      • Efficiency: economic evaluation\n
                          \n
                        • Is the intervention efficient?<\/li>\n<\/ul>\n<\/li>\n
                        • Implementation: implementation analysis<\/li>\n<\/ol>\n
                            \n
                          • \n
                              \n
                            • How does the context influence the implementation of the intervention and the production of the results?<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n

                              Figure 4.2<\/strong> Evaluative Research Questions<\/em><\/p>\n

                              \"Flow
                              Source: Adapted from Champagne, F., Contandriopoulos, A., Brousselle, A., Hartz, Z., & Denis, J. L. (2011c). L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes. In A. Brousselle, F. Champagne, A.-P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes (2nd edition ed., pp. 35-56). Presses de l\u2019Universit\u00e9 de Montr\u00e9al: 62.<\/figcaption><\/figure>\n

                              Monitoring questions<\/h3>\n

                              Monitoring is used to assess the fidelity of the intervention in comparison to the original plans (which includes the costs), the reach, the quality of the intervention, whether the goals were achieved, and the impacts on planetary health. Table 4.1 summarizes the different options for monitoring the intervention.<\/p>\n\n\n\n\n\n\n
                              <\/td>\nFidelity
                              \n(includes costs)<\/td>\n                              		Reach<\/td>\nQuality<\/td>\nGoal Achievement<\/td>\nImpacts on planetary health<\/td>\n<\/tr>\n
                              Resources<\/td>\nX<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
                              Process<\/td>\nX<\/td>\nX<\/td>\nX<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
                              Results<\/td>\n<\/td>\n<\/td>\n<\/td>\nX<\/td>\nX<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                              Table 4.1: Possible Focus for Monitoring
                              \n<\/em><\/sub><\/p>\n

                              Source: Adapted from Champagne, F., Hartz, Z., Brousselle, A., & Contandriopoulos, A. P. (2011d). L\u2019appr\u00e9ciation normative. In A. Brousselle, F. Champagne, A. P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation: concepts et m\u00e9thodes. Deuxi\u00e8me \u00e9dition mise \u00e0 jour.<\/em> Les Presses de l\u2019Universit\u00e9 de Montr\u00e9al: 98.<\/sub><\/p>\n

                              Note that monitoring questions focus on the intervention\u2019s components, whereas evaluative research questions examine the relationships or links between those components (see Figures 4.2 and 4.3).<\/p>\n

                              Figure 4.3<\/strong> Focus of monitoring<\/em><\/p>\n

                              \"Flow
                              Source: Adapted from Champagne, F., Contandriopoulos, A., Brousselle, A., Hartz, Z., & Denis, J. L. (2011c). L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes. In A. Brousselle, F. Champagne, A.-P. Contandriopoulos, & Z. Hartz (Eds.), L\u2019\u00e9valuation dans le domaine de la sant\u00e9 : concepts et m\u00e9thodes (2nd edition ed., pp. 35-56). Presses de l\u2019Universit\u00e9 de Montr\u00e9al: 62.<\/figcaption><\/figure>\n

                              Be aware that assessing the effects in a monitoring exercise is different from an effect analysis. An effect analysis<\/em> will analyze the causal relationship between the intervention and the observed effects. In monitoring, no causal relationship is assessed; effects are monitored but could potentially be due to another element than the intervention, such as another competing intervention or a change in the characteristics of the context.<\/p>\n

                              Understanding the distinction between evaluative research and monitoring is important. However, in real life evaluators will often mix the use of the two approaches as they are different but complementary ways of analyzing an intervention. For example, in implementation analysis, the first stage may include an assessment of the program\u2019s implementation (fidelity and reach) before explaining the implementation with a process analysis (Patton, 1997; Scheirer, 1994). The distinction between evaluative research and monitoring is mostly a heuristic approach to learning evaluation.<\/p>\n

                              Media Attributions<\/h2>
                              • Figure 4.1 Evaluation Questions <\/li>
                              • Figure 4.2 Evaluative Research Questions <\/li>
                              • Figure 4.3 Focus of monitoring <\/li><\/ul><\/div>","protected":false},"author":20,"menu_order":6,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[49],"contributor":[],"license":[],"class_list":["post-153","chapter","type-chapter","status-publish","hentry","chapter-type-numberless"],"part":141,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/153","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/users\/20"}],"version-history":[{"count":7,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/153\/revisions"}],"predecessor-version":[{"id":628,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/153\/revisions\/628"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/parts\/141"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapters\/153\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/media?parent=153"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/pressbooks\/v2\/chapter-type?post=153"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/contributor?post=153"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/foundationsofevaluation\/wp-json\/wp\/v2\/license?post=153"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}