5.4 Drafting an evaluation plan

“An evaluation plan is a written document that describes how you will monitor and evaluate your program, as well as how you intend to use evaluation results for program improvement and decision making” (Centers for Disease Control and Prevention, 2011, p. 1). It consists of a written document of a few pages that results from extensive consultations to find the right arrangement to fit the evaluation purpose and context.

Elaborating a plan will help you prepare but also negotiate agreement among different uses and participants on the expectations. This step can support building trust among the different actors. It can also be a safeguard for potential subsequent influence from different actors. This entire exercise begins with the expressed interest of a commissioner, who has typically identified an evaluation question. Before proposing an evaluation plan, the evaluator must conduct their own research to gain a rapid yet thorough understanding of the topic and the evaluation context.

Coherence between the different components of an evaluation plan is key. Many evaluation questions, approaches, and methods exist. Therefore, finding an adequate combination of plan elements to meet the characteristics of the context is essential (see Figure 5.2).

Figure 5.2 The Design Triangle

Digging into the topic

What is this project about? What is the existing state of knowledge on the problem the intervention or similar interventions target? What is the scientific consensus and what are potential tensions or disagreements? What impacts on planetary health can be anticipated?

Conducting a short review of the scientific literature and summarizing the information is the first step to familiarize yourself with the project topic. Some topics are very broad, and you will need to narrow the scope of your search. If you have no expertise on the topic, consider consulting an expert to get guidance on the most recent scientific knowledge and on how to orient your search. Be mindful that online resources can be biased. When you conduct a web search try to prioritize sites that make scientific articles accessible publicly, for example, PubMed in the health sector (https://pubmed.ncbi.nlm.nih.gov). Widely available and peer-reviewed information ensures both credibility and barrier-free access to information for all users.

Mapping the context

Mapping the context includes:

Identify who should be involved

Identifying who should be around the evaluation project table is an important step. The evaluation should ask if some voices are currently missing and who should be invited.

Determining the evaluation’s focus

Once the previous information is gathered, the evaluator can, in collaboration with the commissioner and potentially other actors, discuss the purpose and intended use of the evaluation. This process supports the confirmation or formulation of evaluation question(s) and facilitates the selection of an approach that best aligns with the expectations and context.

At this stage we suggest evaluators include a discussion about integrating planetary health considerations, specifically the intervention’s impacts on human and natural systems. By leading these conversations evaluators can raise awareness of the importance of adopting a holistic evaluation approach and the potential for designing and implementing interventions with this aim. Raising this option early will facilitate getting an agreement on the type of assessment done.

International commitments exist to protect and promote biodiversity and to reduce greenhouse gas emissions (United Nations, 2015, 2022). Solutions also exist in all sectors and at all levels from the individual to community to regional, even for corporations, and governments that are establishing policies and regulations. When an organization takes steps to first assess the whole impact of their intervention(s) and then takes action to enhance its positive impacts on the environment and population, they can be positioned favorably with funders and other organizations. More organizations are becoming accountable for their program impacts (see for example Environmental, Social and Governance [ESG] investment criteria). The planetary health assessment doesn’t need to be extensive. Planetary health dimensions can be included at all evaluation stages starting with the logic model.

The approaches and tools presented in this book are intended to support evaluators in offering a planetary health perspective to their clients. The consideration of planetary health dimensions shouldn’t represent a cost for clients, as the approach and tools offered in this book do not involve specific expertise or extensive data collection. Once practiced, the evaluator will have enough experience to apply planetary health lenses to any kind of project with minimal additional effort. The evaluator could even present this layer as a bonus to the evaluation project. Organizations willing to engage in assessing their planetary health impacts will create a path for others and develop a reputation as innovators and responsible investors.

Choosing the evaluation design

One component of the evaluation project’s validity or credibility is the coherence between the context and choices of approach, questions, design, and methods. Many options exist. While some combinations work well, others are just not possible. For example, you won’t use the same research designs for an effect analysis and an implementation analysis. The choice of methods will be determined by your research design. Similarly, some evaluation approaches will determine the choice of questions and research design. For example, in community-based projects or Utilization-Focused Evaluations (Patton, 1997) participants or users determine the direction of the evaluation project early in the process. When working in Indigenous contexts, the entire process must be shaped according to existing protocols and with respect for the communities’ cultures. Working with Indigenous peoples and communities will determine the evaluation process, particularly the choice of approach, frameworks, ownership of data, and ethical procedures (Chilisa & Mertens, 2021; Chouinard & Cram, 2020; Cram, 2018a, 2018b; Evans et al., 2020; Government of Canada, 2014). How you engage with different parties and define your own role as an evaluator will be determined by the specific context of evaluation and the type of approach that will be relevant for this context.

Once the right combination is determined and agreed on with the evaluation partners, the evaluator can continue designing the project by identifying the analysis strategy, getting agreement on the knowledge-sharing activities, the timeline, the budget and meeting the ethical approval requirements. Again, creating the evaluation plan presents an opportunity to engage with the different parties and build trust while facilitating agreement of the evaluation’s shape and its different components. The written document summarizes what should be a negotiated understanding of what the evaluation project aspires to look like.