4.7 Conclusion

Coherence is key when conceiving an evaluation project. Coherence refers to the right alignment between evaluation context, questions, approaches, and methods. There is no real starting point to align these pieces and the final proposal is often the result of many adjustments and negotiations.

Finding the right combination requires a deep understanding of the context and of the expectations of the commissioners and various groups involved. Being knowledgeable about different evaluative approaches will increase the evaluator’s agility for choosing among the many possibilities. Once the evaluation questions have been identified in collaboration with the commissioners, the evaluator will need to identify the research design for answering the question and then identifying the methods to use. The research design will depend on the evaluation question. For example, if you plan to conduct an effect analysis, you will likely choose between experimental designs, quasi-experimental designs, or contribution analysis. If you plan to conduct a logic analysis, you will use a knowledge synthesis approach as your study design. Evaluation questions are key in determining the study design and the study design will be key in choosing appropriate methods. In some contexts, the choice may be decentralized or co-determined with multiple parties; in such cases, the evaluator acts a facilitator offering guidance on what approaches would work or not work for addressing potential questions.

Given the advocacy, sense-making and value-creation roles that evaluation can play, the evaluator also needs to remain mindful of the broader context and consider which parameters can be influenced to ensure the evaluation has a positive impact. In particular, participative and collaborative processes, culturally responsive practices, or engagement processes can be important components of the project design, beyond evaluation questions. However, the ability to choose what would be an adequate approach for a given context depends on the evaluator’s ability to use different models and their awareness and depth of understanding the context. As evaluators, we would like this first phase to be rational and straightforward. The reality is much messier.