5.3 Evaluation use and the importance of mapping the context

A useful evaluation is one that is influential. The field has extensively debated the importance of use and its determinants. Various types of evaluation use have been identified: (1) symbolic use is when evaluation is caught in a political game; (2) instrumental use is when results are used to contribute to policy or decision-making; and (3) process use when the process itself can be used to achieve some objectives such a empowering communities (Contandriopoulos & Brousselle, 2012). Even if the evaluation project is planned with specific expected uses in mind, some users may focus on elements of the project that we, as evaluators, did not anticipate. Of course, use can also change over time, depending on components of evaluation projects (Patton, 1997). Users vary too; they are not a homogenous group and the evaluator may have little control over the aspects of the project that are used and/or who uses them. And, in some cases, no use will occur and the evaluation project is forgotten as soon as it is finished.

Use is mainly determined by the characteristics of the context (see Figure 5.1) (Contandriopoulos & Brousselle, 2012). This explains why some evaluators experience difficulties using the results to influence the situation studied. It is not the process that seems to be the main determinant of use, but rather the characteristics of the context. Long-standing debates in the field of evaluation have centered around the issue of use, one of the most notable being the debate between Patton and Weiss, both highly respected evaluators (Patton, 1988; Weiss, 1988). Patton argued that the process was a main determinant of use, while Weiss suggested that the process could have very little impact. This debate can be better understood by analyzing the characteristics of the contexts in which Weiss and Patton worked.

Some contexts are more favourable to use, others less. “As soon as users are willing to bear most of the costs, the potential for use increases dramatically” (Contandriopoulos & Brousselle, 2012, p. 70). Collaborative and participatory approaches work well in those favorable contexts. However, these approaches are unlikely to work as expected in other types of contexts.

Low-polarized contexts in which the evaluator bears most of the costs are associated with very little potential for use. Results produced in this quadrant [as shown in Figure 5.1 and described as ‘the knowledge-driven swamp’] are likely to join the ever-growing pile of ignored advice and shelved reports. (Contandriopoulos & Brousselle, 2012, p. 70)

In practice, evaluators often specialize in a particular evaluation approach and tend to choose to work in contexts which are a fit for these approaches. Ideally an evaluator should be able to work in a diversity of contexts and choose the evaluation approach that fits the context.

Figure 5.1 Evaluation Use, Evaluation Models and Contextual Characteristics

The current post-truth context is creating extraordinary conditions and creating new challenges for evaluators.