Effect analysis and related approaches
8.5 Contribution analysis
Contribution analysis was initially developed by John Mayne when he was working in the Office of Auditor General of Canada. He was working on strategies to increase the relevance of evaluations in the Federal Government of Canada.
Government programs are intended to produce certain outcomes: more jobs, a healthier public, better living conditions, etc. Effective programs are those that make a difference in meeting these kinds of objectives – they contribute to the intended outcomes that citizens value. In trying to measure the performance of a program, we face two problems. We can often—although frequently not without some difficulty—measure whether or not these outcomes are actually occurring. The more difficult question is usually determining just what contribution the specific program in question made to the outcome. How much of the success (or failure) can we attribute to the program? What has been the contribution made by the program? (Mayne, 1999, pp. 2-3)
Contribution analysis has often been associated with the analysis of complex programs.
Contribution analysis is based on the existence of, or more usually, the development of a postulated theory of change for the intervention being examined. The analysis examines and tests this theory against logic and the evidence available from results observed and the various assumptions behind the theory of change, and examines other influencing factors. The analysis either confirms – verifies – the postulated theory of change or suggests revisions in the theory where the reality appears otherwise. The overall aim is to reduce uncertainty about the contribution an intervention is making to observed results through an increased understanding of why results did or did not occur and the roles played by the intervention and other influencing factors.(Mayne, 2012a, p. 271)
Contribution analysis examines, through causal claims, the contribution, rather than the attribution, a complex program is making to outcomes and impacts (Delahais & Toulemonde, 2012; Mayne, 2001, 2008, 2011, 2012a, 2012b, 2015). When an intervention is sufficiently complex that it is difficult to isolate direct causal relationships, contribution analysis can help construct a plausible story about what happened. It explains the observation of outcomes and impacts of the intervention and identifies key threats to the chain of effects, other contributing factors and rival explanations. The overall objective is for the evaluator to build confidence that the observed effects and impacts are associated with the intervention, while considering other potential influences and alternative explanations.
Contribution analysis involves six steps:
-
Setting out the cause–effect issue to be addressed;
-
Developing the postulated theory of change and risks to it, including rival explanations;
-
Gathering evidence on the theory of change;
-
Assembling and assessing the contribution claim and challenges to it;
-
Seeking out additional evidence; and
-
Revising and strengthening the contribution story. (Mayne, 2012a, p. 272)
Mayne indicates that to be confident the intervention led to the observed effects, the evaluation will need to check the following elements:
-
plausibility of the theory of change;
-
implementation as outlined in the theory of change;
-
evidentiary confirmation of key elements;
-
identification and examination of other influencing factors; and
-
the extent to which key alternative explanations have been disproved. (Mayne, 2011, p. 7)
