Introduce presentation: The way a focus on measuring ‘impact’ plays out is not suitable in the context of many projects and programmes -> we need to recognise the limits of a project’s influence, and shape our planning, learning, and accountability functions around “outcomes”, which are further ‘upstream’ from impacts. Looking from the point of view of a project, we see Sphere of control = operational environment Sphere of Influence = Relationships & Interactions Sphere of Interest = social, economical, environmental states & trends DIRECT CONTROL DIRECT INFLUENCE INDIRECT INFLUENCE This relates to concepts you may be familiar with from the log frame, along the results chain through to intended impacts. The premise is -> we can’t control everything we’d like to see change -> this is not something unscientific: complexity theory (and common sense!) tells us that real, sustainable change involves the combination of a number of different factors, and is a product of the interaction of many different actors and stakeholders -> Outcome Mapping is concerned with the level where a programme has direct influence Complexity cross-reference: Systems with multiple actors, inter-related and connected with each other and with their environment Various forces interacting with each other, interdependent (e.g. political and social dimensions) In these situations, change occurs because of the interaction of multiple actors and factors; can’t be controlled by one programme Very difficult to predict what ‘impacts’ might be achieved in advance; SDOIC means inherent unpredictability, that isn’t unscientific but based on careful investigation Common mistakes include trying to deliver clear, specific, measurable outcomes; better to work with inevitable uncertainty than to plan based on flimsy predictions Russell Ackoff : 3 kinds of problems: Mess, problem and puzzle. MESS has no defined form or structure, not a clear understanding of what’s wrong, often involves economic, technological, ethical and political issues. Common mistake is to carve off part of a mess, deal with it as a problem and solve it as if it was a puzzle (as the simple causal chain from inputs to impact tries to do) -> need to recognise messy realities
Now, why is it important to support boundary partners and measure your influence on them? Here we see the results chain from input, through activities, to outputs, which then lead to outcomes (behavioural changes in our boundary partners) and in turn, hopefully, to impact on the target beneficiaries. At the inputs level, partners have least influence (project design, location, timing etc), but then as funding flows partners and beneficiaries become more committed and have more prominent roles. For the outcomes of the project to lead to long term, large scale, sustainable benefits (i.e. impact) local ownership and influence need to become effective and dominant. So, a difficulty exists: the more successful a project is, the sooner and the more its influences disappear Why Behaviour Change? Measurable, observable and influencable not about claiming a change is down just to one programme, but rather encoruaging supporting local structures and contributing to processes of change; Sympathetic to the difference between influence and control, attribution and contribution
Here are spheres of control, influence and interest again. For the project, we have the various stakeholders, partners, ultimate beneficiaries, etc. [labelled as illustrative example] BOUNDARY PARTNERS ARE: Those individuals, groups, or organisations with whom the program Interacts with directly to effect change Anticipates opportunities for influence Engages in mutual learning -> With outcome mapping, the focus for learning and accountability are the boundary partners . plan, monitor, and evaluate our efforts around them -> look for changes in their behaviour that would contribute to the wider change you’d like to see Behaviour is defined as… Actions Activities Interactions Relationships … of your Boundary Partners Complexity cross-reference: Emphasises the importance of relationships; in a complex systems relationships are key for: - carrying information (central to many feedback effects) Constitutive of power in a system Co-evolution. “co-evolution” is where the evolution of one domain or entity is partially dependent on the evolution of other related domains and entities. It’s a long-term process of interactions and mutual influence. Concept comes from biology, e.g. bees and flowers. Warning from co-evolution in ‘homeostasis’- about how trying to change just one element in a system often causes the system to adapt to keep certain key variables constant. For example, introduction of the hoover didn’t reduce the amount of drudgery for women in many countries; lead to higher standards of cleanliness.
conclusion: « impact » is a highly politicized concept in development. OM focuses on outcomes not impact there are other methods to do impact assessment at OM African Users Workshop in Niamey, January 2007: OM not only about P,M&E but about the way you conceptulize development
Planning: What are we trying to accomplish? Why? Who? How? Monitoring: What do we want to know? Evaluation: What do we want to learn?
A reality check: people used to using LFA, a tool with real practical value, many donors insist on a log frame. So, an important issue is how to integrate OM and the log frame, or in many instances, integrate principles from OM into a log frame? One example: VECO Indonesia Works on the promotion of sustainable agriculture, including institutional development Had used logframes to guide their work- was criticised by internal+external evaluations+studies. wasn’t allowing for flexibility, learning, sustainable change Made an OM framework, but their donors changed their mind: Belgian government insisted all funded NGO projects used log frames The decision was made to go halfway between OM and the log frame, using OM largely for their (internal) planning a system to translate OM elements into reports for log frame structured the programme around 4 main Objectives; For each of these objectives they identified the relevant boundary partners, and for the boundary partners outcome challenges, progress markers, and strategy maps. So, log frame where aim for behavioural change, and build planning around boundary partners Much more work going on in this area. E.g. Daniel Roduner drafting a synthesis model of OM and logframe, e.g. paper exploring different philosophical modes of understanding logframe: taking a more ‘interpretivist’ stance means looking at behaviour change, limits to influence, easier to M&E.
Outcome Mapping Complexity & Aid 9 July 2008 London Simon Hearn (email@example.com) ODI, London
Focus of Outcome Mapping Sphere of Control Sphere of Influence Sphere of Interest LFA focus RBM focus OM focus Inputs Activities Outputs Outcomes Impacts
Focus of Outcome Mapping Outcome Mapping Local partners / beneficiaries ownership increases Program influence decreases Inputs Activities Outputs Outcomes Impacts
The Limits of Influence Programme Sphere of Control Sphere of Influence Sphere of Interest Beneficiaries Stakeholders Boundary Partners
The Problem with Impact Impact implies… The reality is… Cause & effect Open system Positive, intended results Unexpected positive & negative results occur Focus on ultimate effects Upstream effects are important Credit goes to a single contributor Multiple actors create results & need credit Story ends when program obtains success Change process never ends