Downloaded 250 times
Uploaded bySimon Hearn
OM for policy influencing
Outcome Mapping (OM) is a participatory method for planning, monitoring, and evaluating projects focused on social and organizational learning. It is oriented towards changes in behavior of those the project works with rather than direct results. OM recognizes that policy change is complex, unstable, non-linear, and beyond a project's control but subject to influence. It focuses on a project's contribution rather than attribution, and identifies and focuses efforts on key agents of change. OM monitors behavioral changes in target actors and their environment over time.
More Related Content
Similar to OM for policy influencing
OM for policy influencing
- 1. Outcome Mapping Monitoringand Evaluating Policy Influencing 8 December 2009 Simon Hearn, s.hearn@odi.org.uk
- 2. Brief definition ofOM A participatory method for planning, monitoring and evaluation Focused on changes in behaviour of those with whom the project or program works Oriented towards social & organizational learning
- 4.
- 5. Policy change canbe : Complex (involve a confluence of actors and factors) Unstable (independent of project duration) Non-linear (unexpected, emergent, discontinuous) Two-way (intervention may change) Beyond control (but subject to influence) Incremental, cumulative (watersheds & tipping points)
- 6. 1. Contribution, notAttribution Change processes involve interactions among multiple actors and factors
- 7. 2. There isa limit to our influence Sphere of Control Sphere of Influence Sphere of Interest Inputs Activities Outputs Outcomes Impacts
- 8. 3. Identify andfocus on key agents of change Develop enthusiasm to address topic Learn in partnership Develop awareness and enthusiasm Challenge existing beliefs High General level of alignment Low Low High Interest in specific topic Map actors on the matrix Identify which are the most influential Who do you work with directly?
- 9. 4. Outcomes asprogressive behaviour changes (Deep transformation) (Active engagement) (Early positive responses) Love to see Like to see Expect to see
- 10. ...policy change examplesDiscourse changes Procedural changes Content changes Attitudinal changes Behavioural changes
- 11. 5. Use abroad range of strategies causal persuasive supportive Aimed at the actor Aimed at the actors environment
- 12. 6. M&E onthree levels Initiative Policy Actor outcomes (behaviour changes in the actors) implementation (interventions by the program) relevance & viability (actions of the program) Strategies
- 13. 7. Can’t provecausation But can make a plausible estimate based on: Timing : Did the change happen after the activity? Logic : Is it reasonable to expect that these inputs would contribute to the change? Expert Judgment : Do knowledgeable people – including those involved – agree with the contribution claim? Alternative Explanations : What other factors could explain the change?
- 14. 8. Clarify intentfrom the outset Need a systematic approach to choosing areas for influencing, setting objectives and defining outcomes This will enable a systematic method for collecting monitoring information and evidence of change
- 15. Evaluating REGLAP RAPIDOutcome Assessment Outcome Mapping Episode Studies Most Significant Change Tracks key changes in target actors and surrounding environment Makes a judgement on the contribution of project activities on the changes Recognises external events
- 16. Evaluating REGLAP Documentreview Interviews with project team Workshop with project team, key stakeholders, external experts
- 17. Evaluating REGLAP Clarify...Policy objectives Key policy actors targeted Desired changes in and around these actors Collect evidence of... Activities undertaken and outputs of strategies Internal changes within the programme Changes in behaviour of key actors Other changes in the policy environment Map the evidence and discuss linkages
- 18.
- 19.
- 20. Evaluating REGLAP StrategyManagement Outputs Uptake Outcomes and Impact Somewhat
- 21. Step 1: Describethe policy environment
- 22. Step 2: Identifykey policy actors
- 23. Step 3: Describethe behaviour of the key actors
- 24. Step 4: Mapthe key changes in behaviour
- 25. Step 5: Mapproject changes and external influences
- 26. Step 6: Determinelevel of influence Influences: Direct Indirect External
- 27. Value for money?Multiple perspectives of value Counterfactual is near impossible No direct cause and effect OM provides evidence Decision rests with evaluator
Editor's Notes
- #2 A set of tools and principles for PME of social and policy change interventions. Developed by IDRC in Canada almost 10 years ago.
- #3 Main Messages In a results’ chain OM helps you think about the « outcomes » section Definition of Program: A group of related projects and activities with a specific set of resources (human, financial, capital) directed to the achievement of a set of goals within a specified period of time Can be used by projects, organizations, and communities too
- #4 They were looking for an approach that could solve an ongoing problem they were having when they were evaluating the research they were funding...
- #5 a development intervention starts with an input and ends with an output but the processes that turn the input into the output were either being ignored or weren’t understood. Another way to think about this is that at the beginning of a project you have objectives, inputs and activities At the end you have results, outputs and impact And there’s a process to get from one to the other – but this is often a black box, what goes on here is a mystery Outcome Mapping aims to de-mystify this black box
- #6 The reality is … In summary these boil down to 3 main challenges: Establishing attribution Tracking learning along the way – new problems and solutions emerge as you learn Spin and counter-spin (fear of failure and loss of funding rather than desire for feedback) In the face of such complexity: How can we show that we have made a difference with our research / project? Proving causality is difficult, especially when there are several factors and actors at work How can we reduce the unknowns regarding our contribution? How can we share the credit?
- #8 The way a focus on measuring ‘impact’ plays out is not suitable in the context of many projects and programmes -> we need to recognise the limits of a project’s influence, and shape our planning, learning, and accountability functions around “outcomes”, which are further ‘upstream’ from impacts. Limits depend on time, geography, resources, contacts, politics Looking from the point of view of a project, we see Sphere of control = operational environment Sphere of Influence = Relationships & Interactions Sphere of Interest = social, economical, environmental states & trends DIRECT CONTROL DIRECT INFLUENCE INDIRECT INFLUENCE This relates to concepts you may be familiar with from the log frame, along the results chain through to intended impacts. The premise is -> we can’t control everything we’d like to see change -> this is not something unscientific: complexity theory (and common sense!) tells us that real, sustainable change involves the combination of a number of different factors, and is a product of the interaction of many different actors and stakeholders -> Outcome Mapping is concerned with the level where a programme has direct influence Complexity cross-reference: Systems with multiple actors, inter-related and connected with each other and with their environment Various forces interacting with each other, interdependent (e.g. political and social dimensions) In these situations, change occurs because of the interaction of multiple actors and factors; can’t be controlled by one programme Very difficult to predict what ‘impacts’ might be achieved in advance; SDOIC means inherent unpredictability, that isn’t unscientific but based on careful investigation Common mistakes include trying to deliver clear, specific, measurable outcomes; better to work with inevitable uncertainty than to plan based on flimsy predictions Russell Ackoff : 3 kinds of problems: Mess, problem and puzzle. MESS has no defined form or structure, not a clear understanding of what’s wrong, often involves economic, technological, ethical and political issues. Common mistake is to carve off part of a mess, deal with it as a problem and solve it as if it was a puzzle (as the simple causal chain from inputs to impact tries to do) -> need to recognise messy realities
- #9 OM identifies where outcomes are likely to occur
- #10 We set out a set of possible changes – in the actors we are influencing - that we hope will contribute to our goal Behaviours means relationships, actions, practices or....
- #11 Discursive changes: These refer to changes in the labels or narratives of policy actors. They reflect a new or improved understanding of a subject -- even if it does not imply an effective change of policy or practice. Procedural changes: These refer to changes in the way certain processes are undertaken. For example, the incorporation of consultations to otherwise closed processes, or small changes in the way that national policies are implemented in the field. Content changes: These refer to changes in the content of policies including strategy papers, legislation and budgets. These are formal changes in the policy framework. Attitudinal changes: These refer to changes in the way policy actors think about a given issue. This might be an important change to target in the event that key stakeholders have high influence but lack interest in a policy area or are not necessarily aligned with the policy objectives of the programme. Behavioural changes: These refer to more durable changes in the way that policy actors behave (act or relate to others) as a consequence of formal and informal changes in discourse, process and content.
- #12 For maximum effectiveness, may have to think widely.
- #13 Use journals and other tools to gather information
- #16 OM is flexible, can bring in many different tools. For ex-post, OM needs support.
- #19 For example, a map of changes developed in dialogue with the project team and other key informants.
- #20 The linkages between outcomes can be estimated – this shows direct influence (blue), indirect (green) and external (red)
- #21 Somewhat – can demonstrate the contribution of activities to certain outcomes. Strategy map can be useful for assessing relative strengths of different types of intervention. No. OM has a tool for planning particular management practices, mostly KM, comms and partnerships but it doesn’t provide a technique for evaluating these. No. Fairly straight forward. Other methods available. By placing the lens close the interventions, OM can quickly locate uptake of key messages among key actors Depends on quality of the evidence, but by focussing on key actors that the programme can influence, and by having specific behaviour changes in mind, the collection of evidence can be more intentional and effective. OM says: Multiple perspectives on value No direct cause and effect Counterfactual is near impossible Be Idealist Realists