Outcome Mapping: Monitoring and Evaluation Tool


Published on

A method for planning and assessing the social effects and internal performance of projects, programs, and organizations.“A project should see itself as a part of an interconnected web of actors, factors and relationships” (Sarah Earl, 2008 IDRC)

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Adopted by organizations where monitoring and evaluation are intended to help with project learning and improvement.
  • These 5 words and 4 arrows might look familiar to some of you. This is a linear-based cause and effect logic that many evaluation frameworks are built upon. Logframe design: is a hierarchical causal-effect chain presented at 4 levels. Concrete and not fluidy. Strength of the LFA is that it forces people to think through their theory of change – a process which many people find difficult. It encourages clear thinking. It encourages clear formulation of outcomes and goals as well as precise definition of quantifiable targets. Enhances accountability between implementors, beneficiaries and donors. The weakness of the LFA is attribution of cause and effects between the levels of outcome and impact. In reality this cannot be conclusively determined in most cases. This uncertainty in attribution is known as the attribution gap and is inherent to the logic model of the logframe. New modalities of cooperation. (i) From direct poverty alleviation to capacity building and social development. Increasingly difficult to establish a plausible and direct cause-efefct relationship between the use of donor resources and poverty alleviation. (ii) From direct implementation to multistakeholder approach. (difficult to attribute each individual actor and the overall impact of a project). Impacts result from a confluence of efforts of several actors: Other donors, partner organizations, government departments, communities, organizations, and groups within the community Linear causality, cause effect thinking of LFA is disputed among practitioners.
  • 1. Weakneses inherent in the existing project tools particularly in the monitoring and evaluation of development effects have motivated IDRC (International Development Research Centre, Canada) to develop a different approach. “OUTCOME MAPPING”. 2. OM developed by IDRC in Canada 3. They needed to demonstrate results 4. In the late 90’s they came across a tool called Outcome Engineering which (Barry Kibel) was being used to plan and measure drug rehabilitation, from the perspective of behaviour changes as outcomes. 5. Over the next few years, the Evaluation Unit adapted this approach, piloted the methodology in several development projects. 6. In 2000, they released the OM manual. Since then, use of OM has been supported globally through trainings and through the virtual learning community.
  • By using OM a project is not claiming achievement of development impacts, rather the focus is on its contributions to outcomes. These outcomes enhance the possibility of development impacts - but the relationship is not necessarily a direct one of cause and effect. Ultimately long term development impacts is rarely accomplished by one single actor (especially an external donor agency). The complexity of the development process makes it extremely difficult to assess impact especially for an external donor agency seeking attribution. 3. Outcomes can be logically linked to a programs activities, although they are not necessarily directly caused by them. These changes are aimed at contributing to specific aspects of human and ecological well-being by providing partners with new tools, techniques and resources to contribute to the development process.
  • These milestones indicate possible ways to achieve change on a bigger map. They are used for learning and reflection. 2. A graduated set of statements describing a progression of changed behaviours in the boundary partner. 3. Describe changes in actions, activities and relationships leading to the ideal outcome. 4. Can be monitored & observed 5. Permit on-going assessment of partner’s progress (including unintended results)
  • The actors are boundary partners because even though the program works with them to effect a change, it cannot control them. The power to influence development rests with them.
  • 2. OM charts the process necessary to get to a place where the project influence meets with the endogenous actors. That’s when the outcomes start to be observed. 3. Typically, at the “inputs” end of the process, the externally supported project has most control over decisions and events - things such as project budget and design, choice of partners, location, timing, etc. At this stage, the local partners and beneficiaries have the least influence. 4. However, once funding flows, activities start and local participants become increasingly active the balance of influence should begin to change. 5. If the project progresses as intended, local actors become more committed and their roles become more prominent. 6. For the outcomes of the project to be relevant and lead to long term, large scale, sustainable benefits - i.e. create impact - then local ownership and influence need to become effective and dominant. 7. This model suggests that the more successful the project, the sooner, and the more, its influences disappear. Results move to the impact stage as project influence is supplanted by endogenous activities and institutions.
  • OM is divided into 3 stages. Stage1: Intentional design helps a project to establish a consensus on the macro-level changes it will help to bring about and plan the strategies it will use. Stage 2: Outcome and performance monitoring provides a framework for ongoing project monitoring of the project actions and boundary partners achievement of outcomes.- based largely on systematized self-assessment. Stage 3: Evaluation planning helps the program identify evaluation priorities and develop an evaluation plan.
  • OM defines 3 distinct but highly interrelated sets of activities and offers tools to monitor each one. Besides monitoring changes in boundary partners, it also monitors the projects strategies and organisational practices to enhance understanding as to how the change has come about. Tools: Performance journal – how the project is operating to fulfil its mission Staregy journal: Strategies employed to encourage change in boundary partners. Otcomes journal: AChievemnts of the progress markets for each outcome
  • other uses: PRA tool for participatory development work with women`s self help groups in India
  • OM is a relatively new method and there are no systematic studies on its effectiveness or efficiency to date. It has some obvious drawbacks: A one-dimensional focus on behaviour changes of partners is not sufficient. What are needed are clear impact hypotheses and indicators at the level of developmental results, particularly the MDGs.
  • Goal of the synthesis is to combine the advantages and strenghts of both models to make it applicable in diferent contextx in a more multifaceted way. Strengths of OM (capacity building) and LFA (focus on development results).
  • Outcome Mapping: Monitoring and Evaluation Tool

    1. 1. Outcome Mapping: Monitoring and Evaluation Tool Mini Symposium on Outcome and Impact Assessment, R4D Planning week. Emily Ouma. 24 November 2010.
    2. 2. Linear-based cause and effect frameworks <ul><li>Difficulty in attributing development impacts to the outputs provided by individual projects – new modalities of cooperation. </li></ul><ul><li>Principle of linear causality seen as too rigid and unrealistic as a basis for planning and management </li></ul><ul><li>also depends on endogenous socio-cultural, economic, political, and environmental factors of the project. </li></ul>
    3. 3. Origins of Outcome Mapping <ul><li>1990’s : IDRC had a need to demonstrate results. </li></ul><ul><li>1998 : Discovered Outcome Engineering – used to plan and measure the results of drug rehabilitation. </li></ul><ul><li>2000 : publication of the OM manual in English </li></ul><ul><li>Global training & using OM globally. </li></ul><ul><li>2006 : www.outcomemapping.ca </li></ul>
    4. 4. Outcome Mapping A method for planning and assessing the social effects and internal performance of projects, programs, and organizations. Looks at the bigger picture “ A project should see itself as a part of an interconnected web of actors, factors and relationships ” (Sarah Earl, 2008 IDRC)
    5. 5. Outcome Mapping <ul><li>Focus is on a particular category of results: </li></ul><ul><ul><li>Changes in behaviors of people, groups and organizations with whom a project/program works directly. Referred to as “ Outcomes ”. </li></ul></ul><ul><ul><li>Development projects/programs can claim contributions to achievement of outcomes rather than attribution of development impacts. </li></ul></ul>
    6. 6. Underlying principles of Outcome Mapping <ul><li>Contribution rather than attribution. </li></ul><ul><li>Focuses on the logical links between interventions and behavioural change. </li></ul><ul><li>Changes are complex and are not linear, </li></ul><ul><ul><li>not based on a cause–effect framework, it recognizes that multiple, nonlinear events lead to change. </li></ul></ul>
    7. 7. Underlying principles of Outcome Mapping <ul><li>Development is done by and for the people. </li></ul><ul><li>A project can: </li></ul><ul><ul><li>influence achievement of outcomes but </li></ul></ul><ul><ul><li>cannot control them as the ultimate responsibility rests with people affected </li></ul></ul>
    8. 8. Outcome Mapping Terminologies <ul><li>Boundary partners </li></ul><ul><ul><li>The individuals, groups, and organizations with whom a project interacts directly to effect change and with whom the project can anticipate some opportunities for influence. </li></ul></ul><ul><li>Outcome challenge </li></ul><ul><ul><li>Describes how behaviors, relationships, activities, actions of an individual, group, or institution will change if the project is successful. </li></ul></ul><ul><li>Progress markers </li></ul><ul><ul><li>- A gradual set of milestones describing a progression of changed behavior in a project partner. </li></ul></ul>Love to see Like to see Expect to see
    9. 9. Boundary partners PROGRAM Boundary partner THE REAL WORLD
    10. 10. Relative Influence along the Results Chain Behavioural Changes (focus of OM) Endogenous actors Project/program Low High (Community ownership and capacity increases) (Project ownership declines) Inputs Activities Outputs Outcomes Impacts
    11. 11. Outcome Mapping Framework How do we know we are on-track? What do we want to learn? What are we trying to accomplish and how?
    12. 12. Project (performance journal) Boundary Partners (outcomes journal) Spheres of M&E with Outcome Mapping Relevance and viability (of the project) Internal practices in order to remain effective Implementation (interventions by the project) Outcomes (behaviour changes in the partners) Strategies (strategy journal)
    13. 13. Primary Uses of Outcome Mapping <ul><li>Planning Articulate goals & define activities </li></ul><ul><li>Monitoring Track project performance & partners’ progress </li></ul><ul><li>Evaluation Design & conduct a use-oriented evaluation </li></ul>
    14. 14. Methodologically, what does Outcome Mapping bring in? <ul><li>Focus on measurable outcomes. </li></ul><ul><ul><li>Clear formulation of responsibilities, roles and measurable milestones. </li></ul></ul><ul><ul><li>Each boundary partner develops a set of activities (milestones) that allow for RBM. </li></ul></ul><ul><li>Focus on learning and participation. </li></ul><ul><ul><li>Iterative and participatory planning for learning based management. </li></ul></ul><ul><ul><li>Information management involving all stakeholders. </li></ul></ul>
    15. 15. <ul><li>Expanding accountabilities. </li></ul><ul><li>Using OM for M&E can help in managing multiple accountabilities (up, down, internal) by: </li></ul><ul><ul><li>being able to document & communicate outcomes. </li></ul></ul><ul><ul><li>engagement in social learning with boundary partners, thereby anhancing ownership. </li></ul></ul>Methodologically, what does Outcome Mapping bring in?
    16. 16. Drawbacks of Outcome Mapping <ul><li>“ Changes in behavior” (though fundamental to sustainable development) cannot be an objective or an end in itself. </li></ul><ul><ul><li>Behavior changes should induce higher level improvements in situations at a higher level. </li></ul></ul><ul><li>It requires that project structures and activities constantly adapt to changing context (“fluidy”). </li></ul>
    17. 17. Towards a synthesis of LFA, and OM frameworks? (Roduner et al., 2008)
    18. 18. Accountability and Learning: A Balancing Act
    19. 19. Thank You