M&e for ciat seminar
Upcoming SlideShare
Loading in...5
×
 

M&e for ciat seminar

on

  • 338 views

 

Statistics

Views

Total Views
338
Views on SlideShare
337
Embed Views
1

Actions

Likes
0
Downloads
3
Comments
0

1 Embed 1

http://dapa.ciat.cgiar.org 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Motivation of having this presentation. This “results- based management system” is being designed by the consortium in a top- down manner (as it requires deliberation and decision making at a high strategic level), and we want to do a first “sweep” for the center, organizing the materials that are coming out.
  • The circle marks the emphasis of this system on development outcomes and impact, and next and end users, for planning activities and outputs.
  • NONE of the definitions from now on in this presentation are FINAL or OFFICIAL. This is a stage of the thinking that is going on in the task force of the consortium and experts from around the CG. Final and Official definitions will be presented when they are considered to be ready for sharing. In words of group members, this is not “set in stone”. It is intended as guidance, and it will have to be adapted to each of the particular programs or projects using it.
  • Something we cannot stress enough- the importance of the greenarrows. Research outputs, for the most part, (again, this needs to be adapted to the particular program or project) can, and maybe should be, produced in partnership with the next users, to increase the chances that the product will really be used, and how, by the next user groups.
  • Questions we have- where, for example, do we want to place Learning Alliances? I took it out of capacity development “events”. Are Learning Alliances even an output?
  • Questions we have - where, for example, do we want to place Innovative Platforms? Are these referring to communication platforms, or to things like Innovation Platforms, which would then NOT be an output?
  • Will probably happen in chains- of actors, or of time, or of geographical scope
  • This is not all the types of practice changes, but very important
  • Have to specify-reduced or increased- by how much? Consumption by whom? Etc.
  • We need a better definition of Institutions
  • There are many different terms to define these groups, even within discussion in the consortium. I have picked next and end, personal preference, but you can call them however you like.There is no solid line dividing who should be Next or End- depends of the project or program we are talking about, including particular outputs and stages of the project. The same group can be considered at different moments to be implementers, next and then end users.
  • Emphasis on – now we will focus on Development outcomes, including IDOS, until now those were just research outcomes.
  • Maybe confusion as to where this type of IDOs stop and Impacts begin…
  • Standalone= benefit in and of its own, not “just” a step necessary to achieve
  • Standalone= benefit in and of its own, not “just” a step necessary to achieve
  • This tries to show where, until now, the emphasis of M&E is being placed. Likely to be adjusted still.
  • We= each CRP

M&e for ciat seminar M&e for ciat seminar Presentation Transcript

  • Outcomes thinking in CIAT and CRPs March 2013 CIAT Presented by Sophie Alvarez www.ciat.cgiar.org Science to cultivate change
  • Achievement of Development Outcomes CIAT is committed to emphasizing the achievement of development outcomes. A results-based program management in the CGIAR shifts the focus from outputs or products into how these products will be used, and by whom, to ultimately contribute to CGIAR’s overall goals of poverty and hunger alleviation.
  • Achievement of Development Outcomes This has implications in the way WE: •Plan projects and initiatives, • Implement them, especially regarding clarity on who we have to work with and who we work for, and strategies to do it, • Learn & document and report, and • Measure performance.
  • Generic Impact Pathway and Actors inResearch for Agricultural Development Research Research DevelopmentActivities Outputs Outcomes Impact Outcomes Implementers Next users End users
  • What are Impact Pathways and Theories of Change? Impact Pathways describe results chains, showing the linkages between the sequence of results in getting to impact, but do not explain the ‘theory’ of why the intervention is expected to work i.e. the links in the pathway. Research Research Development Activities Impact Outputs Outcomes Outcomes Set of Outcome Outcomeactivities WHY Output 2 WHY 2 WHY 4 WHY Impact A theory of change (ToC) complements impact pathways by describing the causal linkages through which it is expected that an intervention will bring about the desired results. Put simply, a ToC is a causal model of how the intervention worked or is expected to work.
  • Example of an Impact Pathway USED BY… Results in… Other CG Development scientists doing needs analysis with women Evidence of CGIAR scientists Other Outcomes value in use farmers of HR cassava Increased productivity for NARS breeders knowManuals and Evidence of the beneficiaries"cookbooks" value in use how to select HR NARS 1 cassava in field Protocols for HR cassava KAS and Practice Changes in social development Train host changes arrangement, such as in scientist to Needs support gender norms and analysis home NARS breeders using HR power structures institute cassava breeding Tech transfer from other NARS 2 materials crop systems New varieties released by govt.
  • Research Activities (includes participatory and action) Capacity Building (for key target groups, partners, centers, CRPs) Engagement (communicate and/or influence, co-learning, demonstrating goodImplementers practices, and networking)
  • ImplementersActivities Us and partners: CENTERS CRPs NGOs, NARS, Private …. Working in partnerships and networks
  • New information and Research Outputs understanding (CGIAR research outputs) New technologies and practices Research information (knowledge and basic understanding)Implementers Next users New understanding for putting research into action
  • Research Outputs Capacity development events On the job training and activities Professional development courses and curriculaImplementers Next users
  • Research Outputs Engagement events and networks Communication campaigns Innovative platformsImplementers Next users
  • 2 Major types of Research Outcomesoutcomes: Capacity change (changes in knowledge, attitudes, and skills) that underpin or support….. Key practice/ behavioral changes, practice change* Development Outcomes Direct benefits to beneficiaries ImplementersNext and end users Changes in the enabling environment
  • Research Outcomes Key capacity changes(knowledge, attitudes and skills) Changes in capacity of Understandusersimplications of evidence- •Next the •End users based research results and knowledge •Other CG scientists and CRPs Have the ability to apply the results andFor example, knowledge into policypolicy makers Have improved capacities to negotiatein area x: science-based results into policy agendas Are aware of and convinced by results relevant to their policy area and understand policy implications
  • Research Outcomes Key capacity changes(knowledge, attitudes and skills) Describe things such as: – Understand, accept, – Are prepared to use, know how to do… – Have improved capacities to participate, to negotiate… – Are aware of, see added value of… – Are motivated, “believe”, are convinced… – Have an improved attitude towards learning about… and Have a defined next user group, such as NARS researchers or Policy Makers, or ourselves
  • Key practice/ behavioral Research Outcomes changes X# Farmers in the (-) region adopt and adapt improved nutrient management practices for higher yields and improved soil qualityLand use planners using GIS maps and decision support models NARS breeders using CGIAR-developed maize breeding materials, and Smallholder farmers adopt improved varieties released maize varieties
  • Key practice/ behavioral Research Outcomes changes Describe actions such as: – Using, making improved use of… • And we need to know (at least descriptively) how and if they are using it now – Adapting... Adopting • And we need to know what we will consider significant and sustainable adoption And also Have a defined next user group, such as NARS researchers or Policy Makers
  • Development Outcomes Direct benefits to beneficiariesIncreased productivity for the beneficiariesImproved distribution of income, food security andnutrition benefits to the targeted poor and womenReduced degradation of natural resources, betterfunctioning or conservation of ecosystems andlandscapes
  • Development Outcomes Direct benefits to beneficiaries Reduced deforestation; increased carbon sequestration (an ISPC IDO) Reduced erosion, better water quality, and flood control Increased consumption of biofortified foods (an ISPC IDO)Increased income for smallholder farmers from adopting improved varieties
  • Development OutcomesChanges in the enabling environmentNew policies and policy instruments mechanisms such as:new regional and national policies,regulations and administrative rules with a realistic implementation planNew or better functioning institutions both formal and informal:new or improved formal governance arrangements, such as managementsystems, networks, planning bodies, government institutions, innovationplatforms, etc.new or improved private sector institutions, markets and practiceschanges in social arrangement, such as in gender norms and power structureschanged patterns of interactions within and between organisations and socialunitschanges in global public goods and the R4D paradigm related to agriculturalresearch
  • Development OutcomesChanges in the enabling environment Increased market opportunities for poor from a specific value chain (an ISPC IDO) Increased accessFunctioning seed markets (an to assets for ISPC IDO) women in country XPolicies National policies thatcontrolling support pro-poorillegal logging value chainadopted (an development andISPC RO) credit schemes for women (an ISPC RO)
  • End users/ Next users Beneficiaries/ Intermediaries Indirect boundary partners/ Primary boundary partners Next usersResearch development partners (also implementers)Groups which will use our research outputs End Users Examples: •Civil Society groups Individuals and •Communities and regions of communities households •Advanced research institutions (farmers, fishers, •CGIAR researchers from different centers and women) disciplines •NARES individuals/groups (local researchers, officials) •Private sector groups (seed producers, microfinance agencies, etc.) •Regional/national policy makers •Local, national and international NGOs
  • Generic Impact Pathway and Actors Research Research DevelopmentActivities Outputs Outcomes Impact Outcomes Includes IDOS Implementers Next users End users
  • What are Intermediate Development Outcomes (IDOs)?IDOs can relate to either or both types of Development Outcomes
  • What are Intermediate Development Outcomes (IDOs)?IDOs can relate to either or both types of Development Outcomes
  • What are Intermediate Development Outcomes (IDOs)?Are the result of multiple activities by diverse actorsboth inside and outside the CGIARCapacity and behavioral changes are NOT developmentoutcomes, therefore NOT IDOs- (for the most part)Are intended to affect positively the welfare of thetargeted population or environmentCan cut across individual impact pathways, e.g., an IDOon enabling environment could apply to several impactpathways
  • What are Intermediate Development Outcomes (IDOs)?Occur as a result of adopting improved varieties orbetter farm management approaches and relate tothose directly targeted by the specific activities.There must be a link between the IDOs and theactivities undertaken - are attributable to outputs inthat the contribution made can be assessedRepresent major milestones in the individual CRPs’impact pathways towards the SLOs
  • What are Intermediate Development Outcomes (IDOs)? CRP Strategic Goal= CRP IDO impactTimeframe 5-7 years 10-15 Scale ≧10s of thousands(numbers ≧ millions to 100s of thousandsimpacted) “Standalone” result: Specific contributing Type aggregation of outcome outcomes
  • What are CRP impacts (CRP Strategic Goals)? CRP Strategic Goal= CRP IDO impactTimeframe 5-7 years 10-15 Scale ≧10s of thousands to(numbers ≧ millions 100s of thousandsimpacted) “Standalone” Specific contributing Type result: aggregation outcome of outcomes
  • What are CRP impacts (CRP Strategic Goals) and SLOs?CRP impacts are aggregates across the CRP and reflect theinteractions among several factors.CRP impacts can be thought of as specific instances of SystemLevel Outcomes (SLOs)Are a purpose unto themselves, not a step to achievingsomething else. In that sense, increased stability-connectivity- or redistribution (of power, of benefits) are nota good CRP impact or SLO.High-level SLOs are aspirational and the theoretical impactpathways to them from research are very long.
  • Theory of changeA theory of change (ToC) complements impact pathways by describingthe causal linkages through which it is expected that an interventionwill bring about the desired results. Put simply, a ToC is a causal modelof how the intervention worked or is expected to work.Useful to distinguish:Types of strategies being used. These could be around the various typesof engagement or capacity development efforts undertaken acrossprojects and the specific results sought through research activities.By target groups. Focusing on activities aimed at the different targetgroups (such as research partners, households, women). For each a ToCcould be readily developed and used as the basis for the impact storyassociated with each group.
  • Theory of changeAre time dependent. Reflect understanding and knowledge up to that pointAre based on prior research and stakeholder views. ToC are based on acombination of prior social science research, experience and on stakeholderviewsAre usually developed in a participatory manner. Usually the processinvolves some or all of the various stakeholdersNeed to recognize uncertainties. In fact, ToC need to be thought of inprobabilistic terms: that the ToC model is likely sufficientHave different purposes. develop a common understanding , and tocommunicate with others, on how and why an intervention works, as a basisfor a monitoring system or impact evaluation, for making causal claimsCan be either ex ante or ex post. Can be envisaged both before theintervention has been implemented and also after the intervention has beenin place for some time.
  • Measuring- M&E Ex- post socio- econ impact assessments and impact The pathway from research activities and outputs evaluations, done to research outcomes will be where most of a some time after the CRP’s monitoring effort would be focused. intervention has been put in place and often once the project is completed. CRP monitoring will be tracking progress to the direct benefit IDOs through, for example, adoption studies and surveys of changes in income levels.Knowing the role played by an outcome in getting to impacts is important in assessing which of many possible outcomes are most useful to track
  • Measuring- M&E and contribution claimsSome sorts of contribution claims we will be able to make: Some early changes (effects of the CRP activities, such as aspects of capacity development—the relevant aspects of changes in knowledge, attitudes, skills, and opportunities for the targeted groups, incremental effect on community innovation rate, self-reliance and their natural resource base for a number of years) have been realized in line with the ToC. The upstream elements of the ToC are robust, i.e. are supported by empirical evidence. Therefore, contributions to the upstream outcomes and impacts are likely to occur.
  • What are we doing now?1. Developing CRP Results Strategy FrameworksEach CRP setting out its results strategy framework showing its: strategic goals, main (few) impact pathways and their outcomes, the general assumptions and linkages among the impact pathways and the CGIAR system-level outcomes, and CRP IDOs.… accompanied by narratives describing: the development challenges being addressed, why are you involved in each area? What is the CRP comparative advantage? the rationale for the impact pathways used and the major assumptions associated with these pathways (external events and conditions needed for the impact pathway to be successful) What IDOs and impacts do we expect to accomplish with each IP?
  • What are we doing now?2. Grouping CRP Outcomes3. Identifying IDOs4. Developing ToC for the CRP Results Strategies (impact pathways and ToCs)5. ???
  • Lots of clarification still needed…Some are simply LANGUAGE:End users/ Next users, Beneficiaries/ Intermediaries, Indirect boundary partners/Primary boundary partners, Key target groups, Significant othersBehavioral- practice changes = ISPC research outcomes/ immediate outcomes? /intermediate outcomes?Others are not:Commodity vs other more systemic CRPs- are IP and ToC, etc. just as good for both?Greater participation of women in decision-making in a targeted domain - what isthis? An IDO? A practice outcome? A CRP outcome? An impact?How to measure aspects of capacity development (the relevant aspects of changesin knowledge, attitudes, skills, innovative capacity and opportunities) —as “stepone” in achieving outcomes?What is the role of baselines?
  • ThanksMaterials adapted from:Discussions and materials in the CGIAR CRP IDO discussion groupPIPA and OMCRPs (especially RTB, CCAFS, AAS) planning materialsConversations in the KS and CS and ISS – DAPA teams
  • Some participant questions• Probability of success- may make us work towards “safer” outcomes- low risk, low pay-off• System seems to make CRPs working together harder- collaboration vs. competition.• Does the consortium have a TOC?• CRP level- vs institutional level– what can we do internally (at CIAT)?• We are having this presentation because this is part if our core business. This is a first awareness- raising, next we will: – Share this PPT with the regions and CIAT – ISPC meetings to happen in Cali, further clarification – Sit down with CRP focal points and discuss what with ISPC • Follow up meeting – Other meeting (general) –specific to each CRP/ knowledge base – In theory we can understand: but how to manage + make operational- we have to pull this together.• Do not call a new paradigm- we have always worked towards outcomes and impact• How do we go about fitting something into the CRP? How to feedback into a CRP?