Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Session: The role of the evaluator in results based evaluation
ney                                                                     xpay  ers’ mo                                     ...
Why Results? Assessment - measure whether achieved what was planned Accountability - communicate the impact of developme...
Everyone’s at it…“The Government has made a priority ofachieving better results from developmentassistanceUlla Tørnæs, Dan...
…and without being hard-hearted, we will also be hard-headed, and make sure our aid money is directed atthose things which...
• …Strengthen our efforts to achieve concrete and  sustainable results. This involves better managing for results,  monito...
Results are… A describable or measurable change in state that is derived  from a cause and effect relationship. Results a...
The Key Achievementssection at the start of thisreport sets out a number ofaggregate results based onDFID’s standard indic...
Opportunities/Threats with the Results agenda          Opportunities                             ThreatsOpportunity for ev...
The ‘Value for Money’ debate...“    our commitment to reaching 0.7% of    national income on aid by 2013, and    enshrinin...
Value for Money
What is Value for Money (VfM)?                  Often VfM isResources /      understood by                                ...
Opportunities/Threatswith VFM agenda          Opportunities                            Threats(Re) promote an evaluation a...
Transparency
W is aid transparency important? hy  ACCOUNTABILITY                         OWNERSHIP        EFFICIENCY                  F...
International Aid Transparency                Initiative (IATI)1.   A standard for data on aid flows - NOT a new database2...
Background: Aid Information Today      Lots of people want data                       There is a lot of data out therewww....
… the worst of all worlds
How does IATI work ? IATI is a standard, not a database or a reporting system:        1) agreement on what will be publish...
IATI: publish once, use often
IATI “Beast and the Beauty”
Conclusions and… more questions? Individual agency data of little use; collaborative data better value Exchange data to ...
Opportunities/Threats with the Transparency agenda            Opportunities                                  ThreatsEvalua...
New opportunitiesNew skills needed (in all three areas)New partnerships (data people for Transparency)The dangers of o...
Thank you!
Evaluating impact, maximising Value for Money (VfM) and being transparent
Evaluating impact, maximising Value for Money (VfM) and being transparent
Evaluating impact, maximising Value for Money (VfM) and being transparent
Evaluating impact, maximising Value for Money (VfM) and being transparent
Evaluating impact, maximising Value for Money (VfM) and being transparent
Evaluating impact, maximising Value for Money (VfM) and being transparent
Upcoming SlideShare
Loading in …5
×

Evaluating impact, maximising Value for Money (VfM) and being transparent

1,772 views

Published on

Responding to the current drivers in UK international development in UK international development. Julian Barr and Sam McPherson - Itad

  • Be the first to comment

Evaluating impact, maximising Value for Money (VfM) and being transparent

  1. 1. Session: The role of the evaluator in results based evaluation
  2. 2. ney xpay ers’ mo oun d of ta e of value.” Aid mus t show re for every p e 100 penc value for al re that strat money “Ensu , we demon we spend “Overseas aid to be“Interna tional spent through new nt secretary system of payment by results” de velopme t to ac pledges ure ly to ens ‘THE £3.7billion h ruthless oney for overseas aid could ike in r m value fo paid for 200 new sc have rs” taxpaye kept 123,333 cops hools, Corr on the uptio beat or locked up an extra accu n 97,368 criminals,.. s Glob ations hi can reveal.’’ The Sun al Fu t the nd ‘Corru p UK ai tion ruinin d bu d g get ‘Value of aid to war-torn countries ‘cannot easily be assessed’
  3. 3. Why Results? Assessment - measure whether achieved what was planned Accountability - communicate the impact of development assistance to Parliament, taxpayers, civil society, & recipients Decision making - make evidence-based decisions by knowing what works and what doesn’tGlobal shift to accountability and decision-making driven by demand for results and evidence of resultsLonger-term shift towards Results-Based Management
  4. 4. Everyone’s at it…“The Government has made a priority ofachieving better results from developmentassistanceUlla Tørnæs, Danish Minister for Development Cooperation“The question we ask today is not whether our government istoo big or too small, but whether it works …those of us whomanage the public’s dollars will be held to account - tospend wisely, reform bad habits, and do our business in thelight of day – because only then can we restore the vital trustbetween a people and their government.”
  5. 5. …and without being hard-hearted, we will also be hard-headed, and make sure our aid money is directed atthose things which are quantifiable andmeasureable…… so we really know we are getting results.David Cameron, Lagos, 19th July 2011
  6. 6. • …Strengthen our efforts to achieve concrete and sustainable results. This involves better managing for results, monitoring, evaluating and communicating progress; as well as scaling-up our support, strengthening national capacities and leveraging diverse resources and initiatives in support of development results.• We will partner to implement a monitor progress, evaluate impact, ensure sound, results-focused public sector management, and highlight strategic issues for policy decisions
  7. 7. Results are… A describable or measurable change in state that is derived from a cause and effect relationship. Results are the same as outcomes, and are further qualified as immediate, intermediate, or ultimate.•Sustained improvement in development outcomes at thecountry level (e.g., families lifted out of poverty…).•Results: The output, outcome or impact (intended orunintended, positive and/or negative) of a developmentintervention
  8. 8. The Key Achievementssection at the start of thisreport sets out a number ofaggregate results based onDFID’s standard indicatorset…
  9. 9. Opportunities/Threats with the Results agenda Opportunities ThreatsOpportunity for evaluators to ‘step Indicator driven development:up’ to the plate...now is our time doing what is easy to measure…(in development)!Focus on theory of change to Results agenda dominated byensure program results are possible ‘output’ measurementInvest in developing innovative Focus only on positive resultsways to measure results (comms messages)Needs very good results Failure to show results = failure ofmeasurement systems (M&e) evaluators rather than programmes (or the fact that doing this is hard!...)
  10. 10. The ‘Value for Money’ debate...“ our commitment to reaching 0.7% of national income on aid by 2013, and enshrining this in law, imposes an even greater duty on us, more than any other Department in Whitehall, to get value for money, to bear down on waste, and to ensure that aid secures 100 pence of value for every hard-earned British taxpayers pound we spend.Andrew Mitchell, UK Secretary of State for International Development. Oct 2010
  11. 11. Value for Money
  12. 12. What is Value for Money (VfM)? Often VfM isResources / understood by Service & WiderInvestment comparing unit costs Outcomes Money Economic People Inputs Outputs SocialEnvironment Real VfM is achieved Environmental by comparing outcomes with investment
  13. 13. Opportunities/Threatswith VFM agenda Opportunities Threats(Re) promote an evaluation agenda VFM agenda does not ‘learn’ from(with new language!) the experience of the evaluation sector (ignore counterfactual, attribution etc)Help clarify and operationalise Multiple interpretations of VFM (e.g. technical v allocative VF)Create innovative and useful tools Simple, high-level definitions, withto help people use VFM as little guidance on measurement;management tool; recognisesdifferent ‘values’X-Whitehall initiative Focus on the easy to measure (economy & efficiency) - perverse incentives
  14. 14. Transparency
  15. 15. W is aid transparency important? hy ACCOUNTABILITY OWNERSHIP EFFICIENCY FEEDBACK RESULTS PREDICTABILITY CORRUPTION EFFECTIVENESS DIVERSION Traceability BUDGET PLANNING COORDINATION Demonstrating Results
  16. 16. International Aid Transparency Initiative (IATI)1. A standard for data on aid flows - NOT a new database2. A common exchange format3. Open data as a platform4. Multi-stakeholder initiative
  17. 17. Background: Aid Information Today Lots of people want data There is a lot of data out therewww.aidtransparency.net BUT... Data is not accessible to users
  18. 18. … the worst of all worlds
  19. 19. How does IATI work ? IATI is a standard, not a database or a reporting system: 1) agreement on what will be published; 2) common definitions for publication; 3) a common, open, electronic data format; 4) a Framework for Implementation. The approach: • Aid providers publish their aid information (HQ or country office) •Open Aid Register • Link to the IATI Registry (an index that points users searching for aid information to its location) The underlying principle is publish once, use many times www.aidtransparency.net
  20. 20. IATI: publish once, use often
  21. 21. IATI “Beast and the Beauty”
  22. 22. Conclusions and… more questions? Individual agency data of little use; collaborative data better value Exchange data to improve performance Increase public support?  Set up a collaborative platform to tell the story of UKAid (in HIV), visualising ‘results’ (focusing on financials now is probably a tactical mistake)
  23. 23. Opportunities/Threats with the Transparency agenda Opportunities ThreatsEvaluation findings can (finally) get the transparency can still be "gamed“ (justexposure ‘we’ would want! as targets can...)The focus on getting the data out is only Transparency can result in ‘damage’ tothe first part of the job...we still need development agenda (e.g. negativeexperts to analyse and interpret the results used to justify political agendas)data (that’s our job!)Evaluations should be easier (since core Being ‘transparent’ requires significantdata should be available...) technical and financial resources. Opportunity cost and whos funding this?Chance to push transparency aroundnull/failed or uncompleted evaluationsagenda (e.g. IE registries of IE)
  24. 24. New opportunitiesNew skills needed (in all three areas)New partnerships (data people for Transparency)The dangers of over-emphasis on results (& measurement) driving evaluation out of fashion [being pushed back]Need to be better communicators – all three agendas converge around communicating to wider audiences
  25. 25. Thank you!

×