Latest Trends in 
Independent Evaluation 
at ADB 
Olivier Serrat 
2010 
The views expressed in this presentation are the v...
The Changing Operations 
Evaluation Landscape 
There is new emphasis on aid effectiveness from increased 
demand for bette...
The Changing Operations 
Evaluation Landscape 
The growing influence of nongovernment and civil society 
organizations is ...
Agents of Evaluation in ADB 
In ADB, self-evaluation is carried out by the 
operations departments responsible for designi...
Audiences for Evaluation 
ADB Operations Departments 
(monitoring and evaluation, results 
agenda, applied learning) 
Deve...
Purpose of Independent 
Evaluation 
Development Effectiveness—to contribute to 
development results, i.e., maximize the de...
Evolution of 
Independent Evaluation in ADB 
1972—A unit is established in 
the Economics Office. 
1978—A Post-Evaluation ...
The Role of the Development 
Effectiveness Committee 
The Development Effectiveness Committee in ADB was 
established in D...
The Role of IED 
The core task of IED is 
the systematic and 
objective assessment of 
policies, strategies, 
programs, an...
A Few Influential Evaluations 
Building a Results-Based 
Management Framework, 
2003— 
Promoting Portfolio 
Performance, 2...
Issues in Independent Evaluation 
The balance between 
accountability and learning, 
and between independence 
and engagem...
Issues in Independent Evaluation 
How to improve 
credibility— 
through 
stronger 
evidence? 
How to make 
better use of 
...
Knowledge Management for 
Independent Evaluation 
Evaluation findings only add value when they are used. 
To be used, eval...
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Matching Feedback Vehicles to 
Accountability and Learning
Improving Evaluation-Based 
Learning 
ADB's strategic frameworks are moving toward 
greater concentration on country, sect...
Improving Evaluation-Based 
Learning 
ADB must systematize and institutionalize 
feedback systems for evaluation-based 
le...
Improving Evaluation-Based 
Learning 
ADB should continue to publicly 
release proposed evaluation approach 
papers at an ...
Possible Directions in Evaluation 
Assigning resources to broader evaluation studies 
Harmonizing evaluation standards 
(J...
Possible Directions in Evaluation 
Evaluating business processes 
Disseminating findings and recommendations 
Promoting mo...
Further Reading 
• ADB. 2007. Independent Evaluation at the Asian Development 
Bank. Available: www.adb.org/publications/i...
Olivier Serrat 
Principal Knowledge Management Specialist 
Regional and Sustainable Development Department 
Asian Developm...
Upcoming SlideShare
Loading in...5
×

Latest Trends in Independent Evaluation at ADB

325

Published on

Independent evaluation has changed from the beginnings of evaluation activities in ADB some 30 years ago. Initially, the focus of evaluation was on assessing whether implementation was consistent with the intentions reflected in the appraisal of a project, and the extent to which the project achieved the expected economic and social benefits. Independent evaluation now shapes decision making throughout the project cycle and in ADB as a whole. This presentation offers more than an account of the early steps: it describes recent accomplishments and looks to the future.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
325
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Latest Trends in Independent Evaluation at ADB

  1. 1. Latest Trends in Independent Evaluation at ADB Olivier Serrat 2010 The views expressed in this presentation are the views of the author/s and do not necessarily reflect the views or policies of the Asian Development Bank, or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of the data included in this presentation and accepts no responsibility for any consequence of their use. The countries listed in this presentation do not imply any view on ADB's part as to sovereignty or independent status or necessarily conform to ADB's terminology.
  2. 2. The Changing Operations Evaluation Landscape There is new emphasis on aid effectiveness from increased demand for better development results. The aid community faces challenges caused by the crowded policy agenda, e.g., conflict reduction and prevention, fair international trade, environmental agreements, gender, a new international financial architecture, etc., many of which suffer from definitional problems and unclear time horizons, making them intrinsically difficult to evaluate. The aid community finds now often works with new partners in different partnership configurations; multi-agency consortia are becoming more common and agencies are moving beyond government-to-government links.
  3. 3. The Changing Operations Evaluation Landscape The growing influence of nongovernment and civil society organizations is adding dimensions, often positive, to debates about aid and its delivery. New instruments, e.g., sector-wide approaches, comprehensive development frameworks, the livelihoods approach, etc., are being tried to change the way the aid community works; they require significant shifts in how evaluation is approached. Aid delivery has been shifting to new horizons—from the project to the program to the country levels; there is also an array of issues, e.g., climate change, trade, migration, etc., that require regional or global approaches.
  4. 4. Agents of Evaluation in ADB In ADB, self-evaluation is carried out by the operations departments responsible for designing and implementing country partnership strategies, programs, projects, and technical assistance activities. Independent evaluation is conducted by the Independent Evaluation Department to systematically evaluate policies, strategies, programs, and projects, e.g., design, implementation, results, and associated business processes, to determine relevance, efficiency, effectiveness, impact, and sustainability.
  5. 5. Audiences for Evaluation ADB Operations Departments (monitoring and evaluation, results agenda, applied learning) Developing Member Countries (use of evaluations, evaluation capacity development, joint Independent Evaluation Department (evaluations, monitoring and evaluation expertise, self-assessment) ADB Management (use and follow-up of evaluations, influence on ADB directions) ADB Board of Directors (oversight) evaluations) International Evaluation Community (harmonization and joint evaluations)
  6. 6. Purpose of Independent Evaluation Development Effectiveness—to contribute to development results, i.e., maximize the development effectiveness of ADB's operations. Accountability —to confirm that ADB is doing the right things, that resources are properly allocated and used, that intended outcomes are realized. Generating Knowledge/Learning Lessons—to identify or derive lessons to improve the development impact of future policies, strategies, and operations.
  7. 7. Evolution of Independent Evaluation in ADB 1972—A unit is established in the Economics Office. 1978—A Post-Evaluation Office is set up. 1999—An Operations Evaluation Office is formed. 2009—An Independent Evaluation Department is created whose head is to be appointed by the Board. 2004—The Evaluation Head reports to the Board of Directors of ADB through its Development Effectiveness Committee. 2001—An Operations Evaluation Department is constituted.
  8. 8. The Role of the Development Effectiveness Committee The Development Effectiveness Committee in ADB was established in December 2000. The DEC's mandate is to help the Board of Directors of ADB carry out its responsibility of ensuring that the programs and activities of ADB achieve development effectiveness. Development effectiveness is assessed in part through independent evaluation. For the purpose of the DEC's work, development effectiveness is the measure of (i) whether ADB's programs and activities have resulted in the desired outcomes; and (ii) whether these programs and activities have made efficient use of ADB's available resources.
  9. 9. The Role of IED The core task of IED is the systematic and objective assessment of policies, strategies, programs, and projects, including their design, implementation, results, and associated business processes, to determine their relevance, effectiveness, efficiency, sustainability, and impact. • It evaluates the development effectiveness of operations. • It derives lessons to improve future operations. • It contributes to ADB's accountability for results. • It provides assistance to the DEC as the focal point for interaction with the Board of Directors. • It seeks to improve self-evaluation in ADB.
  10. 10. A Few Influential Evaluations Building a Results-Based Management Framework, 2003— Promoting Portfolio Performance, 2006— Improving Country Partnership Strategies, 2000s— Contributing to Safeguard Policy Update Involuntary Resettlement, 2006 Environment, 2006 Indigenous Peoples, 2007
  11. 11. Issues in Independent Evaluation The balance between accountability and learning, and between independence and engagement Conflict between closer engagement with developing member countries and Management Independence from the Board— is that advisable? Attribution or contribution—not all evaluations can address attribution issues How far should one move on self-evaluation and validation? How might one improve buy-in? Conduct fewer individual public sector project evaluations? Launch more country, sector, impact, and policy evaluations? Prepare more private sector evaluations?
  12. 12. Issues in Independent Evaluation How to improve credibility— through stronger evidence? How to make better use of evaluation results? Which evaluation methods to apply, e.g., qualitative or quantitative; rigorous or non-rigorous; experimental or non-experimental)? How to assess impact and sustainability? How to strengthen the accountability of Management? Recommendations must be clear, actionable, and monitorable How about the accountability of IED itself?
  13. 13. Knowledge Management for Independent Evaluation Evaluation findings only add value when they are used. To be used, evaluation findings must be available when needed by decision makers, in user-friendly formats. A Knowledge Management Unit was set up in IED in late 2006. Efforts are being made to improve dissemination of findings. IED invests continuingly in a searchable lessons database system that includes well over 1,000 lessons to date. Efforts to increase external dissemination through www.adb.org/site/evaluation/main, among other platforms, are well received.
  14. 14. Matching Feedback Vehicles to Accountability and Learning
  15. 15. Matching Feedback Vehicles to Accountability and Learning
  16. 16. Matching Feedback Vehicles to Accountability and Learning
  17. 17. Matching Feedback Vehicles to Accountability and Learning
  18. 18. Matching Feedback Vehicles to Accountability and Learning
  19. 19. Matching Feedback Vehicles to Accountability and Learning
  20. 20. Matching Feedback Vehicles to Accountability and Learning
  21. 21. Matching Feedback Vehicles to Accountability and Learning
  22. 22. Matching Feedback Vehicles to Accountability and Learning
  23. 23. Improving Evaluation-Based Learning ADB's strategic frameworks are moving toward greater concentration on country, sector, and thematic areas, and greater decentralization. ADB sees itself as a learning organization where operations evaluation plays a key part, and is given greater institutional independence and greater priority. More knowledge is generated from evaluations using more sophisticated methods. More impact-orientation is needed for quality improvement, conceptual advancement of development, and greater external accountability.
  24. 24. Improving Evaluation-Based Learning ADB must systematize and institutionalize feedback systems for evaluation-based learning and accountability. Efforts are needed to manage the quickening flood of data and information, network the systems, and cater for upward feedback. ADB must intensify the internalization of lessons from evaluation. Implementation monitoring, that is, implementation of findings and recommendations from evaluation, should be strengthened by means of a Management action record system.
  25. 25. Improving Evaluation-Based Learning ADB should continue to publicly release proposed evaluation approach papers at an early stage and post the comments of external stakeholders. ADB should continue to recognize the need for greater participation of in-country partners and stakeholders in the feedback process. Moving from project evaluations to a higher-level country, sector, and thematic focus is an important way of increasing potential evaluation impact.
  26. 26. Possible Directions in Evaluation Assigning resources to broader evaluation studies Harmonizing evaluation standards (Jointly) Evaluating country/sector assistance programs Validating country partnership strategy and project/program completion reports Evaluating impact Developing evaluation capacity Promoting portfolio performance
  27. 27. Possible Directions in Evaluation Evaluating business processes Disseminating findings and recommendations Promoting monitoring and measuring of outcomes Institutional evaluations , e.g., human resource function, information technology, knowledge management, Asian Development Bank Institute Rigorous impact evaluations
  28. 28. Further Reading • ADB. 2007. Independent Evaluation at the Asian Development Bank. Available: www.adb.org/publications/independent-evaluation- asian-development-bank • ADB. 2008. Conducting After-Action Reviews and Retrospects. Available: www.adb.org/publications/conducting-after-action-reviews- and-retrospects • ADB. 2009. Learning from Evaluation. Available: www.adb.org/publications/learning-evaluation • ADB. 2009. The Most Significant Change Technique. Available: www.adb.org/publications/most-significant-change-technique • ADB. 2011. Learning Histories. Available: www.adb.org/publications/learning-histories
  29. 29. Olivier Serrat Principal Knowledge Management Specialist Regional and Sustainable Development Department Asian Development Bank knowledge@adb.org www.adb.org/knowledge-management www.facebook.com/adbknowledgesolutions www.scribd.com/knowledge_solutions www.twitter.com/adbknowledge
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×