Latest Trends in Independent Evaluation at ADB
Upcoming SlideShare
Loading in...5
×
 

Latest Trends in Independent Evaluation at ADB

on

  • 438 views

Independent evaluation has changed from the beginnings of evaluation activities in ADB some 30 years ago. Initially, the focus of evaluation was on assessing whether implementation was consistent with ...

Independent evaluation has changed from the beginnings of evaluation activities in ADB some 30 years ago. Initially, the focus of evaluation was on assessing whether implementation was consistent with the intentions reflected in the appraisal of a project, and the extent to which the project achieved the expected economic and social benefits. Independent evaluation now shapes decision making throughout the project cycle and in ADB as a whole. This presentation offers more than an account of the early steps: it describes recent accomplishments and looks to the future.

Statistics

Views

Total Views
438
Views on SlideShare
438
Embed Views
0

Actions

Likes
0
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Latest Trends in Independent Evaluation at ADB Latest Trends in Independent Evaluation at ADB Presentation Transcript

  • Latest Trends in Independent Evaluation at ADB Olivier Serrat 2010 The views expressed in this presentation are the views of the author/s and do not necessarily reflect the views or policies of the Asian Development Bank, or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of the data included in this presentation and accepts no responsibility for any consequence of their use. The countries listed in this presentation do not imply any view on ADB's part as to sovereignty or independent status or necessarily conform to ADB's terminology.
  • The Changing Operations Evaluation Landscape • There is new emphasis on aid effectiveness from increased demand for better development results. • The aid community faces challenges caused by the crowded policy agenda, e.g., conflict reduction and prevention, fair international trade, environmental agreements, gender, a new international financial architecture, etc., many of which suffer from definitional problems and unclear time horizons, making them intrinsically difficult to evaluate. • The aid community finds now often works with new partners in different partnership configurations; multi-agency consortia are becoming more common and agencies are moving beyond government-to-government links.
  • The Changing Operations Evaluation Landscape • The growing influence of nongovernment and civil society organizations is adding dimensions, often positive, to debates about aid and its delivery. • New instruments, e.g., sector-wide approaches, comprehensive development frameworks, the livelihoods approach, etc., are being tried to change the way the aid community works; they require significant shifts in how evaluation is approached. • Aid delivery has been shifting to new horizons—from the project to the program to the country levels; there is also an array of issues, e.g., climate change, trade, migration, etc., that require regional or global approaches.
  • Agents of Evaluation in ADB • In ADB, self-evaluation is conducted by the operations departments responsible for designing and implementing country partnership strategies, programs, projects, and technical assistance activities. • Independent evaluation is conducted by the Independent Evaluation Department to systematically evaluate policies, strategies, programs, and projects, e.g., design, implementation, results, and associated business processes, to determine relevance, effectiveness, efficiency, and sustainability, and impact.
  • Audiences for Evaluation ADB Operations Departments (monitoring and evaluation, results agenda, applied learning) Developing Member Countries (use of evaluations, evaluation capacity development, joint evaluations) International Evaluation Community (harmonization and joint evaluations) ADB Management (use and follow-up of evaluations, influence on ADB directions) ADB Board of Directors (oversight) Independent Evaluation Department (evaluations, monitoring and evaluation expertise, self-assessment)
  • Purpose of Independent Evaluation • Development Effectiveness—to contribute to development results, i.e., maximize the development effectiveness of ADB's operations. • Accountability —to confirm that ADB is doing the right things, that resources are properly allocated and used, that intended outcomes are realized. • Generating Knowledge/Learning Lessons—to identify or derive lessons to improve the development impact of future policies, strategies, and operations.
  • Evolution of Independent Evaluation in ADB • 1972—A unit was established in the Economics Office. • 1978—A Post-Evaluation Office was set up. • 1999—An Operations Evaluation Office was formed. • 2001—An Operations Evaluation Department was constituted. • 2004—The Evaluation Head reports to the Board of Directors of ADB through its Development Effectiveness Committee. • 2009—An Independent Evaluation Department was created whose head is to be appointed by the Board.
  • The Role of the Development Effectiveness Committee • The Development Effectiveness Committee was established in December 2000. • The DEC's mandate is to help the Board of Directors of ADB carry out its responsibility of ensuring that the programs and activities of ADB achieve development effectiveness. • Development effectiveness is assessed in part through independent evaluation. • For the purpose of the DEC's work, development effectiveness is the measure of (i) whether ADB's programs and activities have resulted in the desired outcomes; and (ii) whether these programs and activities have made efficient use of ADB's available resources.
  • The Role of IED • The core task of IED is the systematic and objective assessment of policies, strategies, programs, and projects, including their design, implementation, results, and associated business processes, to determine their relevance, effectiveness, efficiency, sustainability, and impact. • IED evaluates the development effectiveness of operations. • It derives lessons to improve future operations. • It contributes to ADB's accountability for results. • It provides assistance to the DEC as the focal point for interaction with the Board of Directors. • It seeks to improve self-evaluation in ADB.
  • A Few Influential Evaluations • Building a Results-Based Management Framework, 2003— • Promoting Portfolio Performance, 2006— • Improving Country Partnership Strategies, 2000s— • Contributing to Safeguard Policy Update • Involuntary Resettlement, 2006 • Indigenous Peoples, 2002 • Environment, 2006
  • Issues in Independent Evaluation • The balance between accountability and learning, and between independence and engagement • Conflict between closer engagement with developing member countries and Management • Independence from the Board—is that advisable? • Attribution or contribution—not all evaluations can address attribution issues • How far should one move on self-evaluation and validation? • How might one improve buy-in? • Conduct fewer individual public sector project evaluations? Launch more country, sector, impact, and policy evaluations? Prepare more private sector evaluations?
  • Issues in Independent Evaluation • How to improve credibility—through stronger evidence? • How to make better use of evaluation results? • Which evaluation methods to apply? – Qualitative or quantitative – Rigorous or non-rigorous – Experimental or non-experimental • How to assess impact and sustainability? • How to strengthen the accountability of Management? • Recommendations must be clear, actionable, and monitorable • How about the accountability of IED itself?
  • Knowledge Management for Independent Evaluation • Evaluation findings only add value when they are used. • To be used, evaluation findings must be available when needed by decision makers, in user-friendly formats. • A Knowledge Management Unit was established in IED in late 2006. • Efforts are being made to improve dissemination of findings. • IED invests continuingly in a searchable lessons database system that includes well over 1,000 lessons to date. • Efforts to increase external dissemination through www.adb.org/site/evaluation/main, among other platforms, are well received.
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Matching Feedback Vehicles to Accountability and Learning
  • Improving Evaluation-Based Learning • ADB's strategic frameworks are moving toward greater concentration on country, sector, and thematic areas, and greater decentralization. • ADB sees itself as a learning organization where operations evaluation plays a key part, and is given greater institutional independence and greater priority. • More knowledge is generated from evaluations using more sophisticated methods. • More impact-orientation is needed for quality improvement, conceptual advancement of development, and greater external accountability.
  • Improving Evaluation-Based Learning • ADB must systematize and institutionalize feedback systems for evaluation-based learning and accountability. • Efforts are needed to manage the quickening flood of data and information, network the systems, and cater for upward feedback. • ADB must intensify the internalization of lessons from evaluation. • Implementation monitoring, that is, implementation of findings and recommendations from evaluation, should be strengthened by means of a Management action record system.
  • Improving Evaluation-Based Learning • ADB should continue to publicly release proposed evaluation approach papers at an early stage and post the comments of external stakeholders. • ADB should continue to recognize the need for greater participation of in-country partners and stakeholders in the feedback process. • Moving from project evaluations to a higher-level country, sector, and thematic focus is an important way of increasing potential evaluation impact.
  • Possible Directions in Evaluation • Assigning resources to broader evaluation studies • Harmonizing evaluation standards • (Jointly) Evaluating country/sector assistance programs • Validating country partnership strategy and project/program completion reports • Evaluating impact • Developing evaluation capacity • Promoting portfolio performance
  • Possible Directions in Evaluation • Evaluating business processes • Disseminating findings and recommendations • Promoting monitoring and measuring of outcomes • Institutional evaluations , e.g., human resource function, information technology, knowledge management, Asian Development Bank Institute • Rigorous impact evaluations
  • Further Reading • ADB. 2007. Independent Evaluation at the Asian Development Bank. Available: www.adb.org/publications/independent-evaluation- asian-development-bank • ADB. 2008. Conducting After-Action Reviews and Retrospects. Available: www.adb.org/publications/conducting-after-action-reviews- and-retrospects • ADB. 2009. Learning from Evaluation. Available: www.adb.org/publications/learning-evaluation • ADB. 2009. Most Significant Change Technique. Available: www.adb.org/publications/most-significant-change-technique • ADB. 2011. Learning Histories. Available: www.adb.org/publications/learning-histories
  • Knowledge Management Center Olivier Serrat Principal Knowledge Management Specialist Knowledge Management Center Regional and Sustainable Development Department Asian Development Bank knowledge@adb.org www.adb.org/knowledge-management www.facebook.com/adbknowledgesolutions www.scribd.com/knowledge_solutions www.twitter.com/adbknowledge