Latest Trends in Independent Evaluation at ADB
Upcoming SlideShare
Loading in...5

Latest Trends in Independent Evaluation at ADB



Independent evaluation has changed from the beginnings of evaluation activities in ADB some 30 years ago. Initially, the focus of evaluation was on assessing whether implementation was consistent with ...

Independent evaluation has changed from the beginnings of evaluation activities in ADB some 30 years ago. Initially, the focus of evaluation was on assessing whether implementation was consistent with the intentions reflected in the appraisal of a project, and the extent to which the project achieved the expected economic and social benefits. Independent evaluation now shapes decision making throughout the project cycle and in ADB as a whole. This presentation offers more than an account of the early steps: it describes recent accomplishments and looks to the future.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Latest Trends in Independent Evaluation at ADB Latest Trends in Independent Evaluation at ADB Presentation Transcript

    • The views expressed in this presentation are the views of the author/s and do not necessarily reflect the views or policies of the AsianDevelopment Bank, or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of the data includedin this presentation and accepts no responsibility for any consequence of their use. The countries listed in this presentation do not imply anyview on ADBs part as to sovereignty or independent status or necessarily conform to ADBs terminology.Latest Trends inIndependent Evaluationat ADBOlivier Serrat2010
    • The Changing OperationsEvaluation Landscape• There is new emphasis on aid effectiveness from increaseddemand for better development results.• The aid community faces challenges caused by the crowdedpolicy agenda, e.g., conflict reduction and prevention, fairinternational trade, environmental agreements, gender, a newinternational financial architecture, etc., many of which sufferfrom definitional problems and unclear time horizons, makingthem intrinsically difficult to evaluate.• The aid community finds now often works with new partnersin different partnership configurations; multi-agency consortiaare becoming more common and agencies are moving beyondgovernment-to-government links.
    • The Changing OperationsEvaluation Landscape• The growing influence of nongovernment and civil societyorganizations is adding dimensions, often positive, to debatesabout aid and its delivery.• New instruments, e.g., sector-wide approaches,comprehensive development frameworks, the livelihoodsapproach, etc., are being tried to change the way the aidcommunity works; they require significant shifts in howevaluation is approached.• Aid delivery has been shifting to new horizons—from theproject to the program to the country levels; there is also anarray of issues, e.g., climate change, trade, migration, etc.,that require regional or global approaches.
    • Agents of Evaluation in ADB• In ADB, self-evaluation is conducted by theoperations departments responsible fordesigning and implementing countrypartnership strategies, programs, projects,and technical assistance activities.• Independent evaluation is conducted by theIndependent Evaluation Departmentto systematically evaluate policies, strategies,programs, and projects, e.g., design, implementation, results,and associated business processes, to determine relevance,effectiveness, efficiency, and sustainability, and impact.
    • Audiences for EvaluationADB Operations Departments(monitoring and evaluation, resultsagenda, applied learning)Developing Member Countries(use of evaluations, evaluationcapacity development,joint evaluations)International EvaluationCommunity(harmonization and jointevaluations)ADB Management(use and follow-up of evaluations,influence on ADB directions)ADB Board of Directors(oversight)Independent EvaluationDepartment(evaluations, monitoring andevaluation expertise,self-assessment)
    • Purpose of Independent Evaluation• Development Effectiveness—tocontribute to development results,i.e., maximize the developmenteffectiveness of ADBs operations.• Accountability —to confirm thatADB is doing the right things, thatresources are properly allocatedand used, that intended outcomesare realized.• Generating Knowledge/Learning Lessons—to identify orderive lessons to improve the development impact of futurepolicies, strategies, and operations.
    • Evolution ofIndependent Evaluation in ADB• 1972—A unit was established in the Economics Office.• 1978—A Post-Evaluation Office was set up.• 1999—An Operations Evaluation Office was formed.• 2001—An Operations Evaluation Department was constituted.• 2004—The Evaluation Head reports to the Board of Directorsof ADB through its Development Effectiveness Committee.• 2009—An Independent Evaluation Department was createdwhose head is to be appointed by the Board.
    • The Role of the DevelopmentEffectiveness Committee• The Development Effectiveness Committee was established inDecember 2000.• The DECs mandate is to help the Board of Directors of ADBcarry out its responsibility of ensuring that the programs andactivities of ADB achieve development effectiveness.• Development effectiveness is assessed in part throughindependent evaluation.• For the purpose of the DECs work, development effectivenessis the measure of (i) whether ADBs programs and activitieshave resulted in the desired outcomes; and (ii) whether theseprograms and activities have made efficient use of ADBsavailable resources.
    • The Role of IED• The core task of IED is the systematicand objective assessment of policies,strategies, programs, and projects,including their design, implementation,results, and associated businessprocesses, to determine theirrelevance, effectiveness, efficiency,sustainability, and impact.• IED evaluates the development effectiveness of operations.• It derives lessons to improve future operations.• It contributes to ADBs accountability for results.• It provides assistance to the DEC as the focal point forinteraction with the Board of Directors.• It seeks to improve self-evaluation in ADB.
    • A Few Influential Evaluations• Building a Results-Based Management Framework, 2003—• Promoting Portfolio Performance, 2006—• Improving Country Partnership Strategies, 2000s—• Contributing to Safeguard Policy Update• Involuntary Resettlement, 2006• Indigenous Peoples, 2002• Environment, 2006
    • Issues in Independent Evaluation• The balance between accountability and learning, andbetween independence and engagement• Conflict between closer engagement with developing membercountries and Management• Independence from the Board—is that advisable?• Attribution or contribution—not all evaluations can addressattribution issues• How far should one move on self-evaluation and validation?• How might one improve buy-in?• Conduct fewer individual public sector project evaluations?Launch more country, sector, impact, and policy evaluations?Prepare more private sector evaluations?
    • Issues in Independent Evaluation• How to improve credibility—through stronger evidence?• How to make better use of evaluation results?• Which evaluation methods to apply?– Qualitative or quantitative– Rigorous or non-rigorous– Experimental or non-experimental• How to assess impact and sustainability?• How to strengthen the accountability of Management?• Recommendations must be clear, actionable, and monitorable• How about the accountability of IED itself?
    • Knowledge Management forIndependent Evaluation• Evaluation findings only add value when they are used.• To be used, evaluation findings must be available whenneeded by decision makers, in user-friendly formats.• A Knowledge Management Unit was established in IED in late2006.• Efforts are being made to improve dissemination of findings.• IED invests continuingly in a searchable lessons databasesystem that includes well over 1,000 lessons to date.• Efforts to increase external dissemination, among other platforms,are well received.
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Matching Feedback Vehicles toAccountability and Learning
    • Improving Evaluation-Based Learning• ADBs strategic frameworks are moving toward greaterconcentration on country, sector, and thematic areas, andgreater decentralization.• ADB sees itself as a learning organization where operationsevaluation plays a key part, and is given greater institutionalindependence and greater priority.• More knowledge is generated from evaluations using moresophisticated methods.• More impact-orientation is needed for quality improvement,conceptual advancement of development, and greaterexternal accountability.
    • Improving Evaluation-Based Learning• ADB must systematize and institutionalize feedback systemsfor evaluation-based learning and accountability.• Efforts are needed to manage the quickening flood of dataand information, network the systems, and cater for upwardfeedback.• ADB must intensify the internalization of lessons fromevaluation.• Implementation monitoring, that is, implementation offindings and recommendations from evaluation, should bestrengthened by means of a Management action recordsystem.
    • Improving Evaluation-Based Learning• ADB should continue to publicly release proposed evaluationapproach papers at an early stage and post the comments ofexternal stakeholders.• ADB should continue to recognize the need for greaterparticipation of in-country partners and stakeholders in thefeedback process.• Moving from project evaluations to a higher-level country,sector, and thematic focus is an important way of increasingpotential evaluation impact.
    • Possible Directions in Evaluation• Assigning resources to broaderevaluation studies• Harmonizing evaluation standards• (Jointly) Evaluating country/sectorassistance programs• Validating country partnership strategyand project/program completion reports• Evaluating impact• Developing evaluation capacity• Promoting portfolio performance
    • Possible Directions in Evaluation• Evaluating business processes• Disseminating findings and recommendations• Promoting monitoring and measuring of outcomes• Institutional evaluations , e.g., human resource function,information technology, knowledge management,Asian Development BankInstitute• Rigorous impact evaluations
    • Further Reading• ADB. 2007. Independent Evaluation at the Asian DevelopmentBank. Available:• ADB. 2008. Conducting After-Action Reviews and Retrospects.Available:• ADB. 2009. Learning from Evaluation.• ADB. 2009. Most Significant Change Technique.• ADB. 2011. Learning Histories.
    • Knowledge Management CenterOlivier SerratPrincipal Knowledge Management SpecialistKnowledge Management CenterRegional and Sustainable Development DepartmentAsian Development