Independent evaluation has changed from the beginnings of evaluation activities in ADB some 30 years ago. Initially, the focus of evaluation was on assessing whether implementation was consistent with the intentions reflected in the appraisal of a project, and the extent to which the project achieved the expected economic and social benefits. Independent evaluation now shapes decision making throughout the project cycle and in ADB as a whole. This presentation offers more than an account of the early steps: it describes recent accomplishments and looks to the future.

1. The views expressed in this presentation are the views of the author/s and do not necessarily reflect the views or policies of the Asian
Development Bank, or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of the data included
in this presentation and accepts no responsibility for any consequence of their use. The countries listed in this presentation do not imply any
view on ADB's part as to sovereignty or independent status or necessarily conform to ADB's terminology.
Latest Trends in
Independent Evaluation
at ADB
Olivier Serrat
2010

2. The Changing Operations
Evaluation Landscape
There is new emphasis on aid effectiveness from increased
demand for better development results.
The aid community faces challenges caused by the crowded
policy agenda, e.g., conflict reduction and prevention, fair
international trade, environmental agreements, gender, a new
international financial architecture, etc., many of which suffer
from definitional problems and unclear time horizons, making
them intrinsically difficult to evaluate.
The aid community finds now often works with new partners in
different partnership configurations; multi-agency consortia are
becoming more common and agencies are moving beyond
government-to-government links.

3. The Changing Operations
Evaluation Landscape
The growing influence of nongovernment and civil society
organizations is adding dimensions, often positive, to debates
about aid and its delivery.
New instruments, e.g., sector-wide approaches, comprehensive
development frameworks, the livelihoods approach, etc., are
being tried to change the way the aid community works; they
require significant shifts in how evaluation is approached.
Aid delivery has been shifting to new horizons—from the project
to the program to the country levels; there is also an array of
issues, e.g., climate change, trade, migration, etc., that require
regional or global approaches.

4. Agents of Evaluation in ADB
In ADB, self-evaluation is carried out by the
operations departments responsible for designing
and implementing country partnership strategies,
programs, projects, and technical assistance
activities.
Independent evaluation is conducted by the
Independent Evaluation Department to
systematically evaluate policies, strategies,
programs, and projects, e.g., design,
implementation, results, and associated business
processes, to determine relevance, efficiency,
effectiveness, impact, and sustainability.

5. Audiences for Evaluation
Independent Evaluation Department
(evaluations, monitoring and
evaluation expertise, self-assessment)
ADB Operations Departments
(monitoring and evaluation, results
agenda, applied learning)
ADB Management
(use and follow-up of evaluations,
influence on ADB directions)
ADB Board of Directors
(oversight)
International Evaluation Community
(harmonization and joint evaluations)
Developing Member Countries
(use of evaluations, evaluation
capacity development, joint
evaluations)

6. Purpose of Independent
Evaluation
Development Effectiveness—to contribute to
development results, i.e., maximize the development
effectiveness of ADB's operations.
Accountability —to confirm that ADB is doing the right
things, that resources are properly allocated and used,
that intended outcomes are realized.
Generating Knowledge/Learning Lessons—to identify or
derive lessons to improve the development impact of
future policies, strategies, and operations.

7. Evolution of
Independent Evaluation in ADB
1972—A unit is established in
the Economics Office.
1978—A Post-Evaluation Office
is set up.
1999—An Operations Evaluation
Office is formed.
2001—An Operations Evaluation
Department is constituted.
2004—The Evaluation Head
reports to the Board of Directors
of ADB through its Development
Effectiveness Committee.
2009—An Independent
Evaluation Department is
created whose head is to be
appointed by the Board.

8. The Role of the Development
Effectiveness Committee
The Development Effectiveness Committee in ADB was
established in December 2000.
The DEC's mandate is to help the Board of Directors of ADB carry
out its responsibility of ensuring that the programs and activities
of ADB achieve development effectiveness.
Development effectiveness is assessed in part through
independent evaluation.
For the purpose of the DEC's work, development effectiveness is
the measure of (i) whether ADB's programs and activities have
resulted in the desired outcomes; and (ii) whether these
programs and activities have made efficient use of ADB's
available resources.

9. The Role of IED
The core task of IED is
the systematic and
objective assessment of
policies, strategies,
programs, and projects,
including their design,
implementation, results,
and associated business
processes, to determine
their relevance,
effectiveness, efficiency,
sustainability, and
impact.
• It evaluates the development
effectiveness of operations.
• It derives lessons to improve
future operations.
• It contributes to ADB's
accountability for results.
• It provides assistance to the
DEC as the focal point for
interaction with the Board of
Directors.
• It seeks to improve self-
evaluation in ADB.

10. A Few Influential EvaluationsBuildingaResults-Based
ManagementFramework,
2003—
PromotingPortfolio
Performance,2006—
ImprovingCountry
PartnershipStrategies,
2000s—
ContributingtoSafeguard
PolicyUpdate
Involuntary
Resettlement, 2006
Environment, 2006
Indigenous Peoples,
2007

11. Issues in Independent Evaluation
The balance between
accountability and learning,
and between independence
and engagement
Conflict between closer
engagement with developing
member countries and
Management
Independence
from the Board—
is that advisable?
Attribution or
contribution—not all
evaluations can address
attribution issues
How far should
one move on self-
evaluation and
validation?
How might one
improve buy-in?
Conduct fewer individual public sector
project evaluations? Launch more country,
sector, impact, and policy evaluations?
Prepare more private sector evaluations?

12. Issues in Independent Evaluation
How to improve
credibility—
through
stronger
evidence?
How to make
better use of
evaluation
results?
Which evaluation methods to
apply, e.g., qualitative or
quantitative; rigorous or non-
rigorous; experimental or non-
experimental)?
How to assess
impact and
sustainability?
How to strengthen
the accountability
of Management?
Recommendations
must be clear,
actionable, and
monitorable
How about the accountability of IED itself?

13. Knowledge Management for
Independent Evaluation
Efforts to increase external dissemination through
www.adb.org/site/evaluation/main, among other platforms,
are well received.
IED invests continuingly in a searchable lessons database
system that includes well over 1,000 lessons to date.
Efforts are being made to improve dissemination of findings.
A Knowledge Management Unit was set up in IED in late 2006.
To be used, evaluation findings must be available when needed
by decision makers, in user-friendly formats.
Evaluation findings only add value when they are used.

14. Matching Feedback Vehicles to
Accountability and Learning

15. Matching Feedback Vehicles to
Accountability and Learning

16. Matching Feedback Vehicles to
Accountability and Learning

17. Matching Feedback Vehicles to
Accountability and Learning

18. Matching Feedback Vehicles to
Accountability and Learning

19. Matching Feedback Vehicles to
Accountability and Learning

20. Matching Feedback Vehicles to
Accountability and Learning

21. Matching Feedback Vehicles to
Accountability and Learning

22. Matching Feedback Vehicles to
Accountability and Learning

23. Improving Evaluation-Based
Learning
ADB's strategic frameworks are moving toward
greater concentration on country, sector, and
thematic areas, and greater decentralization.
ADB sees itself as a learning organization where
operations evaluation plays a key part, and is given
greater institutional independence and greater priority.
More knowledge is generated
from evaluations using more
sophisticated methods.
More impact-orientation is needed for quality
improvement, conceptual advancement of
development, and greater external accountability.

24. Improving Evaluation-Based
Learning
ADB must systematize and institutionalize
feedback systems for evaluation-based
learning and accountability.
Efforts are needed to manage the quickening
flood of data and information, network the
systems, and cater for upward feedback.
ADB must intensify the
internalization of
lessons from evaluation.
Implementation monitoring, that is, implementation of findings
and recommendations from evaluation, should be strengthened
by means of a Management action record system.

25. Improving Evaluation-Based
Learning
ADB should continue to publicly
release proposed evaluation approach
papers at an early stage and post the
comments of external stakeholders.
ADB should continue to recognize the need for
greater participation of in-country partners and
stakeholders in the feedback process.
Moving from project evaluations to a higher-level country, sector,
and thematic focus is an important way of increasing potential
evaluation impact.

26. Possible Directions in Evaluation
Assigning resources to broader evaluation studies
Harmonizing evaluation standards
(Jointly) Evaluating country/sector assistance programs
Validating country partnership strategy and
project/program completion reports
Evaluating impact
Developing evaluation capacity
Promoting portfolio performance

27. Possible Directions in Evaluation
Evaluating business processes
Disseminating findings and recommendations
Promoting monitoring and measuring of outcomes
Institutional evaluations , e.g., human resource
function, information technology, knowledge
management, Asian Development Bank Institute
Rigorous impact evaluations

28. • ADB. 2007. Independent Evaluation at the Asian Development
Bank. www.adb.org/publications/independent-evaluation-
asian-development-bank
• ——. 2008. Conducting After-Action Reviews and Retrospects.
www.adb.org/publications/conducting-after-action-reviews-
and-retrospects
• ——. 2009. Learning from Evaluation.
www.adb.org/publications/learning-evaluation
• ——. 2009. The Most Significant Change Technique.
www.adb.org/publications/most-significant-change-technique
• ——. 2011. Learning Histories.
www.adb.org/publications/learning-histories
Further Reading

29. Quick Response Codes
@ADB
@ADB Sustainable
Development Timeline
@Academia.edu
@LinkedIn
@ResearchGate
@Scholar
@SlideShare
@Twitter