The document discusses the challenges of evaluating blended finance, including management and organizational diversity, methodological difficulties, and the need for stronger coordination in monitoring and evaluation practices. It highlights the necessity of definitions for additionality, as well as the varying approaches to evaluation among different financial institutions. The authors recommend enhancing access to data, adopting new methodologies, and fostering collaboration between stakeholders to improve the effectiveness of blended finance evaluations.

1. BLENDED FINANCE EVALUATION
Irene Basile, OECD
Magdalena Orth, DEval
Preview of the forthcoming OECD working paper
Governance and Methodological
Challenges

2. Introduction
Structure:
• Management and organizational challenges
• Evaluation, impact and additionality
• Review of a sample of evaluations
• Conclusions and potential next steps
Authors: Ole Winckler Andersen, (DIIS), Irene Basile (OECD),
Antonie de Kemp (IOB), Gunnar Gotz (Deval), Erik Lundsgaarde
(DIIS), Magdalena Orth (Deval).

3. Background and purpose
• Two observations in OECD (2018):
”.. the evidence based on blended finance is still quite limited”
”.. monitoring and evaluation of blended finance funds and facilities
are less developed than for other development co-operation
activities”
• Objective:
– Contribute to ongoing consultation process on the OECD
Blended Finance Principle 5
– Foster dialogue between evaluation and development finance
communities

4. MANAGEMENT AND
ORGANIZATIONAL
CHALLENGES

5. Institutional and organizational factors
influence evaluation practice
• Organizational diversity
• Complex governance patterns affect monitoring and
evaluation
• The lack of harmonisation in monitoring tools
hampers comparability
→ significant variety in evaluation practice,
including access to financial resources,
evaluation capacity and degree of
independence.

6. DFI
Development
Bank or Export
Credit Agency
Ministry in charge of development cooperation and/or finance
MDBs
Client
End beneficiaries
Other governments
Client Client Client
Aid Agency
Client ClientClient
Complex governance patterns affect
monitoring and evaluation

7. EVALUATION, IMPACT
AND ADDITIONALITY

8. Objectives and Evaluation Design
Different evaluation objectives
• Assess the developmental impact with the goal of
a) improving the impact of the interventions or
b) guiding future investement strategies
• Consider whether selected instruments are adequate in given context
Evaluation Design
• Should be tailored to blended finance instruments (debt, equity or technical
assistance)
• e.g. adjustment of evaluation scope and judgment criteria

9. Methodological approaches
Funds typically not allocated randomly
Challenge: Find a valid control group
Experimental
designs:
often not feasible
Quasi-experimental
designs:
can offer ways to
artificially construct
a control group
Integrated approaches help to reflect this complexity
Challenge: complexity of blended finance initiatives
Causalanalysis
Theory-
focused
approach
Cross-case causal
inference:
Comparative case
Studies, Systematic
Reviews
Within-case causal
inference:
Contribution
Analysis, Process
Tracing
Comparison, relationships
Valuation,
weighting
Consistency,
coherence
Non-causal
analysis
Statistical modelling: input-output modelling

10. Methodological approaches
Funds often not allocated randomly
Challenge: Find a valid control group
Experimental designs
and randomised
control trials: often
not feasible
Quasi-experimental
designs:
can offer ways to
artificially construct
a control group
Theory-based evaluation, integrated approaches
Issue: complexity in blended finance initiatives
CausalAnalysis
Theory-
focused
approach
Cross-case causal
inference:
Comparative Case
Studies, Systematic
Reviews
Within-case causal
inference:
Contribution
Analysis, Process
Tracing
Comparison, relationships
Valuation,
weighting
Consistency,
coherence
Non-causal
analysis
e.g. in-depth impact analysis
on the development of one
market
e.g. assess and compare similar
projects in different markets or
assess the effectiveness
e.g. verify the alignment of
blended finance with
(inter)national development
strategies
e.g. estimate the impact of
investments on the entire
economy
Statistical modelling: input-output modelling
Different evaluation objectives
make different approaches more suitable

11. Additionality in blended finance
evaluation
OECD definition requires blended finance to
- mobilize additional finance and
- use it for sustainable development
Finance is mobilised and invested
into activities that would not have
materialized otherwise
Outcome and impact of investment
that goes beyond what would have
been achieved in absence of
additional finance
Financial additionality Development additionality
OECD, 2018
in praxis, concepts closely related

12. • Assessment whether
and to which degree
additional finance
was mobilised
• Showing causality
remains very
challenging
Situation
• Mobilisation often
assumed rather than
observed
In practice
• Using mixed-
methods, or pragmatic
approaches
• Requires a common
definition of
additionality
• More comprehensive
framework for
comparing projects
Recommendation
Key challenges in measuring financial
additionality

13. • Measuring
development
additionality
comparable to impact
evaluations
• No aggregate
monitoring and
reporting on
additionality
Situation
• Simple before and
after comparisons
do not allow for
attribution
In practice
• Analysis at project level
and system level
needed
Recommendation
Key challenges in measuring
development additionality

14. • Comparison of
alternatives to assess
efficiency
• combination of risk and
returns competitive to
alternative investments:
no case for blending
Situation
• Difficult to use higher
leverage rates:
blending might have
made investments too
attractive and crowded
out private investment
In practice • Harmonised
approach applied by
all DFIs
• Comprehensive
framework in
assessing efficiency to
compare projects
Recommendation
Key challenges in measuring efficiency

15. REVIEW OF A SAMPLE OF
EVALUATIONS

16. Approach of the review
• Purpose:
• review applied evaluation approaches and identify
challenges
• not to asses the quality of individual evaluations or the
potential effects of blended finance interventions
• Purposive sample of completed evaluations and
studies (27)
• Review method: team identified tendencies across a
number of evaluations

17. Purpose and scope of the evaluations
Scope
No common
terminology
on additionality
Internal
accountability
rather than
learning
Purpose
DAC criteria
not
systematically
applied
Indirect
effects
and time
horizon rarely
considered
Relevance
assessed
differentlyt

18. Approach and issues of the evaluations
Theory-based
without explicit
assumptions
TOCs
often re-
constructed
Approach
Acces to
information
an issue due to
policies on
disclosure of info
“Before-and-
after“
assessments
Issues

19. Methodology of the evaluations
Contribution
analysis
• Stakeholder
Interviews
• Portfolio analysis
• Document and
literature reviews
Quantitative
Methods
to asses
outcome and
impact
Methodology
Mixed-
methods
with different
applications
Case studies
without details
on analytical
approach
Limitations
only a few
reflections

20. CONCLUSIONS AND
POTENTIAL NEXT STEPS

21. • In order to establish an evidence base, monitoring and evaluation of
blended finance must be strengthened, but also better coordinated.
• Challenges specific to the evaluation of blended finance:
– various management systems,
– wide range of instruments and contexts,
– assessment of financial and development additionality.
• Other evaluation methodologies than those usually applied may be
relevant.
• A common language is emerging, but could benefit from the existing
standards on development evaluation and more interaction with the
evaluation profession.
• Learning will require openess and access to information.
Closing remarks

22. • Adapt or integrate DAC evaluation glossary and
guidelines for development finance
• Improve availability and access to comparable
monitoring data
• Ensure that evaluation reports from DFIs and private
intermediaries are publicly available (e.g. DEREC)
• Assess the relevance of individual evaluation methods
for specific blended finance instruments
• Systematic review of existing evidence on achieved
results by various blended finance instruments
Potential next steps

23. THANK YOU

24. BACK-UP SLIDES

25. Example Theory of Change
Impact
Outcome
Input
Output
Population groups gain
access to the financial
sector or have
additional funding at
their disposal
New products for the
target group or financial
sector development
Loans and capacity
development
Loans granted to MSMEs
Increased employment
Economic growth

26. • The focus of the DFI blended finance guidelines is on blending concessional
funds with non-concessional resources to support private sector operations.
• The OECD approach focuses on combining financing sources that have a
developmental mandate with those that have a commercial mandate, while
the investment could be either private or public.
• Ex. for DFI=additional, OECD=not additional (because mandate is not
purely commercial)
– KfW finances a private sector project with a concessional credit (BMZ funded) and a
promotional loan (KfW own resources)
• Ex. for DFI=not additional, OECD=additional (because public investment)
– KfW finances a private sector project with a concessional credit (BMZ funded) and a
promotional loan (KfW own resources)
Different definitions: Blended Finance

27. • Evaluation Division of the EIB considers an intervention to be additional if
it facilitates or enhances a project from the public welfare (development)
perspective in a way that the market alone would not allow, or at least not to
the same extent, or in the same timeframe
• The EBRD defines additionality as an input that the Bank contributes to
each project, which is expected to foster development impact through its
influence on project design and implementation.
Different definitions: Additionality