Advantages of Hiring UIUX Design Service Providers for Your Business
CDI Seminar: 3d impact analysis
1. 3D impact analysis
A new tool to approach impact evaluations
April 23, 2015
CDI is a joint initiative between:
1
and and
For more information: www.ids.ac/cdi or email: cdi@ids.ac.uk
Rob D. van den Berg
Visiting Fellow, IDS
2. Overview
• What is impact?
• What is evidence?
• What is causality?
• What is attribution/contribution?
• Time
• Space
• Scale
2
3. Impact
• Impact is an ordinary word in the English
language
– “the effective action of one thing or person on
another; the effect of such action; influence;
impression”
• Its meaning cannot be scientifically claimed
• Demand for impact evidence can refer to a
wide variety of effects, influences and
impressions
3
4. Evidence
• Evidence is an ordinary word in the English
language
– “the quality or condition of being evident;
clearness; evidentness”
• Its meaning cannot be scientifically claimed
• Demand for impact evidence can refer to a
wide variety of qualities or conditions
4
5. Causality
• The word “cause” is an ordinary word in the
English language
– “A person or thing that gives rise to an action,
phenomenon, or condition”
• Its meaning cannot be scientifically claimed
• Demand for evidence of cause and effect can
refer to a wide variety of actions, phenomena
and conditions
5
6. Attribution / Contribution
• Both words are ordinary words in the English
language, with great variety in meaning
– Attribution: in copyright law, requiring an author
to be credited; in marketing, assigning a value to a
marketing activity based on desired outcome;
journalism, practice of attributing information to
its source
– Contribution: donation, sharing, payment,
publication, a song by Mica Paris
6
7. Impact Evaluation
• Focus of Impact Evaluations:
– Impact = evidence of causality between an
intervention and the desired effect by establishing
a counterfactual through controlled
experimentation, which attributes part of the
effect to the intervention
• This partially meets the demand for impact
evidence in politics, the media and society
• So what to do with other demands?
7
8. Meeting impact demand
• Broaden the concepts of impact and causality
• Broaden the range of scientific methods and
tools
• Develop a framework for understanding
demand for impact evidence
• Incorporate issues of time, space and scale
• This is urgent, given the adoption of the
Sustainable Development Goals in September
2015
8
9. Understanding causality
• Schaffer (2013) proposes two kinds of causality:
“difference” and “production”
– Difference: with/without (counterfactual) analysis
– Production: A “produces” B (natural systems, physics &
technology)
• Concepts that include causality:
– Catalytic roles (the change agent speeds up change but is
not involved in the change itself)
– Dynamic and chaotic systems (feedback loops, iterative
processes, Fibonacci sequences)
• Terry Pratchett: “hardly anything important has a single
cause”
9
10. Issues of time, space and scale
• Some changes can be observed immediately – others take
decades
– Short-term results: vaccinations, technology transfer, new
livelihood approaches etc.
– Either short- or long-term: market transformations, societal
change
– Long-term results: health trends, ecosystem services, ozone
layer
• Some changes are local, other regional, national or even
global
• Some changes concern one actor, intervention or
institution, others involve multiple actors or institutions,
and multiple sectors
• Sustainable development involves longer time horizons,
overlapping locations and many scales
11. Matrix of evaluable impact
• Impact can be evaluated at different moments in time: ex
ante, in real time and ex post
– These can be refined: ex ante tends to be done once, but real
time and ex post have many possibilities
– Different processes tend to have different time horizons
• Geographical space runs from local to global
– These can also be refined: the boundaries of societies,
economies and natural systems are different from each other
and may overlap
• Scales of involvement can go from one actor to a
multiplicity, from one market to a full economic system,
from one governance level (or sector) to many
– Actors, markets and governance may not fully overlap
14. Matrix dimensions space and time
14
Ex ante Inception Mid-term End of
project
Ex post <
2 years
Ex post 5-
8 years
Local
National
Regional
Global
Ecosystem
(overlap with
other rows)
Experimentation
Mixed methods /
theory of change
approaches
Monitoring and data
analysis (including “big
data”)
15. Matrix dimensions space and scale
15
One inter-
vention
Multiple
inter-
ventions
Enabling
environ-
ment
Market
change
Market
transform
-ation
Climate
change
Local
National
Regional
Global
Ecosystem
(overlap with
other rows)
RCTs
Data
analysis
Monitoring
Mixed methods / theory of change
Double evaluand evaluations
16. Matrix dimensions scale and time
1616
One inter-
vention
Multiple
inter-
ventions
Enabling
environ-
ment
Market
change
Market
transform
-ation
Climate
change
Ex-ante
Inception
Real-time
End-of-
project
Ex-post
RCTs and
quasi-
experimental
Data
analysis
Monitoring
Mixed methods / theory of change
