This document discusses monitoring and evaluation (M&E) for development projects in peace precarious situations. It begins with an agenda that includes strategic contingency approaches, case studies, and lessons learned. Normal development is defined in contrast to peace precarious development, which occurs in contexts of ongoing power disputes and instability. Key aspects for M&E in these situations include volatility, variability of parameters, and complexity of stakeholders. A workshop discussion focuses on specific cases in Afghanistan and Guinea and challenges of developing theories of change, measurement, and information use in volatile contexts. The document concludes by discussing gaps in understanding these contexts and strategies for designing useful M&E to manage expectations and validate results given limitations.

1. M&E and evaluation
for development
in peace precarious situations
Catherine Elkins, PhD, MALD
Belling the Cat LLC–Duke University
EES Workshop 01 October 2012

2. Assisting development: Is our territory cognita?
2

3. Our day’s agenda
• Sessions and schedule in handouts
– Strategic contingency approaches
– Case studies over time
– Six-dimensional analytical model
– Lessons for peace precarious situations and lessons
from peace precarious situations

4. What is ‘normal’ development?
• “Think–pair–share”
• Define
– elements of typical development context
– typical development goals
– typical development stakeholders
– typical development implementers / partners

5. Peace Precarious Situations
photo © Joe Holmes 5

6. What is ‘peace precarious’ development?
• Differences we experience in
– context
– goals
– stakeholders
– implementers / partners

7. Peace precarious situations
• May overlap circumstances labeled in
other discussions as fragile, low-level
violence, conflict, post-conflict, failed
states
7

8. Peace precarious situations
• Power disputes
• recent, ongoing, or chronic
• violent at least sometimes
• Destabilized but not obliterated core
institutions
• No directional assumptions
• No ‘state’ assumptions
8

9. Introductions: Catherine Elkins
• http://linkedin.com/in/seaelkins
• Professor, Duke University; Consultant
• About 20 years’ experience working in
developing countries
• Afghanistan, Cambodia, Eritrea, Guinea,
Guyana, Iraq, Kazakhstan, Lebanon,
Mauritania, Pakistan, Rwanda, South Africa,
Uganda, West Bank, Zimbabwe
• Belize, Jordan, Senegal, Tanzania, Thailand

10. Introductions: Colleagues
• Name, main occupation
• Evaluation (or M&E) experience
• Development experience
• Peace-precarious experience

11. Introductions: Cases 1, 2
11

12. Introductions: Case 3

13. Development in peace precarious situations
• Extreme cases: International
development efforts ongoing despite
widespread internal or regional areas
characterized by a “mid-range of
violence”
• “Instability still precludes heavy NGO
involvement, but […] is not so acute that
combat operations predominate”
USAID 2006, Provincial Reconstruction Teams in Afghanistan: an Interagency Assessment (PN-ADG-252) . 13

14. Critical dimensions
14

15. Peace precarious situations
Key aspects for M&E/Evaluation
• Volatility
• Internal and external ‘constants’ are
potentially/probably variable
• And especially: Multiple parameters can
change at once

16. Activity
• Afghanistan – external evaluation
– What do we know about the 6 dimensions?
– What else do we need to know?
– Ideas for evaluation plan
• In principle
• 20 questions

17. Focus on stakeholders
17

18. Stakeholder complexity & sensitivities
Sources for volatility
• Who has a stake in project failure?
• Who has mixed interests, hedged bets?
• What additional political or other concerns do
those supporting success need to consider?
18

19. Lunch
19

20. Theories of change
• Hypothesized relationships among influential
factors shaping conditions or behavior of
interest
• Probabilistic causal expectations
• Observable, measurable, falsifiable model
• Mechanisms expected to produce results
20

21. Implementation – or so we like to think
21

22. Theories of change and
peace precarious development
• “Normal” development interventions
– Complicated
– Complex
• “Peace precarious” interventions
– Complicated, complex, and volatile
– Relationships between good program design &
good M&E/evaluation design in peace-precarious
situations
22

23. Peace precarious stakeholders and planning
• From initial exercise
• From Afghan case exercise
• From own experiences
• Relationship dynamics and contingencies

24. Activity
• Guinea governance M&E
– Dynamics over time
Legislative
elections
Coup Pres
Project Coup attempt election fall ’11**
spr’08 Dec’08 Dec’09 June’10 (not yet!)
Army Massacre 2 intra- Run-off
mutiny Sep’09 coup results
sum‘08 transitions final
Jan-Feb’10 Nov’10
– http://www.reuters.com/article/2010/06/27/us-guinea-events-
idUSTRE65Q0ZP20100627

25. Activity (cont’d)
• Guinea
– Initial design
– Big disruption
– Ongoing disruptions

26. M&E strategy
• Guinea governance M&E
– Model and measurement
– Elasticities

27. Afghanistan and Guinea
• M&E and evaluation dynamics
– Same or different challenges?
– Documentation

28. Model (theory of change) and measurement
‘Normal’ vs peace precarious challenges
• If–then hypotheses of predicted
relationships
– Falsifiable using empirical evidence
– Discriminating paths among multiple influences
– Quasi-experimental counterfactual; may be very
loose “other things being equal”
• Baseline, implementation, results
28

29. Fidelity and slippage
• Contextual constants and variables
• Cost and implementation elasticities
• Articulation of expectations, execution, M&E,
and evaluation including changes from design
through close-out
• Learning systems
29

30. Egypt, girls’ education, and “revolution”?
globalgiving.org

31. Recap: Exploring the cases
Punctuated equilibria
• Full or partial disruptions, interruptions,
unpredictable timing
• Expansions and curtailments of components,
flexible scope
• Nature and timing of specific shocks or factor
changes (context, institutions, stakeholder/
counterpart composition)
31

32. Exploring the cases – coping under stress
Cost factors
• Fluctuation in budget
• Fluctuation in security expenses
• Staffing through interruptions, evacuations
• Recruiting, screening, incentives, and turnover
32

33. Measurement risk factors
Data quality and precision
• Human capital and stress effects on timeliness,
completeness, accuracy
• Perceived risk and risk attitudes
– Risks to participants, beneficiaries,
enumerators
– Transparency and accountability risks
• Existence, accessibility, reliability of secondary
data sources
33

34. Information uses
Lower confidence in data and…
• Potential for political or other unintended uses of
project results and evaluation findings?
• Likelihood that certain actors and audiences will
misunderstand or misuse findings?
• Potential damage to any stakeholders from use or
misuse of results or findings?
34

35. Information and stakeholders in context
• International context and global dynamics
– Goals, uses, audiences, information/security
dimensions affecting development results
information
– Six dimensions across our cases
– Appropriate, useful M&E/evaluation?
• Possible?
• Likely?
• Under what conditions?
35

36. Cross-cutting conclusions and challenges
36

37. Gaps analysis and theory building
• What are we missing?
• What else can we bring to bear?
37

38. Strategy
Where program and evaluation concerns meet
• Design evaluation around challenges to project
and ways challenges are most likely to affect
evaluation
• Identify specific decision points
– More than one theory of change?
– Client understanding or learning curve?
– Variance across stakeholder interests?
– Data needs for methodology options?
– Budget requirements for strategic options?
38

39. Design for development
handout 39

40. Useful M&E/evaluation (peace precarious)
• Questions:
– Adequacy of development models for (some)
peace-precarious situations
– Adequacy of M&E/evaluation approaches for
(some) peace-precarious situations
– Managing expectations and validating deliverables
– Ethics and other issues
40

41. Final Questions?
Thanks!
41