John Gaventa
World Bank Institute Seminar
November 22, 2010
So what difference does it
make?
Assessing the impact of
parti...
Taking a Citizen – Led Approach:
10 Years of DFID – funded research on
Citizenship, Participation and
Accountability (www....
Active citizens build effective states
- not (only) the other way around
• Much has been learned about citizens view the s...
What difference does citizen engagement
make? The plea for evidence
The idea that good governance cannot be achieved witho...
What difference does citizen engagement
make?
Results from a meta-synthesis of 100 case studies in
20 countries
Drawing fr...
Distribution of positive and negative outcomes
0
10
20
30
40
50
60
70
80
90
Construction of
citizenship
Practices of citiz...
Positive Negative
Increased civic and
political knowledge
Greater sense of
empowerment and
agency
Increased knowledge
depe...
Outcome 2
More Effective Participation
Positive Negative
Increased capacities for
collective action
New forms of participa...
Positive Negative
Greater access to state
services and resources
Greater realisation of
rights
Enhanced state
responsivene...
Positive Negative
Inclusion of new actors and
issues in public spaces
Greater cohesion across
social groups
Reinforcement ...
Types of outcomes and types of engagement
Citizens engage in multiple ways
– Local associations
– Social movements and cam...
Distribution of positive and negative
outcomes across type of citizen engagement
Outcomes sorted by type of citizen engage...
Types of citizen engagement
(n=100)
Positive outcomes
sorted by outcome
categories
Local
association
s
(n=29)
Social
movem...
The relationships of outcomes to context
Grouping of countries across regime (Polity
IV, Freedom House, Economist Intellig...
Distribution of positive and negative outcomes across country types
0
10
20
30
40
50
60
70
80
90
100
Tier 1 Tier 2 Tier 3 ...
Types of citizen
engagement
Positive Outcomes in Tier Three countries
(n=273)
Construction
of citizenship
(n=96)
Practices...
Implications
1. Citizen engagement makes a difference,
but not always. We need to understand
more the factors that lead to...
Key Findings
4. Citizen engagement makes a difference
across regimes, not just in more
democratic settings.
5. The role of...
Part II: What is the impact of
citizen-led accountability
initiatives?
Over a decade of rapid growth and
spread of transpa...
... Particular focus:
Citizen-led, demand-side and ‘social
accountability’ activities and their connection to
state actors...
... Particular focus:
Citizen-led, demand-side and ‘social
accountability’ activities and their connection to
state actors...
Some evidence of impact, but highly uneven….
The positive story - in some conditions TAIs
demonstrably contribute to:
– Gr...
Challenge 1:
Aims, claims, assumptions and expectations
(or, against what are we assessing impact and
effectiveness?)
Aims...
Challenge 2:
Methodological issues: How do we know what
we know?
We found some:
Quantitative surveys
Analysis of aggregate...
Methodological issues (cont.)
Limited evidence, few comparators, difficulty of
valid counterfactuals
Untested assumptions ...
Challenge 3: Factors that make a difference
Little evidence that supports generalisations of
the kind ‘initiative x produc...
Challenge 3 (cont.): Factors
State (supply side)
Level of democratisation
Level of political will
Broader enabling legal
f...
Challenge 3 (cont.): Factors
Thinking beyond the dichotomies (state-society;
voice–response; supply-demand)
New thinking i...
Key Lessons
The evidence base (+/-) is weak - but that doesn’t mean
that TAIs are not significant. The challenge is to dee...
Key lessons (cont.)
On factors for greater impact:
• Deepen understandings of synergies between T, A,
participation and vo...
For links to these studies
www. drc-citizenship.org
www.ids.ac.uk
• Gaventa and Barrett, ‘ So what difference does it
make...
Upcoming SlideShare
Loading in …5
×

IDS John Gaventa at World Bank Institute 2010

1,410 views
1,313 views

Published on

So what difference does it make? Assessing the impact of participation, transparency and accountability
IDS Research Fellow, John Gaventa
World Bank Institute Seminar November 22, 2010

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,410
On SlideShare
0
From Embeds
0
Number of Embeds
179
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

IDS John Gaventa at World Bank Institute 2010

  1. 1. John Gaventa World Bank Institute Seminar November 22, 2010 So what difference does it make? Assessing the impact of participation, transparency and accountability
  2. 2. Taking a Citizen – Led Approach: 10 Years of DFID – funded research on Citizenship, Participation and Accountability (www.drc-citizenship.org). First phase, much attention was on dynamics of state-society relationship Second phase, more on how citizens mobilise and empower themselves, often outside of the state In this presentation draw from this work, but also highlight two recent projects and their implications for voice and accountability – Mapping outcomes of citizen engagement – The impact of transparency and accountability
  3. 3. Active citizens build effective states - not (only) the other way around • Much has been learned about citizens view the state, and about the state-society relationship • Citizens can help to build democratic institutions, legitimacy, responsiveness, capability, accountability • ‘societal opportunities’ create possibilities of political reform • But we need more focus on the society side: • how to ‘build’ active, empowered citizens • how active citizens mobilise to change development policies, build responsive states, and do things for themselves
  4. 4. What difference does citizen engagement make? The plea for evidence The idea that good governance cannot be achieved without the active involvement of citizens and civil society actors has gained growing consensus in recent years. Many donors and NGOs now support "participatory governance", "social accountability" or "demand for good governance" programmes aimed at promoting the active involvement of citizens/CSOs in public decision- making and holding government accountable [...] I'm currently involved in a research project to gather evidence of the results and/or impact of such initiatives. E-mail to author from World Bank consultant, 2009 . ‘Our number one challenge is to demonstrate what difference citizen engagement makes.’ - Representative of large donor agency 2008.
  5. 5. What difference does citizen engagement make? Results from a meta-synthesis of 100 case studies in 20 countries Drawing from widely accepted approaches of systematic review, qualitative case study analysis and synthesis Coding of 800 ‘outcomes’ Evidence points to largely positive contributions With risks of negative outcomes
  6. 6. Distribution of positive and negative outcomes 0 10 20 30 40 50 60 70 80 90 Construction of citizenship Practices of citizen participation Responsive and accountable states Inclusive and cohesive societies Total Percentage Positive Negative
  7. 7. Positive Negative Increased civic and political knowledge Greater sense of empowerment and agency Increased knowledge dependencies Disempowerment and reduced sense of agency Outcome 1 Better Citizens
  8. 8. Outcome 2 More Effective Participation Positive Negative Increased capacities for collective action New forms of participation Deepening of networks and solidarities New capacities used for ‘negative’ purposes Tokenistic or ‘captured’ forms of participation Lack of accountability and representation in networks
  9. 9. Positive Negative Greater access to state services and resources Greater realisation of rights Enhanced state responsiveness and accountability Denial of state services and resources Social, economic and political reprisals Violent or coercive state response Outcome 3 More Responsive and Accountable States
  10. 10. Positive Negative Inclusion of new actors and issues in public spaces Greater cohesion across social groups Reinforcement of social hierarchies and social exclusion Increased horizontal conflict and violence Outcome 4 More inclusive and cohesive societies
  11. 11. Types of outcomes and types of engagement Citizens engage in multiple ways – Local associations – Social movements and campaigns – Formal governance spaces – Multiple forms of engagement Local associations and movements are particularly important for positive outcomes Multiple forms of engagement are particularly important for realising responsive and accountable states Beyond Putnam – not just the density but the nature of the quality and quantity of the association that counts
  12. 12. Distribution of positive and negative outcomes across type of citizen engagement Outcomes sorted by type of citizen engagement (n=828) Outcome type Local associations (n=324) Social movements and campaigns (n=233) Formal participatory governance spaces (n=153) Multiple (n=118) Positive 90% 71% 55% 68% Negative 10% 29% 45% 32% Total 100% 100% 100% 100%
  13. 13. Types of citizen engagement (n=100) Positive outcomes sorted by outcome categories Local association s (n=29) Social movements and campaigns (n=29) Formal participator y governance spaces (n=19) Multiple (n=23) Construction of citizenship 36% 35% 33% 29% Practices of citizen participation 26% 24% 30% 22% Responsive and accountable states 29% 33% 25% 44% Inclusive and cohesive societies 9% 8% 12% 5% Total 100% 100% 100% 100% Distribution of positive outcomes across types of citizen engagement
  14. 14. The relationships of outcomes to context Grouping of countries across regime (Polity IV, Freedom House, Economist Intelligence Unit|) Positive outcomes are not linearly associated with level of democratisation – highest proportion of positive outcomes are found in most and least democratic countries Associations are particularly strong for least democratic settings
  15. 15. Distribution of positive and negative outcomes across country types 0 10 20 30 40 50 60 70 80 90 100 Tier 1 Tier 2 Tier 3 Total Percentage Positive Negative
  16. 16. Types of citizen engagement Positive Outcomes in Tier Three countries (n=273) Construction of citizenship (n=96) Practices of citizen participatio n (n=66) Responsiv e and accountabl e states (n=83) Inclusive and cohesive societies (n=28) Local associations 89% 92% 83% 78% Social movements and campaigns 0% 2% 2% 0% Formal participatory governance spaces 6% 0% 4% 11% Multiple 5% 6% 11% 11%
  17. 17. Implications 1. Citizen engagement makes a difference, but not always. We need to understand more the factors that lead to positive vs negative change 2. These gains do not emerge automatically: pay more attention must be paid to building citizenship e.g. of empowerment, as a component of other action 3. Only through empowered citizens will accountability relationships likely occur.
  18. 18. Key Findings 4. Citizen engagement makes a difference across regimes, not just in more democratic settings. 5. The role of associations in fragile settings is particularly important for building cultures of citizenship 6. Citizen engagement is often met by reprisals: it is critical to protect the democratic space for engagement if developmental outcomes are to be achieved.
  19. 19. Part II: What is the impact of citizen-led accountability initiatives? Over a decade of rapid growth and spread of transparency and accountability work in development and aid circles and development academia Apparent promise of T&A as the cures for many ‘evils’ As yet, little clarity about what is being achieved, what works, how it works, and how best to fulfil that promise…
  20. 20. ... Particular focus: Citizen-led, demand-side and ‘social accountability’ activities and their connection to state actors, institutions and processes Connections between T, A and participation Methodological challenges of impact assessment in the T&A field ‘effectiveness’ Vs ‘impact’ Service delivery, budget processes, FoI, natural resource governance, aid
  21. 21. ... Particular focus: Citizen-led, demand-side and ‘social accountability’ activities and their connection to state actors, institutions and processes Connections between T, A and participation Methodological challenges of impact assessment in the T&A field ‘effectiveness’ Vs ‘impact’ Service delivery, budget processes, FoI, natural resource governance, aid
  22. 22. Some evidence of impact, but highly uneven…. The positive story - in some conditions TAIs demonstrably contribute to: – Greater state responsiveness – Lower corruption – Building spaces for engagement and empowering local voices – Better budget utilization and delivery of services But not always: – Evidence is uneven and scattered – Initiatives are new and impacts unknown – Much focus on effectiveness rather than impact – Positive evidence in one case not corroborated by studies in another How do we enhance demonstrable impact?
  23. 23. Challenge 1: Aims, claims, assumptions and expectations (or, against what are we assessing impact and effectiveness?) Aims vary: Developmental outcomes, democratic outcomes, voice  empowerment outcomes Assumptions vary: eg on links between T, A and participation; about ‘citizens’; hierarchies of objectives; how explicit/implicit; etc The need for sharper theories of change
  24. 24. Challenge 2: Methodological issues: How do we know what we know? We found some: Quantitative surveys Analysis of aggregated survey data, multivariate analysis Experimental approaches (RCTs) Qualitative case studies and case study analysis Stakeholder interviews Indices and rankings We found a lack of: Comparative studies Ex-post long-term evaluations Appropriate use of baselines Rigorous participatory approaches Complexity-aware approaches eg Most Significant Change; Outcome Mapping; narrative-based Methodological mixes
  25. 25. Methodological issues (cont.) Limited evidence, few comparators, difficulty of valid counterfactuals Untested assumptions and poorly articulated theories of change: blurry goalposts Correlation Vs causality; attribution Vs contribution Indicators – what we want to measure Vs what we can realistically measure Whose perspectives, which impacts count? Upward or downward accountability in the impact assessment process itself Complexity, contingency, uncontrollability
  26. 26. Challenge 3: Factors that make a difference Little evidence that supports generalisations of the kind ‘initiative x produces outcome y’.... A more useful question to ask: Which factors (enabling and disabling) shape the possibility that TAIs will achieve their stated goals in a particular context?
  27. 27. Challenge 3 (cont.): Factors State (supply side) Level of democratisation Level of political will Broader enabling legal frameworks, incentives and sanctions Citizen voice (demand side) Capabilities of citizens and civil society Interaction of TAIs with other mobilisation and collective action Embeddedness of TAIs in broader policy processes Linking Mechanisms
  28. 28. Challenge 3 (cont.): Factors Thinking beyond the dichotomies (state-society; voice–response; supply-demand) New thinking in governance would urge us to pay more attention to: Multiple actors, accountability coalitions, networked approaches to governance Changing norms and cultures of accountability in state, private sector and civil society simultaneously Looking across levels and scales: Linking the local, national, regional, international Bringing politics back in: – Power – The black box of political will and political economy – Links to parties, elections and political regimes
  29. 29. Key Lessons The evidence base (+/-) is weak - but that doesn’t mean that TAIs are not significant. The challenge is to deepen the evidence and the methods for developing it. On the state of the evidence: • Develop new approaches to assessment, with complexity perspective, that combine methods and approaches • Explore further user-centred and participatory approaches • Support comparative in-depth work across contexts and TAIs, multi-case and other synthetic analysis • Strengthen capacities of researchers and practitioners to develop and build on innovative approaches • Build into new TAIs ToCs, baselines, comparators, etc
  30. 30. Key lessons (cont.) On factors for greater impact: • Deepen understandings of synergies between T, A, participation and voice • Move beyond dichotomies to build new knowledge on cross-cutting accountability coalitions • Apply to T&A field the cutting-edge thinking on governance, especially networked governance; interaction of global – national – local; and private sector • Explore whether initiatives can travel across context, method and issue
  31. 31. For links to these studies www. drc-citizenship.org www.ids.ac.uk • Gaventa and Barrett, ‘ So what difference does it make? Mapping the outcomes of Citizen Engagement, ’ IDS Working paper 247 • Review of Impact and Effectiveness of Transparency and Accountability Initiatives, Rosemary McGee & John Gaventa with contributions from Greg Barrett, Richard Calland, Ruth Carlitz, Anuradha Joshi and Andrés Mejía Acosta, IDS, October 2010

×