This was presented by Guy Grossman from the University of Pennsylvania at the Impacts of Civic Technology Conference (TICTeC 2018) in Lisbon on 19th April 2018. You can find out more information about the conference here: http://tictec.mysociety.org/2018

1. Crowdsourcing Accountability: ICT for Service Delivery
Guy Grossman1
Melina Platas Izama2
Jonathan Rodden3
1
UPenn 2
NYU Abu Dhabi 3
Stanford
April 19, 2018
TICTeC-2018 (Lisbon)
Grossman (Penn) ICT for Service Delivery April 19, 2018 1 / 26

2. Overview
Motivation: mobile revolution
0
20
40
60
80
(per100people)
1985 1990 1995 2000 2005 2010 2015
sub-Saharan Africa Uganda
Mobile cellular subscriptions
Figure: Source: The World Bank; International Telecommunication Union, World
Telecommunication/ICT Development Report and database.
Grossman (Penn) ICT for Service Delivery April 19, 2018 2 / 26

3. Overview
Research question
Can innovations in Information Communication Technologies (ICTs) be
used to improve service delivery outcomes in low-income countries?
Grossman (Penn) ICT for Service Delivery April 19, 2018 3 / 26

4. Overview
Preview
We study the effect of a community reporting (ICT) platform on local public
goods provision (education, health and water)
Grossman (Penn) ICT for Service Delivery April 19, 2018 4 / 26

5. Overview
Preview
We study the effect of a community reporting (ICT) platform on local public
goods provision (education, health and water)
The platform enabled villagers to report service delivery problems via
text-messages to district government officials
Grossman (Penn) ICT for Service Delivery April 19, 2018 4 / 26

6. Overview
Preview
We study the effect of a community reporting (ICT) platform on local public
goods provision (education, health and water)
The platform enabled villagers to report service delivery problems via
text-messages to district government officials
The platform was introduced to over 100 Ugandan villages using a field
experimental research design
Grossman (Penn) ICT for Service Delivery April 19, 2018 4 / 26

7. Overview
Preview
We study the effect of a community reporting (ICT) platform on local public
goods provision (education, health and water)
The platform enabled villagers to report service delivery problems via
text-messages to district government officials
The platform was introduced to over 100 Ugandan villages using a field
experimental research design
Though the program registered relatively high take-up, we find little
evidence that it had lasting effects on service delivery outcomes
Grossman (Penn) ICT for Service Delivery April 19, 2018 4 / 26

8. Overview
Status quo
Starting point: persistent abysmal public service delivery
Grossman (Penn) ICT for Service Delivery April 19, 2018 5 / 26

9. Overview
Status quo
Starting point: persistent abysmal public service delivery
high absenteeism rates
low effort levels
abuse is rampant (health)
Grossman (Penn) ICT for Service Delivery April 19, 2018 5 / 26

10. Overview
Status quo
Starting point: persistent abysmal public service delivery
high absenteeism rates
low effort levels
abuse is rampant (health)
Government has means and mandate to discipline service providers. But. . .
Grossman (Penn) ICT for Service Delivery April 19, 2018 5 / 26

11. Overview
Status quo
Starting point: persistent abysmal public service delivery
high absenteeism rates
low effort levels
abuse is rampant (health)
Government has means and mandate to discipline service providers. But. . .
Public officials lack reliable, targeted information about ‘shirking’
Grossman (Penn) ICT for Service Delivery April 19, 2018 5 / 26

12. Overview
Status quo
Starting point: persistent abysmal public service delivery
high absenteeism rates
low effort levels
abuse is rampant (health)
Government has means and mandate to discipline service providers. But. . .
Public officials lack reliable, targeted information about ‘shirking’
Citizens are reluctant to complain against service providers
Grossman (Penn) ICT for Service Delivery April 19, 2018 5 / 26

13. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

14. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free):
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

15. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

16. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
2 Anonymous:
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

17. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
2 Anonymous: reduce social costs; ‘cut-out’ middle-man
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

18. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
2 Anonymous: reduce social costs; ‘cut-out’ middle-man
3 Immediate and dynamic:
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

19. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
2 Anonymous: reduce social costs; ‘cut-out’ middle-man
3 Immediate and dynamic: government can signal responsiveness
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

20. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
2 Anonymous: reduce social costs; ‘cut-out’ middle-man
3 Immediate and dynamic: government can signal responsiveness
4 ‘Comparative advantage’:
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

21. Overview
ICT and community monitoring
ICT “complaint” platforms may help address these problems:
1 Low cost (free): reduce the monetary costs associated with reporting
2 Anonymous: reduce social costs; ‘cut-out’ middle-man
3 Immediate and dynamic: government can signal responsiveness
4 ‘Comparative advantage’:
citizens are more informed
public officials are better positioned to enforce service delivery standards
Grossman (Penn) ICT for Service Delivery April 19, 2018 6 / 26

22. Overview
Hypotheses
Access to an ICT platforms for political communication will increase:
H1 monitoring of health and public education facilities
H2 effort of frontline staff in public health and education facilities
H3 inputs availability in health, education, and water
Grossman (Penn) ICT for Service Delivery April 19, 2018 7 / 26

23. Research Design
Research Design
Grossman (Penn) ICT for Service Delivery April 19, 2018 8 / 26

24. Research Design
Study area: Arua district (Uganda)
Districts: highest tier of subnational government, responsible for administering
local public services (e.g. health, education, water)
Grossman (Penn) ICT for Service Delivery April 19, 2018 9 / 26

25. Research Design
U-Bridge Platform
Collaborated wt Governance, Accountability, and Participation Program
Project launched in August 2014 and ended in June 2016
Introduced in over 100 villages in Arua district using random assignment
Treatment: encouragement to use a service ‘open to all’
Unit of randomization: village clustered around Arua’s 48 health centers
Grossman (Penn) ICT for Service Delivery April 19, 2018 10 / 26

26. Research Design
Design: treatment (yellow); control (blue)
Grossman (Penn) ICT for Service Delivery April 19, 2018 11 / 26

27. Research Design
U-Bridge implementation
Training of district officials: login dashboard, message response
Inception community meetings in 24 treatment clusters
Quarterly community dialogue meetings in treatment clusters
Registration exercise in 91 treatment villages
Periodic SMS polls with close to 4000 registered users (companion paper)
Grossman (Penn) ICT for Service Delivery April 19, 2018 12 / 26

28. Research Design
Inception meeting
Grossman (Penn) ICT for Service Delivery April 19, 2018 13 / 26

29. Research Design
U-Bridge messages take-up
Monthly
Cumulative
Aug. 14 Oct. 14 Dec. 14 Feb. 15 Apr. 15 Jun. 15 Aug. 15 Oct. 15
0
1000
2000
3000
50
100
150
200
Numberofmessages
Type of message Actionable Relevant
Grossman (Penn) ICT for Service Delivery April 19, 2018 14 / 26

30. Research Design
Example messages
Not relevant:
“Hi ubridge”
“We are for election”
Relevant:
“I greet you all, but our major problem is sickness”
“The tobbacco farmers are misserable how can Ubridge help them?”
Actionable:
“The Only Borehole in Ogboa Village is broken”
“NURSES DONT ATTEND PATIENTS DURING SAT AND sun in Opia HC”
Grossman (Penn) ICT for Service Delivery April 19, 2018 15 / 26

31. Research Design
Sectors
1 Government health facilities (HCII and HCIII)
2 Government primary schools
3 Water services
Grossman (Penn) ICT for Service Delivery April 19, 2018 16 / 26

32. Research Design
Main evaluation outcomes
1 Monitoring (by district)
Calls, visits from district officials
Inspector reports
2 Effort (by facilities)
Absenteeism
Engagement (outreach, classroom teaching)
3 Inputs (in facilities)
Drug stock-outs
Supplies
Grossman (Penn) ICT for Service Delivery April 19, 2018 17 / 26

33. Research Design
Data sources
1 Unannounced audits in schools and health centers
2 Administrative data collection
3 Survey of 16 treatment villages, 4,259 respondents
4 Focus group discussions in 8 villages
Grossman (Penn) ICT for Service Delivery April 19, 2018 18 / 26

34. Research Design
Summary of core findings
1 We find no evidence of treatment effects in health services
Grossman (Penn) ICT for Service Delivery April 19, 2018 19 / 26

35. Research Design
Summary of core findings
1 We find no evidence of treatment effects in health services
2 We find suggestive evidence of short-term effects in education and water
Grossman (Penn) ICT for Service Delivery April 19, 2018 19 / 26

36. Research Design
Summary of core findings
1 We find no evidence of treatment effects in health services
2 We find suggestive evidence of short-term effects in education and water
3 Effects in education likely did not last beyond year-1
Grossman (Penn) ICT for Service Delivery April 19, 2018 19 / 26

37. Research Design
Summary of core findings
1 We find no evidence of treatment effects in health services
2 We find suggestive evidence of short-term effects in education and water
3 Effects in education likely did not last beyond year-1
4 Anecdotal evidence of fixing specific problems
problems addressed via u-bridge may have been addressed anyway by
alternative means
Grossman (Penn) ICT for Service Delivery April 19, 2018 19 / 26

38. Research Design
Summary of core findings
1 We find no evidence of treatment effects in health services
2 We find suggestive evidence of short-term effects in education and water
3 Effects in education likely did not last beyond year-1
4 Anecdotal evidence of fixing specific problems
problems addressed via u-bridge may have been addressed anyway by
alternative means
5 u-bridge clearly did not produce wholesale (equilibrium) change
Grossman (Penn) ICT for Service Delivery April 19, 2018 19 / 26

39. Research Design
Possible reasons for null effect on health service
1 People might care more about education and water than health
2 Harder for citizens to identify problems in the health sector
3 Harder for the district to address problems in health
4 Health systems are already “technologically wired” (e.g., mTrac)
5 Health workers have greater job security
6 Small-n problem (n = 1 districts)
Grossman (Penn) ICT for Service Delivery April 19, 2018 20 / 26

40. Research Design
Possible reasons for why education effects did not persist
1 Novelty effects
2 Mobilization efforts (e.g., reminders) more intense in first year
3 Citizens are disappointed and begin to disengage
On one hand – 84% response rate ; on the other hand. . .
Citizens’ messages were too vague to address
Citizens’ messages could not be addressed within the budgetary cycle
Citizens were referred to lower level government or local institutions
Among users level of satisfaction is not high (39% satisfied; 35% not)
Grossman (Penn) ICT for Service Delivery April 19, 2018 21 / 26

41. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

42. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

43. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
responsiveness: citizens believe they can prompt public officials to act
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

44. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
responsiveness: citizens believe they can prompt public officials to act
55% of respondents report the district is responsive to citizens requests
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

45. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
responsiveness: citizens believe they can prompt public officials to act
55% of respondents report the district is responsive to citizens requests
marketing: citizens know about the service and the associated costs
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

46. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
responsiveness: citizens believe they can prompt public officials to act
55% of respondents report the district is responsive to citizens requests
marketing: citizens know about the service and the associated costs
31% of villagers knew about the platform (38% in high take-up)
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

47. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
responsiveness: citizens believe they can prompt public officials to act
55% of respondents report the district is responsive to citizens requests
marketing: citizens know about the service and the associated costs
31% of villagers knew about the platform (38% in high take-up)
take-up: users send actionable complaint messages
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

48. Research Design
Testing assumptions (survey of > 4000 villagers)
capacity: citizens believe that the district can fix the problem
84% of respondents believed the LG was capable to improve services
responsiveness: citizens believe they can prompt public officials to act
55% of respondents report the district is responsive to citizens requests
marketing: citizens know about the service and the associated costs
31% of villagers knew about the platform (38% in high take-up)
take-up: users send actionable complaint messages
Majority of messages were insufficiently actionable
Grossman (Penn) ICT for Service Delivery April 19, 2018 22 / 26

49. Research Design
Lessons learned: citizen demand and usage
1 Demand: there is clear underlying demand, even in poor rural areas, to use
mobile phones to register complaints
Grossman (Penn) ICT for Service Delivery April 19, 2018 23 / 26

50. Research Design
Lessons learned: citizen demand and usage
1 Demand: there is clear underlying demand, even in poor rural areas, to use
mobile phones to register complaints
2 Anonymity is extremely important for reporting initiatives
reluctance to provide specific details following initial complaint when prompted
by district response
Grossman (Penn) ICT for Service Delivery April 19, 2018 23 / 26

51. Research Design
Lessons learned: citizen demand and usage
1 Demand: there is clear underlying demand, even in poor rural areas, to use
mobile phones to register complaints
2 Anonymity is extremely important for reporting initiatives
reluctance to provide specific details following initial complaint when prompted
by district response
3 Tension between ‘all-purpose’ platform and effective usage
actionability of messaging cannot be assumed
Grossman (Penn) ICT for Service Delivery April 19, 2018 23 / 26

52. Research Design
Public (district) officials
1 District officials report satisfaction & intangible benefits
Grossman (Penn) ICT for Service Delivery April 19, 2018 24 / 26

53. Research Design
Public (district) officials
1 District officials report satisfaction & intangible benefits
2 Mismatch between district capacity and resources and citizen expectations
Grossman (Penn) ICT for Service Delivery April 19, 2018 24 / 26

54. Research Design
Public (district) officials
1 District officials report satisfaction & intangible benefits
2 Mismatch between district capacity and resources and citizen expectations
3 District officials heavily relied on support of a third-party
Grossman (Penn) ICT for Service Delivery April 19, 2018 24 / 26

55. Research Design
Additional lessons
1 Need to ensure case tracking is possible, and that system users receive
updates on case, at all stages
Grossman (Penn) ICT for Service Delivery April 19, 2018 25 / 26

56. Research Design
Additional lessons
1 Need to ensure case tracking is possible, and that system users receive
updates on case, at all stages
2 Need to include lower level governments (subcounties) to help increase
system effectiveness
Grossman (Penn) ICT for Service Delivery April 19, 2018 25 / 26

57. Research Design
Additional lessons
1 Need to ensure case tracking is possible, and that system users receive
updates on case, at all stages
2 Need to include lower level governments (subcounties) to help increase
system effectiveness
3 Need to make sure facilities get feedback on their performance (don’t know
what messages are being sent)
Grossman (Penn) ICT for Service Delivery April 19, 2018 25 / 26

58. Research Design
Additional lessons
1 Need to ensure case tracking is possible, and that system users receive
updates on case, at all stages
2 Need to include lower level governments (subcounties) to help increase
system effectiveness
3 Need to make sure facilities get feedback on their performance (don’t know
what messages are being sent)
4 Need to create common knowledge among citizens (don’t know what
messages are sent /response rate)
Grossman (Penn) ICT for Service Delivery April 19, 2018 25 / 26

59. Research Design
Conclusions
1 U-Bridge promising in terms of demand but not yet reaching full potential
with respect to service delivery outcomes
2 G4: penetration of Smartphones likely would be consequential
3 Important to explore take-up variability
Grossman (Penn) ICT for Service Delivery April 19, 2018 26 / 26