Listening to the community.Using information and communication technologies for program monitoring<br />Papersubmittedtoth...
1.Introduction<br />Not enough timely data; limited reporting capacities at district levels<br />What is “effective” is co...
<ul><li>72 percentof all mobile subscriberslocated in DCs (ITU 2010)
“Frogleap development” in telecommunications; no alternative to mobile phones in DCs </li></ul>1. Introduction (2)<br />Re...
1.Introduction (3)<br />-Case studies: one pilot for healthcare in Kenya, and two pilots for water services in Guatemala a...
Were providers able to actually make use of the resulting information to improve their services? </li></ul>-Notframedasrig...
2. Literature<br />Sourcing of information for monitoring purposes(“remotesensing”, “geographical mapping”, “crowdsourcing...
3. Framework<br />Descriptive indicator / quantitative <br />Concept<br />Values<br />measure<br />Description of the ICT ...
4. Data<br /><ul><li>ChildCount is an integrative technological platform
Use of SMS text messages from mobile phones by 108 community health workers (CHWs) for patient registration and health rep...
Central web-based interface containing aggregated health data from the community</li></ul>Case 1: Healthcarepilot in Kenya...
4. Data (2)<br /><ul><li>Over 20,000 SMS-based health reports reaching the web-based interface (patient registrations, nut...
Daily data supply
10 per cent of all messages supplied in an improperly structured format, resulting in their rejection by the system
Analysis of health-related macrodata of the community such as birth rates, nutritional trends and seasonal variability of ...
4. Data (3)<br /><ul><li>Field Level Operations Watch (FLOW), a cloud-based data collection system to replace paper-based ...
Use of electronic surveys installed on Adroid-phones by Water For People staff, field workers of local partner NGOs and co...
Web-based system for mapping, data analysis and aggregation</li></ul>Case 2: Waterpointmonitoringpilot in Guatemala <br />...
4. Data (4)<br /><ul><li>Over 1000 data points: 116 water points 112 school water facilities, 1000 household surveys
Data on one water point is provided at least once a year (or more frequently)
Upcoming SlideShare
Loading in …5
×

Listening to the community. Using ICTs for program monitoring

1,235 views

Published on

Presentation about a paper submitted to 3rd International Social Innovation Research Conference, London 2011.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,235
On SlideShare
0
From Embeds
0
Number of Embeds
423
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Listening to the community. Using ICTs for program monitoring

  1. 1. Listening to the community.Using information and communication technologies for program monitoring<br />Papersubmittedtothe International SocialInnovation Research Conference, London, 2011<br />Zoltán Ferenczi<br />betterplace lab<br />gut.org gAG<br />Berlin, 10405 Germany<br />E-Mail: zfe@betterplace.org<br />Susanna Krüger<br />betterplace lab<br />gut.org gAG<br />Berlin, 10405 Germany<br />E-Mail: skr@betterplace.org<br />
  2. 2. 1.Introduction<br />Not enough timely data; limited reporting capacities at district levels<br />What is “effective” is contested; “multiple realities” of different stakeholders<br /><ul><li>Need for more direct, real-time, community level data about social services in DCs </li></ul>Developing countries face information related challenges in <br />social program monitoring (healthcare, water supply, etc.)<br />
  3. 3. <ul><li>72 percentof all mobile subscriberslocated in DCs (ITU 2010)
  4. 4. “Frogleap development” in telecommunications; no alternative to mobile phones in DCs </li></ul>1. Introduction (2)<br />Relevance of mobile phones for the developing world<br />
  5. 5. 1.Introduction (3)<br />-Case studies: one pilot for healthcare in Kenya, and two pilots for water services in Guatemala and Tanzania<br />-Questions:<br /><ul><li>To what extent has the application of ICTs enabled service providers to remotely gather timely feedback data from program beneficiaries?
  6. 6. Were providers able to actually make use of the resulting information to improve their services? </li></ul>-Notframedasrigorousresearch: Empiricaldatasetsareyetproblematic<br />The aim of the paper: to explore thepotential of ICTs, <br />especially mobile phones<br />
  7. 7. 2. Literature<br />Sourcing of information for monitoring purposes(“remotesensing”, “geographical mapping”, “crowdsourcing”) <br />Existing mechanisms of citizen reporting (Citizen Report Cards)<br />Participatory Monitoring & Evaluation (e.g. Guba and Lincoln 1989)<br />Real-time monitoring and evaluation<br />Research about mobile phones and data collection<br />Five core themes <br />
  8. 8. 3. Framework<br />Descriptive indicator / quantitative <br />Concept<br />Values<br />measure<br />Description of the ICT used in the case <br />“Remoteness”<br />“Yes” / “No”<br />(for example, simple mobile phone with <br />SMS capabilities)<br />Description of people involved in <br />“Traditionalinformationsourcing” / “Key <br />reporting for monitoring (e.g., citizens, <br />informants” (bounded crowdsourcing) / <br />community health workers, local civil <br />“Degree of <br />inclusiveness<br />”<br />“Beneficiaryinvolvement” <br />society members, trained employees of <br />(<br />c<br />rowdsourcing<br />)<br />NGOs/IOs, researchers, external <br />evaluators)<br />Frequency of data transmission (for <br />“Non real time” / “Moderate” / “Real<br />-<br />“Immediacy”<br />example, 3 times a day, daily, weekly, <br />time”<br />monthly, etc.)<br />Number of transmittedreportsinrelation<br />“Size of feedbackdataset”<br />Quantitative measure<br />tothetargetgroup<br />Number of error entries relative to all <br />“Errorproneness”<br />Quantitative measure<br />entries*<br />“None” / “Service provision” / “Strategic<br />Existence of policy document or rhetoric <br />“Evidenceonchange”<br />use”/ “Both”<br />demonstrating use of data<br />
  9. 9. 4. Data<br /><ul><li>ChildCount is an integrative technological platform
  10. 10. Use of SMS text messages from mobile phones by 108 community health workers (CHWs) for patient registration and health reporting
  11. 11. Central web-based interface containing aggregated health data from the community</li></ul>Case 1: Healthcarepilot in Kenya - Involvement, remoteness, immediacy<br />6<br />
  12. 12. 4. Data (2)<br /><ul><li>Over 20,000 SMS-based health reports reaching the web-based interface (patient registrations, nutrition screening reports, vaccination registration)
  13. 13. Daily data supply
  14. 14. 10 per cent of all messages supplied in an improperly structured format, resulting in their rejection by the system
  15. 15. Analysis of health-related macrodata of the community such as birth rates, nutritional trends and seasonal variability of malaria rates; performance monitoring of CHWs</li></ul>Case 1: Healthcarepilot in Kenya- Resulting data set and <br />action on data <br />7<br />
  16. 16. 4. Data (3)<br /><ul><li>Field Level Operations Watch (FLOW), a cloud-based data collection system to replace paper-based surveys
  17. 17. Use of electronic surveys installed on Adroid-phones by Water For People staff, field workers of local partner NGOs and community members
  18. 18. Web-based system for mapping, data analysis and aggregation</li></ul>Case 2: Waterpointmonitoringpilot in Guatemala <br />- Involvement, remoteness, immediacy<br />8<br />
  19. 19. 4. Data (4)<br /><ul><li>Over 1000 data points: 116 water points 112 school water facilities, 1000 household surveys
  20. 20. Data on one water point is provided at least once a year (or more frequently)
  21. 21. Zero errors at data entry
  22. 22. Data provides the basis for regional meetings among Water For People staff, local partner organisations and the local government; data is regularly included into reports</li></ul>Case 2: Waterpointmonitoringpilot in Guatemala <br />- Resulting data set and action on data<br />9<br />
  23. 23. 4. Data (5)<br /><ul><li>Approach to empower rural people to become change agents themselves and demand accountability
  24. 24. Use of text messages and simple mobile phones by citizens
  25. 25. Central web-based interface</li></ul>Case 3: Waterpointmonitoringpilot in Tanzania<br />- Involvement, remoteness, immediacy<br />10<br />
  26. 26. 4. Data (6)<br /><ul><li>800 text messages from beneficiaries
  27. 27. Approx. 2.5 reports a day
  28. 28. All reports were fed into the system; however, many messages were inappropriately formatted or contained incomplete information
  29. 29. Use of data for political purposes; 200 messages were passed on to the local government in order to start repairing broken water points</li></ul>Case 3: Waterpointmonitoringpilot in Tanzania<br />- Resulting data set and action on data<br />11<br />
  30. 30. 5. Analysis andsummary<br />12<br />Case<br />“Remote<br />-<br />“Inclusiveness”<br />“Immediacy”<br />“Size of <br />“Error <br />“Evidence based <br />ness”<br />feedback data <br />proneness”<br />on data”<br />set”<br />ChildCount, <br />Simple <br />Local CHWsaskey<br />real<br />-<br />time<br />Over 20000 SMS <br />10% of all <br />Both service <br />Kenya<br />mobile <br />informants (bounded<br />(3 reports by <br />by CHWs<br />messages were <br />provision <br />phones <br />crowdsourcing)<br />CHWs daily <br />rejected<br />(treatments ) and <br />(SMS)<br />on average)<br />strategic use (health <br />programming)<br />Water for <br />Android <br />Local volunteers and <br />moderate <br />Ca. 200 data <br />Zero rejection<br />Both service <br />People, <br />capable <br />community <br />(every water <br />points, each <br />provision (repairing <br />Guatemala<br />phones<br />members (bounded <br />point regularly <br />updated several <br />water points) and <br />crowdsourcing)<br />updated, at <br />times= 1000+ <br />strategic use <br />least once a <br />feedback set<br />(programmatic <br />year)<br />decisions)<br />Maji Matone, <br />Simple <br />Beneficiaries<br />real<br />-<br />time <br />800 SMS by <br />Zero rejection. <br />Both service <br />Tanzania<br />mobile <br />(Crowdsourcing)<br />(approx. 2.5 <br />Tanzanian <br />Many messages <br />provision (at least <br />phones <br />reports a day <br />citizens<br />were <br />12 water points <br />(SMS)<br />during pilot)<br />inappropriately <br />repaired) and <br />formatted or <br />strategic use <br />contained <br />(organizational <br />incomplete <br />learning)<br />information<br />
  31. 31. 6. Discussion<br /><ul><li>Treat results with caution; success may be context-specific
  32. 32. Under specific conditions: great potential in the field of international development
  33. 33. However; important to establish the “specific conditions” under which ICT pilots are successful
  34. 34. Need for quantitative studies with large-N comparisons
  35. 35. Implementing NGOs should be aware of the “dark side” of mobile phones; conduct thorough multidisciplinary analysis of the target area prior to pilot launch</li></ul>More researchisneeded!<br />13<br />
  36. 36. 7. Suggestionsforfutureresearch<br />-Outreachand promotionalactivities<br />-Training<br />-Settingthe right incentives<br />-Ownership/commitment<br />-Data validation<br />Areas of institutional design that may influence outcome:<br />14<br />
  37. 37. 9. References<br />Amin, S., J. Das, et al. (2007). Are You Being Served? New Tools for Measuring Service Delivery. Washington, DC:, The World Bank. <br />Berg, M., J. Wariero, et al. (2009). Every Child Counts - The use of SMS in Kenya to support the community based management of acute malnutritition and malaria in children under five, Millenium Villages Project, Earth Institute at Columbia University. <br />Bonbright, D. (2006). "Not learning from beneficiaries." Alliance, 11. (2). <br />Cars, M. (2006). Project Evaluation in Development Cooperation: A Meta-Evaluative Case Study in Tanzania, Stockholm University, Institute of International Education<br />Caseley, J. (2003). Blocked Drains and Open Minds: Multiple Accountability Relationships and Improved Service Delivery Performance in an Indian City. IDS Working Paper. Brighton, UK, Institute of Development Studies. <br />Chambers, R. (1994). "Participatory rural appraisal (PRA): Analysis of experience." World Development, 22. (7). <br />Chambers, R. (1997). Whose reality counts? Putting the first last. London, Intermediate Technology.<br />Cornwall, A. and V. S. P. Coelho (2007). Spaces for Change? The Politics of Participation in New Democratic Arenas.<br />Cornwall, A., V. Schattan, et al. (2004). New Democratic Spaces?<br />Coyle, D. and P. Meier (2009). New Technologies in Emergencies and Conflicts: The Role of Information and Social Networks. Washington, D.C. and London, UK, UN Foundation-Vodafone Foundation Partnership. <br />Cracknell, B. E. (2000). Evaluating Development Aid. Issues, Problems and Solutions. New Delhi, Thousand Oaks, London, SAGE Publications. <br />Daraja (2009a). Raising the Water Pressure - Programme Strategy Paper. Harnessing citizens’ agency to promote accountability, equity and sustainability in rural water supply. <br />Daraja (2009b). Raising the Water Pressure. A Concept Note. Harnessing new technology, the power of information and citizens’ agency to promote equity and functionality in rural water supply.<br />Donner, J., K. Verclas, et al. (2008). Reflections on MobileActive08 and the M4D Landscape. In Perspective. Proceedings of 1st International Conference on M4D 2008. <br />Eagle, N. and A. S. Pentland (2009). "Eigenbehaviors: identifying structure in routine." Behavioral Ecology and Sociobiology, (63): 1057–1066. <br />Fals-Borda, and Rahman, M. A. Eds. (1991). Action and Knowledge: Breaking the Monopoly with Participatory Action-Research. New York, Apex Press.<br />15<br />
  38. 38. 9. References<br />Forss, K. and J. Carlsson (1997). "The Quest for Quality - Or Can Evaluation Findings Be Trusted." Evaluation, (3). <br />Freire, P. (1972). Pedagogy of the Oppressed. New York, Herder & Herder.<br />Goetz, A.-M. and J. Gaventa (2001). Bringing citizen voice and client focus into service delivery. IDS Working Paper 138. Brighton, Institute of Development Studies. <br />Guba, E. G. and Y. S. Lincoln (1989). Fourth Generation Evaluation. London, Sage Publications. <br />Hellström, J. (2008). Mobile phones for good governance – challenges and way forward. http://www.w3.org/2008/10/MW4D_WS/papers/hellstrom_gov.pdf, <br />Holland, J. and J. E. Blackburn, Eds. (1998). Whose Voice? Participatory Research and Policy Change. London, Intermediate Technology Publications.<br />Howe, J. (2006). "The Rise of Crowdsourcing." Wired, (14.06.). <br />ITU (2010). World Telecommunication/ICT Development Report 2010. Monitoring the WSIS targets. A mid-term review. <br />Kothari, U. (2001). Power, Knowledge and Social Control in Participatory Development. Participation. The New Tyranny? B. Cooke and U. Kothari. London, Zed Boks: 139-152.<br />Krüger, S. and S. Teggemann (2008). Institutional Leadership in a Multistakeholder International Development Setting. Leadership as a Vocation. Houben/Rusche, NomosVerlag.<br />Lundberg, M. (2008). Client Satisfaction and the Perceived Quality of Primary Health Care in Uganda Mattias Lundberg. Are you being served? New Tools for Measuring Service Delivery: 323.<br />Martin, C. (2009). Put up a billboard and ask the community: Using mobile tech for program monitoring and evaluation. MobileActive.org. 2009, October 31. http://mobileactive.org/put-billboard-and-ask-community-using-mobile-tech-program-monitoring-and-evaluation, 15.03.2011<br />McGee, R. and J. Gaventa (2010). Review of Impact and Effectiveness of Transparency and Accountability Initiatives. Sythesis report. Transparency and Accountability Initiative Workshop, Institute of Development Studies. <br />Mohan, G. (2001). Beyond Participation: Strategies for Deeper Empowerment. Participation. The New Tyranny? U. Cooke Bill; Kothari. London, Zed Books: 163-167.<br />Munyua, A. W. and M. Mureithi (2008). "Harnessing the power  of the cell phone by women entrepreneurs: new frontiers in the gender equation in kenya. grace project research report." <br />16<br />
  39. 39. 9. References<br />Norheim-Hagtun, I. and P. Meier (2010). "Crowdsourcing for Crisis Mapping in Haiti." Innovations: Technology, Governance, Globalization 5 (4): 81-89. <br />Patnaik, S., E. Brunskill, et al. (2008). Evaluating the Accuracy of Data Collection on Mobile Phones: A Study of Forms, SMS, and Voice. <br />Patton, M. (2011). Developmental Evaluation. Applying Complexity Concepts to Enhance Innovation and Use. New York, London, The Guilford Press.<br />Paulos, E., R. J. Honicky, et al. (2009). Citizen Science: Enabling Participatory Urbanism. Handbook of Research on Urban Informatics: The Practice and Promise of the Real-Time City. M. Foth. Hershey, New York, Information Science Reference, IGI Global.<br />Power, M. (1997). The Audit Society, Rituals of Verification. New York, Oxford University Press.<br />Ravindra (2004). An Assessment of the Impact of Bangalore Citizen Report Cards on the Performance of Public Agencies, ECD Working Paper Series <br />Rebien, C. (1996). Evaluating Development Assistance in Theory and Practice. Avebury, Aldershot<br />Rudqvist, A. and P. Woodford-Berger (1996). Evaluation and Participation. Sida Studies in Evaluation 96/1. Stockholm, Department for Evaluation and Internal Audit, Sida<br />Schuster, C. and C. Perez-Brito (2011). "Cutting costs, boosting quality and collecting data real-time. Lessons from a Cell Phone-Based Beneficiary Survey to Strengthen Guatemala’s Conditional Cash Transfer Program." En Breve, World Bank LAC, (166). <br />Sutton, P., D. Roberts, et al. (1997). "A Comparison of Nighttime Satellite Imagery and Population Density for the Continental United States." Photogrammetric Engineering and Remote Sensing, (63): 1303-1313. <br />Turner, M. (2011). Mobilizing Development. Washington, D.C., Berkshire, UK, The UN Foundation and Vodafone Foundation Technology Partnership. <br />Ulbricht (2011). MobileActive.org, http://www.mobileactive.org/darker-side-mobiles-women-part-two-potential-dangers, 06.06.2011<br />UNDP (1997). Who Are the Question-Makers? A Participatory Evaluation Handbook. <br />Water for People (2007). Water for People - Guatemala Country Strategy. <br />Weiss, C. (1997). "How can theory-based evaluation make greater headway?" Evaluation, 21. 501-524. <br />WHO (2006). The World Health Report 2006 - Working Together for Health. Geneva, World Health Organization<br />WHO (2008). Safer Water, Better Health. <br />World Bank (2004). World Development Report. Making Services Work for Poor People. <br />17<br />
  40. 40. 9. Interviews<br />-Interviewwith Ben Taylor, Executive Director of Daraja, Tanzania<br />-Interviewwith Keri Kugler, programmatic data manager at Water for People, Denver <br />-Interview with Dr. James O. Wariero, ChildCount, Kenya, Nairobi<br />18<br />

×