Data Quality in Remote     Monitoring  A comparative analysis of  experiences in Somalia and  Eastern BurmaMona Fetouh (Co...
Remote Monitoring   Humanitarian space restricted in many situations; risks    have increased   Both delivery and monito...
Comparable Contexts   Somalia     Classic remote management situation     Difficulty of access due to insecurity, heavy...
Different Data Environments (1)   Somalia:     Main   concern is availability of data.     Little opportunity to collec...
Different Data Environments (2)   Eastern Burma:     Main concern is management and quality of data     Data are abunda...
Comparative Analysis Similar contexts require different  approaches Presentation with details on     Third party verifi...
Somalia – Third Party Verification (1)   First Level: Information from partners and    networks     Implementing  Partne...
Somalia – Third Party Verification (2)   Second Level: Third Party Verification      Flagged issues are scheduled for th...
Somalia – Third Party Verification (3)   Third Level: Follow up on concerns from Level 1 and 2     Third party informati...
Eastern Burma – StrengtheningMonitoring Quality (1)   Correlating data in geographical information    systems     Main  ...
Eastern Burma – StrengtheningMonitoring Quality (2)   Regular surveys are expanded in scope    and feed into the informat...
Eastern Burma – StrengtheningMonitoring Quality (3)   Increased linkages between implementing    partners, and improved M...
Eastern Burma – StrengtheningMonitoring Quality (4)   Post-facto review of health centre    logbooks and patient files   ...
Eastern Burma – StrengtheningMonitoring Quality (5)   Increased use of photographic and video    evidence of implementati...
Lessons learned, and looking towardsthe future   Similar contexts, different data environments   Good examples of when s...
Thank you!For further questions:Mona Fetouh - fetouh@un.orgVolker Hüls - volker@makingaidwork.comChristian Balslev Olesen ...
Upcoming SlideShare
Loading in …5
×

Data quality in remote monitoring (Mona Fetouh, UNOIOS)

614 views
508 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
614
On SlideShare
0
From Embeds
0
Number of Embeds
87
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Data quality in remote monitoring (Mona Fetouh, UNOIOS)

  1. 1. Data Quality in Remote Monitoring A comparative analysis of experiences in Somalia and Eastern BurmaMona Fetouh (Co-Author and Presenter), Christian Balslev-Olesen (Co-Author), and Volker Hüls (Co- Author)
  2. 2. Remote Monitoring Humanitarian space restricted in many situations; risks have increased Both delivery and monitoring of programmes affected Remote monitoring and cross-border approaches required in a number of countries in recent years Increased reliance on local actors Reduced ability to collect and verify information Often results in compromises on data quantity & quality Although remote management is increasingly common, evidence on best practice is still emerging
  3. 3. Comparable Contexts Somalia  Classic remote management situation  Difficulty of access due to insecurity, heavy reliance on national civil society (NGOs, communities) for aid delivery (still largely managed from neighbouring Kenya)  Risk to external monitors; monitoring relies heavily on local partners – risk of bias Eastern Burma  Hard to impossible to access from capital due to government restrictions and long-running conflicts between ethnic groups in the East and the government  Heavy reliance on civil society (LNGOs, communities) for aid delivery that is managed from neighbouring Thailand  Little to no access for INGO staff, including national staff  Risk to external monitors, monitoring relies heavily on local partners – risk of bias
  4. 4. Different Data Environments (1) Somalia:  Main concern is availability of data.  Little opportunity to collect data regularly, even for local partners  Local partners have varied capacity to produce quality data, severe education gap results in overall low staff capacity  Focus of strengthening objective monitoring was through investment in independent systems (―third party verification‖).
  5. 5. Different Data Environments (2) Eastern Burma:  Main concern is management and quality of data  Data are abundant, both monitoring data and surveys  Local partner capacity is varied but good – strong ethnic and professional exchange with Thai border area; better access to education  Information remains in technical silos; local NGOs are largely confined to ethnic areas; access opportunities are often limited to a particular sector.  Few mechanisms for independent verification/ triangulation of data.  Focus of strengthening objective monitoring was on the quality and verification of information  Access is improving due to ceasefire agreements
  6. 6. Comparative Analysis Similar contexts require different approaches Presentation with details on  Third party verification in Somalia  Quality assurance of monitoring information in Eastern Burma Experiences presented are based on of work of UNICEF (Somalia) and International Rescue Committee and the Border Consortium (Thailand/Burma)
  7. 7. Somalia – Third Party Verification (1) First Level: Information from partners and networks  Implementing Partner Reports are main source of primary performance data  Reviewed against:  Previous track record of partner / confidence level  Specific concerns about partner performance  Reporting complete and realistic?  Specific issues flagged in or apparent from requiring follow-up  Comparison to occasional information from staff contacts in the filed (email, telephone)
  8. 8. Somalia – Third Party Verification (2) Second Level: Third Party Verification  Flagged issues are scheduled for third party monitoring  3rd Party Systems use field monitors that are  not affiliated with any implementing partner  often outside of the aid business  And therefore  can move more freely with less risk  are not as qualified to judge details of implementation  used mostly for verification of easily obtainable information  Third party monitors, are ‗blind‘ tasked to avoid fabrication of reports  Were successfully tasked to track leakage of relief goods into markets, including quantities and pricing.
  9. 9. Somalia – Third Party Verification (3) Third Level: Follow up on concerns from Level 1 and 2  Third party information is kept confidential and assessed for the risk level of a particular performance issue.  Depending on risk level, issues are taken up with the partner:  without revealing source  e.g. dedicated open monitoring mission at next opportunity Main reasons for staggering:  First level flags issues, but is in itself not sufficient for reliable data  Second level is costly, and can be targeted to only flagged issues  Third level is costly and not timely, and should only be used with knowledge of problems Low-level third party networks have worked well in Somalia for other purposes, e.g. for food price monitoring
  10. 10. Eastern Burma – StrengtheningMonitoring Quality (1) Correlating data in geographical information systems  Main limitation to data correlation / triangulation is sector-based systems (Health, Education, Relief information management systems)  Sector IMS are basis for cross-sectoral GIS solution  Platform maps service delivery data of all sectors to village location  Allows analysis of performance data from all sectors per location  Allows sectors to engage in cross-monitoring and data sharing
  11. 11. Eastern Burma – StrengtheningMonitoring Quality (2) Regular surveys are expanded in scope and feed into the information system  Key strength of the Eastern Burma programmes is history of conducting regular (sector) surveys.  All partners are now supporting the expansion of the scope and the coverage of these surveys.  Example: Annual Poverty Survey
  12. 12. Eastern Burma – StrengtheningMonitoring Quality (3) Increased linkages between implementing partners, and improved M&E capacity  Effortsin recent years to connect local organizations across sector and ethnic group  Discussion on cross-monitoring  Comprehensive M&E training for local groups  Improved community feedback/village monitoring (also used by Oxfam in Somalia and Tearfund in Afghanistan)
  13. 13. Eastern Burma – StrengtheningMonitoring Quality (4) Post-facto review of health centre logbooks and patient files  Simple but innovative example:  Can be done remotely and is not time sensitive  Reveals substantial information about quality of support and services  Initiated by the IRC, and now being expanded to whole sector  Variations conceivable for other sectors
  14. 14. Eastern Burma – StrengtheningMonitoring Quality (5) Increased use of photographic and video evidence of implementation  Where possible, local NGOs use  photography  video to document their work.  Use of video started with success by one local NGO  Provides better representation e.g. of trainings and public awareness activities.
  15. 15. Lessons learned, and looking towardsthe future Similar contexts, different data environments Good examples of when similar contexts warrant different approaches Both contexts are changing rapidly – more access in Somalia, political reforms in Burma Improved monitoring systems instill long-term effects to adapt to these changes—stronger information, increased local capacity Changes in Somalia may make Eastern Burma lessons applicable in near future Experience in Somalia valuable for similar situations elsewhere, e.g. Syria
  16. 16. Thank you!For further questions:Mona Fetouh - fetouh@un.orgVolker Hüls - volker@makingaidwork.comChristian Balslev Olesen - christianbalslev@gmail.com

×