Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Use of monitoring data for evidence-based decision making: A factor analysis

173 views

Published on

Presentation given by Marieke Adank during the IRC Symposium All Systems Go! on 14 March 2019. This session was organised by Heather Skilling (DAI), in collaboration with Brain Banks (GETF), Nick Dickinson (WAHSNote) and Marieke Adank (IRC).

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Use of monitoring data for evidence-based decision making: A factor analysis

  1. 1. WASH SYSTEMS SYMPOSIUM ALL SYSTEMS GO! Use of monitoring data for evidence-based decision making: A factor analysis Marieke Adank, IRC 14 March 2019
  2. 2. 2 Setting the scene Source: mowie.gov.et Monitoring dataDecision makers
  3. 3. 3 Setting the scene Is monitoring data used for informing decision-making processes?
  4. 4. 4 Use of monitoring data for decision making • For improvement • For accountability • Data and information • From continuous collection and analysis -> trends • Instrumental • Conceptual • Symbolic • Operations and corrective actions • Planning & resource allocation • Regulation • Policy making
  5. 5. 5 Case: use of monitoring data in small town water supply in Ethiopia Instrumental use Conceptual use Accountability Serviceproviderlevel Utility NationallevelServiceauthority level Utility planning Department / MT Utility staff Billingdata Expenditure data Revenue data Production data Meter readingdata Asset data Staff data Staff performance Data producers Information producers Monthly , quarterly, annual reports WB (DC) MOWIE MIS data Report Decision makers Regional Bureau Water Board MOWIE Development partners IBNET focal persons IBNET data Technical operations Department MT Regional Water bureau Compiled data reports Programme development Resource allocation Resource allocation Policy decisions Projectdecisions Rewarding Support provision Budget approval Strategic plans and budgets Tariff proposals Operational decisions Staffingdecisions MOWIE Annual sector report Brochure X XX     
  6. 6. 6 Setting the scene How can use of monitoring data be improved? What are the factors than enable or hamper use of monitoring data in evidence- based decision making? How do these factors enable or hamper use of monitoring data?
  7. 7. 7 3 Models Data use Well- packaged information Data use Data producer Data user Relationship model Systems model Data use Actors and factors Linear model (Based on Best et al, 2009)
  8. 8. 8 CapacitiesData characteristics Factors Data use Data Relevance Data Accessibility Data Timeliness Motivations Data Quality Data Quantity Incentives Interest Culture Individual capacity Organisational capacity: Financial and logistical resources Organisational capacity: Data and information systems Institutional capacity
  9. 9. 9 Poll: In your experience, what is the most influential factor that hampers the use of monitoring data in evidence-based decision making? Session code: M6 Factor Description Data characteristics Quality Degree to which data reflect reality Relevance Quantity Degree to which data are relevant (actionable) for decision making and are presented in an understandable way Degree to which data quantity is suitable for informing decision-making processes (not too much, not too little) Timeliness Accessibility Degree to which data is available when needed (e.g. at key planning and decision making moments) Degree to which data and information are available to users Capacities Individual capacity Degree of human resource capacity (in terms of quality and quantity) for interpreting and acting on data Organisational capacity – financial and logistical resources Degree to which financial and logistical resources are available for interpreting and acting on data Organisational capacity – data and information systems Degree to which data and information (and where relevant decision-support) systems are present for collection, processing, analysis and storage of data and information Institutional capacity Degree of trust, communication and collaboration between actors and organisations involved in interpreting and acting on data Motivation Incentives Interest Culture Degree to which external incentives are present to stimulate interpreting and acting on data Degree to which actors have intrinsic interest in interpreting and acting on data Degree to which the organisational and institutional culture (norms, values, practices) enable and stimulates evidence-based decision making
  10. 10. 10 Poll results (37 participants) Data characteristics: 39% Capacities: 22.2% Motivations: 38.9%
  11. 11. 11 Assessment of factors in Jinka and Wukro town Utility monitoring system MIS IBNET Jinka Wukro National level Utility level Service authority level Utility level Service authority level Data timeliness High Moderate High Moderate Low Low Data relevance High Low High Moderate moderate Moderate Data quality Moderate Moderate Moderate Moderate Low Low Data quantity Moderate Moderate Moderate Moderate Moderate Moderate Data accessibility Low Low Moderate Low Moderate High Individual capacity Moderate Low Moderate Low Moderate Organisation capacity, finance and logistics Low Low Moderate Low Low Organisation capacity, information systems Low Low Moderate Low Moderate Institutional capacity Low? Moderate Moderate Interest Moderate Low Moderate Low Low Incentives Low Low Low Low Low Culture Low Low Low Low Low
  12. 12. 12 How do the factors relate to each other? • Influence and dependencies (MICMAC) • Centrality • (Causal loops) Data use (Inspired by the “factor mapping” methodology developed by University of Colorado Boulder)
  13. 13. 13 Relationships between factors (Matrix of Direct Influence) Data characteristics factors Capacity factors Motivation factors Outcome factor Influenced Influencing Data relevanc e Data quality Data quantity Data accessibility Data timeliness Individual capacity Organisational capacity - Financial and logistical resources Organisational capacity - Data and information Institutional capacity Incentives Interest Culture Use of data Data relevance 0 0 0 0 0 1 0 0 1 0 2 0 2 Data quality 0 0 0 0 0 1 0 0 1 0 2 0 2 Data quantity 0 0 0 0 0 1 0 0 1 0 1 0 2 Data accessibility 0 0 0 0 0 1 0 0 2 0 2 0 2 Data timeliness 0 0 0 0 0 1 0 0 1 0 2 0 2 Individual capacity 2 2 1 0 2 0 0 1 1 2 2 1 2 Organisational capacity - Financial and logistical resources 1 1 1 1 1 1 0 2 0 2 1 1 2 Organisational capacity - Data and information systems 1 2 1 2 2 0 0 0 1 0 1 1 2 Institutional capacity 2 1 2 2 1 1 1 2 0 1 1 2 1 Incentives 2 2 2 1 2 1 2 2 0 0 2 2 2 Interest 2 2 2 1 2 1 2 2 0 2 0 2 2 Culture 2 1 1 1 1 1 1 2 2 2 2 0 1 Use of data 0 0 0 0 0 2 0 0 2 2 2 1 0
  14. 14. 14 Factor dependency and influence (MICMAC) Dependence Influence Leverage points Low significance Potentially volatile Highly sensitive
  15. 15. 15 Dependence Potentially volatile Data characteristics Influence Leverage points Low significance Highly sensitive Motivations Capacities Factor dependency and influence (MICMAC)
  16. 16. 16 Centrality Label weighted indegree (influenced) weighted outdegree (influencing) betweenness centrality (bridging) Interest 21 21 15.9 Incentives 12 21 1.7 Institutional capacity 13 18 15.9 Individual capacity 13 17 10.2 Culture 11 17 2.6 Organisational capacity - Financial and logistical resources 7 15 0.1 Organisational capacity - Data and information systems 13 13 1.5 Use of data 22 12 4.4 Data accessibility 10 8 0.5 Data relevance 13 7 0.5 Data quality 12 7 0.5 Data timeliness 12 7 0.5 Data quantity 11 6 0.5 Using Gephi 0.9.2 (https://gephi.org/)
  17. 17. 17 Generated using Gephi 0.9.2 (https://gephi.org/) Centrality
  18. 18. 18 Causal loops Data relevance Data quality Data quantity Data accesability Data timeliness Individual capacity Finacial and logistical arrangements Presence of data and information systems Institutional capacity Incentives to use data Interest to use data Culture Use of data
  19. 19. 19 Causal loops Data relevance Data quality Data quantity Data accesability Data timeliness Individual capacity Finacial and logistical arrangements Presence of data and information systems Institutional capacity Incentives to use data Interest to use data Culture Use of data
  20. 20. 20 Causal loops Data relevance Data quality Data quantity Data accesability Data timeliness Individual capacity Finacial and logistical arrangements Presence of data and information systems Institutional capacity Incentives to use data Interest to use data Culture Use of data
  21. 21. 21 Conclusions • Focussing only on improving data characteristics will not ensure improved evidence-based decision making. • There is especially a need to also address: • Culture and incentive structures (including regulatory frameworks); • Institutional capacities; • Individual capacities; • Organisational capacity related to financial and logistical resources for taking and implementing evidence-based decisions.
  22. 22. 22 Thank you! Questions? ://www.ircwash.org/blog For (further) comments, suggestions, inputs and feedback, please contact me: Marieke Adank: adank@ircwash.org
  23. 23. 23 The human barometer • Locate yourself at a place in the room which, in your experience, represents best where the most influencing factor(s) for use of monitoring data in evidence-based decision making are. • Form a group with “like-minded” people (some 6-10 people per group) • Discuss in your group (10 min): - Why this / these factor(s)? - What actions can be taken to strengthen the priority factor(s), in order to improve use of data in decision making processes? (max 3 actions)
  24. 24. 24 Human barometer results Group 1 Group 2 Group 4 Group 3
  25. 25. 25 Group work results
  26. 26. 26
  27. 27. 27
  28. 28. 28
  29. 29. 29

×