Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

8 M&E: Data Sources


Published on

A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.
Part 8 of 11

Published in: Business
  • Be the first to comment

8 M&E: Data Sources

  1. 1. Development Project Planning 6 Monitoring and Evaluation Data Sources
  2. 2. Ground Rules <ul><li>The point is </li></ul><ul><li>1) for you to get benefit from the process </li></ul><ul><li>2) not to disturb other people </li></ul><ul><li>What are the rules? </li></ul>
  3. 3. This Session <ul><li>Recap – Indicators </li></ul><ul><li>Mathematics </li></ul><ul><li>Data Systems and Collection </li></ul><ul><li>Levels of Indicators </li></ul>
  4. 4. What is an Indicator ? <ul><li>a variable … </li></ul><ul><li>that measures one aspect of a project </li></ul><ul><li>that is directly related to the program’s objectives. </li></ul>
  5. 5. DATA SYSTEMS <ul><li>Components of a clearly defined Data System: </li></ul><ul><ul><li>multiple, well defined indicators </li></ul></ul><ul><ul><li>a variety of appropriate data sources </li></ul></ul><ul><ul><li>baseline and target values </li></ul></ul><ul><ul><li>feasible data collection plan and budget </li></ul></ul><ul><ul><ul><li>specified frequency </li></ul></ul></ul><ul><ul><ul><li>identified responsibility </li></ul></ul></ul>
  6. 6. LEVELS OF DATA <ul><li>Policy or Program </li></ul><ul><li>Population </li></ul><ul><li>Service Environment </li></ul><ul><li>Client </li></ul><ul><li>Spatial / Geographic </li></ul>
  7. 7. DATA SOURCES AND TOOLS POLICY/PROGRAM LEVEL <ul><li>Gives information about the ‘big picture’ </li></ul><ul><li>The context, the nation or region </li></ul><ul><li>How countries compare to each other </li></ul>
  8. 8. DATA SOURCES AND TOOLS POLICY/PROGRAM LEVEL <ul><li>Sources: </li></ul><ul><li>Official documents (e.g. government reports, legislative and administrative documents) </li></ul><ul><li>National budgets or other accounts data </li></ul><ul><li>Policy inquiries </li></ul><ul><li>Reputational rankings (e.g. program effort scores) </li></ul><ul><li>Tools: </li></ul><ul><li>Index questionnaires (give rankings e.g. morbidity) </li></ul><ul><li>Special and contract studies </li></ul>
  9. 10. DATA SOURCES AND TOOLS SERVICE ENVIRONMENT LEVEL <ul><li>Gives information about how the services are being run </li></ul>
  10. 11. DATA SOURCES AND TOOLS SERVICE ENVIRONMENT LEVEL <ul><li>Sources: </li></ul><ul><li>Administrative records (e.g., service statistics, HMIS data, financial) </li></ul><ul><li>Service delivery point information (e.g., audit information, inventories, facility survey data) </li></ul><ul><li>Staff or provider information (performance / competency assessments, training records, staff/provider data, quality of care data) </li></ul><ul><li>Client visit registers/compilations </li></ul><ul><li>Tools: </li></ul><ul><li>Health Service Information Systems </li></ul><ul><li>Facility sample surveys </li></ul><ul><li>Performance monitoring reports </li></ul><ul><li>Service Delivery Point records </li></ul>
  11. 12. HEALTH MANAGEMENT INFORMATION SYSTEMS (HMIS) <ul><li>An important way of monitoring routine data over time is through a Health Management Information System . </li></ul><ul><li>HMIS is a system for routine reporting on service delivery. </li></ul><ul><li>Often it is national. Best if data collected from a full set of service delivery points, and on topics like: </li></ul><ul><ul><li>Costs </li></ul></ul><ul><ul><li>Stockouts </li></ul></ul><ul><ul><li>Births </li></ul></ul><ul><ul><li>Mortality </li></ul></ul><ul><ul><li>Morbidity </li></ul></ul><ul><ul><li>Numbers of clients seen, referred (inpatient; outpatient) </li></ul></ul><ul><ul><li>Numbers of clients by types of service </li></ul></ul>
  12. 14. DATA SOURCES AND TOOLS INDIVIDUAL LEVEL <ul><li>Gives information about how individual people are being served </li></ul>
  13. 15. DATA SOURCES AND TOOLS INDIVIDUAL LEVEL <ul><li>Sources: </li></ul><ul><li>Case surveying (e.g., epidemiology of disease) </li></ul><ul><li>Medical records </li></ul><ul><li>Interview data </li></ul><ul><li>Provider-Client interactions (check the clinical, technical or interpersonal skills) </li></ul><ul><li>Tools: </li></ul><ul><li>Case reports </li></ul><ul><li>Client register analysis </li></ul><ul><li>Patient flow analysis </li></ul><ul><li>Direct observation </li></ul>
  14. 17. DATA SOURCES AND TOOLS POPULATION LEVEL <ul><li>Gives information about everyone in the country </li></ul><ul><ul><li>How things change over time </li></ul></ul><ul><ul><li>how countries compare </li></ul></ul>
  15. 18. DATA SOURCES AND TOOLS POPULATION LEVEL <ul><li>Sources: </li></ul><ul><li>Government Census Office </li></ul><ul><li>Vital registration systems (e.g., birth and death certificates) </li></ul><ul><li>Sentinel systems – watch a specific thing (e.g. H2N1) </li></ul><ul><li>Sample households or individuals </li></ul><ul><li>Special population samples (demographic or occupational group, or geographic sector) </li></ul><ul><li>Tools: </li></ul><ul><li>Birth certificates </li></ul><ul><li>Household/Individual/Special surveys </li></ul><ul><li>Census forms </li></ul>
  16. 20. DATA SOURCES AND TOOLS SPATIAL / GEOGRAPHIC <ul><li>Sources: </li></ul><ul><li>Satellite imagery and aerial photography </li></ul><ul><li>Digital line graphs and elevation models </li></ul><ul><li>Cadastral maps (land ownership) </li></ul><ul><li>Tools: </li></ul><ul><li>Global Positioning System GPS </li></ul><ul><li>Computer software programs GIS </li></ul>
  17. 22. Take a break <ul><li>Back in 15 minutes </li></ul>
  18. 23. Some Maths <ul><li>A couple of statistical ideas </li></ul>
  19. 24. PERCENTAGES <ul><li>Consist of a fraction – the number measured divided by the maximum possible </li></ul><ul><li>times by 100 ( per cent = for each hundred) </li></ul><ul><li>Number counted = “Numerator” </li></ul><ul><li>Number possible = “Denominator” </li></ul>% = Numerator Denominator x 100
  20. 25. Example: PERCENTAGES <ul><li>Number of projects at HU Ratanakiri: 7 </li></ul><ul><li>Number of EU funded projects: 4 </li></ul><ul><li>What percentage of projects at HU RTK ate EU funded? </li></ul><ul><li>Numerator = 4, Denominator = 7 </li></ul>% = 4 7 x 100 = 57 %
  21. 26. Percentile <ul><li>A percentile tells you where an individual or subgroup fits into a larger group as a percentage </li></ul><ul><li>If you say “Sam’s IQ is at the 66 th percentile” it means that his IQ is in the first 66% of the population </li></ul>
  22. 27. Normal Distribution <ul><li>Many population statistics rely on the rule that the measure results will scatter in a “normal distribution” </li></ul><ul><li>There is an average (“mean”) with half above and half below </li></ul><ul><li>Most individuals are close to the mean </li></ul>
  23. 28. Normal Distribution Percentile 0.1 2.2 15.8 50 84.1 97.7 99.9
  24. 29. Quintile <ul><li>Divide a group into five equal groups according to a ranking e.g income </li></ul><ul><li>Each group is a quintile </li></ul><ul><li>Enables comparisons between locations </li></ul><ul><li>e.g someone in Cambodia earning $2 a week and someone in the USA earning $40 a week will both be in the lowest quintile for income. </li></ul>
  25. 30. Example: Disabilities by income quintile (Indigenous Australia)
  26. 31. DIFFERENT DATA SOURCES - SAME INDICATOR <ul><li>Different data sources can be used to measure the same indicator. In some cases, changes to the metric are required depending on data sources selected. </li></ul><ul><li>% of live births attended by a trained TBA in last 12 months </li></ul><ul><li>Option 1 – using health service data: </li></ul><ul><li>Numerator: # of live births in the district attended by trained TBA in last 12 months </li></ul><ul><li>Denominator: # of live births in the district in last 12 months </li></ul><ul><li>Option 2 – using information from village visits: </li></ul><ul><li>Numerator: # of women having a live birth in last 12 months reporting being attended by a trained TBA </li></ul><ul><li>Denominator: # of women having a live birth in last 12 months </li></ul>
  27. 32. DIFFERENT DATA SOURCES - SAME INDICATOR <ul><li>When a choice is available look at the + and - of different data sources. </li></ul><ul><li>% of children 12-23 months receiving all three polio vaccines </li></ul><ul><li>Numerator: # of children 12-23 months receiving all three polio vaccines </li></ul><ul><li>Denominator: # of children 12-23 months </li></ul><ul><li>Data sources: </li></ul><ul><li>Option 1: numerator from individual client records at facilities, denominator from census </li></ul><ul><li>Option 2: numerator from HMIS, denominator from census </li></ul><ul><li>Option 3: numerator and denominator from sample household survey (interview with mother) </li></ul><ul><li>Option 4: numerator and denominator from sample household survey (review of child’s immunization card) </li></ul>
  28. 34. Some MEASURE Indicators <ul><li>The Handouts come from USAID MEASURE program, which promotes good M&W practice </li></ul><ul><li>Disaggregate: to separate into parts </li></ul><ul><ul><li>how this indicator is specific to one group </li></ul></ul>Population Health Environment
  29. 35. Value-Added Indicators <ul><li>Value-added indicators show groups of people or sectors that may not have been targeted in the intervention but have an impact from the intervention. </li></ul>
  30. 36. Value-Added Indicator: Example <ul><li>The project may intend to increase the use of condoms by providing condoms to local merchants and providing merchants with sales training. </li></ul><ul><li>In the process, the livelihood of that merchant is helped by providing an alternative stream of income. </li></ul><ul><li>The diversification of livelihood for that merchant is a value-added result. </li></ul>
  31. 37. Integration Indicators <ul><li>Integrated programs have several advantages over stand-alone population, child health, or environment programs. </li></ul><ul><li>Integrated programs are cost-effective compared to the cost of single-sector programs. </li></ul><ul><li>Integrated programs also recruited a greater number of men to family planning efforts and a greater number of women and adolescents to environment/ conservation efforts. Integrated programs also improved the perceived valueof family planning efforts by packaging them with health interventions. </li></ul><ul><li>Example: Number of cross sector linkages formed </li></ul>
  32. 38. Global Fund Guide Lists many indicators
  33. 39. Thankyou
  34. 40. Produced by Tony Hobbs Health Unlimited, Ratanakiri, Cambodia With the support of Australian Volunteers International © 2009 HU. Use with Acknowledgement