Making the Connection : Monitoring and Evaluation in the Context of Integrated Health Services and Systems


Published on

Presented by Beth Sutherland, David Boone and Cristina de la Torre at the MEASURE Evaluation End-of-Phase-III Event.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Making the Connection : Monitoring and Evaluation in the Context of Integrated Health Services and Systems

  1. 1. Making the Connection Monitoring and Evaluation in the Context of Integrated Health Services and Systems MEASURE Evaluation End-of-Phase-III Event May 22, 2014
  2. 2. Introductions  Elizabeth Sutherland, MS, PhD  Cristina de la Torre, MPH, ScD  David Boone, MPH, PhD
  3. 3. Outline  Background  OverviewofMEASURE Evaluation’s work inintegration  Discussion ofMEASURE Evaluation’s workin  Monitoring referrals  Strengthening referralnetworks  Integrating health informationsystems  Take home messages and discussion
  4. 4. The Way We Were…  What is integration?
  5. 5. The Way We Were…  Why integration?  What should be integrated?  Where and how does integration happen?  What is the value added of integration?
  6. 6. Clients Have Multiple Health Needs  HIV  Malaria  TB  Growth  Nutrition  Family planning  Immunization  Respiratory illness  Diarrhea  Fever
  7. 7. Making the Link Between Clients and Services  One stop shop  Referrals and Referral Networks  Community vs facility models  Combinations
  8. 8. Making the System Work  System made of many elements  Together system elements support each other  System moves people, resources, and information up and down the levels of the system
  9. 9. So Where Are We Now?  We understand that integration operates among and within all levels of the health system  We know that goals and mechanisms for integration will vary by context
  10. 10. MEASURE Evaluation’s Work  Development and application of standardized tools and approaches to M&E of Integration  Support of development of interagency USG resources on M&E of integration  Developments of tools and techniques for monitoring referrals and strengthening referral networks
  11. 11. MEASURE Evaluation’s Work  Development and application of standardizedtools and approaches to M&E of Integration  Case study approaches to documenting best practices and lessons learned in integration  Integrating health information systems and using integrated data to facilitate data use
  12. 12. Referrals to Strengthen Service Integration
  13. 13. Models of Integration
  14. 14. M&E of Referral Systems Organizational Network Analysis Referral System Monitoring
  15. 15. Organizational Network Analysis (ONA)  Who is in the network  Service gaps or redundancies  How organizations are linked  Information sharing  Resource sharing  Referrals across organizations
  16. 16. ONA Application
  17. 17. Referral Assessment and Monitoring (RSAM) Toolkit Guidelines for  Establishing a routine monitoring system of referrals  Assessing overall functioning of the referral system  Can be adapted to any type of referral system
  18. 18. Focus on processes and systems Consists of interviews and document review to determine: RSAM TOOLKIT Referral System Assessment  How the referral system is structured  Whether referral protocols and guidelines exist  The processes providers follow to refer and counter-refer clients  How well referrals are tracked and followed up  Barriers to referral initiation and referral completion
  19. 19. RSAM TOOLKIT Referral System Monitoring Consists of routine data collection at facility  How often referrals are made to different services (initiation)  What types of services are clients most often referred to  Are clients able to take advantage of the referrals (completion)  Is adequate follow-up provided after the fact (counter-referral)
  20. 20. Routine Monitoring of Referral Systems Key indicators:  Referral initiation o % clients referredfromservice Ato service B  Referral completion o %ofreferredclients who complete referral  Counter-referral o % ofclients who complete referralwho areseenagain by initiating provider
  21. 21. Referral Systems
  22. 22. COLUMN Y TOTAL NUMBER CLIENTS SEEN AT REFERRING SERVICE CLIENTS REFERRED TO RECEIVING SERVICES Service 1 (FP) Service 2 (VCT) Service 3 (STI) Service 4 (ART) Service 5 Service 6 REFERRING SERVICE Service 1 (FP) Service 2 (VCT) Service 3 (STI) Service 4 (ART) Service 5 Service 6 (TO BE COMPLETED BY REFERRING SERVICE) PAGE 1 of 3 Name of organization and facility: _____________________________________ Geographic unit: _______________________________ Reporting period—Month: ______ Year: __________ Prepared by: ________________________ 1. Number of clients referred by type of service Group for which data are reported—Age range: _______________ Sex: ______________
  23. 23. Illustrative Monitoring Data
  24. 24. Illustrative Data
  25. 25. Benefits of Monitoring and Assessing Referrals Aid in Identifying:  under or over-utilized services  providers who are not referring patients  access or quality issues that impede service utilization  linkages between services that are not sufficiently established Aid in planning, resource allocation
  26. 26. Future Directions Increase evidence that these tools:  Help in referral strengthening  Impact client outcomes Better understand how they can be used in different contexts
  27. 27. Integrating Health Information Systems
  28. 28. Integration and Interoperability of Health Information Systems  Integration = combining two (or more) different systems to create one system  Interoperability = making two (or more) different systems work together to give the appearance of integration
  29. 29. Integration of Information Systems  Information systems  Data elements  Indicators  Data collection tools  Reporting protocols, procedures  Harmonization, rationalization of data and indicators  Data use
  30. 30. Interoperability (1)  Horizontal – between different systems at the same level  Vertical – between sub-units of the system at different levels of the health system  Semantic – do the terms we use mean the same thing?  Vocabularies o E.g. LOINC, SNOMED, HL7, ICD10  Syntactic – what language are we speaking?  E.g. XML, SDMX-HD
  31. 31. CHW in the VillageLocal Clinic Community Hospital Clinical Record System Rapid SMS Hospital Record System Shared Record Coordinated service delivery Two-way information flow Continuity of person-centred care Source: Open HIE
  32. 32. Source: Open HIE
  33. 33. Integration of IS at Community Level  MEASURE Evaluation HAITI  CBIS 2006-2010 o Identified landscape of interventions and actors o Identified needs of information o Harmonized/rationalized data and indicators o Harmonized data collection tools, reporting protocols o Monitored and supported implementation through supervision/capacity building o CLPIR toolkit
  34. 34. Integration at Facility Level  Beneficiary management  Linking services  Integrating data collectiontools, reportingforms  Master client Index (unique id)  Electronic patient recordssystems (e.g. OpenMRS)  Links to otherelectronicsystems(e.g. HR management) on client ID  Example (WHO/MEASUREEvaluation-3ILPMS)
  35. 35. Integration/Interoper- ability at District Level  Facility and system management  Data warehouse  Master facility list with attribute data  Examples: o MEASURE Evaluation Ethiopia (SNNPR) HMIS Scale-up o MEASURE Evaluation Cote d’Ivoire – integration of HIV/AIDS IS into RHIS
  36. 36. Integration/Interoper- ability at National Level  Policy development and Planning  Data warehouse  Monitoring and evaluation o E.g. MDGs  HMIS governance  Coordination of donors and other stakeholders  Local IS enterprise architects  Sustainable, scalable, incremental implementation  Example: RHIS Data management standards on Integration/interoperability
  37. 37. Wrap Up
  38. 38. Key Messages  Integration can operate at all levels of the health system and can include interventions to all building blocks of the health system.  Integration can take many forms and is inherently country-owned, country-led, and context specific
  39. 39. Key Messages  Despite the variability in integration models, however, there are standardized tools, approaches, and techniques that can be applied to integrated health and development  MEASURE Evaluation has worked to identify and develop these resources, including pioneering efforts to develop framework, indicators, tools, and systems related to M&E of integration.
  40. 40.  Continue to develop, apply, and refine resources intended to help countries to design, implement, and evaluate integrated health interventions, including integrated service delivery and integrated health information systems Future Directions
  41. 41.  What are the pressing needs in M&E of integration right now?  Where should M&E of integrated health interventions be going?
  42. 42. Resources For links to resources and references relevant to this presentation (including MEASURE Evaluation and Non-MEASURE Evaluation resources) see: