A Systematic Approach to Monitoring andEvaluating Integrated Health Interventions in   the Era of the Global Health Initia...
Rationale
Definition of integration             Linkages                          PrimaryComprehensive             care         One ...
Defining Integration The effort, within any building block of the health  system, to improve the continuum of care for  c...
Health System Building Blocks  and Integration   Health financing       Direct funding by external donors -> General hea...
Health System Building Blocks  and Integration (con’t)   Work-force       Providers and supervisors with specialized -> ...
Health Systems and Integration Health systems do not need to be integrated to  result in integrated care Health systems ...
So now what? How do we operationalize integration in a  systematic fashion? How do we monitor and evaluate integrated  i...
No Need to Panic!
Existing M&E Best Practices                                 Apply                    Are we doing                         ...
Key M&E Steps for Integration1. Begin with end in mind2. Define essential packages of services3. Develop logic model4. Imp...
1. Begin with the end in mind Key health outcomes and impacts   MDGs 4, 5, and 6   National priorities and targets   P...
2. Define essential packages ofservices Built around specific health care entry points   ANC/maternity   HIV testing an...
2. Essential packages of services(con’t) Precedent setting examples of such  packages exist ANC     Women presenting in...
2. Essential packages of services(con’t)   Need international guidance on service    packages      Choice of package det...
3. Develop logic models Defines how and where integration occurs at each  level of intervention    Inputs, processes, ou...
IHP+ Common M&E Framework    Inputs and Processes                              Outputs          Outcomes          Impacts ...
4. Improve health informationsystems Support provider access to client health  information    Electronic medical records...
4. Health information systems (con’t) Indicators     Derived from logic model     Access, readiness, coverage, health o...
4. Health information systems (con’t) Consistent with efforts to strengthen the  broader M&E system     IHP+, CHeSS, eva...
5. Use the data Inform program decision making Refine logic model inputs, processes, indicators Strengthen the evidence...
Research and Evaluation Agenda
Current state of the evidence Some improved client-level outcomes noted    increased uptake of services    increased cl...
Research agenda Conduct outcome and impact evaluations    At scale/going to scale    Across several models and countrie...
Research agenda Evaluate effective capacity building approaches to  intervention    For building human resources to prov...
Conclusions Integration is fundamentally client-centered Interventions should improve continuum of care Approach assume...
Conclusions Health system needs to be strong but not  necessarily integrated Integrated interventions take place within ...
MEASURE Evaluation is a MEASURE project funded by theU.S. Agency for International Development and implemented bythe Carol...
Upcoming SlideShare
Loading in …5
×

A Systematic Approach to Monitoring and Evaluating Integrated Health Interventions in the Era of the Global Health Initiative

1,110 views
954 views

Published on

Presented by Heidi Reynolds and Elizabeth Sutherland for the USAID Brown Bag Series for GHI Principles.

Published in: Health & Medicine, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,110
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • The definition of integration, what we mean by integration, bogs down the dialogueIt is really an umbrella term for many concepts and operationalizations
  • Many definitionsDefinition must supersede specific disease areas or health outcomesClient at center as opposed to program or donor prioritiesClient has easy access, perceives seamless care, her/his needs are met
  • Interventions to improve the continuum of care take place within the health system building blocks, but the health system building blocks do not themselves have to be integrated to improve the continuum of care for the client(Building blocks from WHO, definitions of integration within building blocks mixed from Atun and Mitchell, and this may not be exhaustive and certainly is not definitive)
  • We propose a systematic approach that will inform operationalization, M&E, and data for decision making.
  • Many frameworks exist, this is one (MDG 6)and there are others (e.g. Bryce MCH approach to evaluate the scale up for MDGs 4 and 5) These present a series of important questions to be asked when addressing M&E. Namely, we must id the problem, plan the response to the problem, monitor implementation of the response, collect and analyze data that will allow us to revise the response as needed and assess the effectiveness of the response.This is a dynamic process and not just a matter of reporting on indicators and putting together a report.
  • How would we approach this process for integration? There are several key principles for applying these types of frameworks/processes to integration. This outlines some practical steps that must be taken to appropriately operationalize M&E for integration. Highlight three principles to discuss in greater depth.
  • Roles of international and national public health community in each of these.
  • Some useful packages already created (e.g. ANC and to some extent HIV) but others still needed. Not all packages can be implemented in all countries immediately. Priorities determined by country level health needs as evidenced by national plans and targets.
  • Lack evidence for some may have to rely on expert opinion while evidence is generated for specific packages
  • Logic model provides a template for integration and coordination. This is a planning document that allows for each stage of implementation (inputs and processes to be planned in terms of coordination/integration roles and responsibilities). This is where the integrated interventions are planned for, with the aim of improving the continuum of care for clientsLogic models describe plausible pathways for causal effects. They are informed by theory and evidence. They lay out a logical expected pathway for activities to influence intermediate outcomes and collectively work toward expected impact.
  • Common framework adapted to show how the logic model can show the elements of the health care system working together to influence service delivery/care and ultimate health impacts.
  • Information that allows providers to follow clients health information over time will be important in the case of screening and referrals.
  • Outcome/impact evaluateions are needed to generate evidence for decision making for program management and scaleup and also for global adaptation and adoption. To understand the benefit of investment in integration must also understand cost effectiveness and Must also understand the ideal interventions within each health building block to maximize the benefits of integration interventions for health
  • (Re: last bullet: Case studies of on-going integration efforts can help understand what changes have been made to the health system building blocks, uncover other’s ideas about plausible pathways for effects of integrated interventions on client outcomes, build the “integration theory”, tap into the measures they are testing. This information can broadly inform interventions and determine impact evaluation priorities and designs)
  • A Systematic Approach to Monitoring and Evaluating Integrated Health Interventions in the Era of the Global Health Initiative

    1. 1. A Systematic Approach to Monitoring andEvaluating Integrated Health Interventions in the Era of the Global Health Initiative Heidi Reynolds, MPH, Ph.D. and Elizabeth Sutherland, Ph.D.
    2. 2. Rationale
    3. 3. Definition of integration Linkages PrimaryComprehensive care One stop shop Coordination Holistic Interoperable Not vertical Synergies Case management
    4. 4. Defining Integration The effort, within any building block of the health system, to improve the continuum of care for clients over the life course.• Integration is from the client’s perspective Client• Goal is to improve health outcomes
    5. 5. Health System Building Blocks and Integration  Health financing  Direct funding by external donors -> General health care budget  Leadership and governance  Disease policies -> integrated health policies  Decisions made without -> with consideration of general health care activities  Health services  Single purpose ->multi-purpose serviceAtun et al 2009;Mitchell et al 2004
    6. 6. Health System Building Blocks and Integration (con’t)  Work-force  Providers and supervisors with specialized -> generalized knowledge  Medical products, vaccines and technologies  Vertical -> general systems  Health information systems  Single purpose reporting ->patient centered systemAtun et al 2009; Mitchell 2004
    7. 7. Health Systems and Integration Health systems do not need to be integrated to result in integrated care Health systems do need to be strong Whether and how health systems will be integrated is context specific
    8. 8. So now what? How do we operationalize integration in a systematic fashion? How do we monitor and evaluate integrated interventions? How do we use the data to adapt program response and inform the global evidence base for integration?
    9. 9. No Need to Panic!
    10. 10. Existing M&E Best Practices Apply Are we doing 8. Are collective efforts them on a large impacting the epidemic? enough scale? 7. Are Interventions making a difference? 6. Are we implementing the program as Are we doing planned? them right? 5. What are we doing? Are we doing it correctly? 4. What interventions and resources are needed? Are we doing the right things? 3. What interventions can work (efficacy & effectiveness)? 2. What are the contributing factors? 1. What is the problem?Adapted from: Organizing a framework for a functional national HIVmonitoring and evaluation system. A report. UNAIDS. April, 2008.
    11. 11. Key M&E Steps for Integration1. Begin with end in mind2. Define essential packages of services3. Develop logic model4. Improve health information systems5. Use the data
    12. 12. 1. Begin with the end in mind Key health outcomes and impacts  MDGs 4, 5, and 6  National priorities and targets  Proximate health outcome indicators where appropriate  E.g. delivery with skilled birth attendants
    13. 13. 2. Define essential packages ofservices Built around specific health care entry points  ANC/maternity  HIV testing and treatment  Curative or ambulatory services  Child wellness
    14. 14. 2. Essential packages of services(con’t) Precedent setting examples of such packages exist ANC  Women presenting in pregnancy  WHO package includes range of services  HIV testing and screening for other STIs  Blood pressure and anemia screens  Tetanus toxoid injection and malaria prophylaxis
    15. 15. 2. Essential packages of services(con’t)  Need international guidance on service packages  Choice of package determined by health needs  Tailored for country-specific priorities and epidemiology  Service delivery guidelines for different service delivery levels
    16. 16. 3. Develop logic models Defines how and where integration occurs at each level of intervention  Inputs, processes, outcomes, impacts  National, district hospital, health facility, community  Health system building blocks Promotes stakeholder buy in at national-level
    17. 17. IHP+ Common M&E Framework Inputs and Processes Outputs Outcomes Impacts • Service • Improved Governance • Infrastructure Readiness • Coverage Financing Health • Workforce Outcomes • Commodities • Access • Prevalence • Information of risk • Efficiency Systems • Quality of factors CareAdapted from: Monitoring the building blocks of thehealth system. WHO Report. October 2010.
    18. 18. 4. Improve health informationsystems Support provider access to client health information  Electronic medical records  3 interlinked patient monitoring systems  ANC client cards Strong linked/interoperable routine health information systems still needed  Track progress in service delivery
    19. 19. 4. Health information systems (con’t) Indicators  Derived from logic model  Access, readiness, coverage, health outcomes New indicators are needed  Quality  Met needs and prevention  Referral
    20. 20. 4. Health information systems (con’t) Consistent with efforts to strengthen the broader M&E system  IHP+, CHeSS, evaluation platform, etc. Map data needs (from indicators in logic model) to existing survey data, surveillance, RHIS, etc. Determine what new data collection efforts are necessary
    21. 21. 5. Use the data Inform program decision making Refine logic model inputs, processes, indicators Strengthen the evidence base of what works
    22. 22. Research and Evaluation Agenda
    23. 23. Current state of the evidence Some improved client-level outcomes noted  increased uptake of services  increased client satisfaction Pilot tests  Little info on how to implement/effectiveness at scale Value added  Little/no info on relative value of integration
    24. 24. Research agenda Conduct outcome and impact evaluations  At scale/going to scale  Across several models and countries Evaluate essential packages of services  What should they contain?  What is the effectiveness of package in improving key health outcomes? Evaluate effectiveness of improved patient monitoring tools on  Continuum of care  Provider access to client health info
    25. 25. Research agenda Evaluate effective capacity building approaches to intervention  For building human resources to provide integration Evaluate effectiveness of referral mechanisms Conduct case studies of integrated interventions to inform impact evaluations
    26. 26. Conclusions Integration is fundamentally client-centered Interventions should improve continuum of care Approach assumes integration will be country led  Context specific and tailored to epidemiology Experience needed to understand how to adapt approach to reality of country setting Role for international community to guide and help build evidence base
    27. 27. Conclusions Health system needs to be strong but not necessarily integrated Integrated interventions take place within health system building blocks Intersects with innovative National Evaluation Platform design, IHP+ initiative, HSS questions, and the CHeSS initiative Leadership is needed from GHI on expectations for implementing the integration principle
    28. 28. MEASURE Evaluation is a MEASURE project funded by theU.S. Agency for International Development and implemented bythe Carolina Population Center at the University of North Carolinaat Chapel Hill in partnership with Futures Group International,ICF Macro, John Snow, Inc., Management Sciences for Health,and Tulane University. Views expressed in this presentation do notnecessarily reflect the views of USAID or the U.S. Government.MEASURE Evaluation is the USAID Global Health Bureausprimary vehicle for supporting improvements in monitoring andevaluation in population, health and nutrition worldwide.

    ×