Successfully reported this slideshow.
Your SlideShare is downloading. ×

Evaluating National Malaria Programs’ Impact in Moderate- and Low-Transmission Settings: Technical Approaches

More Related Content

More from MEASURE Evaluation

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Evaluating National Malaria Programs’ Impact in Moderate- and Low-Transmission Settings: Technical Approaches

  1. 1. Evaluating National Malaria Programs’ Impact in Moderate- and Low-Transmission Settings: Technical Approaches Background New evaluation approaches that are adaptive and flexible are needed to accurately measure the progress and achievements of malaria programs in low-, moderate-, and heterogeneous-transmission settings. These settings present unique challenges because of the level of granularity, mosaic of intervention packages, and number of implementing partners focusing on different transmission zones in heterogeneous settings. The timing of impact evaluation in relation to the maturity of the interventions is another challenge because high population coverage of the intervention might be less of an issue. In these contexts, evaluation approaches should include both process and impact evaluations. Debra Prosnitz, MPH1 ; Ruth Ashton, PhD2 ; Andrew Andrada, MSPH1 ; Yazoumé Yé, PhD1 1 MEASURE Evaluation, ICF; 2 MEASURE Evaluation, Tulane University Refined Evaluation Design The approach to refined, adaptive, and flexible evaluation design includes both process and impact evaluations (Figure 1). Process evaluations give programs evidence for corrective action to improve program performance (coverage, intensity, and quality of malaria interventions) and will also help determine the timing of an impact evaluation, which will measure the reduction of malaria burden. For impact evaluations, confirmed malaria incidence is recommended as the primary impact indicator in low- or heterogeneous-transmission settings due to its sensitivity to detect changes. Methods to measure impact will include interrupted time series, district-level dose-response analyses, and constructed control, supported by triangulation of multiple data sources to strengthen the plausibility argument. Access the full Framework for Evaluating National Malaria Programs in Moderate- and Low‑Transmission Settings Limitations The challenges, which are priority areas for further research, are as follows: benchmarking “adequate quality” routine health information system data, defining intervention maturity of malaria programs, setting thresholds for implementation strength, fully accounting for endogeneity, and determining at what level of program coverage measurable impact is expected. Summary The framework for evaluating national malaria programs highlights the importance of routine surveillance data for evaluation and use of confirmed malaria incidence to measure impact in low-, moderate-, and heterogeneous-transmission settings. In many low-transmission settings in which the program has already achieved scale-up of key interventions, evaluation activities are likely to take the form of continuous process evaluation, complemented by impact evaluation when a substantial change in policy, intervention, or strategy has taken place. Emphasizing a continuous cycle along the causal pathway, linking process evaluation to impact evaluation and then programmatic decision making, the framework provides practical guidance in evaluation design, analysis, and interpretation to ensure that the evaluation responds to national malaria program priority questions and guides decision making at national and subnational levels. Definitions: NMSP=national malaria strategic plan; ITNs=insecticide-treated nets; IRS=indoor residual spraying; IPTp=intermittent preventive treatment in pregnancy; SMC=seasonal malaria chemoprevention; SME=surveillance, monitoring, and evaluation; SBC=social behavior change; HIS=health information system; MDA/MSAT/FSAT=mass drug administration/mass screening and treatment/focal screening and treatment Figure 1. Theory of change for national malaria programs across the transmission spectrum Evaluation questionsEnabling environment Malaria program implementation Health system (e.g., government expenditure on health, health facility infrastructure and resources) Macro socioeconomic (e.g., political situation, GDP per capita, transport, gender, and communication infrastructure) Environmental (e.g., extreme weather events, other disease outbreaks) Process evaluation All settings implement NMSP? All settings Program managed and implemented well? All settings Interventions in NMSP implemented as expected? Interventions good quality? Low Implementing the right interventions in the right places? Surveillance system functioning well? Factors influencing outcomes Health system (e.g., access to and use of health services, availability of essential drugs and commodities) Macro and micro sociocultural and economic (e.g., poverty,migration, householdwealth, sleepinghabits, gender) Outcome and impact evaluation Moderate or high Reached high enough coverage? Interventions equitable? Low Coverage sustained at high enough levels? Interventions reaching the right people? All settings Program have an impact? Do alternative explanations exist for the observed impact? Factors influencing impact Health system of other health interventions) Environmental (e.g., climate, temperature) Epidemiological (e.g., malnutrition, disease prevalence, population movement) HIGH MODERATE LOW • Identification of high-risk groups/subnational areas Funding, health workforce, infrastructure, HIS, commodities, governance, policy and guidelines 1. INPUTS • Delivery of prevention activities (ITNs, IRS, larviciding, MDA/MSAT/ FSAT) • Surveillance system strengthening/adaptation • Delivery of prevention activities (ITNs, IRS, larviciding, IPTp, SMC, MDA/MSAT/FSAT) • Surveillance system development • Delivery of prevention activities (ITNs, IRS, larviciding, IPTp, SMC) • Surveillance system development SBC, case management, supervision, supply chain management, training, SME and research, diagnostics quality assurance 2. PROCESSES • Diagnostic tests retested • Surveillance data generated, reviewed and acted on • Work force trained • M&E data generated and reviewed • Chemoprevention delivered • Work force trained • M&E data generated and reviewed Diagnostics and treatment delivered, ITNs distributed, IRS conducted, supervision, coordinated implementation of activities, utilization of services, research generated and reviewed 3. OUTPUTS • Uptake of interventions (ITNs, IRS, MDA/MSAT/ FSAT) • Case-based surveillance system • Uptake of interventions (ITNs, IRS, IPTp, SMC, MDA/MSAT/FSAT) • Routine surveillance system • Uptake of interventions (ITNs, IRS, IPTp, SMC) • Routine surveillance system Diagnostics and treatment coverage, quality data used for decision-making, diagnostic proficiency, functioning and responsive surveillance system 4. OUTCOMES • Interrupt malaria transmission • Decrease malaria incidence and prevent outbreaks • Prevent malaria- attributable mortality • Decrease all-cause child mortality Decrease malaria transmission, decrease malaria incidence, decrease malaria-attributable mortality 5. IMPACT *May include net durability monitoring and IRS application quality monitoring using cone bioassay. WHO provides specific guidance on entomological surveillance, and intervention monitoring and evaluation available at Transmision Setting Acknowledgments—This poster presents work initiated by the Evaluation Taskforce of the Roll Back Malaria Monitoring and Evaluation Reference Group. This research has been supported by the President’s Malaria Initiative (PMI) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AIDOAA-L-14-00004. The opinions expressed are those of the authors and do not necessarily reflect the views of USAID, or the United States Government. For information, contact: