MONITORING AND EVALUATION         TOOLKITHIV/AIDS, TUBERCULOSIS AND MALARIA                 DRAFT 14/01/2004  World Health...
Table of contentsWHY THIS KIT?...............................................................................................
Why this kit?With the global momentum to scale up the response to the three main infectious diseases,HIV/AIDS, tuberculosi...
In addition, this toolkit addresses frequently asked questions in relation to implementingM&E for HIV/AIDS, TB and malaria...
or dissemination of data collection can reduce overlap in programming and increasecooperation between different groups, ma...
DATA                      −   An overall national level data dissemination planDISSEMINATION             −   A well-dissem...
Table 1 presents a generalised M&E framework for AIDS, TB and malaria. Examples ofthe areas measured at each level, key qu...
Level          Area              Key questions                      Indicator example               Drugs and      •   Dru...
Level               Area               Key questions                     Indicator example                   People on    ...
prevalence is consistently over 1% in pregnant women, the target population could verywell be the general population. Howe...
• Rationale for use   • Definition, including numerator and denominator   • Measurement - i.e. details on instrument and p...
Blood safety and                                 universal precautions                                 Antiretroviral trea...
Reduced Malaria specific             Reduced percentage of                              morbidity             high risk gr...
indicators covering injecting drug users (IDU) and HIV prevalence. The IDU indicator isapplicable to countries where injec...
Summary table for HIV/AIDS                  Service                  Delivery                    Area      Input          ...
Service           Delivery            Area        Input   Process           Output                      Outcome           ...
Service                                Delivery                                  Area         Input   Process            O...
Service             Delivery               Area          Input    Process              Output                        Outco...
•   The UNAIDS Estimates, Modelling and Projections Reference Group and       UNAIDS/WHO working group on surveillance and...
GuidelinesThe major sources for guidelines cited below are UNAIDS, WHO, USAID, CDC,MEASURE and FHI. The latest versions of...
TuberculosisThis section provides an overview of the core indicators for TB control and offersresources for more in-depth ...
Smear-positive TB                                                                           cases registered under        ...
Monitoring,                                   Number of training          evaluation, and                               ac...
−   Stop TB Partnership Working Groups: Three operational working groups provide a    focus for coordinated action and sup...
−   World Health Organization (2002). Global Tuberculosis Control: WHO Report       2003.       http://www.who.int/gtb/pub...
inaccurate; and many deaths, especially in young children, may be malaria relatedrather than attributable to malaria exclu...
Summary table for malaria               Sub-             Inpu               Programme          t                          ...
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Monitoring and evaluation toolkit
Upcoming SlideShare
Loading in...5
×

Monitoring and evaluation toolkit

398

Published on

Monitoring and evaluation toolkit - Conférence de la 2e édition du Cours international « Atelier Paludisme » - TUSEO Luciano - World Health Organization / Roll Back Malaria - maloms@iris.mg

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
398
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Monitoring and evaluation toolkit

  1. 1. MONITORING AND EVALUATION TOOLKITHIV/AIDS, TUBERCULOSIS AND MALARIA DRAFT 14/01/2004 World Health Organization
  2. 2. Table of contentsWHY THIS KIT?.......................................................................................................................................3WHO IS IT FOR?......................................................................................................................................3WHAT ARE ITS CONTENTS? ..............................................................................................................3BASIC ELEMENTS OF M&E ................................................................................................................4 BOX 1: CHECKLIST OF FEATURES OF A GOOD M&E SYSTEM. .................................................................5OVERALL FRAMEWORK FOR M&E ................................................................................................6 TABLE 1: THE M&E FRAMEWORK, WITH EXAMPLE AREAS, KEY QUESTIONS, AND INDICATORS ............7LEVELS OF MONITORING AND EVALUATION..........................................................................10HIV/AIDS .................................................................................................................................................13 SUMMARY TABLE FOR HIV/AIDS.........................................................................................................15 General resources .............................................................................................................................18 Technical assistance.........................................................................................................................19 Software products ............................................................................................................................19 Guidelines.........................................................................................................................................20TUBERCULOSIS ....................................................................................................................................21 SUMMARY TABLE FOR TUBERCULOSIS ..................................................................................................21 General resources .............................................................................................................................23 Technical assistance.........................................................................................................................24 Software products ............................................................................................................................24 Guidelines.........................................................................................................................................24MALARIA ................................................................................................................................................25 SUMMARY TABLE FOR MALARIA ...........................................................................................................27 General resources .............................................................................................................................29 Technical assistance and software products ....................................................................................29 Guidelines.........................................................................................................................................29FREQUENTLY ASKED QUESTIONS ................................................................................................32 TECHNICAL QUESTIONS .........................................................................................................................32 OPERATIONAL QUESTIONS .....................................................................................................................36ANNEX A..................................................................................................................................................40ANNEX B..................................................................................................................................................62ANNEX C..................................................................................................................................................74This document was based on a collaboration between WHO, UNAIDS, The GlobalFund to Fight AIDS, Tuberculosis and Malaria, USAID, CDC, UNICEF and theWorld BankM&E toolkit, Draft 14.01.04 2
  3. 3. Why this kit?With the global momentum to scale up the response to the three main infectious diseases,HIV/AIDS, tuberculosis (TB) and malaria, public health practitioners need to providevarious levels of accountability for their activities or policies to a variety ofconstituencies. It is becoming increasingly important for countries to be able to reportaccurate, timely and comparable data to donors and national authorities in order to securecontinued funding for expanding health programmes and, most importantly, to utilize thisinformation locally to strengthen evolving programmes. In sum, this toolkit is one steptowards assuring that countries are able to measure, report, and use good quality healthinformation in a manner that meets both donor and country needs. It is particularlyimportant for national programme implementers and managers to have access to thequality information they need to make adjustments and programmatic and technicaldecisions.Much progress has been made in the monitoring and evaluation (M&E) of these threediseases through the international community from global disease partnerships such asUNAIDS, StopTB, and Roll Back Malaria. Existing M&E guidelines and materials havebeen developed through the collaborative work of many partnership constituents such asWHO, bilateral agencies and NGOs. Developed with the support of international fundersand M&E experts, the purpose of this toolkit is to gather in one place the "essentials" ofagreed upon best practice in M&E, by applying a common M&E framework for the threediseases and providing users with references to key materials and resources. Althoughlabeled as a "monitoring and evaluation" toolkit, this document will focus mainly on themonitoring component.This toolkit aims to assist countries in the following:• Formulation of a national M&E strategy by providing an overview of key issues to consider;• Design of sustainable M&E systems that can be used to report on results and impact during the implementation stages of scaled up programmes;• Implementation and quality control of M&E systems and reporting of progress; and• Evaluation, review and improvement of M&E systems over time as the scale up of interventions to reduce morbidity and mortality associated with HIV/AIDS, TB and malaria occurs.Who is it for?This information package aims to provide those working at the country level on M&Esystems linked to expanded HIV/AIDS, TB and/or malaria programmes with rapid accessto key resources and standard guidelines. Users include national disease programmemanagers and project leaders and donor agencies, technical and implementing agenciesand NGOs to better harmonise information demands.What are its contents?The toolkit includes a standard framework for the development of a range of M&Eguidelines and tools, a summary of agreed upon illustrative core indicators for the threediseases, and references to more detailed indicator manuals on specific programme areas.M&E toolkit, Draft 14.01.04 3
  4. 4. In addition, this toolkit addresses frequently asked questions in relation to implementingM&E for HIV/AIDS, TB and malaria programmes.Note to users: The illustrative indicators presented have been developed for the nationallevel, although many of them can be used at various levels. Country users should designor modify their health information collection system bearing in mind the differentinformation that needs to be collected for use at different levels in order to construct “thebig picture” that these indicators allow. Additionally, new technologies anddevelopments will result in the need to periodically revise and update the illustrativeindicators presented here. This is the first time indicators from these diseases have beenbrought together in one manual. The approach includes the need to develop and improveindicators at different levels over time based on feedback. It is therefore a work inprogress. We will identify areas which require refinement in future editions asnecessary. This document is available electronically at the websites of partners involvedin its production.Basic elements of M&EWhile significant progress has been made in country M&E, much disease-specific M&Ehas been done in a vertical, isolated fashion that is often not linked or triangulated withother sources. For example, a surveillance system for HIV may be in place but notfunctioning well, and behavioral studies may have been done, though not necessarilyusing the same sampling methodologies or indicators. Extensive evaluation of a donor-sponsored project may have been carried out in an important area of programming,without the results ever being shared with others in the field. In short, the utility of muchof the disease-related measurement in a country may be lost because there is no coherentM&E system that can be used to capture necessary information on multiple diseases forusers at different levels. In addition, many countries rely on population surveys such asDHS and MICS that are funded through external donors to gather information on theimpact of their own and donor-supported programmes. This produces data that may bevaluable in the broader M&E context, but may not be well integrated with traditionalsources of health information, such as national health information and surveillancesystems.A common, comprehensive and coherent M&E system has several advantages. Itcontributes to more efficient use of data and resources by ensuring, for example, thatindicators and sampling methodologies are comparable over time and by reducingduplication of effort. Where resources are scarce, this is an important asset. Datagenerated by a comprehensive M&E system ought to serve the needs of manyconstituents, including programme or project managers, researchers or donors,eliminating the need for each to repeat baseline surveys or evaluation studies when theymight easily use existing data.From the point of view of the national programme, a coherent M&E system helps ensurethat donor-funded M&E efforts best contribute to national needs, rather than simplyserving the reporting needs of specific international donors or organizations. A furtheradvantage is that it encourages coordination and communication between differentgroups involved in the national response to diseases. Agreement among the major donor,technical and implementing agencies on the basic core M&E framework will reduce theburden of requests for data from different agencies. Shared planning, execution, analysisM&E toolkit, Draft 14.01.04 4
  5. 5. or dissemination of data collection can reduce overlap in programming and increasecooperation between different groups, many of whom may work more efficientlytogether than in isolation.Countries have different M&E needs, dictated in part by the state of their HIV, TB,and/or malaria disease burdens. Yet successful M&E systems will share commonelements. A list of some of these elements is given in Box 1.Box 1: Checklist of features of a good M&E system.M&E UNIT − An established M&E unit within the Ministry of Health with designated technical and data management staff − A budget for M&E that is between 5 and 10 percent of the combined national HIV/AIDS, TB, and malaria budgets from all sources − A significant national contribution to the national M&E budget (not total reliance on external funding sources) − A formalised (M&E) link, particularly with appropriate line ministries, NGOs and donors, and national research institutions aimed at enhancing operations research efforts − A multisectoral working group to provide input and achieve consensus on indicator selection and various aspects of M&E design and implementation − Epidemiological expertise in the M&E unit or affiliated with the unit − Behavioural/social science expertise in the M&E unit or affiliated with the unit − Data processing and statistical expertise in the M&E unit or affiliated with the unit − Data dissemination expertise in the M&E unit or affiliated with the unitCLEAR GOALS − Well-defined national programme or project plans with clear goals, targets and operational plans − Regular reviews/evaluations of the progress of the implementation of the national programme or project plans − Guidelines and guidance to districts and regions or provinces for M&E − Guidelines for linking M&E to other sectors − Coordination of national and donor M&E needsINDICATORS − A set of priority indicators and additional indicators at different levels of M&E − Indicators that are comparable over time − A number of key indicators that are comparable with other countriesDATA − An overall national level data collection and analysis plan,COLLECTION & including data quality assuranceANALYSIS − A plan to collect data and periodically analyse indicators at different jurisdictional levels of M&E (including geographical) − Second-generation surveillance, where behavioural data are linked to HIV/STI surveillance dataM&E toolkit, Draft 14.01.04 5
  6. 6. DATA − An overall national level data dissemination planDISSEMINATION − A well-disseminated, informative annual report of the M&E unit − Annual meetings to disseminate and discuss M&E and research findings with policy makers, planners and implementers − A clearinghouse for generation and dissemination of findings − A centralised database or library of all HIV/AIDS, TB, and malaria-related data collection, including ongoing research − Coordination of national and donor M&E dissemination needsOverall framework for M&EIndicators are used at different levels to measure what goes into a programme orproject and what comes out of it. The most commonly used framework for theselection of indicators for M&E is the input-process-output-outcome-impactframework illustrated below. For a programme or project to achieve its goals, inputssuch as money and staff time must result in outputs such as stocks and deliverysystems for drugs and other essential commodities, new or improved services, trainedstaff, information materials, etc. These outputs are often the result of specificprocesses, such as training sessions for staff, that should be included as key activitiesaimed at achieving the outputs. If these outputs are well designed and reach thepopulations for which they were intended, the programme or project is likely to havepositive short-term effects or outcomes, for example increased condom use withcasual partners, increased use of insecticide-treated nets (ITNs), adherence to TBdrugs, or later age at first sex among young people. These positive short-termoutcomes should lead to changes in the longer-term impact of programmes, measuredin fewer new cases of HIV, TB, or malaria. In the case of HIV, a desired impactamong those infected includes increased survival time and behavioral change. Foradditional information on M&E frameworks, readers may be interested in visiting thefollowing UNDP and MEASURE Evaluation sites:http://cfapp1.undp.org/undpweb/eo/evalnet/docstore3/yellowbook/http://www.cpc.unc.edu/measure/publications/evalman/Note: In many instances, behavioral change is considered a outcome of an HIVprogramme’s efforts. For the purposes of this guide, however, behavioral change isconsidered an impact indicator.Measuring impact requires extensive investment in evaluation, and it is often difficult toascertain the extent to which individual programmes, or individual programmecomponents, contribute to overall reduction in cases and increased survival. In order toestablish a cause-effect relationship for a given intervention, studies with experimental orquasi-experimental designs are necessary to demonstrate the impact.Therefore, focus is given here to output and outcome indicators, which are often moreeasily collected than impact indicators and used in the short to medium term forprogramme strengthening and reporting. As a programme or intervention matures, usersmay consider evaluating impact, using information that has been collected through theprogramme’s life and/or by undertaking special evaluation studies. As impact evaluationis not the focus of this document, the undertaking of such studies is not discussed here.M&E toolkit, Draft 14.01.04 6
  7. 7. Table 1 presents a generalised M&E framework for AIDS, TB and malaria. Examples ofthe areas measured at each level, key questions to answer, and indicators are provided.The aim of Table 1 is to familiarize users with this framework in order to facilitate theuse of this toolkit. This is particularly relevant for users familiar with otherinterpretations of the different levels. For example, the Global Fund to Fight AIDS, TBand Malaria (GFATM) generally defines process as a mixture of inputs and outputs, andcoverage as a mixture of outputs and outcomes. Depending on the level of programmedevelopment, there may be some overlap in indicators to measure inputs, processes andoutputs. For example, where trained personnel are available to the programme, theyrepresent an input for the programme. However, where human resources are lacking,trained personnel may be an output for the programme.Table 1: The M&E framework, with example areas, key questions, andindicators Level Area Key questions Indicator example Policy • National strategic plans • Policy and guidelines in place for each disease, at national level Disbursement including M&E and • Funding availability and operations research plans releaseINPUT Infrastructure exist • Distribution node selected • Policy and guidelines • Sentinel site selected(strategies,policies, exist • Providers selected Coordination • Funds have been • Coordination mechanism inguidelines, disbursed place for technical andfinancing) • Supply chain is in place operational issues • Coordination is • Coordination mechanism in established place for political issues • Infrastructure and equipment Human • Human resources for • Number of people trained per resources health services delivery number of people initially and supervision are targeted by training recruited, adequately • Number of people trainedPROCESS motivated, trained and according to national standards(human deployed for an interventionresources, • Human resources for • Number of people trained fortraining, supportive environment an intervention per 1,000commodities) are trained and deployed people in need of the intervention • Number with adequate supervision and motivationM&E toolkit, Draft 14.01.04 7
  8. 8. Level Area Key questions Indicator example Drugs and • Drugs are consistently • % of drug distribution nodes commodities available to consumers at reporting on stock status the right time and place (repletion, shortage, • Providers are equipped consumption, quality, losses) and available to people on a monthly basis seeking care • % of drug distribution • Standard treatment nodes/facilities reporting no guidelines and utilization drug shortage manuals have been • % of selected providers developed and produced equipped for the intervention (laboratories, nursing, others) Service • Intervention is • Number of districts or other delivery, accessible in a large administrative unit with at least technologies number or majority of one drug distribution center districts or other • % of districts or other administrative unit administrative unit with at least • Resources are available one drug distribution center for supervision • Number of districts or other administrative unit with the required number of providers of the intervention • % of districts or other administrative unit with the required number of providersOUTPUTS of the intervention(services, Knowledge, • Target population knows • Number of districts or othernumbers skills and about the benefit of the administrative unit withreached, practice intervention designated sentinel/providercoverage) operating according to guidelines for the intervention • % of districts or other administrative unit with designated sentinel/provider operating according to guidelines for the intervention • Number of target population with desired health behaviour/attitude • % of target population with desired health knowledge/attitudeM&E toolkit, Draft 14.01.04 8
  9. 9. Level Area Key questions Indicator example People on • A majority of target • Number of target population treatment, population is covered by covered by intervention people the intervention • % of target population covered benefiting from by intervention intervention • Number of target population receiving first line therapeutic regime • % of target population receiving first line therapeutic regime • Ratio of first to second lineOUTCOMES treatment regime(changed Changed • Increased number or • Number of target populationbehaviours, behaviour proportion of target with desired health seekingcoverage) population adopting behaviour (risk reduction, behaviours which reduce health care seeking) their vulnerability to • % of target population with infection, morbidity, desired health seeking and/or mortality behaviour (risk reduction, • Increased number or health care seeking) proportion of target population adopting beliefs and practices that create a supportive environment Morbidity, • Majority of target • Number of target populationIMPACT mortality population is in better showing clinical (and(biology and health as a result of the measurable) signs of recoveryquality of life) intervention after 6, 12 months • % of people showing clinical (and measurable) signs of recovery after 6, 12 months • Disease prevalence at regional or national levelsA note on target populations and denominators: In many cases, it may be difficult todetermine the denominator, or population, to use when assessing, for example, coverage.We have therefore focused on numerators, or the subset of the population that is affectedor benefits from interventions. In this toolkit, though, denominators should also beincluded where possible (if percentages are given, numerators should also always bereported to allow assessment of coverage). The publication Estimating the Size ofPopulations at Risk for HIV (UNAIDS/IMPACT/FHI, 2002) may help readers inaddressing the challenges faced in determining denominators when working with hiddenpopulations.In this toolkit, the term target population refers to the group of people who benefit froman intervention. The target population can be the total population or a smaller group suchas youth. In designing interventions, efforts should be made to clearly define the targetpopulation. Definition of these is usually based on knowing whom diseases affect most,directly and indirectly. For example, the definition of a target population for HIV/AIDSinterventions is often based on the epidemic state. In generalized epidemics where HIVM&E toolkit, Draft 14.01.04 9
  10. 10. prevalence is consistently over 1% in pregnant women, the target population could verywell be the general population. However, in concentrated and low-level epidemics whereHIV prevalence is concentrated within groups with specific risk behaviors, the targetgroup may be defined as a sub-group of the general population that shares these samebehaviors.Levels of Monitoring and EvaluationThis section presents illustrative core output, outcome, and impact indicators forHIV/AIDS, TB, and malaria. Users should be aware that these indicators have beendeveloped, discussed and agreed upon by a wide range of international and nationalexperts and donors. They have been developed for the specific purpose of minimizinginformation demands on countries while also assuring that indicators address specificinternational needs. The indicator development process was guided by five majorprinciples: • Building on existing indicators; • Harmonizing with other international frameworks such as the Millennium Development Goals (MDG); • Minimizing the number of indicators to be collected; • Covering a wide range of programme areas and sectors related to the different diseases; and • Addressing country programme needs.Input and process indicators are generally common across the three disease areas and aretherefore not specified for each. While there are some differences across the threediseases, these indicators generally take on the following forms: Generic input indicator: Existence of national policies, guidelines, or strategies. This is a “yes” / “no” question. Reporting of overall budget allocation is included as an input. Generic process indicator: Number of persons trained, number of drugs shipped/ordered, etc.For each disease, general programme areas have been defined. In the case of HIV/AIDS,for example, these include prevention, treatment, care and support, and supportivepolicy/implementation environments. A summary table showing the different programmeareas as well as indicators is presented for HIV/AIDS, TB, and malaria. When looking atthe summary tables, readers should be aware that sub-programmes often contribute tomultiple outcomes and impact. Although the tables give the impression of a linearprogression, assumptions regarding the overall outcome and impact of each sub-programme should be made with caution.Except for some output and outcome measurements (referred to as "counts", see below),specific information is provided for each of the indicators presented in the summarytables. This information can be found in the more detailed explanation of each indicator.Information provided for each indicator includes:M&E toolkit, Draft 14.01.04 10
  11. 11. • Rationale for use • Definition, including numerator and denominator • Measurement - i.e. details on instrument and process • Data collection platform, sources – i.e. survey, vital registration, in/out-patient registers, facility surveys, inventories, surveillance and sentinel reports • Recommended periodicity • Resources – i.e. reference groups, technical assistance sources, guidelinesOutputs and outcomes here are also monitored and reported as "counts" of increasedcapacity provided against a need that has been estimated as a pre-condition for changeand they can be quantified through direct observation or an annotated inventory. Forexample, it may be easier to collect the number of health providers trained in a specificarea through a record review. For these "counts", the toolkit does not provide a detaileddescription, and the definition of associated terms -where relevant-appears under thedetailed description of outcome indicators.Table 2 provides an overview of the service delivery areas and main objectives forHIV/AIDS, TB and Malaria.Table 2. Overview of service delivery areas and objectives for HIV/AIDS, TB andMalaria Delivery areas HIV/AIDS TB MALARIA Information, Education, Identification of Insecticide-treated nets (ITNs) Communication (IEC) infectious cases Youth Education Prevention of Malaria in pregnancy transmission by treating infectious cases Condom distribution Prevention of TB among Prediction and Containment of PLWHA epidemics Programmes for specific Indoor Residual Spraying Prevention groups Counseling and Information, education & voluntary testing communication (IEC) Prevention of mother to child transmission (PMTCT) STI diagnosis and treatment Post-exposure prophylaxis (PEP)M&E toolkit, Draft 14.01.04 11
  12. 12. Blood safety and universal precautions Antiretroviral treatment Timely detection and Prompt effective antimalarial and monitoring quality treatment of treatment Treatment cases Prophylaxis and Control of drug Monitoring of drug resistance treatment for resistance opportunistic infections HIV/TB Systematic monitoring Home based management of of performance in case malaria management Support for orphans Supporting patients Support through direct Care & observation of treatment Support for the chronically ill Strengthening of Civil Sufficient and quality Monitoring and Operations Society ensured drugs and lab Research supplies Stigma Building and Health systems strengthening maintaining human resource capacity Monitoring, evaluation, and operational research Health systems Supportive Environment strengthening Health systems Coordination and Coordination and partnership strengthening partnership development development (national, (national, community, community, public-private) Coordination and public-private) partnership development (national, community, Monitoring, evaluation, public-private) and operational research Monitoring, evaluation, Operational research and operational research agenda targeting barriers to DOTS Procurement and supply Procurement and supply Procurement and supply management capacity management capacity management capacity building building building Reduced adult HIV Reduced number of Reduced all-cause under 5 prevalence (ages 15-49) smear-positive cases per mortality (endemic areas) 100,000 population Reduced percentage of Reduced number of Reduced Malaria specific young people aged 15- deaths from TB (all mortality 24 who are HIV- forms) per 100,000 infected population per yearM&E toolkit, Draft 14.01.04 12
  13. 13. Reduced Malaria specific Reduced percentage of morbidity high risk groups (sex workers, clients of sex workers, men who have sex with men, injecting drug users) who are HIV infected Reduced percentage of HIV-infected infants born to HIV-infected mothers Increased survival among PLWHA Percentage of young people aged 15-24 reporting the use of a condom during sexual intercourse with a non- regular sexual partner Percentage of young people who have had sex before the age of 15 Percentage of young people who had sex with more than one partner last year Percentage of high risk groups who have adopted behaviours that reduce transmission of HIV Percentage of adults on ARV treatment who gain weight by at least 10% at 6 months after the initiation of treatmentHIV/AIDSThis section of the toolkit provides an overview of the core indicators and general M&Eresources for HIV/AIDS (in addition to those provided for each indicator). Each of theHIV/AIDS core indicators is applicable to all settings, with the exception of theM&E toolkit, Draft 14.01.04 13
  14. 14. indicators covering injecting drug users (IDU) and HIV prevalence. The IDU indicator isapplicable to countries where injecting drug use is an established, significant mode ofHIV transmission. Countries with low HIV prevalence or concentrated epidemics shouldreport on an alternative indicator of HIV prevalence among high-risk behavior groups, asopposed to prevalence among young people obtained from antenatal clinic sentinelsurveillance. Alternative indicators may be found in the UN General Assembly SpecialSession (UNGASS) on AIDS document entitled “Monitoring the Declaration ofCommitment on HIV/AIDS Guidelines on the construction of core indicators” (UNAIDS,2002).M&E toolkit, Draft 14.01.04 14
  15. 15. Summary table for HIV/AIDS Service Delivery Area Input Process Output Outcome Information, HIV/AIDS Increased numbers trained (health personnel, government, non-government, private sector), Commodities purchased (condoms, Drugs, Budget reporting by financial category, Monitoring and Evaluation Framework exists, Strategies and Guidelines developed and used, Education, radio/television Communicati programmes/newspapers on (IEC) produced* HIV/AIDS prevention brochures/booklets distributed* Peer educators active* Youth Schools with teachers Education trained in life-skills based HIV/AIDS education (PI1) Lab supplies [microscopes, reagents, slides]) Young people exposed to HIV/AIDS education Implementing Partners identified in school settings* (under development) Prevention Young people exposed to HIV/AIDS education out of school* (under development) Condom Retail outlets and service distribution delivery points with condoms in stock (PI2) Condoms sold through public sector* Condoms sold through private outlets* Programmes Sex workers & clients for specific exposed to outreach groups programmes* (number and percentage**) MSM exposed to outreach programmes* (number and percentage**)M&E toolkit, Draft 14.01.04 15
  16. 16. Service Delivery Area Input Process Output Outcome Mobile populations exposed to outreach programmes* (number and percentage**) IDUs reached by prevention services (number* and percentage) (PI3) Large companies with HIV/AIDS workplace policies and programmes (number* and percentage) (PI4) Counseling Districts with VCT People requesting and Voluntary services* (PI5) counseling and Testing voluntary testing (PI6)* PMTCT Health facilities offering HIV-infected pregnant minimum package of women receiving a PMTCT* (PI7) complete course of antiretroviral prophylaxis to reduce the risk of MTCT (number* and percentage) (PI8) STI diagnosis STI comprehensive case and treatment management (PI9) Post-exposure Number of people who prophylaxis receive post-exposure (PEP) prophylaxis* Blood safety Districts with access to and universal donor recruitment and precautions blood transfusion (PI10) Transfused blood units screened for HIV (PI11)M&E toolkit, Draft 14.01.04 16
  17. 17. Service Delivery Area Input Process Output Outcome Antiretroviral Health facilities capable People with advanced treatment(AR of providing advanced HIV infection receiving T) and interventions for antiretroviral monitoring prevention and medical combination therapy treatment for HIV (number* and infected persons (TI2) percentage) (TI1) Prophylaxis Health facilities with and treatment capacity to deliver basic Treatment for level counseling and opportunistic medical services for infections HIV/AIDS (number* and (OIs) percentage) (TI 3) HIV/TB Intensified TB case Provision of finding (among cotrimoxazole PLWHA) (TI 4) preventive therapy Counseling and and/or ART (TI 6 and voluntary testing for TB TI 7) patients (TI 5) Support for Orphans and vulnerable orphans children whose households received free basic external support (number* and Care and Support percentage) (CS1) Support for Chronically ill adults the whose households chronically ill received free basic external support (number* and percentage) (undergoing adaptation) Strengthening Number of NGOs of Civil dealing with HIV/AIDS Society services * Supportive Environment Stigma Number of PLWHA support groups fighting against discrimination* Monitoring, evaluation, and operational researchM&E toolkit, Draft 14.01.04 17
  18. 18. Service Delivery Area Input Process Output Outcome Health Number of patients who systems are accurately referred* strengthening Coordination Number of and networks/partnerships partnership involved* development (national, community, public- private) Procurement Percentage of service Unit costs of drugs and and supply delivery points with commodities management sufficient drug supplies capacity (under development) building* Outputs and outcomes here are also measured as "counts" of increased capacity provided against a needthat has been estimated as a pre-condition for change and they can be quantified through direct observationor an annotated inventory. For these "counts" the toolkit does not provide a detailed description in theannexes.,** Both percentages and numbers are required. However, if a denominator can not be obtained, focusshould be on raw numbers (the numerator).Note for HIV/TB service delivery area: TB/HIV programmes are complex in that twoseparate disease programmes are brought together, with each having individualapproaches and reporting mechanisms in place. For full details, refer to "Interim Policyon Collaborative TB/HIV Activities. WHO/HTM/HIV/2004.1 andWHO/HTM/TB/2004.330, and "A Guide to Monitoring and Evaluation for CollaborativeTB/HIV Activities" (WHO, in preparation)Detailed descriptions for each of the indicators listed above are provided in Annex A.General resourcesSince the creation of UNAIDS, a number of M&E resource groups – mainly at globallevel – were established to improve coordination among key M&E players. Currently,there are a total of five groups: • The UNAIDS Monitoring and Evaluation Reference Group (MERG) – composed of cosponsors/Secretariat M&E focal points, bilateral agencies, research institutes, and individual experts – that assists in harmonizing M&E approaches and improving methods.M&E toolkit, Draft 14.01.04 18
  19. 19. • The UNAIDS Estimates, Modelling and Projections Reference Group and UNAIDS/WHO working group on surveillance and estimates for HIV transmission and mortality. • The Inter-Agency M&E coordination working group – composed of key UNAIDS cosponsors, Secretariat and Global Fund to Fight AIDS, TB and Malaria M&E focal points – that assists in improving coordination among global M&E actors. • The UNAIDS Evaluation Unit – composed of UNAIDS Secretariat staff – that assists in the development of generic M&E systems for strategic information sharing. • The Global Monitoring and Evaluation Support Team (GAMET) – composed of World Bank personnel and staff seconded from technical agencies – that focuses on M&E country support in World Bank-supported countries.These resource groups have contributed to the development of the illustrative indicatorspresented here.UNAIDS and partners have been encouraging governments to set up a national levelM&E reference/support group to provide advice on national M&E strategies, and to assistin mobilizing resources for M&E and optimizing the use of data. Where those groupsexist, coordination among partners has tremendously improved.Technical assistanceAlthough technical support to governments is available through M&E technical supportgroups in some countries, additional assistance can be sought from the Evaluation Unit atthe UNAIDS Secretariat at UNGASSindicators@unaids.org for specific questions onUNGASS Declaration of Commitment (DoC) indicators, or at M-E@unaids.org forgeneral M&E questions. Other sources of support for all the diseases include: CDC,Measure Evaluation, Partners for Health Reform Plus (USA), Institute for HealthSystems Development (UK). Further support for HIV/AIDS includes: MeasureEvaluation and Measure DHS, FHI, The Synergy Project.Software productsUNAIDS has put at the disposal of countries a useful tool – the Country ResponseInformation System (CRIS) – that has the potential to house all national data obtained oncore and additional indicators and generate reports on those indicators. The CRISincludes two additional functions: resource tracking and research inventory.To learn more about the process of indicator development and the suggested actionsto implement the DoC M&E framework, readers are encouraged to consult theGuidelines on construction of core indicators that exist in four languages (English,French, Spanish and Russian) and that can be downloaded from UNAIDS web site.For more information on the CRIS, also please visit the UNAIDS web site.M&E toolkit, Draft 14.01.04 19
  20. 20. GuidelinesThe major sources for guidelines cited below are UNAIDS, WHO, USAID, CDC,MEASURE and FHI. The latest versions of these guidelines may be found on the Internetat:http://www.unaids.orghttp://www.who.inthttp://www.cpc.unc.edu/measurehttp://www.fhi.orghttp://www.cdc.govhttp://www.usaid.govUNAIDS/MEASURE (2000). National AIDS Programmes: A Guide to Monitoringand Evaluation. UNAIDS, Geneva.(http://www.cpc.unc.edu/measure/guide/guide.html) [SUGGESTION TO MOVETHIS TO FIRST ON THE LIST]UNAIDS (2002). Monitoring the Declaration of Commitment on HIV/AIDS Guidelineson the construction of core indicators(http://www.unaids.org/UNGASS/docs/JC894-CoreIndicators_en.pdf)UNAIDS/World Bank (2002). National AIDS Councils (NACs) Monitoring andEvaluation Operations Manual. UNAIDS/World Bank, Geneva.(http://www.unaids.org/publications/documents/epidemiology/surveillance/JC808-MonEval_en.pdf)Centers for Disease Control and Prevention (2002). Strategic Monitoring and Evaluation:A Draft Planning Guide and Related Tools for CDC GAP Country Programs. Centers forDisease Control and Prevention, Atlanta.Family Health International (2002). Evaluating Programs for HIV/AIDS Prevention andCare in Developing Countries: A Handbook for Program Managers and DecisionMakers. Family Health International, Arlington.(http://www.fhi.org/en/aids/impact/impactpdfs/evaluationhandbook.pdf)Family Health International (2000). Behavioural Surveillance Surveys (BSS):Guidelines for Repeated Behavioural Surveys in Populations at Risk for HIV. FamilyHealth International, Arlington.(http://www.fhi.org/en/aids/wwdo/wwd12a.html#anchor545312)WHO/UNAIDS (2000). Second Generation Surveillance for HIV: The Next Decade.UNAIDS, Geneva.(http://www.who.int/emc-documents/aids_hiv/docs/whocdscsredc2005.PDF)M&E toolkit, Draft 14.01.04 20
  21. 21. TuberculosisThis section provides an overview of the core indicators for TB control and offersresources for more in-depth consideration of monitoring and evaluation in TB. Theindicators are general in nature and appropriate for monitoring TB control, particularlythrough national TB control programmes. The indicators do not specifically address theadditional monitoring needs of innovations in service delivery such as community-basedcare or engagement of the private sector. Similarly, only a limited number of indicatorsare provided for monitoring TB/HIV interventions and the management of multi drug-resistant TB. Readers are guided to additional references for more comprehensivemonitoring of such activities. A compendium of indicators for monitoring TB controlactivities is under preparation by the Working Group on Indicators whose partners arelisted in the general resources. Many of the indicator definitions provided in this toolkitwere drawn from a draft of the compendium.Summary table for tuberculosis Sub- Programme Input Process Output Outcome international guidelines, financial resources consistent with operational plan, Identification of New smear positive TB infectious cases cases detected under Priority within overall health sector plan, TB policy consistent with supplies [microscopes, reagents, slides]), comprehensive laboratory international guidelines, mid-term operational plan consistent with government, private sector), Commodities purchased (Drugs, Lab DOTS (number* and Increased numbers trained (health personnel, government, non- percentage) (PI 1) Prevention of New smear-positive transmission by cases registered under treating DOTS who smear- Prevention infectious cases convert at 2 months of treatment (number* and percentage) (PI 2) Prevention of TB HIV seroprevalence Individuals dually among PLWHA among TB patients infected with TB and (PI 4) HIV who receive isoniazid preventive therapy (number* and percentage) (PI 3) network established Timely detection Population covered Number of death from TB Treatment and quality by DOTS (number* per 100,000 per treatment of and proportion) (TI 1) year(number* and cases percentage)M&E toolkit, Draft 14.01.04 21
  22. 22. Smear-positive TB cases registered under DOTS who are successfully treated (TI 2) (number* and percentage) Control of drug New smear-positive resistance cases registered under DOTS who default or transfer out of treatment (number* and percentage) (TI 3) (number* and percentage) Systematic Treatment facilities monitoring of submitting accurate, performance in timely and complete case management reports (number* and proportion) (TI 4) Supporting Patients cared for Care & Support patients through with directly directly observed observed therapy therapy (DOT) during intensive phase (number* and proportion) (CS 1) Sufficient and Number of health quality ensured facilities involved in drugs and lab DOTS with sufficient Supportive Environment supplies drug and laboratory supplies** (SE 1 and SE 2) Building and maintaining Number of health human resource facilities and capacity laboratories involved in DOTS with sufficient capacity for DOTS*M&E toolkit, Draft 14.01.04 22
  23. 23. Monitoring, Number of training evaluation, and activities conducted operational as per operational research plan* (focusing on barriers to DOTS Number and implementation) proportion of health workers receiving regular supervisory visits* Number of patients Health systems who are accurately strengthening referred* Health facilities and laboratories with capacity for DOTS implementation (number* and proportion) Coordination and Number of partnership networks/partnerships development involved* (national, community, public-private) Percentage of service Unit costs of drugs and Procurement and delivery points with commodities supply sufficient drug management supplies (under capacity building development)* Outputs and outcomes here are also measured as "counts" of increased capacity provided against a needthat has been estimated as a pre-condition for change and they can be quantified through direct observation oran annotated inventory. For these "counts" the toolkit does not provide a detailed description in the annexes.** Although this information focuses on reporting the number of health facilities, a detailed description ofthe associated indicator reporting on the proportion is provided.The detailed description of each of the indicators listed above is provided in Annex B.General resources− Tuberculosis Monitoring and Evaluation unit of Stop TB Department of World Health Organization: building capacity at country level for monitoring, evaluation and evidence-based planning, conducting global surveillance of epidemiological and financial trends in TB controlM&E toolkit, Draft 14.01.04 23
  24. 24. − Stop TB Partnership Working Groups: Three operational working groups provide a focus for coordinated action and support monitoring and evaluation of country-level activities related to o DOTS expansion, including sub-groups on laboratories and public-private mix o TB/HIV o MDR-TB− Global Working Group on Indicators – a partnership between the World Health Organization, World Bank, U.S. Centers for Disease Control and Prevention, International Union Against Tuberculosis and Lung Disease (IUATLD), Royal Netherlands Tuberculosis Association (KNCV), U.S. Agency for International Development (USAID) and Measure. Contact: cvincent@usaid.govTechnical assistance − International Union Against TB and Lung Diseases (IUATLD): www.iuatld.org − Royal Netherlands Tuberculosis Association (KNCV): www.tuberculose.nl − U.S. Centers for Disease Control: www.cdc.gov (mqualls@cdc.gov) − World Health Organization: www.who.int (dyec@who.int) − World Bank: www.worldbank.org (dweil@worldbank.org)Software products − WHO EpiCentre software to manage quarterly reporting data Contact: WHO SEARO (Nani Nair, nairn@whosea.org) − Electronic TB Register (ETR): a computerized TB register capturing individual patient data available from the U.S. Centers for Disease Control’s “Botusa” project in Africa Contact: Peter Vranken (pbv7@botusa.org). − New Windows (Access) application combines features of EpiCentre and ETR, and is accompanied by "specifications" for software tool development Contact: WHO Geneva (Dan Bleed, bleedd@who.int).Guidelines − World Health Organization (2002). An expanded DOTS framework for effective tuberculosis control. http://www.who.int/gtb/publications/dots/pdf/TB.2002.297.pdf − World Health Organization (1998). Tuberculosis handbook. http://www.who.int/gtb/publications/tbhandbook/index.htmM&E toolkit, Draft 14.01.04 24
  25. 25. − World Health Organization (2002). Global Tuberculosis Control: WHO Report 2003. http://www.who.int/gtb/publications/globrep/index.html − World Health Organization (2003). Management of Tuberculosis Training for health facility staff. − http://www.who.int/gtb/publications/training/management_of_tb/pdf/who_cds_tb _2003_314i.pdf − World Health Organization (1998). Laboratory services in tuberculosis control. http://www.who.int/gtb/publications/whodoc/who_tb-98- 258/en/98.258_org_management- .pdf − World Health Organization (2001). The Use of Indicators for communicable disease control at district level. http://www.who.int/gtb/publications/indicators/tb_2001_289.pdf − World Health Organization (2001). Good practice in legislation and regulations for TB control: An indicator of political will. http://www.who.int/gtb/publications/General/TB_2001_290legisl.pdf − World Health Organization (2000). Guidelines for establishing DOTS-PLUS pilot projects for the management of multidrug-resistant tuberculosis (MDR-TB). http://www.who.int/gtb/publications/dotsplus/dotspluspilot-2000- 279/english/index.htm − World Health Organization (2003). Guidelines for implementing collaborative TB and HIV programme activities. http://www.who.int/gtb/publications/tb_hiv/2003_319/tbhiv_guidelines.pdf − World Health Organization (1998). Guidelines for conducting a review of a national tuberculosis programme. http://www.who.int/gtb/publications/whodoc/who_tb_98.240.pdfMalariaThis section of the toolkit provides a generalized framework for monitoring andevaluation of specific interventions or service delivery areas within malaria controlprogrammes. An overview of the indicators for M & E across interventions ispresented and general resources that are available or in preparation. Each of theindicators is applicable to all malaria endemic settings, with the exception of theindicators covering impact and epidemics. The indicator for the prediction ofepidemics should only be used for countries with epidemic-prone areas. With regardto monitoring impact, the primary indicator to be monitored by all African countriesand high endemic settings is all-cause under-5 mortality, as measured by nationally-representative, household surveys. Malaria-specific mortality cannot be measuredroutinely, as it is difficult to measure in malaria-endemic Africa. Symptoms and signs(such as anemia) are not specific and sensitive, making autopsy and verbal autopsyM&E toolkit, Draft 14.01.04 25
  26. 26. inaccurate; and many deaths, especially in young children, may be malaria relatedrather than attributable to malaria exclusively without concurrent infections.Moreover, a majority of deaths do not occur in hospitals and are not routinelyrecorded in HMIS, and these are unlikely to be picked up in vital registration systems,which are often incomplete..M&E toolkit, Draft 14.01.04 26
  27. 27. Summary table for malaria Sub- Inpu Programme t Process Output Outcome Insecticide- Number of nets, LLNs, Households owning ITN Increased numbers trained (health personnel, government, non-government and private sector), commodities treated nets pretreated nets or (PI1) Budget reporting by financial category, Monitoring and Evaluation Framework exists, Strategies and (ITNs) retreatment kits distributed* Number of nets Children under 5 using retreated* ITN (PI 2) Guidelines developed and used, Implementing Partners identified Number of sentinel sites purchased (drugs, ITNs, insecticides, other, purchasing policy) established for monitoring insecticide resistance* Malaria in Number of nets, LLNs, Pregnant women using pregnancy pretreated nets or ITN (PI 3) retreatment kits distributed* Number of nets Pregnant women Prevention retreated* receiving treatment (IPT) or chemoprophylaxis (PI 4) Number of pregnant women receiving correct IPT* Prediction and Malaria epidemics containment of detected and properly epidemics controlled (PI 5) Indoor Number of homes and Residual areas sprayed with Spraying insecticide* Information, Number of targeted areas education, and with IEC services* communication (IEC)M&E toolkit, Draft 14.01.04 27

×