How Are We Measuring Up?Creating Relevant M&E Frameworks for NACS Amie N. Heap Nutrition Advisor, Office of HIV/AIDS SI Liaison to OGAC, Food and Nutrition TWG U.S. Agency for International Development
Some Thoughts on M&E…“If you are not confused, you are not paying attention.” Tom Peters “True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information.” Winston Churchill “We have an important opportunity to integrate nutrition into care, and we should get lost in the weeds.” Anonymous Colleague
Session Objectives• Global indicators and tools for monitoring and evaluation of NACS programs• Steps for creating a relevant NACS M&E framework• Gaps and future directions for monitoring and evaluation of NACS
Global NACS Reporting LandscapeIdentified Need: A harmonized set of core indicators available forinternational donors, national governments, and program implementers
Creating a Needed and Useful M&E FrameworkAssess the effectiveness Inform and improve Inform and improve of interventions Training program design program designIdentify successful approaches Report results toIdentify successful national approaches governments, donors Referral Links & and others Tracking NACS NACS Human Resources Quality Improvement Advocate for support and expansion of activities
Thematic Areas for NACS M&E Frameworks
Nutrition Assessment, Counseling, and Support M&E Framework Revision or Development Revised of Policy and Policy and Guidelines Guidelines Surveillance, Policies Monitoring M&E and Evaluation System INCREASED Design • Knowledge Guidelines Capacity Capacity of nutrition Building Built and & QA/QI • Number of Reduced QA/QI Support clients with Human, Econ Planned Reduced Morbidity functional omic, working Malnutrition andInfrastructure Clients status Mortality Provideand Technical Nutrition Counseled Resources Assessments and IMPROVED and Counseling Assessed • Dietary practices Nutrition NutritionCommodities Support to Eligible Clients Support Received Formalized Referrals to Clients ES/L/FS Referred Services INPUT PROGRAM OUTPUTS OUTCOMES IMPACT ACTIVITES
Creating Relevant NACS IndicatorsWhat data would data would be mostneeded and useful? ImpactIs an indicator the best method to Outcomescapture the data? OutputsIf so, how can we create indicators Program Activitiesthat are technically sound and feasibleto measure? InputsDoes the indicator set provide data thatis relevant at multiple levels within thehealth system?
Nutrition Care Indicators Indicator Definition Undernutrition in PLHIV: Number and proportion of PLHIV in care and treatment who were identified as undernourished at any point Impact during the reporting period. Provision of Therapeutic or Supplementary Food to Undernourished PLHIV: The number and proportion of undernourished PLHIV who received therapeutic or supplementary food at any point during the reporting period. Nutrition Assessment for PLHIV: The number and proportion of PLHIV in care and treatment who were nutritionally assessed during Output the reporting period. Nutrition Counseling for PLHIV: The number and proportion of PLHIV in care and treatment who were nutritionally assessed with anthropometric measurement who also received nutrition counseling at any point during the reporting period.
PMTCT Indicators Indicator Definition 12-Month Infant HIV-Free Survival (HFS): The percentage of infants born to HIV positive women in PMTCT programs who are Impact alive at 12-months of age and HIV-negative. Maternal Nutrition Status at Postnatal Care: The number and Outcome percent of HIV positive women who have a mid-upper arm circumference (MUAC) of less than 22cm at the first postnatal visit. Infant Nutrition Status: The number and percentage of HIV- exposed infants with acute malnutrition at 12-months of age. Infant Feeding Status: HIV-exposed infants who are: 1) exclusively breastfeeding at 3-months of age, 2) replacement feeding at 3-months of age, and 3) mixed feeding at 3-months of age
Food Access and Referral Indicators Indicator Definition Food Security of PLHIV: The number and proportion of PLHIV Impact receiving care and treatment services whose households have poor access to food based on the household hunger scale Per Capita Household Expenditures in HIV-affected Households: The percentage change in average per capita household expenditures among HIV-affected households Outcome Percentage of Total Expenditures Spent on Food in HIV-affected Households: The average percentage of total household expenditures that are spent on food in HIV-affected households Referral to Food Security Services: The number and percentage of HIV care and treatment clients vulnerable to food insecurity who are referred from clinical facilities to food security services Receipt of Food Security Services: The number and percentage of Output HIV-affected households that receive food security services Referral from Food Security Services to HIV Clinical Services: The number and percentage of clients receiving food security services who are referred to HIV clinical services
Country progress with NACS Indicators NACS at the National Level System Development Early Stages of Collection Established Data CollectionMozambique Malawi Kenya Ethiopia Uganda NACS Pilot Sites System Development Early Stages of Collection Established Data CollectionVietnam Ghana Cote d’IvoireHaiti NamibiaZambia TanzaniaSouth Africa
Creating a Tailored NACS Framework• Is the data needed to measure performance against goals/strategies?• How will the data be used to manage the program/national response?• Will the indicator reliably measure what it is intended to measure?• How should the data be aggregated and reported(site, region, district, national)?• What advocacy and resources are required to integrate nutrition data collection into the broader system?• How can data be made relevant and useful to those collecting it?
Gaps and Next Steps for NACS M&E• Measuring quality of service• Utilizing quality improvement to strengthen M&E systems• Strengthening the capacity establish, monitor, and evaluate links between clinical and community services• Utilizing data to inform and improve service delivery at the site level• Creating a system for sharing best practices within NACS programs
Resources For NACS M&EGuidance for Harmonized Monitoring and Evaluation of Nutrition and HIV Activities (FANTA-3)Still in press—will be available on FANTA-3 website in June 2012GFATM Monitoring and Evaluation Toolkit: HIV, Tuberculosis, Malaria and Health and CommunitySystems Strengthening (GFATM) http://www.theglobalfund.org/en/me/documents/toolkit/A Guide to Monitoring and Evaluation of Nutrition Assessment, Education, and Counseling of PLHIV(FANTA-2) http://www.fantaproject.org/publications/NAEC.shtmlIndicator Standards: Operational Guidelines for Selecting Indicators for the HIV Response (UNAIDS)http://www.unaids.org/en/media/unaids/contentassets/documents/document/2010/4_3_MERG_Indicator_Standards.pdfUNAIDS Indicator Registry: www.indicatorregistry.org