Building New Institutional Capacity in M&E: The Experience of National AIDS Coordinating Authority V F Kemerer Getting To ...
Overview <ul><li>Background </li></ul><ul><li>Use of Assessment Tools for Strategic Planning </li></ul><ul><li>NACA Specif...
Background
Background Coordinate the National HIV DMRS Lead National M&E System Development NAC M&E UNIT National Agencies USG & PEPF...
Strategic Vision: Build effective and sustainable M&E systems to provide comprehensive, high-quality information
Use of Assessment Tools for Strategic Planning <ul><li>SI Assessment </li></ul><ul><li>MESST </li></ul><ul><li>DQA/RDQA </...
SI Assessment Activities <ul><li>Map national HIV output indicators and identify data flow </li></ul><ul><li>Outline chall...
Linking SI Capacity and SI Planning with SI Products <ul><ul><li>Timely, high quality strategic  </li></ul></ul><ul><ul><l...
Use of Assessment Tools for Strategic Planning <ul><li>SI Assessment </li></ul><ul><li>MESST </li></ul><ul><li>DQA/RDQA </...
A new approach to M&E Assessments  Donor-driven Country-driven Focused exclusively on capacities of specific silos Expande...
How does the M&E Strengthening Tool work? Step 1:  Diagnosis Step 3:  Action Plan <ul><li>Development of a Costed Action p...
Expected Benefits <ul><li>Better identify M&E capacity gaps   and corresponding strengthening measures , including through...
Use of Assessment Tools for Strategic Planning <ul><li>SI Assessment </li></ul><ul><ul><li>DDIU Tools </li></ul></ul><ul><...
DQA METHODOLOGY – 2 Protocols <ul><li>The methodology for the DQA includes two (2) protocols:   </li></ul>Data Verificatio...
<ul><li>PURPOSE:   Identify potential risks to data quality created by the data- management and reporting systems at: </li...
Objective of the Data Quality Assessment Tool The Data-Quality Assessment (DQA) Tool is designed to: <ul><li>assess the ab...
NACA Specific Capacity Building Interventions MESST  RDQA  TA for DQ  SI Assessment  M&E Road Map  1/4ly Mentoring (SHAPMoS)
CB INTERVENTIONS
SI Assessment  M&E Road Map  Implementation <ul><li>Incorporation of Data Flow Findings into SHAPMoS  </li></ul><ul><li>In...
SI Assessment  M&E Road Map  Implementation <ul><li>ToT M&E Basic Concepts </li></ul><ul><li>4 quarterly mentoring & coach...
Institutional Capacity Strengthened – Regional M&E Officers <ul><li>Internal monitoring systems developed and in place </l...
CB INTERVENTIONS
Timeline: TA to Malawi 2007 2008 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 Data Quality Plan for  MESST MESST: Ma...
RDQA Tool To Operationalize Statement of Work <ul><ul><li>Support finalization of the Draft Data Quality Protocol  </li></...
Institutional Capacity to Support Data Quality Developed <ul><li>Data quality plan written that includes guidance, reporti...
Conclusions <ul><li>NACA M&E Unit institutional capacity strengthened at National and Sub-National levels </li></ul><ul><l...
MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) through Cooperative Agreement GPO-A-...
Upcoming SlideShare
Loading in...5
×

Building New Institutional Capacity in M&E: The Experience of National AIDS Coordinating Authority

794

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
794
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Building New Institutional Capacity in M&E: The Experience of National AIDS Coordinating Authority

    1. 1. Building New Institutional Capacity in M&E: The Experience of National AIDS Coordinating Authority V F Kemerer Getting To Results IDEAS Global Assembly Johannesburg, South Africa 18 March 2009
    2. 2. Overview <ul><li>Background </li></ul><ul><li>Use of Assessment Tools for Strategic Planning </li></ul><ul><li>NACA Specific Capacity Building Interventions </li></ul><ul><li>Measurement and Determination of Effectiveness and Outcomes </li></ul>
    3. 3. Background
    4. 4. Background Coordinate the National HIV DMRS Lead National M&E System Development NAC M&E UNIT National Agencies USG & PEPFAR GFATM UN Family Misc Bi- & Multi-Laterals CB through TA using various tools & strategies to establish and/or improve performance
    5. 5. Strategic Vision: Build effective and sustainable M&E systems to provide comprehensive, high-quality information
    6. 6. Use of Assessment Tools for Strategic Planning <ul><li>SI Assessment </li></ul><ul><li>MESST </li></ul><ul><li>DQA/RDQA </li></ul>
    7. 7. SI Assessment Activities <ul><li>Map national HIV output indicators and identify data flow </li></ul><ul><li>Outline challenges in the DMRS </li></ul><ul><li>Assess M&E structures </li></ul><ul><li>Stakeholder Analysis – National M&E Framework </li></ul><ul><li>Stakeholder Analysis –HIV researchers, organisations, and research </li></ul>
    8. 8. Linking SI Capacity and SI Planning with SI Products <ul><ul><li>Timely, high quality strategic </li></ul></ul><ul><ul><li>information products </li></ul></ul>Unified SI Strategic Planning and Coordinated SI Support SI Systems Assessments Ongoing Monitoring & Feedback Building of National SI Systems
    9. 9. Use of Assessment Tools for Strategic Planning <ul><li>SI Assessment </li></ul><ul><li>MESST </li></ul><ul><li>DQA/RDQA </li></ul>
    10. 10. A new approach to M&E Assessments Donor-driven Country-driven Focused exclusively on capacities of specific silos Expanded to integration in the wider M&E Systems General approach More systematic approach (based on a checklist of items) Strictly Assessment Tool Broader Assessment, Management, Planning & Capacity Building Tool FROM … TO …
    11. 11. How does the M&E Strengthening Tool work? Step 1: Diagnosis Step 3: Action Plan <ul><li>Development of a Costed Action plan with: </li></ul><ul><li>Identification of Strengths and Weaknesses </li></ul><ul><li>Defining Strengthening Measures : </li></ul><ul><ul><li>Responsibility </li></ul></ul><ul><ul><li>Timeline </li></ul></ul><ul><ul><li>Funding Implication </li></ul></ul><ul><ul><li>T/A Needs </li></ul></ul><ul><li>Diagnosis through completion of a Checklist: </li></ul><ul><li>Answers to Statements/ Questions </li></ul><ul><li>Provision of Comments/Explanations </li></ul>Step 2: Analysis <ul><li>A Summary Dashboard will be automatically generated : </li></ul><ul><li>Displays the overall distribution of answers for each of the checklists’ sections </li></ul><ul><li>Provides a more detailed analysis of the specific gaps that helps respondents complete the ACTION PLAN </li></ul>
    12. 12. Expected Benefits <ul><li>Better identify M&E capacity gaps and corresponding strengthening measures , including through Technical Assistance (TA) </li></ul><ul><li>Ensure that such investments contribute to the strengthening of the National Systems (avoiding parallel reporting systems) </li></ul><ul><li>Guide investments in M&E (within the recommended range of 5%-10% of the overall Budget) </li></ul><ul><li>Enhance the quality of programmatic data to improve program management </li></ul>
    13. 13. Use of Assessment Tools for Strategic Planning <ul><li>SI Assessment </li></ul><ul><ul><li>DDIU Tools </li></ul></ul><ul><li>MESST </li></ul><ul><li>DQA/RDQA </li></ul>
    14. 14. DQA METHODOLOGY – 2 Protocols <ul><li>The methodology for the DQA includes two (2) protocols: </li></ul>Data Verifications (Protocol 1) Quantitative comparison of recounted to reported data and review of timeliness, completeness and availability of reports. 1 Assessment of Data Management Systems (Protocol 2) Qualitative assessment of the strengths and weaknesses of the data-collection and reporting system. 2
    15. 15. <ul><li>PURPOSE: Identify potential risks to data quality created by the data- management and reporting systems at: </li></ul><ul><ul><ul><ul><li>the M&E Management Unit; </li></ul></ul></ul></ul><ul><ul><ul><ul><li>the Service Delivery Points; </li></ul></ul></ul></ul><ul><ul><ul><ul><li>any Intermediary Aggregation Level (District or Region). </li></ul></ul></ul></ul><ul><li>The DQA assesses both (1) the design; and (2) the implementation of the data-management and reporting systems. </li></ul><ul><li>The assessment covers 8 functional areas (HR, Training, Data Management Processes , etc.) </li></ul>PROTOCOL 2: Assessment of Data Management and Reporting Systems M&E Management Unit Service Delivery Sites / Organizations Intermediate Aggregation levels (eg. District, Region) <ul><li>Assess Data Management and Reporting Systems ( Part 1 ) </li></ul>
    16. 16. Objective of the Data Quality Assessment Tool The Data-Quality Assessment (DQA) Tool is designed to: <ul><li>assess the ability of data-management systems to collect manage, and report quality data . </li></ul><ul><li>verify the quality of reported data for key indicators at selected sites ; and </li></ul>The DQA Tool is not intended to assess the entire M&E system; it relates and focuses more in depth into component 10 (i.e. Supportive supervision and data auditing ) of the “Organizing Framework for a Functional National HIV M&E System” . !
    17. 17. NACA Specific Capacity Building Interventions MESST RDQA TA for DQ SI Assessment M&E Road Map 1/4ly Mentoring (SHAPMoS)
    18. 18. CB INTERVENTIONS
    19. 19. SI Assessment M&E Road Map Implementation <ul><li>Incorporation of Data Flow Findings into SHAPMoS </li></ul><ul><li>Incorporation of the Institutional Inventory List into: </li></ul><ul><ul><li>National Programmatic Database based at the NACA </li></ul></ul><ul><ul><li>Supporting primary efforts of WHO’s P-SAM Exercise </li></ul></ul><ul><li>Participation in the Launch of the National HIV M&E Strategy </li></ul>
    20. 20. SI Assessment M&E Road Map Implementation <ul><li>ToT M&E Basic Concepts </li></ul><ul><li>4 quarterly mentoring & coaching visits to Regional M&E Officers </li></ul><ul><li>Support to the NACA to produce QSCR </li></ul>
    21. 21. Institutional Capacity Strengthened – Regional M&E Officers <ul><li>Internal monitoring systems developed and in place </li></ul><ul><li>Trace and verification conducted in each Region at SDPs </li></ul><ul><li>Regional QSCRs produced </li></ul><ul><li>Information products developed for feedback to SDPs </li></ul>
    22. 22. CB INTERVENTIONS
    23. 23. Timeline: TA to Malawi 2007 2008 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 Data Quality Plan for MESST MESST: Malaria MESST: HIV Discussions with NAC: Next Steps Plan for DQ T/A M&E Plan Revision
    24. 24. RDQA Tool To Operationalize Statement of Work <ul><ul><li>Support finalization of the Draft Data Quality Protocol </li></ul></ul><ul><ul><li>Assist to roll-out the data quality protocol </li></ul></ul><ul><ul><li>Conduct data quality training(s) for key implementing agencies </li></ul></ul>Draft Services & Indicators Matrix Draft register(s) to be used by all CBO SDPs Draft clear instructions Draft DQ policies Draft data steps performed at each level of the DMRS Draft guidelines for quality controls Draft written back-up procedures Data Quality Training in 5 Zones
    25. 25. Institutional Capacity to Support Data Quality Developed <ul><li>Data quality plan written that includes guidance, reporting forms, and indicator matrix </li></ul><ul><li>Key stakeholders at national and sub-national levels oriented to – and reviewed – new data quality documents </li></ul><ul><li>NACA M&E Officers trained to train on data quality </li></ul><ul><li>Key staff in all 5 Zones have been trained by NACA on data quality </li></ul>
    26. 26. Conclusions <ul><li>NACA M&E Unit institutional capacity strengthened at National and Sub-National levels </li></ul><ul><li>Data reporting more timely, with increasingly accurate, reliable and complete data </li></ul><ul><li>Foundation for effective data use laid </li></ul><ul><li>Documented systems establish institutional memory </li></ul>
    27. 27. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) through Cooperative Agreement GPO-A-00-03-00003-00 and is implemented by the Carolina Population Center at the University of North Carolina in partnership with Futures Group, John Snow, Inc., Management Sciences for Health, ORC Macro International, and Tulane University. Visit us online at http://www.cpc.unc.edu/measure.

    ×