The document provides information on a M&E training for the Global Fund's Round 9. It discusses key topics including:
- What the Global Fund is and its principles of international health financing and performance-based funding.
- The importance of M&E in grant management and the basic elements of an M&E system including monitoring, evaluation, indicators, and the M&E process.
- Details of Myanmar's Performance Framework for its HIV grant, including objectives, indicators, targets, and periods covered.
- Additional M&E topics like the M&E plan, data collection and management, evaluation, strategic information, capacity building, and coordination.
Collecting the PEPFAR OVC MER Essential Survey Indicators: Frequently Asked Q...MEASURE Evaluation
Gretchen Bachman and Christine Fu (USAID); Lisa Parker, Jenifer Chapman, Lisa Marie Albert, Walter Obiero, and Susan Settergren from MEASURE Evaluation. January 2017 Webinar.
HIV Index Testing: The USAID DISCOVER-Health Project Experience in Zambia JSI
This was presented by Kalasa Mwansa during the Index Testing & Partner Notification for HIV Epidemic Control webinar on April 11th, 2019. The USAID DISCOVER- Health Project Experience aims to increase the use of high quality, integrated health services in specific target groups, and to provide integrated health products and services in a sustainable manner. In addition, it aims to contribute to HIV epidemic control and provides HIV index testing at every ART site.
OVC_HIVSTAT and Linkages to Care for Strengthened Collection, Analysis, and U...MEASURE Evaluation
This webinar focused on explaining the HIV Risk Assessment cascade and how it is related to OVC_HIVSTAT disaggregates. The presenters also provided guidance for how OVC_HIVSTAT data can be analyzed to enhance program outcomes.
Collecting the PEPFAR OVC MER Essential Survey Indicators: Frequently Asked Q...MEASURE Evaluation
Gretchen Bachman and Christine Fu (USAID); Lisa Parker, Jenifer Chapman, Lisa Marie Albert, Walter Obiero, and Susan Settergren from MEASURE Evaluation. January 2017 Webinar.
HIV Index Testing: The USAID DISCOVER-Health Project Experience in Zambia JSI
This was presented by Kalasa Mwansa during the Index Testing & Partner Notification for HIV Epidemic Control webinar on April 11th, 2019. The USAID DISCOVER- Health Project Experience aims to increase the use of high quality, integrated health services in specific target groups, and to provide integrated health products and services in a sustainable manner. In addition, it aims to contribute to HIV epidemic control and provides HIV index testing at every ART site.
OVC_HIVSTAT and Linkages to Care for Strengthened Collection, Analysis, and U...MEASURE Evaluation
This webinar focused on explaining the HIV Risk Assessment cascade and how it is related to OVC_HIVSTAT disaggregates. The presenters also provided guidance for how OVC_HIVSTAT data can be analyzed to enhance program outcomes.
Lessons learned in using process tracing for evaluationMEASURE Evaluation
Access the recording for this Data for Impact (D4I) webinar at https://www.data4impactproject.org/lessons-learned-in-using-process-tracing-for-evaluation/
Evaluation of the Impact of a Social Support Strategy on Treatment OutcomesMEASURE Evaluation
Shared at a data dissemination and data use workshop on the results of the impact evaluation of the Strengthening Tuberculosis Control in Ukraine project. Access another presentation at https://www.slideshare.net/measureevaluation/evaluation-of-the-tbhiv-integration-strategy-on-treatment-outcomes.
What Makes a Good Performance Management Plan? A new tool for managersMEASURE Evaluation
Led by Tory M. Taylor, a monitoring and evaluation specialist with MEASURE Evaluation from Tulane University.
The webinar introduced a tool to assist project managers in conducting effective Performance Management Plan (PMP) reviews. The tool provides feedback to implementing partners and is a brief, comprehensive checklist that covers the essential elements of a comprehensive PMP.
Philadelphia Department of Public Health HIV Prevention ActivitiesOffice of HIV Planning
Coleman Terrell of the Philadelphia Department of Public Health presented on the PDPH's HIV Prevention Activities at the Philadelphia HIV Prevention Planning Group's December 2014 meeting.
The Science of Delivery: Use of Administrative Data in The HRITF PortfolioRBFHealth
A presentation by Ha Thi Hong Nguyen, delivered during "Transforming Health Systems Through Results-Based Financing," an event held during the Third Global Symposium on Health Systems Research in Cape Town on September 30, 2014. This event was hosted by the Health Results Innovation Trust Fund at The World Bank, in partnership with the PBF Community of Practice in Africa.
The Importance of Paid Search In Omnichannel CampaignsMediaPost
Highmark Health is focused on building the health care of the future through technology and a variety of media such as search, television and radio. In fact radio’s influence is more pronounced for how it drives prospects to search, which in turn leads to completed appointments. Join Erin Mcglynn, senior digital marketing analyst at Highmark Health, as she walks through how the company uses multiple channels to help increase new patient appointments.
Data for Impact hosted a one-hour webinar sharing guidance for using routine data in evaluations. More: https://www.data4impactproject.org/resources/webinars/routine-data-use-in-evaluation-practical-guidance/
Presentation is about the uniqueness of Implementation Research and Role of the Government, specially in Indian context of health programme implementation.
Representatives from the Philadelphia Department of Public Health (PDPH) presented an update on their strategic plan for sexual health at the February 2015 meeting of the Philadelphia Ryan White Part A Planning Council.
Performance-based financing presentation to the Health Financing AcceleratorReBUILD for Resilience
A presentation to the Health Financing Accelerator and the Global Fund in March 2021 on the challenges and opportunities for international agencies investing in performance-based financing and direct facility financing in fragiole and conflict-affected settings and low income settings. By Sophie Witter, Maria Bertone & Karin Diaconu of Queen Margaret University, UK.
Lessons learned in using process tracing for evaluationMEASURE Evaluation
Access the recording for this Data for Impact (D4I) webinar at https://www.data4impactproject.org/lessons-learned-in-using-process-tracing-for-evaluation/
Evaluation of the Impact of a Social Support Strategy on Treatment OutcomesMEASURE Evaluation
Shared at a data dissemination and data use workshop on the results of the impact evaluation of the Strengthening Tuberculosis Control in Ukraine project. Access another presentation at https://www.slideshare.net/measureevaluation/evaluation-of-the-tbhiv-integration-strategy-on-treatment-outcomes.
What Makes a Good Performance Management Plan? A new tool for managersMEASURE Evaluation
Led by Tory M. Taylor, a monitoring and evaluation specialist with MEASURE Evaluation from Tulane University.
The webinar introduced a tool to assist project managers in conducting effective Performance Management Plan (PMP) reviews. The tool provides feedback to implementing partners and is a brief, comprehensive checklist that covers the essential elements of a comprehensive PMP.
Philadelphia Department of Public Health HIV Prevention ActivitiesOffice of HIV Planning
Coleman Terrell of the Philadelphia Department of Public Health presented on the PDPH's HIV Prevention Activities at the Philadelphia HIV Prevention Planning Group's December 2014 meeting.
The Science of Delivery: Use of Administrative Data in The HRITF PortfolioRBFHealth
A presentation by Ha Thi Hong Nguyen, delivered during "Transforming Health Systems Through Results-Based Financing," an event held during the Third Global Symposium on Health Systems Research in Cape Town on September 30, 2014. This event was hosted by the Health Results Innovation Trust Fund at The World Bank, in partnership with the PBF Community of Practice in Africa.
The Importance of Paid Search In Omnichannel CampaignsMediaPost
Highmark Health is focused on building the health care of the future through technology and a variety of media such as search, television and radio. In fact radio’s influence is more pronounced for how it drives prospects to search, which in turn leads to completed appointments. Join Erin Mcglynn, senior digital marketing analyst at Highmark Health, as she walks through how the company uses multiple channels to help increase new patient appointments.
Data for Impact hosted a one-hour webinar sharing guidance for using routine data in evaluations. More: https://www.data4impactproject.org/resources/webinars/routine-data-use-in-evaluation-practical-guidance/
Presentation is about the uniqueness of Implementation Research and Role of the Government, specially in Indian context of health programme implementation.
Representatives from the Philadelphia Department of Public Health (PDPH) presented an update on their strategic plan for sexual health at the February 2015 meeting of the Philadelphia Ryan White Part A Planning Council.
Performance-based financing presentation to the Health Financing AcceleratorReBUILD for Resilience
A presentation to the Health Financing Accelerator and the Global Fund in March 2021 on the challenges and opportunities for international agencies investing in performance-based financing and direct facility financing in fragiole and conflict-affected settings and low income settings. By Sophie Witter, Maria Bertone & Karin Diaconu of Queen Margaret University, UK.
Good principles for building customer relationships that stand the test of ti...CharityComms
Dan Dufour, head of brand and engagement, The Good Agency; Natasha Dickinson, head of marketing and communications, RNIB
Visit the CharityComms website to view slides from our past events, see what events we have coming up and to check out what else we do.
http://www.charitycomms.org.uk
How Save the Children is using segmentation to drive value and learning. Audi...CharityComms
Annie Moreton, deputy director individual giving and legacies, Save the Children
Visit the CharityComms website to view slides from our past events, see what events we have coming up and to check out what else we do.
http://www.charitycomms.org.uk
Details benefits of monitoring and evaluation, and how institutional knowledge is built overtime, thus can used in the design, running and effectively delivering development goals.
Evelyn Torres and Sebastian Branca's update on AACO's Client Services Unit (CSU), Housing Services Program (HSP), and Quality Improvement (QI) programs
Evelyn Torres and Sebastian Branca's update on AACO's Client Services Unit (CSU), Housing Services Program (HSP), and Quality Improvement (QI) programs
A monitoring and evaluation system is needed to assess both structural and health sector components of the response to HIV in key populations. It is critical that these systems are practical, not overly complicated, and that they collect information that is current, useful and readily used
PrEP Implementation Planning for the USCHAMP Network
CHAMP Monthly Teleconference Training - PrEP's the Word: Everyone's Talking About It... But What Do We Need To Know, And Do, About Pre-Exposure Prophylaxis
hsns09:The Scottish telecare development programme:the evaluation - Sophie BealeIriss
Sophie Beale,York Health Economics Consortium,University of York.
http://php.york.ac.uk/inst/yhec/?q=contact/keycontacts
Connected Practice Symposium,Human Services in the Network Society,Changes, Challenges & Opportunities. The Institute for Advanced Studies, Glasgow 14-15 September 2009.
http://connectedpractice.iriss.org.uk/
2. What is Global Fund?
M&E training, GFATM
Increase resources to fight AIDS, TB and Malaria
to direct those resources to areas of greatest need
International Health Financing
4. Global Fund adheres to ‘Paris Declaration’ on Aid Effectiveness
1. Country Ownership
2. Alignment with National Systems and Processes
3. Harmonization with partners
4. Managing for Results
5. Mutual Accountability
Global Fund Guiding Principles
M&E training, GFATM
5. GF - Key Principles
Country ownership &leadership
Harmonization with existing systems
Coordination with technical partners
Simplicity - small set of relevant indicators
for reporting on performance
6. • Focus is on Performance
• Ongoing disbursements are linked to the
achievement of clear and measurable results
• No result No Money
• Require Sound Measurement System
Performance Based Funding (PBF)
- one of Core Principles of GF
M&E training, GFATM
7. Performance-based Funding & M&E:
A step based approach
Prove it
Raise it
Invest it
Step 1: raise it
Step 2: invest it
Step 3: prove it
Step 4: raise it
M&E training, GFATM
8. M & E
important & integral part of grant management
M&E training, GFATM
M & E:
Provides information
on program progress and effectiveness
for policy-making and advocacy
to plan future resource needs
Improves program management & decision-making
Ensures effective and efficient use of resources (benefit to the targets)
Accountable to stakeholders, including donors
11. M&E training, GFATM
What is M&E?
Collecting, Storing, Analyzing and finally Transforming Data
into Strategic Information so it can be used to make
informed decisions on program implementation .
12. Why M&E?
Proper M&E ensures most effective & efficient use of resources
M&E training, GFATM
M&E
system
Strategic
Info
Evidence
informed
decision
making
32%
48%
118%
52%
118%
15%
0%
20%
40%
60%
80%
100%
1.2 Condom 1.4 SW 2.1 MSM 5.1 ARV 5.2 CHBC 6.1 TB_ART
13. M&E training, GFATM
Monitoring: Routine tracking on a regular basis.
It looks at what are we doing
Evaluation: Episodic assessment of the change in
targeted results (for Outcome and Impact)
Surveillance is monitoring of epidemic by counting cases
and deaths
14. M&E uses INDICATORS to measure progress
• Measurable variables
• Quantitative indicators – numbers and percentage
• Qualitative indicators – descriptive
• to help assess the performance of a development actor,
• provide with enough information to make informed decisions
M&E training, GFATM
15. For Example:
• Number of FSW reached with HIV prevention
programmes.
• Percentage of FSW who used condom at last sex.
• Percentage of young people aged 15-24 who are
HIV infected.
M&E training, GFATM
16. M & E process
M&E training, GFATM
Program
Development
Data
Program based data
Pop based bio, behavior and
social data
17. 1. Assessment and Planning
• Situation analysis &
• Resource needs/analysis
• M & E plan (details in later slides)
• This planning should be incorporated in the
program design
M&E training, GFATM
Program planning
18. 2. Input (Resources)
After Assessment & Planning,
Inputs are needed in the form of
• human resources,
• funds,
• facilities,
• supplies etc.
– to perform program activities
M&E training, GFATM
19. Activities / Processes
• performed as part of program implementation
with the inputs going into the program.
e.g
– number of training workshops,
– number of treatments or
– commodities procured etc
M&E training, GFATM
20. Output / Coverage
• immediate result of the activities performed
• usually done on a routine basis
– the number of people reached with services
– coverage indicators (the proportion of people
reached by services out of those in need)
M&E training, GFATM
21. Outcome & Impact
• Outcome - the short‐term and medium‐term effect
of an intervention’s outputs, such as change in
knowledge, attitudes, beliefs and behaviors
• Impact - the long‐term, cumulative effect of
programmes/interventions over time on what they
ultimately aim to change, such as a change in HIV
incidence or AIDS‐related morbidity and mortality
M&E training, GFATM
22. Exercise in matching
Indicator description Indicator
# of condoms distributed to Most at Risk Population
# of guidelines for condom distribution under development
# of FSW reached with HIV prevention programmes
US dollar available for HIV and AIDS program
% of youth who are HIV infected
% of youth reporting consistent use of condom in the last year
# of migrants counseled and tested (VCCT)
# of eligible adults and children currently receiving ART
% of youth who have sex before the age of 15 years
% of MSM reached with HIV prevention programs
AIDS-related mortality
% of people expressing (+) attitudes towards PLWHIV
M&E training, GFATM
Output
Process
Output
Input
Impact
Outcome
Output
Output
Outcome
Coverage
Impact
Outcome
23. Mapping Goals, Objectives, SDAs and Main Activities Indicators
Goals
Objectives
Service
delivery areas
Main activities
To reduce HIV transmission and HIV
related morbidity, mortality
Reduce HIV-related risk,
vulnerability and impact among MSM
BCC and community outreach
Workshops on safer sex
for MSM
Implement peer outreach
programmes
% of MSMs who are
HIV infected
Impact indicator
Number of MSM reached by
HIV prevention intervention
% of MSM reached by
HIV prevention intervention
Outcome indicator
% MSM reporting the use of
condom the last time they had
anal sex with a male partner
Setting up DIC and support
to outreach activities
Output indicator
Coverage indicator
25. M&E training, GFATM
Performance Framework (PF) is the legal document, binding Save the
Children (PR) to the targets agreed-up at grant negotiation
•Sub-Recipients (SRs) have to report against PF targets to Save the
Children. Save will consolidate and report to the Global Fund
every 6 months
•use existing national list of indicators, data collection systems and
planned surveys whenever possible to avoid duplication
26. Performance Framework Year 1 & 2: Indicators, Targets, and Periods Covered HIV
Program Details
Country: Union of Myanmar
Disease: HIV/AIDS
Grant number: MYN-S10-G04-H
Principal Recipient: Save the Children Federation, Inc- BURNET INSTITUTE
A. Periods covered and dates for disbursement requests and progress updates (typically completed by the Secretariat during Grant negotiations process)
Period 1 Period 2 Period 3 Period 4 Period 5 Period 6 Period 7 Period 8 Period 9
Period Covered: from 1-Jan-11 1-Apr-11 1-Jul-11 1-Oct-11 1-Jan-12 1-Apr-12 1-Jul-12 1-Oct-12 1-Jan-13
Period Covered: to 31-Mar-11 30-Jun-11 30-Sep-11 31-Dec-11 31-Mar-12 30-Jun-12 30-Sep-12 31-Dec-12 31-Mar-13
Date Progress Update due (typically 45 days after end of period) 15-May-11 14-Aug-11 14-Nov-11 14-Feb-12 15-May-12 14-Aug-12 14-Nov-12 14-Feb-13 15-May-13
Disbursement Request ? (Y,N) Y Y Y Y Y Y Y Y Y
Year 1 Year 2
Audit Report Due Date: 30-Jun-12 30-Jun-13
M&E training, GFATM
27. B. Program Goal, impact and outcome indicators
Goals:
1 To reduce HIV transmission and HIV related morbidity, mortality, disability and social and economic impact
Impact
Indic.
No.
Indicator
Baseline Targets
Comments*
value Year Source Year 1
Report
due
date
Year 2
Report
due
date
Year 3 Year 4 Year 5
1 Percentage of sex workers that are HIV infected 18.40% 2008 HSS 15.2% Nov-11 14.9% Nov-12 14.6% 14.3% 14.0%
Targets are from 2009
estimates using Estimation
and Project Package (EPP)
Data will be collected through
annual HSS for MARPs
2 Percentage of MSMs that are HIV infected 28.80% 2008 HSS 28.0% Nov-11 27.0% Nov-12 26.0% 25.0% 24.0%
Targets are from 2009
estimates using EPP
Data will be collected through
annual HSS for MARPs
3 Percentage of IDUs that are HIV infected 37.50% 2008 HSS 32.0% Nov-11 30.0% Nov-12 29.0% 28.0% 26.6%
Targets are from 2009
estimates using EPP
Data will be collected through
annual HSS for MARPs
4
Percentage of adults and children with HIV known
to be on treatment 12 months after initiation of
antiretroviral therapy
80.0% 2008
NAP
Cohort
81%
2009
cohort
report
82%
2010
cohort
report
83% 84% 85%
Outcome
Indic.
No.
Indicator
Baseline Targets
Comments*
value Year Source Year 1
Report
due
date
Year 2
Report
due
date
Year 3 Year 4 Year 5
1
Percentage of female sex workers reporting the
use of a condom with their most recent client
95% 2008 BSS 95% Dec-12
BSS for sex workers is
planned for 2012 and is
implemented independently
from other BSS
2
Percentage of men reporting the use of condom
the last time they had anal sex with a male
partner
81.55% 2009 BSS 85%
Next BSS for MSMs is
planned for in 2013
3
Percentage of injecting drug users reporting the
use of sterile injecting equipment the last time
they injected
81% 2008 BSS 83% Dec-12
BSS for sex workers is
planned for 2012 and is
implemented independently
from other BSS
M&E training, GFATM
28. M&E training, GFATM
C. Program Objectives, Service Delivery Areas and Indicators
Obj.
No.
Objective description
1 Reduce HIV-related risk, vulnerability and impact among sex workers and their clients
2 Reduce HIV-related risk, vulnerability and impact among men who have sex with men
3 Reduce HIV-related risk, vulnerability and impact among drug users
4 Strengthen the prevention of HIV transmission to children from HIV positive mothers
5 Meeting the needs of people living with HIV for Comprehensive Care, Treatment and Support
6 Reduce the burden of HIV/TB co-infection
7 Strengthen programme management and administration
Obj./
Indic.
No.
Service
Delivery
Area
Indicator
Baseline (if applicable)
PR Tied to
Targets
cumulative
Y-over
program term
Y-cumulative
annually
N-not
cumulative
Baselines
included
in targets
(Y/N)
Top 10
indicat
or
Comments
Value Year Source P2 P4 P6 P8
1.2 Condom
Number of condoms
distributed free of
charge to Most at risk
populations (MSMs,
Sex Workers and
IDUs)
34
millions
2008
National
Progress
Reports
STC
0
26,770 111,538 223,076
GF and
other
donors
Y - cumulative
annually
N Top 10
2.1
BCC -
community
outreach
and
schools
Number of men who
have sex with men
reached by HIV
prevention
interventions
32,890 2008
National
Progress
Reports
STC 2,200 5,400 3,750 6,000
GF and
other
donors
Y - cumulative
annually
N Top 10
Total MSM reached for the
whole year 1 by Burnet will be
the same although the
quarterly targets have been
adjusted.
Link to database
29. Targets in PF
PF includes targets which are:
Targets cumulative
Y-over program term
Y-cumulative annually
N-not cumulative
Tied to (tied or untied)
Tied to national program
Tied to current grant (e.g. GF 9 only)
Tied to GF (e.g. GF 9 + GF 12)
Tied to GF and other donors (e.g. GF 9 + 3DF + AusAids + USAID
Top ten indicators
Top ten
Top ten equivalent
Non top ten
Target
P1 P2 P3 P4
P5 (new
year)
Quarterly (non-cum) 500 500 500 500 600
Cumulative annually 500 1000 1500 2000 600
Cum. over prog-term 500 1000 1500 2000 2600
32. Change in scale or scope of PFs
Examples of Triggers
• a change to the epidemiological pattern
• major fluctuations
• force majeure events
• change in the sources of financing
• acceleration and extension of a program
• a change in PR
M&E training, GFATM
35. M&E training, GFATM
New Grant Architecture (GFATM)
Single Stream of Funding (SSF)
Objectives in undertaking the grant architecture review:
• Simplify the funding architecture
• Contribute to improved alignment and harmonization
• Support and effectively manage growth
36. Phase 1 Phase 2 Grant
Closure
GF funding to Bangladesh malaria program under “old” grant
architecture
Round 9
Round 6 Phase 2
Grant
Closure
Key features:
• 2 different grants for each PR, with 2 different budgets, work plans, performance
frameworks, etc.
• Misalignment with in-country reporting and cycles
• Grant close-out requirements for activities still ongoing with same PR
• Multiple PU/DRs, Audit Reports, Enhancing Financial Reports to be reported at a
time
2010 2011 2012 2013 2014 2015 2016 2017
Progress Update
/ Disbursement
Phase 2 Review
NMCP
(MoF)
Grant
Closure
Phase 2
Grant
Closure
Phase 1 Phase 2 Grant
Closure
Round 9
Round 6
BRAC
37. 1st implementation period 2nd Implementation
period
GF funding to Bangladesh malaria program under “new” grant
architecture
BRAC SSF
1st implementation period 2nd Implementation
period
Key features:
• Consolidation of two grants for each PR (four grants into two)
• 1 SSF grant for each PR, with one budget, work plan, performance framework
• Reporting significantly reduced and more program-based
• More holistic periodic review with aligned timing across PRs
• Periodic review aligned across two PRs to allow for program review
2010 2011 2012 2013 2014 2015 2016 2017
Progress Update
/ Disbursement
Periodic Review
NMCP SSF
(MoF)
41. Key M & E Elements
1. M & E Plan
2. Data Collection
3. Data Management
4. Data Quality
5. Evaluation and Research
6. Data Analysis (Strategic Information)
7. Capacity Building
8. Coordination
9. People
M&E training, GFATM
43. M&E plan
• a nationally agreed document that describes
– the functioning of the national (or Global Fund
grant specific) M&E system, and
– the mechanisms to strengthen it during a
determined period of time
M&E training, GFATM
44. 1. Introduction
2. Indicator definition and measurement
3. Routine Data Collection, Analysis and Reporting
4. Evaluation, Review, Survey, Surveillance or Special Studies
5. Data Quality Assurance Mechanisms and Related Supportive Supervision
6. M & E Coordination
7. Capacity Building
8. M & E Action Plan and Budget
– Annex A – M&E Framework
– Annex B – Indicator Reference Sheet Template
– Annex C – M & E Work-Plan
M&E training, GFATM
M&E plan (PR Save the Children)
45. 1. Introduction
– consistent with the National M&E Plan
– this document has been supported through a participatory M&E
assessment (2010 MESS WS)
– accompanied by an annual costed M&E work plan describing the
strengthening measures planned to improve the M&E system
M&E training, GFATM
M&E plan (contd.)
46. 2. Indicator Definitions and Measurement
Output indicator 1.2: Number of condoms distributed FOC to most at risk populations (MARP -
MSMs, FSWs and IDUs) (Cumulative Annually/ tied to GF and other donors).
Definition: Number of condoms distributed FOC to MARPs – counting condoms distributed FOC to
FSWs, MSMs and IDUs and their partners/clients.
Measurement methods: Data will be entered into condom distribution registers when the condoms
reach the end user.
The measurement is based on condoms distributed to the end user and NOT based on the number
of condoms leaving the warehouse or the number of condoms given to outreach workers.
SRs to report on # of condoms distributed FOC to PLHIV as well as # of condoms distributed to
MARPs. The condoms distributed to PLHIV will not, however, be reported in the PU/DR ‘Actual
Result’. A comment on the condoms distributed to PLHIV is included in the PU/DR
Sources : condom distribution list of OR worker records (PE session, trainings, events, etc)
M&E training, GFATM
M&E plan (contd.)
47. 3. Routine Data Collection, Analysis and Reporting
1. Routine Data Collection and Reporting tools – Standard R&R forms..
2. Report flow and Feedback mechanism – QA
3. Infrastructure for data capturing and reporting – paper based and
computerized data database
4. Information products, timeline and target audience
• Progress Update Report to GF 6 monthly;
• publications for scientific community; briefing doc for political advocacy; brochures,
banners, press release or radio & TV for BCC
• concept notes components within funding application
5. Information dissemination strategy – data analyzed and interpreted
resulting info to ensure any action taken maximizing performance &
impact
M&E training, GFATM
M&E plan (contd.)
48. M&E training, GFATM
4. Evaluation, Reviews, Survey, Surveillance or Special studies
1. External evaluation of GFATM program – at the end of year 5
• Mid Term Evaluation…
2. Operation Research
M&E plan (contd.)
49. M&E training, GFATM
5. Data Quality Assurance and Related Supportive Supervision
1. DQA mechanism – d/r data collection, transfer, compilation, analysis
and storage to ensure accuracy, completeness at every level of
reporting chain
2. Consistency of Primary data during monitoring visit
3. RDQA – data quality & system assessment; RDQA database for FU
and action plan
4. M&E assessments/reviews, meetings
– DQA and OSDV
M&E plan (contd.)
50. M&E training, GFATM
6. M & E Coordination
• Coordination meeting within PR SC and with other PR and SRs
• TWG organized/meeting (PR SC/UNOPS; MoH; WHO; UNAIDS & SRs)
• improves the links between partners and the capacity to collect a wider
set of data in shorter time
7. Capacity Building
• HR and CB measure plan
• Data management training/WS resources & processes
• Learning about M&E @ http://www.theglobalfund.org/en/me/learning/
M&E plan (contd.)
51. M&E training, GFATM
8. M & E Action Plan and Budget
- Costed M&E Action Plan (from MESS workshop 2010)
- Budget: 5% – 10% of the grant, under M&E cost categories
M&E plan (contd.)
57. • M E S S – UNAIDS, WHO, PRs and Stakeholders
• O S D V – LFA
• D Q A – External Audit firm
• R D Q A – Program (PR SR)
M&E training, GFATM
58. Comparing the Data Quality Tools
M&E training, GFATM
MESS OSDV DQA RDQA
Objectives •M&E Assessment &
Strengthening of
country M&E
systems (i.e. Costed
Action Plan)
•MESS tool
•Data
verification;
ensure the
quality of
program results
•LFA-PR
assessment
•“Auditing” Assessment by
funding agency
•Limited input into
recommendations by
programs
•Self-assessment of data
by Program
•Flexible use by programs
for monitoring and
supervision or to prepare
for an external audit
Frequency
and timing
•Once for Grant
Signing (repeated,
as appropriate)
•Once a year for
all Grants
•Anytime during the
lifecycle of Grant(only 12-
20 Grants per year)
Routine monitoring; once
– twice / year
Implementer
Principal Recipient
(PR)
Local Fund
Agent (LFA)
External Audit
(independent; expertise in
public health)
Country staff
(PRSR/SSR)
Main Focus
•M&E Plan
•Data Mx capacities
•Data reporting
•Data verification
(reports)
•Assess system
•PHPM
•Verifying data accuracy
•Assessment of ability of
M&E system
•Reporting performances
•Data verification
•M&E system assessment
•Recommended Action
Plan
59. 1. M&E System Strengthening (MESS)
M&E training, GFATM
60. M&E System Strengthening Assessment Tools
Program
component
Available documents
HIV
Recommended to use the UNAIDS 12-component
assessment tool, developed by UNAIDS in
collaboration with partners, including the Global
Fund. It has been piloted in three countries,
validated by HIV M&E Reference Group (MERG)
and is recommended by UNAIDS for the
assessment of the HIV M&E system.
•MERG 12 Components M&E
System Strengthening Tool
•MERG 12 Components M&E
System Assessment Guidelines
Malaria/TB
The M&E self-assessment tool developed by the
Global Fund and partners enables countries to
conduct a comprehensive assessment of the M&E
system in the country.
•M&E systems strengthening tool
•M&E systems strengthening
checklists
M&E training, GFATM
The table below provides an overview of the recommended M&E systems strengthening tools
62. Key steps in MESS Assessment - Process
Complete the
check-lists
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
Identify
weaknesses
and strengths
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
Identify key
action points
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
Develop
costed M&E
action plan
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
S T A K E H O L D E R W O R K S H O P
(National Program, UN agencies, PRs, SRs)
Share the results and implement
Preparations for the workshop
63. M&E System Strengthening Assessment.
Identify M&E capacity gaps and corresponding strengthening
measures – costed action plan
Ensure that investments in M&E are based on identified
weaknesses and contribute to the strengthening of the
National M&E Systems
Enhance the quality of programmatic data to improve
program management and to support Performance-Based
Funding
65. Guiding Principle
“Ad hoc”
data-
verifications
(anytime)
Trigger Anytime - either suggested by LFAs or requested by FPMs/TLs.
Process
Regular on-site data verification is an integral part of LFA’s responsibilities
regarding verification of implementation (VoI).
LFA expected to perform OSDV when there is a perceived data-quality risk
Planned
data-
verifications
(once a year)
Trigger
Development of the programmatic performance targets in the yearly
Performance Framework
Process
The reporting period to be decided;
The LFA will be informed of the decision to initiate such verification in
advance. The PR will be informed only after submission of the Progress
Update.
LFAs expected to carry out selected OSDV for every Grant at least once a year.
66. LFA On-Site Data Verification (OSDV)
Y1 Y2 Y3 Y4 Y5
Grant
Negotiation Phase 1 Phase 2
LFA onsite
Data
Verificatio
n #1
LFA onsite Data Verification: yearly and/or ad hoc
OBJECTIVES:
Verification of the quality of programmatic data reported by programs
DESCRIPTION:
1)LFAs perform OSDV at the service delivery points, re-aggregate data from primary
records and compare re-counted numbers with results contained in summary reports up
to the national level.
2) PHPM – Pharmaceutical Health Product Management
3) Description of findings and recommendation of strengthening measures
LFA onsite
Data
Verificatio
n #2
LFA onsite
Data
Verificatio
n #5
LFA onsite
Data
Verificatio
n #4
LFA onsite
Data
Verificatio
n #3
67. Example of Bottom-up Audit Trail
Region 1
Monthly Report
SDP 1 45
SDP 2 20
TOTAL 65
Region 3
Monthly Report
SDP 4 50
SDP 5 200
TOTAL 250
Region 2
Monthly Report
SDP 4 75
TOTAL 75
NATIONAL
Monthly Report
Region 1 65
Region 2 75
TOTAL 390
Region 3 250
Service Delivery Point 1
Monthly Report
ARV Nb. 45
Primary
Record 1
Service Delivery Point 2
Monthly Report
ARV Nb. 20
Primary
Record 1
Service Delivery Point 3
Monthly Report
ARV Nb. 75
Primary
Record 1
Service Delivery Point 4
Monthly Report
ARV Nb. 50
Primary
Record 1
Service Delivery Point 5
Monthly Report
ARV Nb. 200
Primary
Record 1
ILLUSTRATION
69. Purpose of DQA
• Protocol – 1
– Assess the quality of the reporting system
• Protocol – 2
– Data verification (2-3 indicators / disease
component)
• Strengths and weaknesses related to both the
reporting system and data verification were
identified and addressed during the audit
M&E training, GFATM
70. DQA Implementation Arrangements
Implementation levels:
- Central/National M&E Unit (PR M&E unit)
- Intermediary Aggregation Levels - e.g., Regions or Districts (SR HQ)
- Service Delivery Sites (SR SDP or SSR)
Report sent directly to the Global Fund
Higher risk of grant management countries – ORM
i) Inadequate M&E and Poor Data Quality;
ii) Inadequate access and promotion of equity & human rights; and
iii) Poor Aid Effectiveness and Sustainability
71. 4. Routine Data Quality Assessment
(RDQA)
M&E training, GFATM
72. Objectives of RDQA tool
1. To assure quality of data reported and availability of source doc in the
SR project sites
• Data verification
2. To assess the M&E structure, data collection and data management
capacity of SR
• M&E system assessment
3. To identify the weakness of SR M&E system
• Recommended strengthening measures – action points – ready for
OSDV
74. Levels of monitoring and Data flow
Intermediate
aggregation level
Service delivery
Data Flow
Monitoring SR
District /
Region 1
District /
Region 2
PR
G F
M&E unit
SD 1 SD 2 SD 3 SD 4
76. RDQA processes
1. Data verification checklist
2. System assessment checklist
3. Recommendations action plan
4. Dashboard
77. Part 1 : Data verification checklist (SDP)
A. Documentation review over primary sources
B. Recounting reported results
C. Cross check reported results with other data sources (examining separate inventory)
Part 1 : Data verification checklist (Aggregate/Regional/ME unit)
Monitoring Checklist
A. Recounting reported results
B. Reporting performance
• % Availability
• % On time
• % Completeness
78. Part 2: System assessment checklist
(I) M&E structure, functions and capabilities
(II) Indicator definitions & reporting guidelines
(III) Data collection & reporting forms and tools
(IV) Data management processes
(V) Link with national reporting system
Editor's Notes
Explain the rationale for the cumulation
The reporting period will be decided by the FPM in collaboration with the M&E Team at the Global Fund;Annually – requirement for each grant. OSDV findings feed into Phase 2 decision.