SlideShare a Scribd company logo
M&E training, GFATM
December 2012
M & E training
GFATM Round 9
PR-Save the Children
What is Global Fund?
M&E training, GFATM
Increase resources to fight AIDS, TB and Malaria
to direct those resources to areas of greatest need
International Health Financing
M&E training, GFATM
Innovative approach of
International Health Financing
Global Fund adheres to ‘Paris Declaration’ on Aid Effectiveness
1. Country Ownership
2. Alignment with National Systems and Processes
3. Harmonization with partners
4. Managing for Results
5. Mutual Accountability
Global Fund Guiding Principles
M&E training, GFATM
GF - Key Principles
 Country ownership &leadership
 Harmonization with existing systems
 Coordination with technical partners
 Simplicity - small set of relevant indicators
for reporting on performance
• Focus is on Performance
• Ongoing disbursements are linked to the
achievement of clear and measurable results
• No result  No Money
• Require Sound Measurement System
Performance Based Funding (PBF)
- one of Core Principles of GF
M&E training, GFATM
Performance-based Funding & M&E:
A step based approach
Prove it
Raise it
Invest it
Step 1: raise it
Step 2: invest it
Step 3: prove it
Step 4: raise it
M&E training, GFATM
M & E
important & integral part of grant management
M&E training, GFATM
M & E:
 Provides information
 on program progress and effectiveness
 for policy-making and advocacy
 to plan future resource needs
 Improves program management & decision-making
 Ensures effective and efficient use of resources (benefit to the targets)
 Accountable to stakeholders, including donors
M&E training, GFATM
M&E basic elements
Framework
System
M&E training, GFATM
M & E Basics
M&E training, GFATM
What is M&E?
Collecting, Storing, Analyzing and finally Transforming Data
into Strategic Information so it can be used to make
informed decisions on program implementation .
Why M&E?
Proper M&E ensures most effective & efficient use of resources
M&E training, GFATM
M&E
system
Strategic
Info
Evidence
informed
decision
making
32%
48%
118%
52%
118%
15%
0%
20%
40%
60%
80%
100%
1.2 Condom 1.4 SW 2.1 MSM 5.1 ARV 5.2 CHBC 6.1 TB_ART
M&E training, GFATM
Monitoring: Routine tracking on a regular basis.
It looks at what are we doing
Evaluation: Episodic assessment of the change in
targeted results (for Outcome and Impact)
Surveillance is monitoring of epidemic by counting cases
and deaths
M&E uses INDICATORS to measure progress
• Measurable variables
• Quantitative indicators – numbers and percentage
• Qualitative indicators – descriptive
• to help assess the performance of a development actor,
• provide with enough information to make informed decisions
M&E training, GFATM
For Example:
• Number of FSW reached with HIV prevention
programmes.
• Percentage of FSW who used condom at last sex.
• Percentage of young people aged 15-24 who are
HIV infected.
M&E training, GFATM
M & E process
M&E training, GFATM
Program
Development
Data
Program based data
Pop based bio, behavior and
social data
1. Assessment and Planning
• Situation analysis &
• Resource needs/analysis
• M & E plan (details in later slides)
• This planning should be incorporated in the
program design
M&E training, GFATM
 Program planning
2. Input (Resources)
After Assessment & Planning,
Inputs are needed in the form of
• human resources,
• funds,
• facilities,
• supplies etc.
– to perform program activities
M&E training, GFATM
Activities / Processes
• performed as part of program implementation
with the inputs going into the program.
e.g
– number of training workshops,
– number of treatments or
– commodities procured etc
M&E training, GFATM
Output / Coverage
• immediate result of the activities performed
• usually done on a routine basis
– the number of people reached with services
– coverage indicators (the proportion of people
reached by services out of those in need)
M&E training, GFATM
Outcome & Impact
• Outcome - the short‐term and medium‐term effect
of an intervention’s outputs, such as change in
knowledge, attitudes, beliefs and behaviors
• Impact - the long‐term, cumulative effect of
programmes/interventions over time on what they
ultimately aim to change, such as a change in HIV
incidence or AIDS‐related morbidity and mortality
M&E training, GFATM
Exercise in matching
Indicator description Indicator
# of condoms distributed to Most at Risk Population
# of guidelines for condom distribution under development
# of FSW reached with HIV prevention programmes
US dollar available for HIV and AIDS program
% of youth who are HIV infected
% of youth reporting consistent use of condom in the last year
# of migrants counseled and tested (VCCT)
# of eligible adults and children currently receiving ART
% of youth who have sex before the age of 15 years
% of MSM reached with HIV prevention programs
AIDS-related mortality
% of people expressing (+) attitudes towards PLWHIV
M&E training, GFATM
Output
Process
Output
Input
Impact
Outcome
Output
Output
Outcome
Coverage
Impact
Outcome
Mapping Goals, Objectives, SDAs and Main Activities  Indicators
Goals
Objectives
Service
delivery areas
Main activities
To reduce HIV transmission and HIV
related morbidity, mortality
Reduce HIV-related risk,
vulnerability and impact among MSM
BCC and community outreach
Workshops on safer sex
for MSM
Implement peer outreach
programmes
% of MSMs who are
HIV infected
Impact indicator
Number of MSM reached by
HIV prevention intervention
% of MSM reached by
HIV prevention intervention
Outcome indicator
% MSM reporting the use of
condom the last time they had
anal sex with a male partner
Setting up DIC and support
to outreach activities
Output indicator
Coverage indicator
Performance Framework
M&E training, GFATM
M&E training, GFATM
Performance Framework (PF) is the legal document, binding Save the
Children (PR) to the targets agreed-up at grant negotiation
•Sub-Recipients (SRs) have to report against PF targets to Save the
Children. Save will consolidate and report to the Global Fund
every 6 months
•use existing national list of indicators, data collection systems and
planned surveys whenever possible to avoid duplication
Performance Framework Year 1 & 2: Indicators, Targets, and Periods Covered HIV
Program Details
Country: Union of Myanmar
Disease: HIV/AIDS
Grant number: MYN-S10-G04-H
Principal Recipient: Save the Children Federation, Inc- BURNET INSTITUTE
A. Periods covered and dates for disbursement requests and progress updates (typically completed by the Secretariat during Grant negotiations process)
Period 1 Period 2 Period 3 Period 4 Period 5 Period 6 Period 7 Period 8 Period 9
Period Covered: from 1-Jan-11 1-Apr-11 1-Jul-11 1-Oct-11 1-Jan-12 1-Apr-12 1-Jul-12 1-Oct-12 1-Jan-13
Period Covered: to 31-Mar-11 30-Jun-11 30-Sep-11 31-Dec-11 31-Mar-12 30-Jun-12 30-Sep-12 31-Dec-12 31-Mar-13
Date Progress Update due (typically 45 days after end of period) 15-May-11 14-Aug-11 14-Nov-11 14-Feb-12 15-May-12 14-Aug-12 14-Nov-12 14-Feb-13 15-May-13
Disbursement Request ? (Y,N) Y Y Y Y Y Y Y Y Y
Year 1 Year 2
Audit Report Due Date: 30-Jun-12 30-Jun-13
M&E training, GFATM
B. Program Goal, impact and outcome indicators
Goals:
1 To reduce HIV transmission and HIV related morbidity, mortality, disability and social and economic impact
Impact
Indic.
No.
Indicator
Baseline Targets
Comments*
value Year Source Year 1
Report
due
date
Year 2
Report
due
date
Year 3 Year 4 Year 5
1 Percentage of sex workers that are HIV infected 18.40% 2008 HSS 15.2% Nov-11 14.9% Nov-12 14.6% 14.3% 14.0%
Targets are from 2009
estimates using Estimation
and Project Package (EPP)
Data will be collected through
annual HSS for MARPs
2 Percentage of MSMs that are HIV infected 28.80% 2008 HSS 28.0% Nov-11 27.0% Nov-12 26.0% 25.0% 24.0%
Targets are from 2009
estimates using EPP
Data will be collected through
annual HSS for MARPs
3 Percentage of IDUs that are HIV infected 37.50% 2008 HSS 32.0% Nov-11 30.0% Nov-12 29.0% 28.0% 26.6%
Targets are from 2009
estimates using EPP
Data will be collected through
annual HSS for MARPs
4
Percentage of adults and children with HIV known
to be on treatment 12 months after initiation of
antiretroviral therapy
80.0% 2008
NAP
Cohort
81%
2009
cohort
report
82%
2010
cohort
report
83% 84% 85%
Outcome
Indic.
No.
Indicator
Baseline Targets
Comments*
value Year Source Year 1
Report
due
date
Year 2
Report
due
date
Year 3 Year 4 Year 5
1
Percentage of female sex workers reporting the
use of a condom with their most recent client
95% 2008 BSS 95% Dec-12
BSS for sex workers is
planned for 2012 and is
implemented independently
from other BSS
2
Percentage of men reporting the use of condom
the last time they had anal sex with a male
partner
81.55% 2009 BSS 85%
Next BSS for MSMs is
planned for in 2013
3
Percentage of injecting drug users reporting the
use of sterile injecting equipment the last time
they injected
81% 2008 BSS 83% Dec-12
BSS for sex workers is
planned for 2012 and is
implemented independently
from other BSS
M&E training, GFATM
M&E training, GFATM
C. Program Objectives, Service Delivery Areas and Indicators
Obj.
No.
Objective description
1 Reduce HIV-related risk, vulnerability and impact among sex workers and their clients
2 Reduce HIV-related risk, vulnerability and impact among men who have sex with men
3 Reduce HIV-related risk, vulnerability and impact among drug users
4 Strengthen the prevention of HIV transmission to children from HIV positive mothers
5 Meeting the needs of people living with HIV for Comprehensive Care, Treatment and Support
6 Reduce the burden of HIV/TB co-infection
7 Strengthen programme management and administration
Obj./
Indic.
No.
Service
Delivery
Area
Indicator
Baseline (if applicable)
PR Tied to
Targets
cumulative
Y-over
program term
Y-cumulative
annually
N-not
cumulative
Baselines
included
in targets
(Y/N)
Top 10
indicat
or
Comments
Value Year Source P2 P4 P6 P8
1.2 Condom
Number of condoms
distributed free of
charge to Most at risk
populations (MSMs,
Sex Workers and
IDUs)
34
millions
2008
National
Progress
Reports
STC
0
26,770 111,538 223,076
GF and
other
donors
Y - cumulative
annually
N Top 10
2.1
BCC -
community
outreach
and
schools
Number of men who
have sex with men
reached by HIV
prevention
interventions
32,890 2008
National
Progress
Reports
STC 2,200 5,400 3,750 6,000
GF and
other
donors
Y - cumulative
annually
N Top 10
Total MSM reached for the
whole year 1 by Burnet will be
the same although the
quarterly targets have been
adjusted.
Link to database
Targets in PF
 PF includes targets which are:
 Targets cumulative
 Y-over program term
 Y-cumulative annually
 N-not cumulative
 Tied to (tied or untied)
 Tied to national program
 Tied to current grant (e.g. GF 9 only)
 Tied to GF (e.g. GF 9 + GF 12)
 Tied to GF and other donors (e.g. GF 9 + 3DF + AusAids + USAID
 Top ten indicators
 Top ten
 Top ten equivalent
 Non top ten
Target
P1 P2 P3 P4
P5 (new
year)
Quarterly (non-cum) 500 500 500 500 600
Cumulative annually 500 1000 1500 2000 600
Cum. over prog-term 500 1000 1500 2000 2600
M&E training, GFATM
M&E training, GFATM
Change in scale or scope of
Performance Frameworks
When?
(Reprogramming)
Change in scale or scope of PFs
Examples of Triggers
• a change to the epidemiological pattern
• major fluctuations
• force majeure events
• change in the sources of financing
• acceleration and extension of a program
• a change in PR
M&E training, GFATM
M&E training, GFATM
Re-Programming of GF 9 in 2011
• Delay procurement
• Targets underachieved
• Indicators – TIED  UNTIED
• Modified PF – revised Budget WP
M&E training, GFATM
M&E training, GFATM
New Grant Architecture (GFATM)
Single Stream of Funding (SSF)
Objectives in undertaking the grant architecture review:
• Simplify the funding architecture
• Contribute to improved alignment and harmonization
• Support and effectively manage growth
Phase 1 Phase 2 Grant
Closure
GF funding to Bangladesh malaria program under “old” grant
architecture
Round 9
Round 6 Phase 2
Grant
Closure
Key features:
• 2 different grants for each PR, with 2 different budgets, work plans, performance
frameworks, etc.
• Misalignment with in-country reporting and cycles
• Grant close-out requirements for activities still ongoing with same PR
• Multiple PU/DRs, Audit Reports, Enhancing Financial Reports to be reported at a
time
2010 2011 2012 2013 2014 2015 2016 2017
Progress Update
/ Disbursement
Phase 2 Review
NMCP
(MoF)
Grant
Closure
Phase 2
Grant
Closure
Phase 1 Phase 2 Grant
Closure
Round 9
Round 6
BRAC
1st implementation period 2nd Implementation
period
GF funding to Bangladesh malaria program under “new” grant
architecture
BRAC SSF
1st implementation period 2nd Implementation
period
Key features:
• Consolidation of two grants for each PR (four grants into two)
• 1 SSF grant for each PR, with one budget, work plan, performance framework
• Reporting significantly reduced and more program-based
• More holistic periodic review with aligned timing across PRs
• Periodic review aligned across two PRs to allow for program review
2010 2011 2012 2013 2014 2015 2016 2017
Progress Update
/ Disbursement
Periodic Review
NMCP SSF
(MoF)
System
M&E training, GFATM
M&E training, GFATM
Key M & E Elements in
M & E System
M&E training, GFATM
Key M & E Elements in M & E System
Key M & E Elements
1. M & E Plan
2. Data Collection
3. Data Management
4. Data Quality
5. Evaluation and Research
6. Data Analysis (Strategic Information)
7. Capacity Building
8. Coordination
9. People
M&E training, GFATM
M & E Plan
M&E training, GFATM
M&E plan
• a nationally agreed document that describes
– the functioning of the national (or Global Fund
grant specific) M&E system, and
– the mechanisms to strengthen it during a
determined period of time
M&E training, GFATM
1. Introduction
2. Indicator definition and measurement
3. Routine Data Collection, Analysis and Reporting
4. Evaluation, Review, Survey, Surveillance or Special Studies
5. Data Quality Assurance Mechanisms and Related Supportive Supervision
6. M & E Coordination
7. Capacity Building
8. M & E Action Plan and Budget
– Annex A – M&E Framework
– Annex B – Indicator Reference Sheet Template
– Annex C – M & E Work-Plan
M&E training, GFATM
M&E plan (PR Save the Children)
1. Introduction
– consistent with the National M&E Plan
– this document has been supported through a participatory M&E
assessment (2010 MESS WS)
– accompanied by an annual costed M&E work plan describing the
strengthening measures planned to improve the M&E system
M&E training, GFATM
M&E plan (contd.)
2. Indicator Definitions and Measurement
Output indicator 1.2: Number of condoms distributed FOC to most at risk populations (MARP -
MSMs, FSWs and IDUs) (Cumulative Annually/ tied to GF and other donors).
Definition: Number of condoms distributed FOC to MARPs – counting condoms distributed FOC to
FSWs, MSMs and IDUs and their partners/clients.
Measurement methods: Data will be entered into condom distribution registers when the condoms
reach the end user.
The measurement is based on condoms distributed to the end user and NOT based on the number
of condoms leaving the warehouse or the number of condoms given to outreach workers.
SRs to report on # of condoms distributed FOC to PLHIV as well as # of condoms distributed to
MARPs. The condoms distributed to PLHIV will not, however, be reported in the PU/DR ‘Actual
Result’. A comment on the condoms distributed to PLHIV is included in the PU/DR
Sources : condom distribution list of OR worker records (PE session, trainings, events, etc)
M&E training, GFATM
M&E plan (contd.)
3. Routine Data Collection, Analysis and Reporting
1. Routine Data Collection and Reporting tools – Standard R&R forms..
2. Report flow and Feedback mechanism – QA
3. Infrastructure for data capturing and reporting – paper based and
computerized data  database
4. Information products, timeline and target audience
• Progress Update Report to GF 6 monthly;
• publications for scientific community; briefing doc for political advocacy; brochures,
banners, press release or radio & TV for BCC
• concept notes components within funding application
5. Information dissemination strategy – data analyzed and interpreted 
resulting info to ensure any action taken maximizing performance &
impact
M&E training, GFATM
M&E plan (contd.)
M&E training, GFATM
4. Evaluation, Reviews, Survey, Surveillance or Special studies
1. External evaluation of GFATM program – at the end of year 5
• Mid Term Evaluation…
2. Operation Research
M&E plan (contd.)
M&E training, GFATM
5. Data Quality Assurance and Related Supportive Supervision
1. DQA mechanism – d/r data collection, transfer, compilation, analysis
and storage to ensure accuracy, completeness at every level of
reporting chain
2. Consistency of Primary data during monitoring visit
3. RDQA – data quality & system assessment; RDQA database for FU
and action plan
4. M&E assessments/reviews, meetings
– DQA and OSDV
M&E plan (contd.)
M&E training, GFATM
6. M & E Coordination
• Coordination meeting within PR SC and with other PR and SRs
• TWG organized/meeting (PR SC/UNOPS; MoH; WHO; UNAIDS & SRs)
• improves the links between partners and the capacity to collect a wider
set of data in shorter time
7. Capacity Building
• HR and CB measure plan
• Data management training/WS  resources & processes
• Learning about M&E @ http://www.theglobalfund.org/en/me/learning/
M&E plan (contd.)
M&E training, GFATM
8. M & E Action Plan and Budget
- Costed M&E Action Plan (from MESS workshop 2010)
- Budget: 5% – 10% of the grant, under M&E cost categories
M&E plan (contd.)
M&E training, GFATM
M&E plan (contd.)
M&E training, GFATM
M&E plan (contd.)
M&E training, GFATM
ANNEX C- M&E Work-Plan
Data Quality Tools and Mechanisms
M&E training, GFATM
Data Quality Framework
M&E training, GFATM
R D Q A
• M E S S – UNAIDS, WHO, PRs and Stakeholders
• O S D V – LFA
• D Q A – External Audit firm
• R D Q A – Program (PR  SR)
M&E training, GFATM
Comparing the Data Quality Tools
M&E training, GFATM
MESS OSDV DQA RDQA
Objectives •M&E Assessment &
Strengthening of
country M&E
systems (i.e. Costed
Action Plan)
•MESS tool
•Data
verification;
ensure the
quality of
program results
•LFA-PR
assessment
•“Auditing” Assessment by
funding agency
•Limited input into
recommendations by
programs
•Self-assessment of data
by Program
•Flexible use by programs
for monitoring and
supervision or to prepare
for an external audit
Frequency
and timing
•Once for Grant
Signing (repeated,
as appropriate)
•Once a year for
all Grants
•Anytime during the
lifecycle of Grant(only 12-
20 Grants per year)
Routine monitoring; once
– twice / year
Implementer
Principal Recipient
(PR)
Local Fund
Agent (LFA)
External Audit
(independent; expertise in
public health)
Country staff
(PRSR/SSR)
Main Focus
•M&E Plan
•Data Mx capacities
•Data reporting
•Data verification
(reports)
•Assess system
•PHPM
•Verifying data accuracy
•Assessment of ability of
M&E system
•Reporting performances
•Data verification
•M&E system assessment
•Recommended Action
Plan
1. M&E System Strengthening (MESS)
M&E training, GFATM
M&E System Strengthening Assessment Tools
Program
component
Available documents
HIV
Recommended to use the UNAIDS 12-component
assessment tool, developed by UNAIDS in
collaboration with partners, including the Global
Fund. It has been piloted in three countries,
validated by HIV M&E Reference Group (MERG)
and is recommended by UNAIDS for the
assessment of the HIV M&E system.
•MERG 12 Components M&E
System Strengthening Tool
•MERG 12 Components M&E
System Assessment Guidelines
Malaria/TB
The M&E self-assessment tool developed by the
Global Fund and partners enables countries to
conduct a comprehensive assessment of the M&E
system in the country.
•M&E systems strengthening tool
•M&E systems strengthening
checklists
M&E training, GFATM
The table below provides an overview of the recommended M&E systems strengthening tools
M&E training, GFATM
12 Components Monitoring & Evaluation System Assessment
Key steps in MESS Assessment - Process
Complete the
check-lists
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
Identify
weaknesses
and strengths
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
Identify key
action points
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
Develop
costed M&E
action plan
For each section:
1. M&E plan
2. Data Mx Capacities
3. Data Reporting system
S T A K E H O L D E R W O R K S H O P
(National Program, UN agencies, PRs, SRs)
Share the results and implement
Preparations for the workshop
M&E System Strengthening Assessment.
 Identify M&E capacity gaps and corresponding strengthening
measures – costed action plan
 Ensure that investments in M&E are based on identified
weaknesses and contribute to the strengthening of the
National M&E Systems
 Enhance the quality of programmatic data to improve
program management and to support Performance-Based
Funding
2. On-Site Data Verification (OSDV)
M&E training, GFATM
Guiding Principle
“Ad hoc”
data-
verifications
(anytime)
Trigger Anytime - either suggested by LFAs or requested by FPMs/TLs.
Process
 Regular on-site data verification is an integral part of LFA’s responsibilities
regarding verification of implementation (VoI).
 LFA expected to perform OSDV when there is a perceived data-quality risk
Planned
data-
verifications
(once a year)
Trigger
Development of the programmatic performance targets in the yearly
Performance Framework
Process
 The reporting period to be decided;
 The LFA will be informed of the decision to initiate such verification in
advance. The PR will be informed only after submission of the Progress
Update.
LFAs expected to carry out selected OSDV for every Grant at least once a year.
LFA On-Site Data Verification (OSDV)
Y1 Y2 Y3 Y4 Y5
Grant
Negotiation Phase 1 Phase 2
LFA onsite
Data
Verificatio
n #1
LFA onsite Data Verification: yearly and/or ad hoc
OBJECTIVES:
Verification of the quality of programmatic data reported by programs
DESCRIPTION:
1)LFAs perform OSDV at the service delivery points, re-aggregate data from primary
records and compare re-counted numbers with results contained in summary reports up
to the national level.
2) PHPM – Pharmaceutical Health Product Management
3) Description of findings and recommendation of strengthening measures
LFA onsite
Data
Verificatio
n #2
LFA onsite
Data
Verificatio
n #5
LFA onsite
Data
Verificatio
n #4
LFA onsite
Data
Verificatio
n #3
Example of Bottom-up Audit Trail
Region 1
Monthly Report
SDP 1 45
SDP 2 20
TOTAL 65
Region 3
Monthly Report
SDP 4 50
SDP 5 200
TOTAL 250
Region 2
Monthly Report
SDP 4 75
TOTAL 75
NATIONAL
Monthly Report
Region 1 65
Region 2 75
TOTAL 390
Region 3 250
Service Delivery Point 1
Monthly Report
ARV Nb. 45
Primary
Record 1
Service Delivery Point 2
Monthly Report
ARV Nb. 20
Primary
Record 1
Service Delivery Point 3
Monthly Report
ARV Nb. 75
Primary
Record 1
Service Delivery Point 4
Monthly Report
ARV Nb. 50
Primary
Record 1
Service Delivery Point 5
Monthly Report
ARV Nb. 200
Primary
Record 1
ILLUSTRATION
3. Data Quality Audit (DQA)
M&E training, GFATM
Purpose of DQA
• Protocol – 1
– Assess the quality of the reporting system
• Protocol – 2
– Data verification (2-3 indicators / disease
component)
• Strengths and weaknesses related to both the
reporting system and data verification were
identified and addressed during the audit
M&E training, GFATM
DQA Implementation Arrangements
 Implementation levels:
- Central/National M&E Unit (PR M&E unit)
- Intermediary Aggregation Levels - e.g., Regions or Districts (SR HQ)
- Service Delivery Sites (SR SDP or SSR)
 Report sent directly to the Global Fund
 Higher risk of grant management countries – ORM
i) Inadequate M&E and Poor Data Quality;
ii) Inadequate access and promotion of equity & human rights; and
iii) Poor Aid Effectiveness and Sustainability
4. Routine Data Quality Assessment
(RDQA)
M&E training, GFATM
Objectives of RDQA tool
1. To assure quality of data reported and availability of source doc in the
SR project sites
• Data verification
2. To assess the M&E structure, data collection and data management
capacity of SR
• M&E system assessment
3. To identify the weakness of SR M&E system
• Recommended strengthening measures – action points – ready for
OSDV
Conceptual FW for the RDQA
Levels of monitoring and Data flow
Intermediate
aggregation level
Service delivery
Data Flow
Monitoring SR
District /
Region 1
District /
Region 2
PR
G F
M&E unit
SD 1 SD 2 SD 3 SD 4
RDQA
Monitoring checklists
1. Service Delivery Site level
2. Aggregation level (district/region)
3. M&E Unit (SR HQ) or PR level
RDQA processes
1. Data verification checklist
2. System assessment checklist
3. Recommendations action plan
4. Dashboard
Part 1 : Data verification checklist (SDP)
A. Documentation review over primary sources
B. Recounting reported results
C. Cross check reported results with other data sources (examining separate inventory)
Part 1 : Data verification checklist (Aggregate/Regional/ME unit)
Monitoring Checklist
A. Recounting reported results
B. Reporting performance
• % Availability
• % On time
• % Completeness
Part 2: System assessment checklist
(I) M&E structure, functions and capabilities
(II) Indicator definitions & reporting guidelines
(III) Data collection & reporting forms and tools
(IV) Data management processes
(V) Link with national reporting system

More Related Content

What's hot

Lessons learned in using process tracing for evaluation
Lessons learned in using process tracing for evaluationLessons learned in using process tracing for evaluation
Lessons learned in using process tracing for evaluation
MEASURE Evaluation
 
Scaling Mobile Community-Based Information Systems
Scaling Mobile Community-Based Information SystemsScaling Mobile Community-Based Information Systems
Scaling Mobile Community-Based Information Systems
MEASURE Evaluation
 
Improvement Collaboratives
Improvement CollaborativesImprovement Collaboratives
Improvement Collaborativesprimary
 
Principles of cost effectiveness webinar: Zimbabwe experience
Principles of cost effectiveness webinar: Zimbabwe experiencePrinciples of cost effectiveness webinar: Zimbabwe experience
Principles of cost effectiveness webinar: Zimbabwe experience
Carmen Figueroa
 
Evaluation of the Impact of a Social Support Strategy on Treatment Outcomes
Evaluation of the Impact of a Social Support Strategy on Treatment OutcomesEvaluation of the Impact of a Social Support Strategy on Treatment Outcomes
Evaluation of the Impact of a Social Support Strategy on Treatment Outcomes
MEASURE Evaluation
 
AWA Experts 2015 Meeting_eMTCT Best Bractices Presentation Rwanda
AWA Experts 2015 Meeting_eMTCT Best Bractices Presentation RwandaAWA Experts 2015 Meeting_eMTCT Best Bractices Presentation Rwanda
AWA Experts 2015 Meeting_eMTCT Best Bractices Presentation RwandaAIDS Watch Africa
 
What Makes a Good Performance Management Plan? A new tool for managers
 What Makes a Good Performance Management Plan? A new tool for managers What Makes a Good Performance Management Plan? A new tool for managers
What Makes a Good Performance Management Plan? A new tool for managers
MEASURE Evaluation
 
Philadelphia Department of Public Health HIV Prevention Activities
Philadelphia Department of Public Health HIV Prevention ActivitiesPhiladelphia Department of Public Health HIV Prevention Activities
Philadelphia Department of Public Health HIV Prevention Activities
Office of HIV Planning
 
The Science of Delivery: Use of Administrative Data in The HRITF Portfolio
The Science of Delivery: Use of Administrative Data in The HRITF PortfolioThe Science of Delivery: Use of Administrative Data in The HRITF Portfolio
The Science of Delivery: Use of Administrative Data in The HRITF Portfolio
RBFHealth
 
Ukes process tracing presentation
Ukes process tracing presentationUkes process tracing presentation
Ukes process tracing presentation
Itad Ltd
 
Snaphsot of ri's situation full version
Snaphsot of ri's situation full versionSnaphsot of ri's situation full version
Snaphsot of ri's situation full version
Mohammad Ismail Zubair MD. MSc
 
Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...
Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...
Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...
MEASURE Evaluation
 
STOP HIV/AIDS Pilot: Program Science and Systems Transformation
STOP HIV/AIDS Pilot: Program Science and Systems TransformationSTOP HIV/AIDS Pilot: Program Science and Systems Transformation
STOP HIV/AIDS Pilot: Program Science and Systems Transformation
amusten
 
Monitoring Scale-up of Health Practices and Interventions
Monitoring Scale-up of Health Practices and InterventionsMonitoring Scale-up of Health Practices and Interventions
Monitoring Scale-up of Health Practices and Interventions
MEASURE Evaluation
 
The Importance of Paid Search In Omnichannel Campaigns
The Importance of Paid Search In Omnichannel CampaignsThe Importance of Paid Search In Omnichannel Campaigns
The Importance of Paid Search In Omnichannel Campaigns
MediaPost
 
Routine data use in evaluation: practical guidance
Routine data use in evaluation: practical guidanceRoutine data use in evaluation: practical guidance
Routine data use in evaluation: practical guidance
MEASURE Evaluation
 
Implementatioin research
Implementatioin researchImplementatioin research
Implementatioin research
Digvijay Trivedi
 
Update on PDPH Strategic Plan – Sexual Health
Update on PDPH Strategic Plan – Sexual HealthUpdate on PDPH Strategic Plan – Sexual Health
Update on PDPH Strategic Plan – Sexual Health
Office of HIV Planning
 
Performance-based financing presentation to the Health Financing Accelerator
Performance-based financing presentation to the Health Financing AcceleratorPerformance-based financing presentation to the Health Financing Accelerator
Performance-based financing presentation to the Health Financing Accelerator
ReBUILD for Resilience
 

What's hot (20)

Lessons learned in using process tracing for evaluation
Lessons learned in using process tracing for evaluationLessons learned in using process tracing for evaluation
Lessons learned in using process tracing for evaluation
 
Scaling Mobile Community-Based Information Systems
Scaling Mobile Community-Based Information SystemsScaling Mobile Community-Based Information Systems
Scaling Mobile Community-Based Information Systems
 
Improvement Collaboratives
Improvement CollaborativesImprovement Collaboratives
Improvement Collaboratives
 
Principles of cost effectiveness webinar: Zimbabwe experience
Principles of cost effectiveness webinar: Zimbabwe experiencePrinciples of cost effectiveness webinar: Zimbabwe experience
Principles of cost effectiveness webinar: Zimbabwe experience
 
Evaluation of the Impact of a Social Support Strategy on Treatment Outcomes
Evaluation of the Impact of a Social Support Strategy on Treatment OutcomesEvaluation of the Impact of a Social Support Strategy on Treatment Outcomes
Evaluation of the Impact of a Social Support Strategy on Treatment Outcomes
 
AWA Experts 2015 Meeting_eMTCT Best Bractices Presentation Rwanda
AWA Experts 2015 Meeting_eMTCT Best Bractices Presentation RwandaAWA Experts 2015 Meeting_eMTCT Best Bractices Presentation Rwanda
AWA Experts 2015 Meeting_eMTCT Best Bractices Presentation Rwanda
 
What Makes a Good Performance Management Plan? A new tool for managers
 What Makes a Good Performance Management Plan? A new tool for managers What Makes a Good Performance Management Plan? A new tool for managers
What Makes a Good Performance Management Plan? A new tool for managers
 
Philadelphia Department of Public Health HIV Prevention Activities
Philadelphia Department of Public Health HIV Prevention ActivitiesPhiladelphia Department of Public Health HIV Prevention Activities
Philadelphia Department of Public Health HIV Prevention Activities
 
The Science of Delivery: Use of Administrative Data in The HRITF Portfolio
The Science of Delivery: Use of Administrative Data in The HRITF PortfolioThe Science of Delivery: Use of Administrative Data in The HRITF Portfolio
The Science of Delivery: Use of Administrative Data in The HRITF Portfolio
 
Ukes process tracing presentation
Ukes process tracing presentationUkes process tracing presentation
Ukes process tracing presentation
 
Snaphsot of ri's situation full version
Snaphsot of ri's situation full versionSnaphsot of ri's situation full version
Snaphsot of ri's situation full version
 
Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...
Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...
Demystifying Disaggregated Data: Factors that Affect Collection and Use of Se...
 
STOP HIV/AIDS Pilot: Program Science and Systems Transformation
STOP HIV/AIDS Pilot: Program Science and Systems TransformationSTOP HIV/AIDS Pilot: Program Science and Systems Transformation
STOP HIV/AIDS Pilot: Program Science and Systems Transformation
 
Monitoring Scale-up of Health Practices and Interventions
Monitoring Scale-up of Health Practices and InterventionsMonitoring Scale-up of Health Practices and Interventions
Monitoring Scale-up of Health Practices and Interventions
 
The Importance of Paid Search In Omnichannel Campaigns
The Importance of Paid Search In Omnichannel CampaignsThe Importance of Paid Search In Omnichannel Campaigns
The Importance of Paid Search In Omnichannel Campaigns
 
Routine data use in evaluation: practical guidance
Routine data use in evaluation: practical guidanceRoutine data use in evaluation: practical guidance
Routine data use in evaluation: practical guidance
 
Implementatioin research
Implementatioin researchImplementatioin research
Implementatioin research
 
APHA_SciMetrika_42x72
APHA_SciMetrika_42x72APHA_SciMetrika_42x72
APHA_SciMetrika_42x72
 
Update on PDPH Strategic Plan – Sexual Health
Update on PDPH Strategic Plan – Sexual HealthUpdate on PDPH Strategic Plan – Sexual Health
Update on PDPH Strategic Plan – Sexual Health
 
Performance-based financing presentation to the Health Financing Accelerator
Performance-based financing presentation to the Health Financing AcceleratorPerformance-based financing presentation to the Health Financing Accelerator
Performance-based financing presentation to the Health Financing Accelerator
 

Viewers also liked

Campaign proposal
Campaign proposalCampaign proposal
Campaign proposal
robdoom
 
Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...
Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...
Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...GOOD Agency
 
March of Dimes Fundraising Plan Presentation
March of Dimes Fundraising Plan PresentationMarch of Dimes Fundraising Plan Presentation
March of Dimes Fundraising Plan Presentation
Rebecca Holden
 
Good principles for building customer relationships that stand the test of ti...
Good principles for building customer relationships that stand the test of ti...Good principles for building customer relationships that stand the test of ti...
Good principles for building customer relationships that stand the test of ti...
CharityComms
 
How Save the Children is using segmentation to drive value and learning. Audi...
How Save the Children is using segmentation to drive value and learning. Audi...How Save the Children is using segmentation to drive value and learning. Audi...
How Save the Children is using segmentation to drive value and learning. Audi...
CharityComms
 
Oxfam PR Plan
Oxfam PR PlanOxfam PR Plan
Oxfam PR Plan
Lionel Ng
 

Viewers also liked (6)

Campaign proposal
Campaign proposalCampaign proposal
Campaign proposal
 
Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...
Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...
Annie Moreton, Head of Individual Giving at Save the Children: Good Bites Big...
 
March of Dimes Fundraising Plan Presentation
March of Dimes Fundraising Plan PresentationMarch of Dimes Fundraising Plan Presentation
March of Dimes Fundraising Plan Presentation
 
Good principles for building customer relationships that stand the test of ti...
Good principles for building customer relationships that stand the test of ti...Good principles for building customer relationships that stand the test of ti...
Good principles for building customer relationships that stand the test of ti...
 
How Save the Children is using segmentation to drive value and learning. Audi...
How Save the Children is using segmentation to drive value and learning. Audi...How Save the Children is using segmentation to drive value and learning. Audi...
How Save the Children is using segmentation to drive value and learning. Audi...
 
Oxfam PR Plan
Oxfam PR PlanOxfam PR Plan
Oxfam PR Plan
 

Similar to M & e training [autosaved]

Components Of M E Systems La 4
Components Of M E Systems La 4Components Of M E Systems La 4
Components Of M E Systems La 4lmwagwabi
 
8_2-Intro-to-IndicatorsFMEF.pdf
8_2-Intro-to-IndicatorsFMEF.pdf8_2-Intro-to-IndicatorsFMEF.pdf
8_2-Intro-to-IndicatorsFMEF.pdf
ssusere0ee1d
 
Benefits of M&E.PDF
Benefits of M&E.PDFBenefits of M&E.PDF
Benefits of M&E.PDF
ssuser1fdbd91
 
strategy_globalfund2023-2028-kpi_handbook_en.pdf
strategy_globalfund2023-2028-kpi_handbook_en.pdfstrategy_globalfund2023-2028-kpi_handbook_en.pdf
strategy_globalfund2023-2028-kpi_handbook_en.pdf
AnanSithivanchai1
 
Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13
Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13
Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13CORE Group
 
Nasiru i. barda data collection tools for emtct
Nasiru i. barda data collection tools for emtctNasiru i. barda data collection tools for emtct
Nasiru i. barda data collection tools for emtct
Nasiru Ibrahim Barda
 
AACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) UpdateAACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) Update
Office of HIV Planning
 
AACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) UpdateAACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) Update
Office of HIV Planning
 
NASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.ppt
NASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.pptNASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.ppt
NASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.ppt
Nasiru Ibrahim Barda
 
PrEP Implementation Planning for the US
PrEP Implementation Planning for the USPrEP Implementation Planning for the US
PrEP Implementation Planning for the US
CHAMP Network
 
Strengthening Information Systems for Community Based HIV Programs
Strengthening Information Systems for Community Based HIV ProgramsStrengthening Information Systems for Community Based HIV Programs
Strengthening Information Systems for Community Based HIV ProgramsMEASURE Evaluation
 
Triangulation.ppt
Triangulation.pptTriangulation.ppt
Triangulation.ppt
ArisamTom
 
Triangulation.ppt
Triangulation.pptTriangulation.ppt
Triangulation.ppt
ArisamTom
 
Mpumalanga HIV Data Triangulation and Use 4 Nov 2014
Mpumalanga HIV Data Triangulation and Use 4 Nov 2014Mpumalanga HIV Data Triangulation and Use 4 Nov 2014
Mpumalanga HIV Data Triangulation and Use 4 Nov 2014
aleidich
 
hsns09:The Scottish telecare development programme:the evaluation - Sophie Beale
hsns09:The Scottish telecare development programme:the evaluation - Sophie Bealehsns09:The Scottish telecare development programme:the evaluation - Sophie Beale
hsns09:The Scottish telecare development programme:the evaluation - Sophie Beale
Iriss
 
JC2763_PopulationBasedSurveys_en
JC2763_PopulationBasedSurveys_enJC2763_PopulationBasedSurveys_en
JC2763_PopulationBasedSurveys_enVelma Lopez
 
Scientific Sessions 2015: IBBS survey 2014
Scientific Sessions 2015: IBBS survey 2014Scientific Sessions 2015: IBBS survey 2014
Scientific Sessions 2015: IBBS survey 2014
Sri Lanka College of Sexual Health and HIV Medicine
 
MAST and its application in RENEWING HEALTH
MAST and its application in RENEWING HEALTHMAST and its application in RENEWING HEALTH
MAST and its application in RENEWING HEALTHAnna Kotzeva
 

Similar to M & e training [autosaved] (20)

Components Of M E Systems La 4
Components Of M E Systems La 4Components Of M E Systems La 4
Components Of M E Systems La 4
 
8_2-Intro-to-IndicatorsFMEF.pdf
8_2-Intro-to-IndicatorsFMEF.pdf8_2-Intro-to-IndicatorsFMEF.pdf
8_2-Intro-to-IndicatorsFMEF.pdf
 
Benefits of M&E.PDF
Benefits of M&E.PDFBenefits of M&E.PDF
Benefits of M&E.PDF
 
strategy_globalfund2023-2028-kpi_handbook_en.pdf
strategy_globalfund2023-2028-kpi_handbook_en.pdfstrategy_globalfund2023-2028-kpi_handbook_en.pdf
strategy_globalfund2023-2028-kpi_handbook_en.pdf
 
Mini Grant Second Quarterly Call
Mini Grant Second Quarterly Call Mini Grant Second Quarterly Call
Mini Grant Second Quarterly Call
 
Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13
Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13
Approaches to Improve Malaria Outcomes_Debra Prosnitz_4.25.13
 
Nasiru i. barda data collection tools for emtct
Nasiru i. barda data collection tools for emtctNasiru i. barda data collection tools for emtct
Nasiru i. barda data collection tools for emtct
 
AACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) UpdateAACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) Update
 
AACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) UpdateAACO's Client Services Unit (CSU) Update
AACO's Client Services Unit (CSU) Update
 
NASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.ppt
NASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.pptNASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.ppt
NASIRU I. BARDA DATA COLLECTION TOOLS FOR EMTCT.ppt
 
PrEP Implementation Planning for the US
PrEP Implementation Planning for the USPrEP Implementation Planning for the US
PrEP Implementation Planning for the US
 
Strengthening Information Systems for Community Based HIV Programs
Strengthening Information Systems for Community Based HIV ProgramsStrengthening Information Systems for Community Based HIV Programs
Strengthening Information Systems for Community Based HIV Programs
 
Triangulation.ppt
Triangulation.pptTriangulation.ppt
Triangulation.ppt
 
Triangulation.ppt
Triangulation.pptTriangulation.ppt
Triangulation.ppt
 
Mpumalanga HIV Data Triangulation and Use 4 Nov 2014
Mpumalanga HIV Data Triangulation and Use 4 Nov 2014Mpumalanga HIV Data Triangulation and Use 4 Nov 2014
Mpumalanga HIV Data Triangulation and Use 4 Nov 2014
 
ppm_information
ppm_informationppm_information
ppm_information
 
hsns09:The Scottish telecare development programme:the evaluation - Sophie Beale
hsns09:The Scottish telecare development programme:the evaluation - Sophie Bealehsns09:The Scottish telecare development programme:the evaluation - Sophie Beale
hsns09:The Scottish telecare development programme:the evaluation - Sophie Beale
 
JC2763_PopulationBasedSurveys_en
JC2763_PopulationBasedSurveys_enJC2763_PopulationBasedSurveys_en
JC2763_PopulationBasedSurveys_en
 
Scientific Sessions 2015: IBBS survey 2014
Scientific Sessions 2015: IBBS survey 2014Scientific Sessions 2015: IBBS survey 2014
Scientific Sessions 2015: IBBS survey 2014
 
MAST and its application in RENEWING HEALTH
MAST and its application in RENEWING HEALTHMAST and its application in RENEWING HEALTH
MAST and its application in RENEWING HEALTH
 

M & e training [autosaved]

  • 1. M&E training, GFATM December 2012 M & E training GFATM Round 9 PR-Save the Children
  • 2. What is Global Fund? M&E training, GFATM Increase resources to fight AIDS, TB and Malaria to direct those resources to areas of greatest need International Health Financing
  • 3. M&E training, GFATM Innovative approach of International Health Financing
  • 4. Global Fund adheres to ‘Paris Declaration’ on Aid Effectiveness 1. Country Ownership 2. Alignment with National Systems and Processes 3. Harmonization with partners 4. Managing for Results 5. Mutual Accountability Global Fund Guiding Principles M&E training, GFATM
  • 5. GF - Key Principles  Country ownership &leadership  Harmonization with existing systems  Coordination with technical partners  Simplicity - small set of relevant indicators for reporting on performance
  • 6. • Focus is on Performance • Ongoing disbursements are linked to the achievement of clear and measurable results • No result  No Money • Require Sound Measurement System Performance Based Funding (PBF) - one of Core Principles of GF M&E training, GFATM
  • 7. Performance-based Funding & M&E: A step based approach Prove it Raise it Invest it Step 1: raise it Step 2: invest it Step 3: prove it Step 4: raise it M&E training, GFATM
  • 8. M & E important & integral part of grant management M&E training, GFATM M & E:  Provides information  on program progress and effectiveness  for policy-making and advocacy  to plan future resource needs  Improves program management & decision-making  Ensures effective and efficient use of resources (benefit to the targets)  Accountable to stakeholders, including donors
  • 9. M&E training, GFATM M&E basic elements Framework System
  • 10. M&E training, GFATM M & E Basics
  • 11. M&E training, GFATM What is M&E? Collecting, Storing, Analyzing and finally Transforming Data into Strategic Information so it can be used to make informed decisions on program implementation .
  • 12. Why M&E? Proper M&E ensures most effective & efficient use of resources M&E training, GFATM M&E system Strategic Info Evidence informed decision making 32% 48% 118% 52% 118% 15% 0% 20% 40% 60% 80% 100% 1.2 Condom 1.4 SW 2.1 MSM 5.1 ARV 5.2 CHBC 6.1 TB_ART
  • 13. M&E training, GFATM Monitoring: Routine tracking on a regular basis. It looks at what are we doing Evaluation: Episodic assessment of the change in targeted results (for Outcome and Impact) Surveillance is monitoring of epidemic by counting cases and deaths
  • 14. M&E uses INDICATORS to measure progress • Measurable variables • Quantitative indicators – numbers and percentage • Qualitative indicators – descriptive • to help assess the performance of a development actor, • provide with enough information to make informed decisions M&E training, GFATM
  • 15. For Example: • Number of FSW reached with HIV prevention programmes. • Percentage of FSW who used condom at last sex. • Percentage of young people aged 15-24 who are HIV infected. M&E training, GFATM
  • 16. M & E process M&E training, GFATM Program Development Data Program based data Pop based bio, behavior and social data
  • 17. 1. Assessment and Planning • Situation analysis & • Resource needs/analysis • M & E plan (details in later slides) • This planning should be incorporated in the program design M&E training, GFATM  Program planning
  • 18. 2. Input (Resources) After Assessment & Planning, Inputs are needed in the form of • human resources, • funds, • facilities, • supplies etc. – to perform program activities M&E training, GFATM
  • 19. Activities / Processes • performed as part of program implementation with the inputs going into the program. e.g – number of training workshops, – number of treatments or – commodities procured etc M&E training, GFATM
  • 20. Output / Coverage • immediate result of the activities performed • usually done on a routine basis – the number of people reached with services – coverage indicators (the proportion of people reached by services out of those in need) M&E training, GFATM
  • 21. Outcome & Impact • Outcome - the short‐term and medium‐term effect of an intervention’s outputs, such as change in knowledge, attitudes, beliefs and behaviors • Impact - the long‐term, cumulative effect of programmes/interventions over time on what they ultimately aim to change, such as a change in HIV incidence or AIDS‐related morbidity and mortality M&E training, GFATM
  • 22. Exercise in matching Indicator description Indicator # of condoms distributed to Most at Risk Population # of guidelines for condom distribution under development # of FSW reached with HIV prevention programmes US dollar available for HIV and AIDS program % of youth who are HIV infected % of youth reporting consistent use of condom in the last year # of migrants counseled and tested (VCCT) # of eligible adults and children currently receiving ART % of youth who have sex before the age of 15 years % of MSM reached with HIV prevention programs AIDS-related mortality % of people expressing (+) attitudes towards PLWHIV M&E training, GFATM Output Process Output Input Impact Outcome Output Output Outcome Coverage Impact Outcome
  • 23. Mapping Goals, Objectives, SDAs and Main Activities  Indicators Goals Objectives Service delivery areas Main activities To reduce HIV transmission and HIV related morbidity, mortality Reduce HIV-related risk, vulnerability and impact among MSM BCC and community outreach Workshops on safer sex for MSM Implement peer outreach programmes % of MSMs who are HIV infected Impact indicator Number of MSM reached by HIV prevention intervention % of MSM reached by HIV prevention intervention Outcome indicator % MSM reporting the use of condom the last time they had anal sex with a male partner Setting up DIC and support to outreach activities Output indicator Coverage indicator
  • 25. M&E training, GFATM Performance Framework (PF) is the legal document, binding Save the Children (PR) to the targets agreed-up at grant negotiation •Sub-Recipients (SRs) have to report against PF targets to Save the Children. Save will consolidate and report to the Global Fund every 6 months •use existing national list of indicators, data collection systems and planned surveys whenever possible to avoid duplication
  • 26. Performance Framework Year 1 & 2: Indicators, Targets, and Periods Covered HIV Program Details Country: Union of Myanmar Disease: HIV/AIDS Grant number: MYN-S10-G04-H Principal Recipient: Save the Children Federation, Inc- BURNET INSTITUTE A. Periods covered and dates for disbursement requests and progress updates (typically completed by the Secretariat during Grant negotiations process) Period 1 Period 2 Period 3 Period 4 Period 5 Period 6 Period 7 Period 8 Period 9 Period Covered: from 1-Jan-11 1-Apr-11 1-Jul-11 1-Oct-11 1-Jan-12 1-Apr-12 1-Jul-12 1-Oct-12 1-Jan-13 Period Covered: to 31-Mar-11 30-Jun-11 30-Sep-11 31-Dec-11 31-Mar-12 30-Jun-12 30-Sep-12 31-Dec-12 31-Mar-13 Date Progress Update due (typically 45 days after end of period) 15-May-11 14-Aug-11 14-Nov-11 14-Feb-12 15-May-12 14-Aug-12 14-Nov-12 14-Feb-13 15-May-13 Disbursement Request ? (Y,N) Y Y Y Y Y Y Y Y Y Year 1 Year 2 Audit Report Due Date: 30-Jun-12 30-Jun-13 M&E training, GFATM
  • 27. B. Program Goal, impact and outcome indicators Goals: 1 To reduce HIV transmission and HIV related morbidity, mortality, disability and social and economic impact Impact Indic. No. Indicator Baseline Targets Comments* value Year Source Year 1 Report due date Year 2 Report due date Year 3 Year 4 Year 5 1 Percentage of sex workers that are HIV infected 18.40% 2008 HSS 15.2% Nov-11 14.9% Nov-12 14.6% 14.3% 14.0% Targets are from 2009 estimates using Estimation and Project Package (EPP) Data will be collected through annual HSS for MARPs 2 Percentage of MSMs that are HIV infected 28.80% 2008 HSS 28.0% Nov-11 27.0% Nov-12 26.0% 25.0% 24.0% Targets are from 2009 estimates using EPP Data will be collected through annual HSS for MARPs 3 Percentage of IDUs that are HIV infected 37.50% 2008 HSS 32.0% Nov-11 30.0% Nov-12 29.0% 28.0% 26.6% Targets are from 2009 estimates using EPP Data will be collected through annual HSS for MARPs 4 Percentage of adults and children with HIV known to be on treatment 12 months after initiation of antiretroviral therapy 80.0% 2008 NAP Cohort 81% 2009 cohort report 82% 2010 cohort report 83% 84% 85% Outcome Indic. No. Indicator Baseline Targets Comments* value Year Source Year 1 Report due date Year 2 Report due date Year 3 Year 4 Year 5 1 Percentage of female sex workers reporting the use of a condom with their most recent client 95% 2008 BSS 95% Dec-12 BSS for sex workers is planned for 2012 and is implemented independently from other BSS 2 Percentage of men reporting the use of condom the last time they had anal sex with a male partner 81.55% 2009 BSS 85% Next BSS for MSMs is planned for in 2013 3 Percentage of injecting drug users reporting the use of sterile injecting equipment the last time they injected 81% 2008 BSS 83% Dec-12 BSS for sex workers is planned for 2012 and is implemented independently from other BSS M&E training, GFATM
  • 28. M&E training, GFATM C. Program Objectives, Service Delivery Areas and Indicators Obj. No. Objective description 1 Reduce HIV-related risk, vulnerability and impact among sex workers and their clients 2 Reduce HIV-related risk, vulnerability and impact among men who have sex with men 3 Reduce HIV-related risk, vulnerability and impact among drug users 4 Strengthen the prevention of HIV transmission to children from HIV positive mothers 5 Meeting the needs of people living with HIV for Comprehensive Care, Treatment and Support 6 Reduce the burden of HIV/TB co-infection 7 Strengthen programme management and administration Obj./ Indic. No. Service Delivery Area Indicator Baseline (if applicable) PR Tied to Targets cumulative Y-over program term Y-cumulative annually N-not cumulative Baselines included in targets (Y/N) Top 10 indicat or Comments Value Year Source P2 P4 P6 P8 1.2 Condom Number of condoms distributed free of charge to Most at risk populations (MSMs, Sex Workers and IDUs) 34 millions 2008 National Progress Reports STC 0 26,770 111,538 223,076 GF and other donors Y - cumulative annually N Top 10 2.1 BCC - community outreach and schools Number of men who have sex with men reached by HIV prevention interventions 32,890 2008 National Progress Reports STC 2,200 5,400 3,750 6,000 GF and other donors Y - cumulative annually N Top 10 Total MSM reached for the whole year 1 by Burnet will be the same although the quarterly targets have been adjusted. Link to database
  • 29. Targets in PF  PF includes targets which are:  Targets cumulative  Y-over program term  Y-cumulative annually  N-not cumulative  Tied to (tied or untied)  Tied to national program  Tied to current grant (e.g. GF 9 only)  Tied to GF (e.g. GF 9 + GF 12)  Tied to GF and other donors (e.g. GF 9 + 3DF + AusAids + USAID  Top ten indicators  Top ten  Top ten equivalent  Non top ten Target P1 P2 P3 P4 P5 (new year) Quarterly (non-cum) 500 500 500 500 600 Cumulative annually 500 1000 1500 2000 600 Cum. over prog-term 500 1000 1500 2000 2600
  • 31. M&E training, GFATM Change in scale or scope of Performance Frameworks When? (Reprogramming)
  • 32. Change in scale or scope of PFs Examples of Triggers • a change to the epidemiological pattern • major fluctuations • force majeure events • change in the sources of financing • acceleration and extension of a program • a change in PR M&E training, GFATM
  • 34. Re-Programming of GF 9 in 2011 • Delay procurement • Targets underachieved • Indicators – TIED  UNTIED • Modified PF – revised Budget WP M&E training, GFATM
  • 35. M&E training, GFATM New Grant Architecture (GFATM) Single Stream of Funding (SSF) Objectives in undertaking the grant architecture review: • Simplify the funding architecture • Contribute to improved alignment and harmonization • Support and effectively manage growth
  • 36. Phase 1 Phase 2 Grant Closure GF funding to Bangladesh malaria program under “old” grant architecture Round 9 Round 6 Phase 2 Grant Closure Key features: • 2 different grants for each PR, with 2 different budgets, work plans, performance frameworks, etc. • Misalignment with in-country reporting and cycles • Grant close-out requirements for activities still ongoing with same PR • Multiple PU/DRs, Audit Reports, Enhancing Financial Reports to be reported at a time 2010 2011 2012 2013 2014 2015 2016 2017 Progress Update / Disbursement Phase 2 Review NMCP (MoF) Grant Closure Phase 2 Grant Closure Phase 1 Phase 2 Grant Closure Round 9 Round 6 BRAC
  • 37. 1st implementation period 2nd Implementation period GF funding to Bangladesh malaria program under “new” grant architecture BRAC SSF 1st implementation period 2nd Implementation period Key features: • Consolidation of two grants for each PR (four grants into two) • 1 SSF grant for each PR, with one budget, work plan, performance framework • Reporting significantly reduced and more program-based • More holistic periodic review with aligned timing across PRs • Periodic review aligned across two PRs to allow for program review 2010 2011 2012 2013 2014 2015 2016 2017 Progress Update / Disbursement Periodic Review NMCP SSF (MoF)
  • 39. M&E training, GFATM Key M & E Elements in M & E System
  • 40. M&E training, GFATM Key M & E Elements in M & E System
  • 41. Key M & E Elements 1. M & E Plan 2. Data Collection 3. Data Management 4. Data Quality 5. Evaluation and Research 6. Data Analysis (Strategic Information) 7. Capacity Building 8. Coordination 9. People M&E training, GFATM
  • 42. M & E Plan M&E training, GFATM
  • 43. M&E plan • a nationally agreed document that describes – the functioning of the national (or Global Fund grant specific) M&E system, and – the mechanisms to strengthen it during a determined period of time M&E training, GFATM
  • 44. 1. Introduction 2. Indicator definition and measurement 3. Routine Data Collection, Analysis and Reporting 4. Evaluation, Review, Survey, Surveillance or Special Studies 5. Data Quality Assurance Mechanisms and Related Supportive Supervision 6. M & E Coordination 7. Capacity Building 8. M & E Action Plan and Budget – Annex A – M&E Framework – Annex B – Indicator Reference Sheet Template – Annex C – M & E Work-Plan M&E training, GFATM M&E plan (PR Save the Children)
  • 45. 1. Introduction – consistent with the National M&E Plan – this document has been supported through a participatory M&E assessment (2010 MESS WS) – accompanied by an annual costed M&E work plan describing the strengthening measures planned to improve the M&E system M&E training, GFATM M&E plan (contd.)
  • 46. 2. Indicator Definitions and Measurement Output indicator 1.2: Number of condoms distributed FOC to most at risk populations (MARP - MSMs, FSWs and IDUs) (Cumulative Annually/ tied to GF and other donors). Definition: Number of condoms distributed FOC to MARPs – counting condoms distributed FOC to FSWs, MSMs and IDUs and their partners/clients. Measurement methods: Data will be entered into condom distribution registers when the condoms reach the end user. The measurement is based on condoms distributed to the end user and NOT based on the number of condoms leaving the warehouse or the number of condoms given to outreach workers. SRs to report on # of condoms distributed FOC to PLHIV as well as # of condoms distributed to MARPs. The condoms distributed to PLHIV will not, however, be reported in the PU/DR ‘Actual Result’. A comment on the condoms distributed to PLHIV is included in the PU/DR Sources : condom distribution list of OR worker records (PE session, trainings, events, etc) M&E training, GFATM M&E plan (contd.)
  • 47. 3. Routine Data Collection, Analysis and Reporting 1. Routine Data Collection and Reporting tools – Standard R&R forms.. 2. Report flow and Feedback mechanism – QA 3. Infrastructure for data capturing and reporting – paper based and computerized data  database 4. Information products, timeline and target audience • Progress Update Report to GF 6 monthly; • publications for scientific community; briefing doc for political advocacy; brochures, banners, press release or radio & TV for BCC • concept notes components within funding application 5. Information dissemination strategy – data analyzed and interpreted  resulting info to ensure any action taken maximizing performance & impact M&E training, GFATM M&E plan (contd.)
  • 48. M&E training, GFATM 4. Evaluation, Reviews, Survey, Surveillance or Special studies 1. External evaluation of GFATM program – at the end of year 5 • Mid Term Evaluation… 2. Operation Research M&E plan (contd.)
  • 49. M&E training, GFATM 5. Data Quality Assurance and Related Supportive Supervision 1. DQA mechanism – d/r data collection, transfer, compilation, analysis and storage to ensure accuracy, completeness at every level of reporting chain 2. Consistency of Primary data during monitoring visit 3. RDQA – data quality & system assessment; RDQA database for FU and action plan 4. M&E assessments/reviews, meetings – DQA and OSDV M&E plan (contd.)
  • 50. M&E training, GFATM 6. M & E Coordination • Coordination meeting within PR SC and with other PR and SRs • TWG organized/meeting (PR SC/UNOPS; MoH; WHO; UNAIDS & SRs) • improves the links between partners and the capacity to collect a wider set of data in shorter time 7. Capacity Building • HR and CB measure plan • Data management training/WS  resources & processes • Learning about M&E @ http://www.theglobalfund.org/en/me/learning/ M&E plan (contd.)
  • 51. M&E training, GFATM 8. M & E Action Plan and Budget - Costed M&E Action Plan (from MESS workshop 2010) - Budget: 5% – 10% of the grant, under M&E cost categories M&E plan (contd.)
  • 52. M&E training, GFATM M&E plan (contd.)
  • 53. M&E training, GFATM M&E plan (contd.)
  • 54. M&E training, GFATM ANNEX C- M&E Work-Plan
  • 55. Data Quality Tools and Mechanisms M&E training, GFATM
  • 56. Data Quality Framework M&E training, GFATM R D Q A
  • 57. • M E S S – UNAIDS, WHO, PRs and Stakeholders • O S D V – LFA • D Q A – External Audit firm • R D Q A – Program (PR  SR) M&E training, GFATM
  • 58. Comparing the Data Quality Tools M&E training, GFATM MESS OSDV DQA RDQA Objectives •M&E Assessment & Strengthening of country M&E systems (i.e. Costed Action Plan) •MESS tool •Data verification; ensure the quality of program results •LFA-PR assessment •“Auditing” Assessment by funding agency •Limited input into recommendations by programs •Self-assessment of data by Program •Flexible use by programs for monitoring and supervision or to prepare for an external audit Frequency and timing •Once for Grant Signing (repeated, as appropriate) •Once a year for all Grants •Anytime during the lifecycle of Grant(only 12- 20 Grants per year) Routine monitoring; once – twice / year Implementer Principal Recipient (PR) Local Fund Agent (LFA) External Audit (independent; expertise in public health) Country staff (PRSR/SSR) Main Focus •M&E Plan •Data Mx capacities •Data reporting •Data verification (reports) •Assess system •PHPM •Verifying data accuracy •Assessment of ability of M&E system •Reporting performances •Data verification •M&E system assessment •Recommended Action Plan
  • 59. 1. M&E System Strengthening (MESS) M&E training, GFATM
  • 60. M&E System Strengthening Assessment Tools Program component Available documents HIV Recommended to use the UNAIDS 12-component assessment tool, developed by UNAIDS in collaboration with partners, including the Global Fund. It has been piloted in three countries, validated by HIV M&E Reference Group (MERG) and is recommended by UNAIDS for the assessment of the HIV M&E system. •MERG 12 Components M&E System Strengthening Tool •MERG 12 Components M&E System Assessment Guidelines Malaria/TB The M&E self-assessment tool developed by the Global Fund and partners enables countries to conduct a comprehensive assessment of the M&E system in the country. •M&E systems strengthening tool •M&E systems strengthening checklists M&E training, GFATM The table below provides an overview of the recommended M&E systems strengthening tools
  • 61. M&E training, GFATM 12 Components Monitoring & Evaluation System Assessment
  • 62. Key steps in MESS Assessment - Process Complete the check-lists For each section: 1. M&E plan 2. Data Mx Capacities 3. Data Reporting system Identify weaknesses and strengths For each section: 1. M&E plan 2. Data Mx Capacities 3. Data Reporting system Identify key action points For each section: 1. M&E plan 2. Data Mx Capacities 3. Data Reporting system Develop costed M&E action plan For each section: 1. M&E plan 2. Data Mx Capacities 3. Data Reporting system S T A K E H O L D E R W O R K S H O P (National Program, UN agencies, PRs, SRs) Share the results and implement Preparations for the workshop
  • 63. M&E System Strengthening Assessment.  Identify M&E capacity gaps and corresponding strengthening measures – costed action plan  Ensure that investments in M&E are based on identified weaknesses and contribute to the strengthening of the National M&E Systems  Enhance the quality of programmatic data to improve program management and to support Performance-Based Funding
  • 64. 2. On-Site Data Verification (OSDV) M&E training, GFATM
  • 65. Guiding Principle “Ad hoc” data- verifications (anytime) Trigger Anytime - either suggested by LFAs or requested by FPMs/TLs. Process  Regular on-site data verification is an integral part of LFA’s responsibilities regarding verification of implementation (VoI).  LFA expected to perform OSDV when there is a perceived data-quality risk Planned data- verifications (once a year) Trigger Development of the programmatic performance targets in the yearly Performance Framework Process  The reporting period to be decided;  The LFA will be informed of the decision to initiate such verification in advance. The PR will be informed only after submission of the Progress Update. LFAs expected to carry out selected OSDV for every Grant at least once a year.
  • 66. LFA On-Site Data Verification (OSDV) Y1 Y2 Y3 Y4 Y5 Grant Negotiation Phase 1 Phase 2 LFA onsite Data Verificatio n #1 LFA onsite Data Verification: yearly and/or ad hoc OBJECTIVES: Verification of the quality of programmatic data reported by programs DESCRIPTION: 1)LFAs perform OSDV at the service delivery points, re-aggregate data from primary records and compare re-counted numbers with results contained in summary reports up to the national level. 2) PHPM – Pharmaceutical Health Product Management 3) Description of findings and recommendation of strengthening measures LFA onsite Data Verificatio n #2 LFA onsite Data Verificatio n #5 LFA onsite Data Verificatio n #4 LFA onsite Data Verificatio n #3
  • 67. Example of Bottom-up Audit Trail Region 1 Monthly Report SDP 1 45 SDP 2 20 TOTAL 65 Region 3 Monthly Report SDP 4 50 SDP 5 200 TOTAL 250 Region 2 Monthly Report SDP 4 75 TOTAL 75 NATIONAL Monthly Report Region 1 65 Region 2 75 TOTAL 390 Region 3 250 Service Delivery Point 1 Monthly Report ARV Nb. 45 Primary Record 1 Service Delivery Point 2 Monthly Report ARV Nb. 20 Primary Record 1 Service Delivery Point 3 Monthly Report ARV Nb. 75 Primary Record 1 Service Delivery Point 4 Monthly Report ARV Nb. 50 Primary Record 1 Service Delivery Point 5 Monthly Report ARV Nb. 200 Primary Record 1 ILLUSTRATION
  • 68. 3. Data Quality Audit (DQA) M&E training, GFATM
  • 69. Purpose of DQA • Protocol – 1 – Assess the quality of the reporting system • Protocol – 2 – Data verification (2-3 indicators / disease component) • Strengths and weaknesses related to both the reporting system and data verification were identified and addressed during the audit M&E training, GFATM
  • 70. DQA Implementation Arrangements  Implementation levels: - Central/National M&E Unit (PR M&E unit) - Intermediary Aggregation Levels - e.g., Regions or Districts (SR HQ) - Service Delivery Sites (SR SDP or SSR)  Report sent directly to the Global Fund  Higher risk of grant management countries – ORM i) Inadequate M&E and Poor Data Quality; ii) Inadequate access and promotion of equity & human rights; and iii) Poor Aid Effectiveness and Sustainability
  • 71. 4. Routine Data Quality Assessment (RDQA) M&E training, GFATM
  • 72. Objectives of RDQA tool 1. To assure quality of data reported and availability of source doc in the SR project sites • Data verification 2. To assess the M&E structure, data collection and data management capacity of SR • M&E system assessment 3. To identify the weakness of SR M&E system • Recommended strengthening measures – action points – ready for OSDV
  • 73. Conceptual FW for the RDQA
  • 74. Levels of monitoring and Data flow Intermediate aggregation level Service delivery Data Flow Monitoring SR District / Region 1 District / Region 2 PR G F M&E unit SD 1 SD 2 SD 3 SD 4
  • 75. RDQA Monitoring checklists 1. Service Delivery Site level 2. Aggregation level (district/region) 3. M&E Unit (SR HQ) or PR level
  • 76. RDQA processes 1. Data verification checklist 2. System assessment checklist 3. Recommendations action plan 4. Dashboard
  • 77. Part 1 : Data verification checklist (SDP) A. Documentation review over primary sources B. Recounting reported results C. Cross check reported results with other data sources (examining separate inventory) Part 1 : Data verification checklist (Aggregate/Regional/ME unit) Monitoring Checklist A. Recounting reported results B. Reporting performance • % Availability • % On time • % Completeness
  • 78. Part 2: System assessment checklist (I) M&E structure, functions and capabilities (II) Indicator definitions & reporting guidelines (III) Data collection & reporting forms and tools (IV) Data management processes (V) Link with national reporting system

Editor's Notes

  1. Explain the rationale for the cumulation
  2. The reporting period will be decided by the FPM in collaboration with the M&E Team at the Global Fund;Annually – requirement for each grant. OSDV findings feed into Phase 2 decision.