PHL_External_Eval-Feb2013_execsum

External Evaluation of Phase 2 of the Philippines Global Fund
Consolidated Malaria Grant: Advancing Malaria Control Towards
Elimination by 2020 (PHL-202-G09-M)
(DRAFT)
Consultants:
Dr Sean Hewitt, VBDC Consulting Ltd., UK
Email: s.e.hewitt@vbdc-consulting.com
Dr James Kelley, Vectorborne Diseases Associates, LLC, USA
Email: jkelley.vbda@gmail.com
WHO Contract PR 2013/296536-0
February 2013

EVALUATION OF GLOBAL FUND CONSOLIDATED MALARIA GRANT PHASE 2, PHILIPPINES
19 February 2013 i
Table of Contents
Executive Summary...................................................................................................................... iii

Acronyms and Abbreviations...................................................................................................... vi

1. Background................................................................................................................................ 1

1.1 Malaria in the Philippines .............................................................................................................. 1

1.2 National Malaria Programme......................................................................................................... 3

a) Administrative structure.............................................................................................................................. 3

b) Malaria control and elimination guidelines ................................................................................................. 3

1.3 Global Fund consolidated malaria grant........................................................................................ 4

a) Objectives and goals .................................................................................................................................. 4

1.4 Technical organization and support .............................................................................................. 4

2. Objectives and Methods ........................................................................................................... 5

2.1 External evaluation........................................................................................................................ 5

a) Objectives................................................................................................................................................... 5

b) Approach and methods .............................................................................................................................. 6

2.2 Roadmap for malaria control beyond 2014 ................................................................................... 7

3. Findings and recommendations .............................................................................................. 8

3.1 Impact............................................................................................................................................ 8

3.2 Diagnostics and treatment........................................................................................................... 10

a) Efficiency and effectiveness of the strategies for diagnosis and treatment.............................................. 11

b) Capacity of health workers, medical technologists and laboratories........................................................ 12

c) Quality Assurance system for diagnostic and treatment services ............................................................ 13

d) Capacity of health care providers to manage malaria .............................................................................. 14

e) Review the progress of monitoring of drug efficacy and drug quality....................................................... 15

f) Effectiveness of procurement and supply management to mitigate stock-outs ........................................ 15

3.3 Vector control .............................................................................................................................. 16

a) LLIN coverage of 80% of the at-risk population ....................................................................................... 17

b) IRS coverage of 70% of houses in target highly endemic villages........................................................... 17

c) Application of vector control measures..................................................................................................... 17

3.4 Community-based malaria control, surveillance and information systems.................................. 20

a) Status and level of integration between the various information systems................................................ 21

b) Malaria surveillance system and its support for epidemic detection and response.................................. 23

c) Project’s activities and relationship with collaborating Centres ................................................................ 24

d) Appropriateness integrated work plan and harmonization with other donors and partners ..................... 25

4. Conclusions ............................................................................................................................. 25

5. Acknowledgements................................................................................................................. 29

Annex 1 – Roadmap for Malaria Control in the Philippines Beyond 2014 ............................. 30

Annex 2 – Health Information Data Flow ................................................................................... 31

Annex 3 – PSFI Organigram ....................................................................................................... 32

Annex 4 – Multi-level Questionnaire Tool ................................................................................. 33

Annex 5 – API in Palawan Barangays........................................................................................ 36

Annex 6 – Field Visits and Schedule: Terms of Reference...................................................... 37

19 February 2013 ii
List of Figures and Tables
Figure 1: P. falciparum API per 1,000 total barangay population in the three provinces................. 1

Figure 2: Malaria cases reported in the Philippines malaria information system (PhilMIS)............. 2

Figure 3: Map of field visit sites: Palawan, Bukidnon and Rizal Provinces (yellow) ........................ 7

Figure 4: Number of P. falciparum cases by year............................................................................ 9

Figure 5: Reported cases of P. falciparum malaria in Palawan since 2006................................... 18

Figure 6: Palawan BMM showing a thin mattress.......................................................................... 19

Figure 7: PhilMIS-PIDSR case conformity..................................................................................... 22

Table 1: PHL-202-G09-M: goal level impact and outcome indicators, targets and results.............. 8

Table 2: Diagnostics and treatment: indicators, targets and results .............................................. 10

Table 3: Vector control: indicators, targets and results.................................................................. 16

Table 4: Community, surveillance and MIS: indicators, targets and results .................................. 21

19 February 2013 iii
Executive Summary
Background
Malaria is endemic in 58 of the Philippines’ 80 provinces: 40 of the 57 receive Global Fund
support. Since 2005 there has been a marked decrease of malaria morbidity and mortality.
However, malaria remains a public health concern. According to the 2011 World Malaria Report
74 million people (~80% of the population) are considered to be at risk but only 6.7 million (7%)
are considered to be at high risk. Malaria is mainly prevalent in hilly, forested and hard to reach
areas. It has been reported that transmission occurs year-round but is typically higher during the
rainy season, which runs from June to December. However, according to 2006-2012 programme
data analysed for this evaluation, the number of reported P. falciparum cases in Palawan is lowest
in December each year and generally reaches a peak in May (this apparent anomaly should be
investigated). Risk groups include upland subsistence farmers, forest workers (including miners),
indigenous people and migrant agricultural workers. There is an ongoing risk of re-introduction of
malaria in disease-free but receptive zones.
The National Malaria Programme has a decentralized health care delivery system led by the
Department of Health and implemented by ‘Local Government Units’. The delivery of anti-malarial
services is a shared responsibility between national and local governments. The Philippines has a
highly decentralized network of malaria specific technical expertise scattered through national
government, institutes, foundations and partner non-governmental organizations.
Global Fund support and the AusAID funded ‘Roll Back Malaria Project” have played key roles in
reducing the burden of malaria in the Philippines. As a result of the advances made the National
Malaria Programme has begun implementing a phased strategy for elimination, outlined in the
Malaria Medium Term Development Plan (2011-2016). By 2011 six provinces were deemed by
the Department of Health to have reached elimination status and 25 provinces were classified as
being at the pre-elimination level.
The current GF malaria grant covers the period 2010 to 2014 under the title Consolidated Malaria
Grant: Advancing Malaria Control Towards Elimination by 2020 (PHL-202-G09-M). Phase 2 of this
grant started in January 2012 and runs until the end of 2014. The sole Principal Recipient of this
Global Fund grant is the Pilipinas Shell Foundation Incorporated. Grant objectives are:
1. Strengthen malaria health care delivery among local health systems in the most at-risk areas
focusing on: a) malaria diagnosis and treatment; b) malaria prevention through integrated
vector control; and c) extensive social mobilization process.
2. Enhancing local governance including the ability to develop and implement a local plan of
action for malaria, sustained by local and external resources, run by local systems and
supported by the community.
3. Establish a standardized system of monitoring and evaluation (local level) hinged on the
developed national programme M&E designed to ensure quality and sustainability.
Grant goals are:
1. Reduce malaria morbidity rate by >70% from 2009-2015;
2. Reduce malaria mortality rate by >90% from 2009-2015;
3. Recognize five provinces as malaria-free by 2014 (baseline 22 provinces in 2007) in line with
the disease-free zones initiative of the DOH.
Objectives
The authors were contracted by the World Health Organization to conduct an external review of
Phase 2 of the PHL-202-G09-M grant. The aims of this review were to assess the project’s

19 February 2013 iv
progress towards achieving its objectives and goals, to identify any issues and to make
recommendations for improving the project’s outcomes and impact. In addition the
consultants were tasked with providing preliminary guidance on the future direction of the
broader National Malaria Programme and preparing a roadmap for developing the next national
strategic plan and securing the necessary financial and technical support for malaria control and
elimination beyond 2014.
Methods and Approach
The review was performed according to ‘Grant evaluation objectives’ set out in the consultants’
terms of reference. Interviews were conducted with a comprehensive range of stakeholders at
central level and during field visits to Palawan, Bukidnon and Rizal provinces. Progress in the
implementation of the project work-plan was verified as far as possible through qualitative
assessments at health facilities and at barangay and household levels. The distribution,
availability, condition and use of project supplies and equipment were assessed and practices of
implementers and beneficiaries were observed where possible. In addition to field visits, the
authors attended a three-day malaria elimination workshop attended by regional and provincial
malaria coordinators, cluster heads and other partners. Background documents were reviewed
and some preliminary analyses of project data were carried out.
Findings and recommendations
Global Fund support has played a critical role in ensuring high coverage with proven interventions
for malaria control in 40 of the 57 endemic provinces in the Philippines. The Global Fund’s support
for logistics, training, surveillance, supervision and monitoring and evaluation have contributed
significantly to the success of the National Malaria Programme. The unusually high level of
community involvement and the dedication of the large volunteer workforce have also contributed
very considerably.
Although the recent reduction in reported malaria burden is widely attributed to programme
activities, unusual weather in 2012 may have played a part. The review team feel that optimism
regarding programme impact should be tempered with caution for the time being. A number of
technical issues relating to various strategies need to be addressed in order to maximize the
impact and cost-effectiveness of the programme. A detailed account of these issues is presented
in section 3 of this report together with recommendations for improvement. Technical working
groups will need to be convened to discuss the various issues identified, to consider
recommendations and revise strategies as appropriate. The extremely decentralized health
system in the Philippines means that technical guidance from the programme at central level must
be explicitly clear to ensure that the management of programme activities at regional, provincial
and municipal levels is technically sound. In addition, the level of expert supervision from central
level will need to be increased significantly if technical standards of implementation are to be
raised to the level required for elimination. Quality assurance of training, IRS operations and
microscopy needs to be strengthened.
The long-term impact of the Global Fund support received so far is questionable. Some
interviewees felt that the project should be doing more to ensure effective transfer of skills and
technology. This applies particularly to procurement and supply management and to the ‘PhilMIS’
malaria information system. Although considerable funds have been allocated to building the
capacity of health staff from regional to barangay level since GF support began in 2003, the
current lack of project emphasis on capacity development appears to have resulted in a
pronounced lack of ownership of project systems at regional level and beyond and unless
addressed this is likely to undermine the sustainability of these systems post-Global Fund support.
The Principal Recipient now needs to shift its emphasis towards building capacity within the

19 February 2013 v
Department of Health and integrating project systems into national ones. This should be an
absolute priority during the next phase of Global Fund support.
There are a number of inconsistencies in the data from the PhilMIS. The Department of Health
should contract an external team to conduct a malaria indicator survey in a small number of
selected provinces to corroborate current reports of disease burden and programmatic coverage
and to provide baseline data for ‘the big push’ towards elimination.
The secretariat of the National Malaria Programme has just one part-time member of staff, the
Director. This is an extraordinary situation. At present the programme is completely reliant on a
number of dedicated individuals employed by the World Health Organization, the ‘Asian
Collaborative Training Network for Malaria’ and the ‘Pilipinas Shell Foundation Inc’. The
Department of Health urgently needs to establish a team at national level to manage the
programme during the next phase of implementation. Elimination will not be achieved through a
‘business as usual’ approach. The World Health Organization needs to take a stronger line on
lobbying the Department of Health to establish a credible National Malaria Programme at central
level.
If the Global Fund were to withdraw its support for the programme from the end of the current
grant the financial shortfall would be very significant and could completely undermine the progress
made to date as well as plans for elimination. The development of a fully costed ‘National
Strategic Plan for Malaria Control and Elimination’ based on a detailed programmatic,
geographical and financial gap analysis is now a priority in order to ensure that the National
Malaria Programme is adequately funded as it moves towards elimination.
Next steps
If the programme is to avoid a gap in Global Fund support then the new National Strategic Plan
will need to be completed and a concept note submitted to Global Fund before the end of 2013. A
draft roadmap detailing the various activities that will be required between now and submission of
the concept note has been developed and is attached as Annex 1.

19 February 2013 vi
Acronyms and Abbreviations
ABER Annual blood examination rate
ACTMalaria Asian Collaborative Training Network for Malaria
APMEN Asia-Pacific Malaria Elimination Network
AusAID Australian Agency for International Development
ACT Artemesinin-based combination therapy
API Annual parasite incidence
BCC Behaviour change communication
BHS Barangay Health Station
BHW Barangay Health Worker
BMMC Barangay Malaria Microscopy Centre
BMM Barangay Malaria Microscopist
CHD Centre for Health Development
CHO City Health Office
CO Community Organizer
DoH Department of Health
EPI Extended Programme on Immunizations
FDA Food and Drug Administration
FETPAFI Field Epidemiology Training Program Alumni Foundation Incorporated
FHSIS Field Health Service Information Surveillance
FPA Fertilizer and Pesticide Agency
G6PD Glucose-6-phosphate dehydrogenase
GoP Government of the Philippines
GFCMG Global Fund Consolidated Malaria Grant
GIS Geographic Information System
GPR Grant Performance Report
HIS Health information system
iCCM Integrated Community Case Management
IDO Infectious Disease Office
IEC Information, education and communication
IMS Information Management System
IP Indigenous population
IRS Indoor residual insecticide spraying
ITN Insecticide treated bed net
KLM Kilusan Ligtas Malaria (Movement Against Malaria)
LFA Local fund agent
LGU Local Government Unit
LLIN Long-lasting insecticide treated bed net
ManCOM Management Committee
MASOVECOS Malaria Surveillance and Vector Control Teams
MHO Municipal Health Officer
MOP Manual of Operational Procedures
MP Malaria Programme, Philippines DOH

19 February 2013 vii
MSF Medecins Sans Frontieres
MTDP National Malaria Medium Term Development Plan, 2011-16
NEC National Epidemiology Centre
NGO Non-government organisation
NMP National Malaria Programme
NSP National strategic plan
PhilMAM Philippines Movement Against Malaria
PhilMIS Philippines Malaria Information System
PHO Provincial Health Office
PIDSR Philippines Integrated Disease Surveillance and Response system
PMC Provincial Medical Centres
PMN Philippines Malaria Network
PR Principal Recipient
PSFI Pilipinas Shell Foundation Inc.
PSM Procurement and supply management
QA Quality assurance
RBM Roll Back Malaria
RCC Rolling continuation channel
RDT Rapid diagnostic test
RDTV Rapid diagnostic test volunteer
RHO Regional Health Office
RHU Rural Health Unit
RITM Research institute for Tropical Medicine
SOP Standard operating procedure
SPEED Surveillance in Post Extreme Emergencies and Disasters
SPR Slide positivity rate
SSA Special Service Agreement
TA Technical assistance
TES Therapeutic efficacy study
ToR Terms of reference
ToT Training of trainers
TWG Technical Working Group
VHW Volunteer health worker
WHO World Health Organization
WPRO Western Pacific Regional Office

19 February 2013 1
1. Background
1.1 Malaria in the Philippines
Malaria is endemic in 57 of the country’s 80 provinces: 40 of the 57 receive Global Fund (GF) support.
Since 2005 there has been a marked decrease of malaria morbidity and mortality. The number of
cases has been reduced by more than 75% from 2005 to 2011 while the number of deaths decreased
more significantly by more than 90% over the same period.1
Reported malaria deaths declined by 99%
between 1999 and 2011, from 755 to only 10.1
However, malaria remains a public health concern.
Confirmed malaria cases fell from 23,655 in 2008 to 9,552 in 2011, giving overall respective
incidences of 26 and 10 per 100,000.
Figure 1: P. falciparum API per 1,000 total barangay population in the three provinces
visited by the review team (Palawan, Bukidnon and Rizal)
Malaria threatens approximately 12-14 million people in the country. Although 80% of the population is
at risk for malaria transmission, only 7% is considered at high risk.1
In 2010 Luzon, the largest and
northernmost island group (31 provinces), contributed nearly 58% of the nation’s malaria cases While
Mindanao, the southernmost island group (25 provinces), contributed 42% of cases, and Visayas, the
central island group (three provinces), contributed less than 1% of cases.(APMEN) Palawan,
administratively part of the Luzon island group, has consistently ranked among the top 3 provinces
having the highest reported annual parasite incidence (API). Even given the steep decrease in API
1
World Malaria Report (WMR), 2012

19 February 2013 2
from 10.3 per 1,000 in 2010 to 2.7 per 1,000 in 2012 (primarily due to reduced cases in northern
municipalities), malaria control remains a challenge in its southern municipalities and barangays
(Figure 1 and Annex 5: close-up).
In 2012 approximately 72% of cases were due to Plasmodium falciparum and 27% were due to P.
vivax (cases of P. malariae, P. ovale, and P. knowlesi as well as mixed P. falciparum/P. malariae and
a surprisingly small number of mixed P. falciparum/P. vivax cases have also been reported). The
proportion P. falciparum to P. vivax has remained relatively constant hovering around 70% with
minimal fluctuation since 2006 (Figure 2).
The primary vector of malaria in the Philippines is Anopheles flavirostris, which breeds in clear, slow-
flowing streams near foothills and forests. Secondary vectors include An. balabacensis, An. litoralis,
An. maculatus, and An. mangyanus.
Malaria is mainly prevalent in hilly, forested and hard to reach areas. It has been reported that
transmission occurs year-round but is typically higher during the rainy season, which runs from June
to December.2
However, according to the 2006-2012 PhilMIS data analysed for this evaluation, the
number of reported P. falciparum cases in Palawan is lowest in December each year and generally
reaches a peak in May (Figure 5). High-risk groups include upland subsistence farmers, forest
workers (including miners), indigenous people (IP) and migrant agricultural workers. And there is an
ongoing risk of re-introduction and outbreaks in receptive disease-free zones by mobile populations.
Figure 2: Malaria cases reported in the Philippines malaria information system (PhilMIS)
by species and year showing proportion of cases caused by P. falciparum (line)
2
Malaria Medium Term Development Plan (MTDP), 2011-2016

19 February 2013 3
1.2 National Malaria Programme
a) Administrative structure
The National Malaria Programme (NMP) has a decentralized health care delivery system led by the
Department of Health (DoH) and implemented by the Local Government Units (LGUs). A network of
private and public health facilities at various levels offer clinical care and public health services.
Tertiary health care is provided by public sector hospitals (managed either by DoH through its
Regional Health Offices (RHOs) or by provincial governments) and by private sector hospitals.
Municipal/city governments are mainly responsible for public health service delivery through Rural
Health Units (RHUs) or health centres. At the peripheral level, Barangay Health Stations (BHS) are
usually staffed by a midwife or nurse and supported by community volunteers including barangay
health workers (BHW), barangay malaria microscopists (BMMs) and Rapid Diagnostic Test Volunteers
(RDTVs). Annex 2 illustrates various levels of the Philippines health delivery system and shows data
flow from barangay to DoH.
The delivery of anti-malarial services is a shared responsibility between national and local
governments. The national government through DoH national and regional offices sets the programme
policies, standards and guidelines, and aims to: provide technical training; augment the logistics
requirements of the LGUs for anti-malaria services,; establish and operate quality assurance for
microscopy, treatment and vector control measures; design health promotion materials and other
approaches; and, conduct monitoring and evaluation. Public health facilities at the provincial,
municipal and barangay levels provide diagnostic and treatment services in all malaria endemic areas.
The same network also allows routine reporting of surveillance data and monitoring of programme
performance.
b) Malaria control and elimination guidelines
Global Fund support and the Roll Back Malaria (RBM) project (managed by the World Health
Organization and funded by AusAID) have played key roles in reducing the burden of malaria in the
Philippines. As a result of the advances made in recent years the NMP has begun implementing a
phased strategy for elimination, which is outlined in the Malaria Medium Term Development Plan
(2011-2016) (MTDP) and Manual of Operating Procedures (MOP). These documents are designed to
provide a guiding framework for elimination, covering programme policy, key strategies and
investment requirements.
Listed in the MOP are eight policy directions and seven key strategies with proposed activities for
achieving the vision of a malaria-free Philippines. The seven MOP key strategies are:
1. Stratification (barangay or sitio level): classify malaria endemic areas/zones according to
transmission history (stable, unstable, sporadic or malaria prone);3
2. Stratification-focused interventions: varying degrees of intervention (diagnosis and treatment,
vector control) according stratification;
3. Surveillance and epidemic management: emphasizes the importance of monitoring the absence
of local transmission;
4. Quality assurance (QA) system of malaria interventions: enhanced for microscopy, insecticide
resistance, vector control operations and drug efficacy;
3
Stable: continuous presence of at least one indigenous malaria case in a month for 6 months or more at any time
during the past three years; Unstable: continuous presence of at least one indigenous malaria case in a month for less
than 6 months at any time during the past three years; Sporadic: presence of at least one indigenous malaria case at
any time in the past 5 years; Malaria Prone Area: absence of indigenous malaria case for 5 past years even in the
presence of malaria vector

19 February 2013 4
5. Intensified health promotion;
6. Built local capacity to manage and sustain malaria programme;
7. Strengthened DoH technical assistance (TA) to local government units (LGUs): harmonize
assistance from DoH and external sources.
The MOP calls for stratification down to peripheral LGUs (including barangays and sitios) in order to
better track malaria cases and improve targeting and promote efficient utilization of commodities.
By 2011 six provinces were deemed by DoH to have reached elimination status and 25 provinces
were classified as being at the pre-elimination level.
1.3 Global Fund consolidated malaria grant
The Government of the Philippines (GoP) has received several grants for the control of malaria from
the GF. The current GF malaria grant, the Consolidated Malaria Grant: Advancing Malaria Control
Towards Elimination by 2020 (PHL-202-G09-M), covers the period 2010 to 2014. Phase 2 of this GF
consolidated malaria grant (GFCMG) started in January 2012 and runs until the end of 2014.
The sole GF Principal Recipient (PR) is the Pilipinas Shell Foundation Inc. (PSFI), which was
established in 1982 and registered with the Securities and Exchange Commission of the Philippines
as a corporate foundation of the Shell companies in the Philippines. Annex 3 shows the PSFI
organizational chart.
a) Objectives and goals
The objectives and targets of the GFCMG are harmonized with those of the national programme as
defined in the MTDP 2011-2016. The GFCMG objectives are:
Objective 1: Strengthen malaria health care delivery among local health systems in the most at-risk
areas focusing on: a) malaria diagnosis and treatment; b) malaria prevention through integrated vector
control; and c) extensive social mobilization process.
Objective 2: Enhancing local governance including the ability to develop and implement a local plan of
action for malaria, sustained by local and external resources, run by local systems and supported by
the community.
Objective 3: Establish a standardized system of monitoring and evaluation (local level) hinged on the
developed national programme M&E designed to ensure quality and sustainability.
The GFCMG goals are:
1. Reduce malaria morbidity rate by >70% from 2009-2015;
2. Reduce malaria mortality rate by >90% from 2009-2015;
3. Recognize five provinces as malaria-free by 2014 (baseline 22 provinces in 2007) in line with the
disease-free zones initiative of the DoH.
Goal level impact and outcome indicators, targets and results are shown in Table 1, section 3 below.
Table 2, Table 3, and Table 4 in the findings section below list objective level indicators, targets and
results.
1.4 Technical organization and support
The Philippines has a highly decentralized network of malaria specific technical expertise scattered
through national government, institutes, foundations and partner non-governmental organizations
(NGOs). It also receives technical support from the World Health Organization (WHO) in the form of

19 February 2013 5
one international advisor and three Special Service Agreement staff (SSAs). Malaria experts from the
DoH, Infectious Disease Office (IDO), PSFI, the Asian Collaborative Training Network for Malaria
(ACTMalaria), WHO and others form a Technical Working Group (TWG), which provides technical
advice and advocacy for the GFCMG and the NMP. The TWG reports to the Management Committee
(ManCOM) which helps the DoH meet its QA obligations for various malaria initiatives and projects.
Also, the Philippines Malaria Network (PMN) supports the Philippines Movement Against Malaria
(PhilMAM), which aims to consolidate and strengthen efforts and resources through sustained public-
private collaborations and partnerships to reduce the burden of malaria.
2. Objectives and Methods
Since 2003 WHO has supported a number of external evaluations of the various GF malaria grants in
the Philippines. At the end of 2003, Dr Kevin Palmer and Dr Eva Maria Christophel of WHO’s Western
Pacific Regional Office (WPRO) evaluated early progress of the Round 2 grant. In 2007, year 4 of GF
grant implementation, Dr Christophel conducted a second evaluation assessing project outcomes and
effectiveness. Early 2011, Dr Raman Velayudhan, Dr Rob Condon and Ms Eireen Villa evaluated the
current GFMCG, one year after implementation of Phase 1 began.
2.1 External evaluation
a) Objectives
This second external evaluation of the GFCMG (one year after Phase 2 began) was designed to
assess progress towards attaining grant goals and objectives and provide recommendations to guide
grant implementation during the remaining two years of Phase 2.
The authors evaluated the performance of the programme in accordance with the Grant Agreement
PHL-202-G09-M objectives and with reference to the indicators and targets set out in the Performance
Framework. The GFCMG performance indicators, targets and results for each grant period to date are
tabulated in the findings section below.
In addition the evaluation team addressed the following Grant evaluation objectives and tasks as set
out in their terms of reference (ToR) (Annex 6):
Grant evaluation objective 1, the provision of universal access to quality diagnostics and
treatment services:
a) Assess the efficiency and effectiveness of the strategies for diagnosis and treatment at all levels;
b) Examine progress and proposed future developments in the capacity of health workers, medical
technologists and laboratories to support diagnosis, training and quality assurance in the project
provinces;
c) Assess progress towards strengthening of QA systems for both the diagnostic and treatment
services in project provinces;
d) Assess the capacity of health care providers to manage malaria in project provinces;
e) Review the progress of monitoring of drug efficacy and drug quality;
f) Assess the effectiveness of procurement and supply management to mitigate stock-outs of anti-
malarial commodities.
Grant evaluation objective 2, scaling up of vector control to reach at least 80% protection of
populations in malaria endemic areas:

19 February 2013 6
a) Assess whether and to what extent this project has achieved LLIN coverage of 80% of the at-
risk population by 2012 in the 40 provinces covered by the project;
b) Assess whether and to what extent this project has achieved IRS coverage of 70% of houses in
target highly endemic villages;
c) Critically appraise the application of vector control measures to guide programme
implementation and recommend improvements.
In accordance to grant evaluation objective 3, strengthening of sustainable community-based
malaria control and strengthening of malaria surveillance and information systems, the authors:
a) Review the status and the level of integration between the Philippines Malaria Information
System (PhilMIS), Philippines Integrated Disease Surveillance and Response (PIDSR) and Field
Health Service Information Surveillance (FHSIS) in the malaria surveillance, and ways in which
this project can complement and work with National Epidemiology Centre (NEC) and other DoH
activities and projects and enhance its outcomes;
b) Assess whether and to what extent this project has strengthened the malaria surveillance
system at the regional level to support epidemic detection and response, including in epidemic
prone areas;
c) Assess the project’s activities and relationship with collaborating centres in the provision of
technical support for diagnosis and entomological services.
d) Assess the need for entomological capacity building in selected regions where other vector
borne diseases are on the rise;
e) With reference the Paris Declaration on Aid Effectiveness (2005) and the Accra Agenda for
Action (2008), assess the appropriateness of the project’s integrated work plan incorporating
inputs of all partners, including how well NMP activities are harmonized with other donors and
partners.
b) Approach and methods
The evaluation consisted of a combination of desk-based review of documentation, face-to-face
interviews, site visits, and analysis of data from the PhilMIS (the assessment schedule is presented in
annex 6). The authors sought views and opinions from staff at DoH and PSFI, from technical partners
at central level and from government leaders and decision-makers at provincial, municipal and LGU
levels.
The assessment included field visits to Palawan, Bukidnon and Rizal provinces (Figure 3). The team
covered regional, provincial, municipal/city, barangay and sitio levels visiting administrative and
technical offices, health facilities and target communities, interviewing officials, DoH and PSFI staff,
volunteers and project beneficiaries (annex 4 provides examples of the questions used during
interviews). The distribution, availability, condition and use of project supplies and equipment were
investigated as far as possible and practices covered by project-supported training were observed. It
should be noted that site selection was carried out by the PR and planned and announced well in
advance of visits. Ideally site selection should be the responsibility of the review team and visits to
peripheral sites should be unannounced or announced only immediately prior to visits.
In addition, the review team attended a three-day workshop coordinated by DoH and ACTMalaria. The
purpose of the workshop was to assess the current malaria situation and plan for elimination,
particularly in malaria prone areas with low transmission. Regional and provincial malaria
coordinators, cluster heads and other partners from around the Philippines were in attendance and so
the meeting provided a useful opportunity to conduct a number of additional interviews.
At the end of their country visit the review team gave a debriefing for DOH, PSFI and WHO and one
week later provided WHO with an aide memoire highlighting findings and recommendations and

19 February 2013 7
incorporating feedback from the debriefing session. The first draft of this report was submitted to WHO
on 22 February 2013.
2.2 Roadmap for malaria control beyond 2014
In addition to the GFCMG evaluation and based on observations and grant evaluation outcomes, the
authors prepared a roadmap for the development of the next National Strategic Plan (NSP) for Malaria
Control and Elimination and for an application for continued financial support from GF under its new
funding mechanism (Annex 1). The roadmap includes ToR for a comprehensive NMP review
scheduled to take place in April-May 2013, designed to inform the development of the new NSP. The
roadmap also sets-out plans for a comprehensive programmatic, geographical and financial gap
analysis that will form the basis for the new NSP.
Figure 3: Map of field visit sites: Palawan, Bukidnon and Rizal Provinces (yellow)
Palawan
Bukidnon
Rizal

19 February 2013 8
3. Findings and recommendations
3.1 Impact
Table 1: PHL-202-G09-M: goal level impact and outcome indicators, targets and results
Table 1 shows GFCMG impact and outcome indicators, targets and results. Actual data figures above
(and in subsequent indicator tables) were obtained from the PhilMIS and online from the PHL-202-
G09-M Grant Performance Report (GPR) external print version dated 25 January 2013.4
Data for 2012
have yet to be audited by the local fund agent (LFA). A massive difference in actual malaria morbidity
rates between 2010 (18) and 2011 (0.1) is the result of an inconsistent formulation and calculation of
the indicator targets and results. As such, the results are misleading and confusing: there is no way to
interpret these data usefully. A retrospective calculation should be done and GPRs updated to provide
consistency between past and present data.
Data collected during the evaluation show a remarkable drop in reported malaria caseload in 2012
(45% lower than in 2011 and 60% lower than in 2010). This is widely attributed to programme
4
http://portfolio.theglobalfund.org/en/Grant/Index/PHL-202-G09-M

19 February 2013 9
activities (community based diagnosis and treatment, ACT, primaquine for radical treatment of P.
vivax and gametocyte clearance in P. falciparum infections, LLINs, IRS) rather than to deforestation
(which occurred largely in the 1960s) or other factors. However, a review of PhilMIS data for Palawan
from the previous several years indicates that malaria is highly seasonal and unstable with erratic
annual fluctuations in disease burden (Figure 5). Optimism regarding recent gains in the field of
malaria control should therefore be tempered with caution for the next year or two at least. Unusual
weather in 2012 may have played a major part in reducing malaria transmission.
There appear to be a number of inconsistencies in the data from the PhilMIS. A preliminary analysis
did not reveal any of the trends that one might expect in an area of falling endemicity, such as falling
proportion of cases amongst children or falling proportion of cases caused by P. falciparum (Figure 4).
The age-sex profile of reported incidence in the Philippines appears similar to that in hyperendemic
settings but incidence seems far too low to account for this. Technical staff at PSFI and within DoH
have suggested that outdoor evening viewing of 'telenovelas' (common in poorer rural areas where
private ownership of televisions is rare) is to blame for elevated caseload amongst youngsters. This
possibility should be further investigated.
Figure 4: Number of P. falciparum cases by year
and percent amongst children under 15 and males
The PhilMIS data indicates an unstable malaria situation in Palawan but this contrasts with the data
from the Philippines presented in Annex 6b of the 2011 World Malaria Report (WMR)5
which indicates
a largely steady decline in malaria burden from almost 100,000 cases in the early 1990s to less than
20,000 cases in 2010. The WMR data do not reveal as many erratic fluctuations in reported burden
between years however this may be a result of consolidated national level data masking province
specific fluctuations.
The DoH should contract an external team to conduct a malaria indicator survey in a small number of
selected provinces (e.g. Palawan, Mindanao and one other), to corroborate current reports of disease
burden and programmatic coverage and to provide baseline data for the elimination effort. The
5
WHO | World Malaria Report 2011 [http://www.who.int/malaria/world_malaria_report_2011/en/]

19 February 2013 10
sampling frame for the survey should allow deliberate sampling of remote areas where services are
absent and of malaria hotspots as identified by PhilMIS. Where security is an issue surveys could
possibly be arranged with the assistance of Medecins Sans Frontieres (MSF). The survey should
include a question on the evening activities of interviewees in an effort to identify reasons for the
unusual age distribution of malaria burden.
3.2 Diagnostics and treatment
Table 2: Diagnostics and treatment: indicators, targets and results
The data presented in table 2 shows that actual figures for number of service providers trained
(indicator 1.1) have consistently exceeded targets and in P4 progress far exceeded expectations (by
1,620 or almost 60%). For indicator 1.2 and 1.3 the number of cases fell below expected levels
reflecting reduced incidence of malaria in 2011 while the % treated for indicator 1.2 fell just short of
the target in P5. Indicator 1.4 (% of suspected malaria cases diagnosed within 48 hours of
consultation at a health facility) is strange. With the widespread availability of microscopy and RDTs
there should be no question of anyone having to wait for any more than an hour or so for parasite-

EVALUATION OF GLOBAL FUND CONSOLIDATED MALARIA GRANT PHASE 2, PHI
19 February 2013
Annex 5 – API in Palawan Barangays
API per 1,000 population (barangay level)
Names of municipalities are shown

19 February 2013 37
Annex 6 – Field Visits and Schedule: Terms of Reference
Engagement of two external consultants through the World Health Organization in the
Philippines to evaluate the ongoing Global Fund Consolidated Malaria Grant Phase 2 (2012-
2014), and to prepare a roadmap for a new funding application to Global Fund beyond 2014
based on a comprehensive national malaria programme review, a national strategy for malaria
elimination and detailed financial gap analysis.
Scope and Objectives of External Evaluation
The evaluation shall be carried out against the Grant Agreement, which is Annex A to the
amended and restated Grant agreement (PHL-202-G09-M) with its objectives and its
performance framework detailing indicators, targets and periods covered. The consultants will
perform the evaluation detailed below, assessing each area and making recommendations for
each grant objective.
Grant evaluation objective 1: Provision of universal access to quality diagnostics and treatment
services
1. Assess the efficiency and effectiveness of the strategies for diagnosis and treatment at all
levels;
2. Examine progress and proposed future developments in the capacity of health workers,
medical technologists and laboratories to support diagnosis, training and quality assurance
in the project provinces;
3. Assess progress towards strengthening of Quality Assurance system for both the
diagnostic and treatment services in project provinces;
4. Assess the capacity of health care providers to manage malaria in project provinces;
5. Review the progress of monitoring of drug efficacy and drug quality;
6. Assess the effectiveness of procurement and supply management to mitigate stock-outs of
anti-malarial commodities.
Grant evaluation objective 2: Scale up of vector control to reach at least 80% protection of
populations in malaria endemic areas
1. Assess whether and to what extent this project has achieved LLIN coverage of 80% of the
at-risk population by 2012 in the 40 provinces covered by the project;
2. Assess whether and to what extent this project has achieved IRS coverage of 70% of
houses in target highly endemic villages;
3. Critically appraise the application of vector control measures to guide programme
implementation and recommend improvements;
Grant evaluation objective 3: Strengthening of sustainable community-based malaria control
and strengthening of malaria surveillance and information systems
1. Review the status and the level of integration between the Philippines Malaria Information
System (PhilMIS), Philippines Integrated Disease Surveillance and Response (PIDSR)
and Field Health Service Information Surveillance (FHSIS) in the malaria surveillance, and
ways in which this project can complement and work with National Epidemiology Centre
(NEC) and other DOH activities and projects and enhance its outcomes;
2. Assess whether and to what extent this project has strengthened the malaria surveillance
system at the regional level to support epidemic detection and response, including in
epidemic prone areas;
3. Assess the project’s activities and relationship with collaborating Centres in the provision
of technical support for diagnosis and entomological services. To assess the need for

19 February 2013 38
entomological capacity building in selected regions where other vector borne diseases are
on the rise.
4. With reference the Paris Declaration on Aid Effectiveness (2005) and the Accra Agenda
for Action (2008), assess the appropriateness of the project’s integrated work plan
incorporating inputs of all partners, including how well NMP activities are harmonized with
other donors and partners.
Developing a roadmap for a new funding application to Global Fund beyond 2014
Based on observations and grant evaluation outcomes, the external consultants are asked to
develop a road map for the preparation of a new malaria grant application to the GF to secure
funds for malaria control and elimination in the Philippines beyond 2014, with the following
components:
1. Prepare draft terms of reference for a comprehensive national programme review,
scheduled to be conducted in April-May 2013, which shall subsequently inform the
development of a national strategy for moving from malaria control to elimination in the
Philippines;
2. Prepare draft terms of reference for a comprehensive programmatic financial gap analysis
for malaria control and elimination in the Philippines;
3. Recommend other steps towards the development of a new malaria grant application to
the GF from the Philippines.
Evaluation approach, outputs and methodology
Approach
The evaluation will be open and consultative, seeking views and opinions from DOH; grant
project personnel; partners; government leaders and decision-makers; provincial, municipal
and other local government unit (LGU) leaders. The review must consider the manner in which
the project addresses cross-cutting issues such as poverty alleviation, gender and
environment in its implementation.
The consultants will adopt the following methodology:
• Field visits will be made to 3 provinces and will form the basis of the evaluation, during
which progress in the implementation of the project work-plan could be verified through
qualitative assessments from on-site visits to functioning health facilities, barangays and
households.
• Random validation to check the distribution, availability, condition and use of project
supplies and equipment; interview health staff and observe the practices that directly
reflect on the project-supported training received and on their understanding of the roles
they have in malaria control.
• Extensive discussions with the major control programme stakeholders, such as the PSFI
(as the PR), DOH, local government unit (LGU) representatives and other stakeholders
deemed crucial in the implementation of the programme.
• Table analysis of project data and documents provided by the Principal Recipient.
• Exit presentations and reports.
Outputs
The outputs of the External Evaluation shall include reporting on the progress of all
components of the project to date, structured in accordance with the scope and reflecting the
objectives of the evaluation. This will comprise three phases of reporting:

19 February 2013 39
• A presentation of preliminary findings and recommendations in the form of an aide
memoire, to the DOH and the Programme Manager of PSFI and/or the ManCOM
(depending on whether a ManCOM meeting can be convened to coincide with the end of
the field mission);
• A draft written report on the implementation of the project to date and recommendations;
• A final report incorporating comments and feedback from DOH, PSFI, WHO and other
PMN stakeholders;
• A road map for a new application to GF for malaria support beyond 2014, including TORs
for comprehensive national programme review and financial gap analysis.
Methodology
DOH, PSFI and WHO requires the following services to be undertaken:
Preparatory work
• Review of all key background documents provided and conduct other research as
appropriate or deemed necessary by DOH, PSFI and WHO;
• Teleconference with DOH Programme Manager, PSFI Management and WHO Medical
Officer for Malaria, Vector-borne and other Parasitic diseases (MVP), Philippines, to
discuss assignment and methodology.
In-country activities
• Meetings with appropriate officials within the DOH, IDO, NEC sections in Philippines, and
within WHO and PSFI,
• Briefing and debriefing with ManCOM in Manila and also with regional Centre for Health
Development (CHD) offices,
• Visits to PSFI-GF project activities in the selected provinces. Security clearances for travel
to these areas will be submitted to United Nations Department of Security Services in
Manila.
• Visits to a representative sample of barangay level project sites.
• Consultations with project personnel including country coordinators, recipient government
authorities, provincial and district leaders, NGOs and other stakeholders as appropriate.
• Consultation by face-to-face meeting in Manila or telephone / teleconference with partners
from provinces that there is not time to visit.
• Consultations with non-project health personnel as appropriate.
Reporting and de-briefing in-country
• Provide full briefing of findings and recommendations as an aide memoire to DOH, PSFI
and WHO.
• Conduct briefings from relevant Government, project staff and programme partners.
The consultants will use a wrap-up meeting in Manila and, as appropriate, CHD regional
offices to provide feedback on major findings and recommendations.
Draft report
• Within 2 weeks of the consultant de-brief, provide a draft report and road map for new
grant application to GF.
Final report
• Finalise the evaluation report within 2 weeks of receiving comments on the report from
DOH, PSFI and WHO.

19 February 2013 40
Evaluation Team
Dr Sean Hewitt (United Kingdom). Serves as Team Leader and Main Report Writer. Total 24
days, including 15 days in-country (from 23rd January to 6th February 2013) and two travel
days.
Dr James Kelley (Hawaii, US). Total 24 days, including 18 days in-country (from 23rd January
to 9th February 2013) and two travel days.
The consultants will plan the evaluation together and divide between them the primary
responsibility for collating inputs and drafting report sections and road maps against the
different evaluation objectives.
Evaluation Schedule
Dates Activity
Wed 23-Jan Arrival of Sean Hewitt and James Kelley in Philippines, Manila;
Meeting with DOH, WHO, PSFI. Background info, data and documents for
review
Thurs 24-Jan Travel to Palawan. Meetings with PSFI; Courtesy call to CHD and PHO.
Field visits
Fri 25-Jan Palawan Field visits
Sat 26-Jan Return to Manila; Document review and report writing
Sun 27-Jan Travel to Bukidnon, Mindanao
Mon 28-Jan Courtesy call CHD and PHO; Field visits
Tues 29-Jan Bukidnon field visits; Travel back to Manila
Wed 30-Jan Participation in NMP consultative meeting on feasibility of malaria elimination,
Manila; Informal meetings with regional coordinators and TWG members
Thurs 31-Jan Participation in NMP consultative meeting on feasibility of malaria elimination;
Informal meetings with regional coordinators and TWG members
Fri 1-Feb Visit to RITM, ACTMalaria, FETPAFI, other partners
Sat 2-Feb Document review and report writing
Sun 3-Feb Document review and report writing
Mon 4-Jan Travel to Rizal Province. Courtesy call to CHD and PHO. Field visits.
Return to Manila
Tues 5-Jan Travel to Rizal Province, field visits
Return to Manila.
Wed 6-Feb Debriefing with the DOH, WHO, PSFI and Malaria Technical Working Group
(discussion and presenting of preliminary evaluation findings). Departure of
Sean Hewitt from the Philippines
7-8 Feb Follow-up activities in Manila as required by James Kelley
Sat 9-Feb Departure of James Kelley from Philippines

PHL_External_Eval-Feb2013_execsum

Recommended

Recommended

More Related Content

Similar to PHL_External_Eval-Feb2013_execsum

Similar to PHL_External_Eval-Feb2013_execsum (20)

PHL_External_Eval-Feb2013_execsum