Portfolio Summary: JSI's Work in Research, Monitoring, Evaluation & HIS

Research,
Monitoring,
Evaluation
& HIS
2013-2014
Portfolio Summary

Innovating & evolving
In the global health community, JSI continues to
improve and adapt our approaches to generating and
using data to monitor, evaluate, and drive public
health interventions and policies.
For several years, demand for rigorous evidence of the effectiveness
of specific health interventions has dominated discussions about
investing in data. Recent debate about the inherent complexity of
driving change in health systems has however suggested a more
balanced approach to measurement – one that explores
implementationas well as impact so that we learn about how best
to deliver life saving health interventions in low resource settings.
With these data we can discover how an intervention works and
how it can be scaled up or replicated, adapting technologies and
implementationstrategies to a particular country, health system,
facility or community.
JSI works at the nexus of research and practice, applying creative
monitoring and evaluation approaches and rigorous analytics to
understand and address operational challenges in public health
programs. We are, at our core, an implementingorganization that
designs and strengthens health system solutions and generates and
uses evidence to guide this process.
This summary of work presents examples of the way in which JSI
values and uses data and analytics for improving health. We hope
this snapshot of our work demonstrates the importance we place on
connecting people, data, and ideas to strengthen health systems
that serve people around the world.
Sincerely,
Anne K. LaFond,
Director, JSI Center for Health Information,
Monitoring & Evaluation.
forward

update: the global state of
research, monitoring, evaluation,
and health information systems
01
table of contents
innovation & collaboration in JSI’s
research. monitoring, evaluation,
and health information portfolio
02
research, monitoring, and
evaluation activity highlights
03
health information system
strengthening activity highlights
04
who we are and what we
do in RM&E and HISpreface

JSI’s workinresearch,
monitoring,evaluation,and
healthinformation system
strengthening connects data
andideasacross the health
sector to strengthen health
systems and improve the
health ofpopulations.

JSI is an industry leader in designing &
implementing data-driven public health
programs
JSI contributes directly to global investments for
achieving an AIDS-free generation, ending
preventable maternal and child deaths, and
ensuring access to family planning, among many
other aspects of population health.
Since its founding, JSI has managed over 500 projects in more than
100 countries, and currently operates 8 U.S. and 81 international
offices with approximately 2500 staff worldwide.
We routinely work to reach vulnerable populations in rural areas
and across urban settings. Our work is conducted in partnership
with global, national, and subnational stakeholders, and we build
capacity of local teams and organizations through our programs.
JSI has become a recognized leader in building effective health
information systems and data management capacity in a range of
public health areas, specializing in information system design and
strengthening; data quality, presentation and strategic use; and
research, monitoring and evaluation (RM&E) for learning and
performance improvement.
JSI’s Center for Health Information, Monitoring, and Evaluation
(CHIME) serves as a central resource for RM&E/HIS assistance to
ensure research, monitoring, and evaluation activities are
conducted to a high technical standard. JSI staff have a wealth of
experience in data management, analysis, and quality assurance.
We conduct landscape analyses; apply mixed methods for impact
evaluation, research-based case studies, and implementation
research; conduct secondary data compilation and analysis; and
build national health information systems (HIS).

Exploring
emerging
opportunities in
RM&E
Strengthening health
information systems
to support the use of
routine health data for
decision making
Making data available,
accessible, and usable
through visualization
Innovating in research
and design including
methods, use of applied
technology, and building
local capacity
Learning through mixed-
methods approaches &
understanding the context
that enables program success
Our research, monitoring, and evaluation (RM&E) portfolio includes program
evaluation, special studies, and capacity development.
For more than 30 years, JSI has designed and implemented RM&E strategies as
part of our domestic and international public health programs, and led
evaluations for governments and their development partners including USAID,
the United Nations, GAVI, the Global Fund for AIDS, TB and Malaria, and the Bill
& Melinda Gates Foundation.
JSI is exploring emerging opportunities in RM&E, looking beyond standard
program evaluations towards new methods for defining effective
implementationstrategies for complex systems and strengthening the
processes and technology through which routine health information is
collected. We work consistently to ensure that data are available in useable,
meaningful formats for key decision makers.

research, monitoring,
evaluation & health
information systems
a summary of policy updates
and changes to the global
landscape, including
emerging areas of interest for
funders, implementers, and
public health professionals
global
update

Global
data use
2015 marks the target year for
the Millennium Development
Goals (MDGs). The global health
community is defining new
global metrics for 2030 and
investing in data systems to
track progress.
The MDGS have defined progress in global
health and social development for the last
fifteen years. In the coming year, secondary
analysis of population level data from routine
health information systems and national
surveys will be used to estimate what was
achieved. A UN report released in July 2014
details ways to still contribute to the MDGs
through the end of 2015.
Looking forward, country stakeholders and
program implementers hope to define new
global objectives in the Sustainable
Development Goals (SDGs).
The SDGs build upon the MDGs post 2015 and
the work to improve the lives of the poor
around the world. These goals continue to
reflect key values such as equity, universal
health care, and accountability.
The proposed list of SDGs includes 17
overarching goals, with 169 sub targets. There
is some concern in the global development
community that this increase in targets will
lead to more confusion than actual progress.
Anticipation for the SDGs has increased focus
on the need for investment in systems to
generate and use data. The 'Data for African
Development Working Group’ met in March
2014 to discuss root causes of slow data
progress across the continent (read more
about this meeting). The working group, in
collaboration with the Center for Global
Development, published a report in August
2014 outlining four key challenges and
identified three strategies to combat those
challenges. You can view that report here.
global update
section 01

Data & evaluation initiatives
global update
International Year of Evaluation
EvalPartners, the global movement to strengthen national
evaluation capacities, proudly announced that 2015 was declared
as the International Year of Evaluation (EvalYear). The aim of
designating 2015 as the International Year of Evaluation is to
advocate and promote evaluation and evidence-based policy
making at international, regional, national and local levels.
At the launch event in December 2014, organizers celebrated the
adoption of the first ever, stand-alone UN resolution on national
evaluation capacity development titled "Capacity building for
Evaluation of Development activities at the Country-level" by the
UN General Assembly, which shows local evaluation capacity
development is going to be a major feature.
Measurement & Accountability for Results in
Health (MA4Health) Summit
In June 2015, USAID, the World Bank, and the WHO will convene to
define a new health measurement and accountability strategy,
which will articulate a common agenda for the global health
community for the Post-2015 Agenda and result in the Roadmap
for Health Measurement and Accountability.
The Summit will address the following four central, crosscutting
themes: (1) lessons learned and gaps in data measurement, (2)
desirable systems, (3) investment priorities and (4) roles and
responsibilities of partners. Each theme explores a series of
questions designed to focus on issues, including capacity building
and data quality.
Fifteen technical areas have been identified to explore as part of
the Roadmap. They range from data sources and platforms,
content-specific approaches and cross-cutting issues in
measurement. Developed collaboratively through input from
partners and experts in the field, these technical areas will inform
the Roadmap, which will be released at the Summit.
section 01

Open data revolution
Data transparency is becoming increasingly important in the
US and among governments globally.
The White House Open Data Initiative was created to improve how the US government
shares information for the benefit of the American people. This initiative, signed by
President Obama in 2013, requires Federal government data to be open by default and
available in machine-readable formats. Today, more than 130,000 datasets reside
on data.gov, the repository for the U.S. Government’s open data.
The USAID Open Data Policy released in October, 2014 is the agency’s response to the
above U.S. government initiative to make data resulting from federally funded programs
more available and easily accessible by the general public. The U.S. government
anticipates that making data more available to the public will encourage
“entrepreneurship, innovation, scientific discovery, and enhanced development
outcomes” and will decrease the likelihood of costly duplicative data collection activities.
JSI supports the efforts by USAID to promote openness and transparency of data, and is
fully committed to making data collected with federal funds open and transparent, while
at the same time respecting the privacy and rights of human subjects and the sovereignty
and ownership rights of local governments and institutions.
.
global update
section 01

Affordable Care Act
Much about the health care system in the United States is changing as states and
the nation began to implement the Affordable Care Act in 2014. The law is
transforming health care by extending health-insurance coverage to millions of
Americans who did not have it before and adapting the way through which care is
delivered.

JSI has been instrumental in health-reform and service delivery redesign initiatives
on the national, state, and local levels, and is helping programs and providers at
community health centers and rural clinics to respond to and help shape the
emerging environment.
JSI has developed an interactive, online map tool for New Hampshire to
geographically illustrate concentrations of people potentially eligible for new
health insurance options under the Affordable Care Act. Eligible groups can be
examined by coverage type, age group, and income range.
The map also pinpoints the location of organizations that offer free, in-person
assistance with enrollment and how to contact them. The New Hampshire
Affordable Care Act Enrollment Resources Map can be customized by turning
separate layers on and off.
global update
section 01

Renewed focus on implementation
In the past few years, the global
health community has focused
more deliberately on assessing
the effectiveness of the
implementation strategies and
processes used to improve
delivery and uptake of life-saving
health interventions and
understanding how they work in
different settings.
To guide investment in health, the global
community requires methods that go beyond
measuring outputs and outcomes and focus
on how investments influence change in
service delivery and uptake. Funders and
implementers are therefore calling for
implementationresearch and advocating for
mixed methods measurement that integrates
a range of quantitative and qualitative
approaches to assess health programs and
health systems.
At the Global Health Systems Research
Symposium (Cape Town, November 2014)
the value of implementation research was
highlighted, focusing measurement on
addressing implementation challenges and
the questions of replication and scale up.
Funders and global health actors can ensure
that implementation research continues to
be cultivated by channeling resources to
teams of researchers and implementers in
organizations where learning and
implementationgo hand in hand.
Developmental Evaluation (DE) is another
RM&E approach that supports learning and
innovation within an organization or
program. DE allows real-time, or close to
real-time, feedback to program staff
facilitating a continuous development loop.
This method offers a powerful approach to
health programming and supports social
innovations by working in partnership with
program decision makers. Read more about
the basics of development evaluation here.
global update
section 01
Photo credit: Robin Hammond

connecting with new
partners & opportunities
Highlights from JSI’s
collaboration with technical
teams and new thought
partners
innovation &
collaboration

Innovating
in research
and design
JSI designs creative research strategies and fosters skills in
measurement and use of data for innovating and learning in public
health. We co-design research strategies with local firms and
investigators and ensure that results are accessible and used in real
time, to guide implementation.
section 02
JSI serves as the Global Research Partner for
the Innovations for Maternal, Newborn, and
Child Health Initiative (Innovations).
Innovations is a multi-year grant, led by
Concern Worldwide, to find and test
ambitious new ideas that have the potential
to overcome barriers to accessing maternal,
newborn and child health (MNCH) services
and significantly improve service delivery.
Under the Initiative, JSI works in partnership
with local research groups to evaluate
program success and conduct process
documentation to test program theory and
inform implementation strategies. Additional
details on Innovations are included in
subsequent sections.
In Ethiopia, the Bill & Melinda Gates
Foundation-funded the Last 10 Kilometers
Project (L10K) which serves about 25 million
peoples in 236 woredas (i.e., districts) of Afar,
Amhara, Oromia, Tigray, and the Southern
Nations, Nationalities and Peoples’ (SNNP)
regions.
In 2013-14, the project conducted three
effectiveness evaluations focusing on:
1) implementing a ‘change process’ to
improve effective care seeking for critical
maternal and newborn conditions in 16
primary health care units; 2) scaling-up
community based data for decision making
strategy from 14 to 115 woredas; and 3)
participatory community quality improvement
strategy that is being implemented in 14
woredas. The evidence from each evaluation
is being used to inform the project’s scale-up
strategy.

Making data
available,
accessible, and
usable through
visualization
JSI designs innovative ways
to visualize data, including
dashboards, infographics,
creative reporting formats,
and tools customized to the
needs of the local users.
JSI’s designers and analytics advisors
routinely collaborate to ensure data
visualizations are anchored in an
understanding of who the audience is for
the product, what their information
needs are, and how they wish to receive
information.
From this user-centered approach has
emerged compelling infographics and
other static products; engaging videos
that tell a data story; and dashboards
designed for use by local stakeholders.
To facilitate regular access and use of
contraceptive security data, JSI, through
the USAID | DELIVER PROJECT, created
and annually updates dashboards of data
linked to the contraceptive security index
which are used by countries around the
world.
Our team also designed Excel-based
dashboards to help district level
stakeholders guide anemia programs and
interactive charts as part of the Pathways
to Better Nutrition case studies created
on the Strengthening Partnerships,
Results and Innovations in Nutrition
Globally (SPRING) project. Additional
dashboard development also supports
the Implant Access Program (IAP).
Making routine health information
accessible and appealing is key to
promoting its use by health managers
and workers. In Liberia and Cote d’Ivoire,
JSI continues to support local adaptation
and customization of DHIS2 dashboards.
JSI technical advisors are actively seeking
out new opportunities to leverage the
capabilities of programs like DHIS2 for
dashboard design to ensure data are
accessible and usable for decision
makers.
section 02

How we innovate
Across our public health programs, JSI looks for
opportunities to innovate using research,
monitoring and evaluation.
JSI promotes the use of innovative processes throughout the
company. Projects implemented both domestically and globally have
changed the collection and use of data for planning and monitoring
health programs. We have designed innovative mixed-method case
studies to explore and compare implementation strategies and are
integrating and studying human-centered design (HCD) techniques
in public health.
We are experimenting with the use of mobile tools for household
and facility surveys, managing logistics systems to streamline data
collection, and documenting the use of advanced health technologies
and practices. Additional information about our innovative work in
health information systems and technology is available in the
subsequent health information systems section.
Using data to drive innovation: In Nepal, JSI led the design and
testing of a community-based distribution model for chlorhexidine, a
tool for reducing newborn sepsis. The project considered the human
factors (a mother’s desire to apply something to the cord after
it was cut) and how to empower women to administer the
treatment themselves. In Nigeria, the community-based distribution
model was replicated and adapted to the local context.
Chlorhexidine was distributed as a spray, rather than a gel, based on
user insights, and misoprostol for the prevention of post-partum
hemorrhage was distributed along with the chlorhexidine.
These projects paved the way for a third chlorhexidine project in
Madagascar, which began in 2013. You can read more about JSI’s
innovative work with chlorhexidine in these three countries here.
Innovating in resource-limited settings: The Timor-Leste Health
Improvement Project (HIP) tested the effectiveness of a non-
pneumatic anti-shock garment (NASG) to treat women experiencing
post-partum hemorrhage in low resource settings. The NASG is a
low-cost, first aid pressure device that decreases blood loss and
stabilizes women to survive transport to emergency care. The HIP
project is piloting the use of the NASG in 26 largely rural facilities in 8
districts. Additionally,the team is interviewing providers to
determine their acceptance to the NASG as a "new tool" using the
Diffusion of Innovations Framework. The World Health Organization
(WHO) has recommended using NASG to help reduce maternal death
in settings where definitive care is not immediately available, and the
device was included in a list of Breakthrough Innovations that Can
Save Women and Children Now.
innovation & collaboration
section 02

Global gains in
maternal, newborn
& child health
From 2008-2014, the Maternal and Child Health Integrated
Program (MCHIP) worked in more than 50 countries in Africa,
Asia, Latin America, and the Caribbean to improve the health of
women and their families. MCHIP, USAID’s flagship maternal
and child health program, focused on accelerating the reduction
of maternal, newborn, and child mortality in USAID priority
countries facing the highest disease burden.
Using data-driven approaches, MCHIP helped countries
determine which innovations would yield the most impact in
reaching the Millennium Development Goals (MDGs) and
supported capacity building and health systems strengthening to
take those interventions to scale. The infographic to the right
highlights some of MCHIP’s achievements in child health.
JSI continues to support global program and evaluation efforts to
improve maternal, newborn, and child health through our work
as a partner on the MCHIP follow on project, the Maternal and
Child Survival Program (MCSP).
section 02

Defining and
testing new
techniques to
improve data
collection,
quality & use
innovation & collaboration Collaboration on the design phase of a Gates-funded initiative was a
notable opportunity to strengthen systems to generate and use data for
decision making.
From 2013-2014, JSI worked alongside our partner PATH in the design
phase of the Better Immunization Data (BID) Initiative. The BID
Initiative’s vision was to empower countries to enhance immunization
and overall health service delivery through improved data collection,
quality, and use. JSI contributed to the design of project activities
focusing on behavioral and organizational drivers of information system
performance.
The BID Initiative proposes a three-pronged strategy: 1) Develop an
effective immunization management information system to collect and
manage immunization data; 2) Provide individuals engaging with the
data system the support they need to both collect and use data; and 3)
Diffuse successes by creating a peer-to-peer BID Learning Network that
enables countries struggling with similar challenges to learn from one
another, adapt, and evolve in a sustainable way.
For the BID Initiative design phase, JSI provided expertise in routine
immunization systems to tailor information solutions to the needs of
program managers and led the design of the project theory of change on
the pathways to improved quality and use of routine immunization
data. The JSI team also conducted a literature review to inform the
behavioral and organizational aspects of the program design, and
contributed to the program RM&E strategy.

Evaluating innovative tools
The Innovations for Maternal, Newborn, and Child Health
Initiative (Innovations) is conducting comparative case studies
to document and assess the influence of human centered
design on maternal, newborn and child health programs.
This research explores the use of design thinking practices and their potential for
improving the implementation and effectiveness of health interventions. As part of the
ongoing data collection, the JSI team works in partnership with local research firms to
interview and observe program staff and community participants in fours pilots: Maker,
Care Community Hub, Community Benefits Health, and Essential Newborn Care. The
studies will be completed by 2016.
section 02
You can learn more
about the Innovations
pilot projects and
ongoing work on the
project website.

Leveraging mobile technology
Mobile technologies streamline data
collection and data use, promote innovation,
and improve health outcomes across
numerous technical areas.
In 2013-2015 JSI leveraged the power of mobile technologies to
achieve positive health outcomes in the United States and across
the globe. Our mobile health, or mHealth, solutions are used to
inventory health supplies, ensure quality supervision at health
facilities, and provide new channels for educating the public about
health topics.
JSI has conducted quarterly health facility surveys using mobile
technology in 9 sub-Saharan countries on behalf of the President's
Malaria Initiative End Use Verification Activity during 2013-14. Data
were collected from more than 5000 health facilities, and findings
used to identify and address supply chain challenges and monitor
treatment practices for malaria.
Additional highlights include: the LAUNCH project in Liberia that
has used MagPi for program monitoring since 2012; SCMS that
performed a NASCA assessment in Mozambique on SurveyCTO
tablets in November 2014; and The USAID | DELIVER Project that
used mobile phones to conduct a Logistics Indicators Assessment
Tool (LIAT) in three countries in 2014.
Innovations used mobile data collection tools to conduct baseline
surveys on maternal health knowledge and practices for Essential
Newborn Care Corps (Sierra Leone) and Community Benefits Health
(Ghana). Read more about overcoming connectivity issues in these
projects on the JSI blog.
SC4CCM implemented the SMS mHealth tool cStock, a reporting
and resupply system to improve the visibility of stock levels of
health commodities and communication between the health
worker and his resupply points. SC4CCM piloted cStock in 6 districts
in Malawi and reached full scale in every district with partner
support in October 2014.
section 02

activity highlights
Throughout 2013 and 2014,
JSI provided research,
monitoring and evaluation
technical assistance services
that generated data to inform
and improve the design and
implementation of public
health programs.
research
monitoring
& evaluation

We frame our
research,
monitoring &
evaluation work
through a health
systems lens
All of our research, monitoring, and evaluation activities
are anchored in a deep understanding of the nuances and
challenges of program implementation in low-resource
settings in the US, Europe, Asia, Africa, Latin America and
the Caribbean.
For over more than 35 years, JSI has designed and implemented rigorous program
monitoring and evaluation strategies and led studies to contribute evidence and
learning in public health. In leading health projects around the globe, JSI has
developed and integrated RM&E strategies that align with and further national health
priorities and strengthen national health and information systems.
JSI is a long-term partner on USAID’s flagship global technical leadership project,
MEASURE Evaluation (1997–2014). MEASURE Evaluation assists country programs to
improve health system performance through increased demand for health
information, improved tools and data collection practices, improved analysis, and,
especially, improved demand for and use of information.
Through MEASURE Evaluation, JSI has helped build capacity in RM&E at the individual
and organizational levels, developed harmonized RM&E tools that address challenges
in implementing health projects, and strengthened health management information
systems as a means to ensuring routinely available and useful data to guide health
programs.
section 03

Collaboration across technical areas
JSI’s commitment to excellence in research,
monitoring, and evaluation is exemplified in the
breadth of our work and collaboration with
technical experts across health domains.
Immunization JSI has been working for decades to strengthen routine
immunization systems, and is invested in finding innovative and
efficient methods to increase vaccination uptake. With funds from
GAVI, the JSI ImmunizationCenter is evaluating the use of different
sized vaccine vials of pentavalent vaccine in Kenya. Mixed vial options
in health facilities may improve the management of vaccine delivery,
reduce missed opportunities for vaccination, and reduce waste.
A Gates Foundation learning grant aims to develop evidence in three
woredas (districts) in Ethiopia on the effectiveness of different
strategies for universal child immunization.
Logistics JSI’s Center for Health Logistics expands and adapts our
work to strengthen supply chains, and JSI remains a go-to technical
assistance provider for logistics initiatives around the world. In the
past two years JSI has conducted multiple in-country supply chain
assessments and global landscaping activities for donors including
UNFPA, GAVI, and Gates Foundation.
The USAID | DELIVER PROJECT implemented a Quality Assurance
Surveillance Plan (QASP) to outline and structure RM&E activities
surrounding the core elements of the project and technical assistance
for supply chain management improvement. This tool is used to
monitor progress toward achieving the project’s primary objectives.
The project has conducted secondary analyses to estimate the
population health impact of the global distribution of various
commodities, and made global data accessible through tools like the
contraceptive security index dashboards.
research, monitoring, & evaluation
section 03
From our US Portfolio
JSI worked with the Ozarks Regional YMCA and its partners to
evaluate an initiative to integrate active transportation
strategies across multiple sectors and improve school and
after-school policy and environments to support healthy
eating and physical activity throughout the City of Springfield,
MO. The evaluation focused on assessing process and
outcomes associated with policy, systems, and
environmental (PSE) changes to increase access to healthier
environments. JSI used data sources from multiple settings
including a household survey, community-led street audits,
afterschool programs, and local city sources.

Program monitoring & evaluation
Designing and implementing rigorous program
monitoring and evaluation plans continues to
be a cornerstone of JSI’s RM&E portfolio.
MEASURE Evaluation Phase IV will build on previous data quality
improvement interventions, working with WHO and The Global
Fund to conceptualize standardized data quality tools within a
country, and testing data management standards .
The Strengthening Partnerships, Results and Innovations in
Nutrition Globally (SPRING) project has conducted research,
evaluation and monitoring related to anemia; agriculture-nutrition
linkages; SBCC; systems and program evaluation at country level.
The Pathways to Better Nutrition case studies in Uganda and Nepal
are collecting longitudinal qualitative and budget data to document
and explore prioritizing nutrition in national policy and financing
efforts. SPRING’s country programs in Uganda and Bangladesh are
collecting panel survey data to evaluate outcomes from SPRING
program interventions.
The Supply Chain Management System (SCMS) project developed
the National Supply Chain Assessment (NSCA), a comprehensive
toolkit to assess the capability, maturity and performance of a
health supply chain. The results of each NSCA enable supply chain
managers and implementing partners to monitor whether program
activities are achieving their expected outcomes and develop
evidence-based strategic and operational plans.
section 03

Program monitoring & evaluation
Research, monitoring and evaluation
strategies generate learning to inform
program strategies.
Live, Learn & Play: Sustainable, Scalable Basketball for Youth
Development (LLP) JSI’s LLP project, a private-public partnership
between USAID and the NBA, is working to promote youth
development and leadership in Africa. JSI has been working closely
with the NBA to develop measures for youth livelihood and
development that reflect the NBA's values as well as the key
youth development domains that LLP is targeting. Focal areas
include: roles for coaches, youth, parents/caregivers; school
readiness assessments; organizational capacity development;
supportive supervision monitoring; and the youth citizenship and
basketball MVP assessment. These tools will be adapted for
different program sites, and will be used to generate evidence on
local NGO capacity to provide sports and youth development
services. The LLP project was recently featured in an article on the
SuperSport website.
section 03
Evaluation of Santa Clara County Community
Transformation Grant JSI completed the process and
outcome evaluation of a local community transformation
grant (CTG). The evaluation assessed outcomes
associated with: 1) educational strategies to promote
healthy eating and active, smoke-free living, and 2) policy,
systems, and environmental (PSE) changes to improve
population health by reducing the prevalence of chronic
disease and health disparities. JSI developed an
evaluation plan, based on the social-ecological model and
Reach Effectiveness Adoption Implementation
Maintenance (RE-AIM) Framework; created/adapted data
collection tools; trained community partners to collect
data; collected and analyzed data; and created
presentations and reports to disseminate findings.

Program success captured by RM&E
Two USAID-funded newborn and child
health projects in Ethiopia and Georgia
showcase examples of where RM&E was an
integral part of documenting success.
The Georgia Sustain Project helped educate over 1800 medical
students in modern OB/GYN and reproductive health care. This
increased capacity has lead to multiple improvements in
maternal and newborn health care including over 90% of
newborns receiving essential newborn care. Additionally, 80% of
rural and semi-rural primary care facilities now provide
comprehensive family planning and reproductive health
services.
The Ethiopia Integrated Family Health Project (IFHP) implemented
a comprehensive package of maternal and newborn health
services, with results showing increased availability & uptake of
services, as illustrated below.
40%
60%
66%
99%
90%
96%
BEmONC PMTCT First ANC Visits
IFHP successfully increased uptake of key maternal health services.
2011 2014
section 03

Evaluating impact of integrated care
The Maine Health Access Foundation
(MeHAF) promotes patient-centered
and seamless care, through the
coordination and integration of primary
care, behavioral health, dental care,
specialty care, and other services.
From our US portfolio
Since 2007, MeHAF awarded 42 grants across the state of Maine to
develop integrated behavioral health programs. After three years, the
integrated care initiative opened 86 new facilities across the state.
The evaluation of MeHAF provided an opportunity to learn from diverse
organizations and about approaches to integration that work in across
different contexts and populations. The evaluation used the framework
of RE-AIM to analyze the Reach, Effectiveness, Adoption, Implementation
and Maintenance of the program.
Key accomplishments of the initiative include:
• Integrated hospital and nursing home care for elderly patients
reduced hospital length of stay from 45 days to 6 days.
• Organization for patients with mental illness integrated primary care
services and self-wellness,resulting in in 50% of patients reaching
their preferred weight goals within 6 months.
• The first pilot for the use of Health and Behavioral codes to financially
support behavioral health services in primary care settings was
conducted in Maine. This approach was then spread to other states.
section 03

Learning
through mixed-
methods
approaches
As a partner and leader on multi-country learning
grants and studies, JSI has applied mixed methods
research approaches to assess impact, explain how and
why program outcomes emerge, and explore and
compare the effectiveness of different implementation
strategies.
Without access to simple, low cost medicines, millions of children die each year
from preventable illnesses. Improving Supply Chains for Community Case
Management of Pneumonia and Other Common Diseases of Childhood
(SC4CCM), was a learning project that identified proven, simple, affordable
solutions to address the unique supply chain challenges of community health
workers (CHWs). SC4CCM tested supply chain interventions in Ethiopia, Malawi,
and Rwanda to improve access to medicines to treat common childhood
illnesses. As part of the final evaluation, case studies were conducted to
understand scalability, institutionalization, integration, and potential
sustainability of the SC4CCM interventions, and the relationships among these.
The findings of the Africa Routine Immunization System Essentials (ARISE)
multi-country mixed methods case studies were published in Health Policy and
Planning and Evaluation, respectively, to ensure lessons from the study are
integrated into policy and implementation strategies related to routine
immunization systems improvement.
section 03

In-depth case study research methods
JSI is a leader in the application of analytical
and in-depth case-based methods, working
across various countries, and initiatives.
In December 2014, JSI collaborated with the George Washington
University School of Public Health, the Institute of Medicine and the
World Bank on a event “Exploring Complexity in the Health Sector: A
Panel on Case-Based Evaluation Approaches.” The speakers
defined case-based approaches as the use of in depth, systematic
and analytical research methods to study real life events in context
using a single case or multiple cases. Presenters gave three
examples of case studies or case-based methods and discussed the
external validity of these methods and their relevance to addressing
questions of replication and scale in global health.
section 03
You can read more about our
event, listen to and view the
presentations given by the
panelists, and discover
additional resources
surrounding case-study
methods here.

Implementation research
JSI integrates rigorous research methods to
improve learning from public health programs
using implementation research and case-based
research.
The Innovations for Maternal, Newborn, and Child Health Initiative
conducts process documentation to understand the ongoing effect of
implementation strategies and study impact pathways defined in a
theories of change for in two projects: Care Community Hub and
Community Benefits Health. The process documentation data will be
analyzed in conjunction with routine monitoring data to gain a better
understanding of how well projects are being implemented and the
extent to which change pathways are emerging as planned.
The Madagascar Health System Strengthening Support Evaluation
(GAVI HSS) utilized a mixed methods approach to explore the steps
taken by country stakeholders to utilize GAVI HSS Support resources
efficiently and transform those resources into improved system and
immunization performance in the context of political upheaval and
transition.
section 03
Healthy Schools Healthy Communities (HSHC). Since 2013, JSI
has assisted the Missouri Foundation for Health (MFH) to
evaluate its statewide initiative to address childhood obesity.
To identify models of practice and assess the functioning of
each of the 25 school districts and community partners, JSI is
employing a mixed-methods evaluation that includes height
and weight measurements, student and household surveys,
pedometer step counts and physical activity time, school
attendance, misconduct, and academic performance.
In addition, we developed a web-based monitoring system for
the school and communities to report work plan performance,
and use the data to assess a core set of data elements,
including organizational roles and progress, leveraging of
resources, and number of meetings and events. To
supplement the monitoring system data, we interviewed lead
agencies and partners and explored processes, challenges and
successes. JSI compiles monthly and annual reports to MFH
and its grantees containing recommendations for future
training and technical assistance.

Data & analytics for policymakers
Compiling, analyzing and making data accessible
transforms information into a tool to guide
investment in health across countries and
projects.
The USAID | DELIVER PROJECT conducted the 5th round of data
collection for the contraceptive security indicators from 43 countries.
The data, maps and global, regional, and country dashboards can be
found on the DELIVER website. The project presented findings at the
international family planning conference in Addis and also APHA in
Boston.
The Strengthening Partnerships, Results and Innovations in Nutrition
Globally (SPRING) team has utilized secondary (DHS) data for many
empirical analyses such as the rapid assessment of distribution and
consumption of IFA and Uganda Anemia Country Analysis . Findings
address questions related to the breakdown in programs that treat or
prevent maternal and child anemia.
Photocredit Robin Hammond
section 03

activity highlights
The use of high quality
information generated
by holistic,integrated and
sustainable routine health
information systems is a core
feature of a thriving health
system.This section features
numerous JSI-led initiatives to
strengthen health information
systems and support the use
of data for improving health.
health
information
systems

Strengthening health information systems
JSI’s experience in HIS strengthening ranges from designing and expanding
basic routine health information systems (RHIS) (e.g. Haiti, Liberia) to
improving advanced systems (e.g. China, South Africa). We have focused on
HIS for specific health programs, improving data use for disease control and
immunization. We have also addressed national health system data needs,
supporting policy makers to monitor changes in health system outcomes and
impact.
JSI has been a partner on USAID’s flagship global monitoring, evaluation, and
HIS strengthening project, MEASURE Evaluation, since 1997. As a partner on
MEASURE Evaluation Phase III (2009-2014), JSI contributed to project impact
strengthening HIS including
• 76 documented examples where regional, national, or sub national
institutions demonstrated increased capacity to independently carry out
M&E/HIS activities
• 37 RHIS or M&E systems demonstrated improvement in system
performance, thanks to project contributions
• 45 documented instances where data was used to inform programs, policy
or advocacy as a result of MEASURE Evaluation activities
health informationsystems
JSI collaborates with country
governments and local partners and
builds local capacity to establish and
strengthen reliable national health
information systems (HIS).

Assessments of HIS
Over the past three decades, JSI’s
International Division has
developed state-of-the-art tools
and approaches to assess and
support improvements in routine
data quality, data use, and service
readiness at national, sub-national,
and service delivery levels.
JSI has supported the development and use of the
Data Quality Audit (DQA), Routine Data Quality
Assessment (RDQA), and Performance of Routine
Information System Management (PRISM) tools for
information systems , as well as the Logistics
Indicator Assessment Tool (LIAT) and Logistics
System Assessment Tool (LSAT) to support supply
chain systems.
In 2013-2014, JSI led a number of assessments and adapted
various global HIS assessment tools to improve the quality
and availabilityof data around the world, including:
• Adapting the RDQA tool in Botswana (MEASURE Evaluation),
Malawi (SC4CCM), and through the JSI World Education Vana
Bantwana project in Zimbabwe.
• Completing PRISM assessments in 5 countries and publishing a
summary paper reviewing PRISM results across countries.
• Redesigning the PRISM tools to reflect the changing reality of HIS
globally, where many countries have adopted electronic systems.
• Working with WHO and The Global Fund to conceptualize
standardized data quality tools within a country, particularly with
HMIS (MEASURE Evaluation IV).
• Testing data management standards.

Global leadership:
Data management standards
In MEASURE Evaluation Phase III, JSI, as the
principal contributor to Routine Health
Information Systems (RHIS) in the MEASURE
Evaluation consortium, continued its decade-
long mission of developing and testing
methods and tools for RHIS performance
improvement.
In 2012, MEASURE Evaluation led an experts' workshop in South
Africa to identify standards and best practices in RHIS data
management. The workshop focused on strengthening RHIS data
management generally, and more specifically, provided guidance
for the development of improvement plans following a
performance assessment. Data management has been identified as
one of the weakest areas of RHIS globally.
The output of the South Africa meeting was synthesized into
guidelines of RHIS data management standards. These standards
and guidelines were validated in two countries, Nigeria (2013) and
Bangladesh (2014).
In MEASURE Evaluation Phase IV, JSI will work to operationalize the
RHIS data management standards through incorporating them into
existing tools and methods, such as PRISM, and developing new
tools to measure adherence to standards.

Information system design
As part of JSI’s comprehensive approach to health
information system strengthening, projects and
teams define HIS specifications, designs, and
facilitate customization of electronic HIS platforms,
including DHIS2.
As a partner in the fourth phase of MEASURE Evaluation, JSI is
leading on HIS strengthening and will play a major role in
supporting the PEPFAR roll out of the new DATIM system. The
team will continue to support countries to strengthen their HIS
and ensure data are available and accessible.
Working with PEPFAR information systems has been a core feature
of JSI’s work in South Africa for more than seven years through the
design and continued adaptation of the Partnership Information
Management System (PIMS) for partners to report their quarterly
program data.
The complementary MEASURE Evaluation SIFSA associate award
provides capacity development to the National Department of
Health, following on the work of JSI’s Enhancing Strategic
Information project.
Even in remote environments, like rural Ethiopia, JSI has found
innovative solutions to increasing the availability and use of
routine health information through designing electronic systems.
In 2014 the, Ethiopia L10K Project established an mHealth based
HMIS for its project portfolio, adapting DHIS2 to collect monthly
and quarterly performance reports and generate automated
analysis of the program performance to identify performance gaps
and provide feedback.
In addition, the Liberia Rebuilding Health Services (RBHS)
supported the MOHSW and trained County Health and Social
Welfare teams on the DHIS2 system and the use of data for
program management, and developed the Community HMIS.

Building Routine HIS Capacity
Building local capacity to support and sustain a
national RHIS is a cornerstone of JSI’s work on
the MEASURE Evaluation, Maternal and Child
Survival Project (MCSP) and USAID | DELIVER
projects.
In Namibia, MCSP and it’s predecessor, the Maternal and Child
Health Integrated Program (MCHIP), supported (1) Development
and implementation of an HIS strategy for Strategic Information, (2)
Review and update of Essential Health indicators, and (3) Business
Process Analysis for restructuring the HIS. MCHIP also provided
technical support to the MOH and other stakeholders to develop
health information tools to improve HIS management performance
issues and facility and community level.
Across Tanzania and Zambia, JSI has designed and supported the
introduction and expansion of the electronic logistics management
information system (eLMIS), which enhanced the management of
health commodities by transitioning facilities from a paper-based
system of data management to an electronic format. The rollout
emphasized engagement with, and capacity development of, local
stakeholders to ensure the sustainability of the new systems.
The eLMIS, built on an open source platform, is a revolutionary and
cost-effective system that will foster better, faster, and more
accurate reporting of supply chain data, reduce stockouts of health
commodities, and ultimately provide increased access to medicines
to improve health outcomes. The USAID|DELIVER PROJECT and
Supply Chain Management System (SCMS) are among the partners
supporting this initiative. Learn more here.

Strengthening
the use of
routine
information in
South Sudan
health informationsystems In JSI’s work under USAID and the Health
Pooled Fund in South Sudan, we saw
collection, verification and reporting of
data at the health facility and local
government levels but very little use of
these data for strategic planning, program
management, or program adaptation.
The use of a data discussion component
during routine facility technical support
visits encouraged local data use practices.
When we routinely engaged facility
workers and local leaders in discussions
using data emerging from facility registers,
staff began to explore service
performance issues, participate actively
in routine planning and problem solving.
In South Sudan, JSI applied a
straightforward process based on three
components:
1) Engage local actors in identifying a few
focal indicators, and define a process for
reviewing data talking about results.
Tying the data use process to strategies to
improve service quality drives home the
“so what” element of reviewing basic
service statistics.
The process depends on making sure the
right people participate, the ability to
distill key findings from data, and using
and updating basic data visuals.
2) Wrap up with feedback on the process
and collaborative action planning where
priority areas are defined with time-
bound responsibilities assigned.
3) Follow-up: Little happens without it.
Action plans are the basis for future
supervision visits or team meetings.
Here’s a small data victory snapshot from
South Sudan:
During a support visit, our project worked
with staff and local government to review
and discuss data on maternal care leading
to a plan to introduce access to 24 -hour
maternity care access. Skilled deliveries
were soon on the rise.
—Kristin Eifler, M&E Advisor, JSI

Community information systems
JSI makes quality data available to decision
makers from national to community levels.
Liberia In 2011, JSI in collaboration with the Liberian MOHSW, and
under the USAID- funded Rebuilding Basic Health Services (RBHS)
project, developed a National Community Health Policy and
Plan that elaborates on the community health components of the
Ten-Year National Health Plan. During this time, they designed a
Community Health Management Information System (C-HMIS). The
C-HMIS links information from the services provided in the
community to facility level information. Design and the
implementationof C-HMIS is an important step toward making the
national HMIS inclusive and comprehensive. It is expected that the
new C-HMIS will give a major boost to the performance of the
Community Health Program and improve the quality of the services
provided by the CHVs.
Ethiopia In Phase III, MEASURE Evaluation trained over 2,100
planners from all zones, woredas/districts and health facilities on
indicator definition and the use of the HMIS as the data source for
preparing woreda-based annual plans. The project installed the
electronic HMIS (eHMIS) in 150 woredas in addition to 14 zonal
health departments, four special woreda health offices and one
town administration.
By utilizing eHMIS at the zonal and woreda level, data are now
readily available and accessible and have been used for woreda
based planning throughout the region. The HMIS is now
contributing to decision making and improvement of service
delivery. The eHMIS is a home grown solution developed by
Ethiopian IT experts taking advantage of expertise made available
by the Project. It can be introduced in the other Regions in Ethiopia
and has interoperability capacity with any other system used by the
Ethiopian Federal Ministry of Health. This HMIS Scale-Up Project is
now housed under the Advancing Partners and Communities (APC)
Project, a transition that took place in 2014 at the start of
MEASURE Evaluation’s fourth phase.

Community Health Systems Catalog
Innovation in community health
information systems promotes
data use and accessibility.
JSI is working in partnership with FHI360 to implement the
USAID-funded Advancing Partners & Communities (APC) , a five-
year project that invests in and supports community health
programs particularly in family planning.
APC has developed the Community Health Systems Catalog as a
resource for ministries of health, program managers,
researchers, and donors interested to learn about the current
state of community health systems around the world. The
Community Health Systems Catalog is a web-based interactive
reference tool on community health systems that includes
information about policy, resource commitments and results
related to community health. The Catalog covers USAID-priority
countries related to population and reproductive health and
community-based family planning.

Innovating through HIT
Within our domestic portfolio, the health
information technology (HIT) team has
developed and monitored multiple
responsive, user-centered systems for
managing and linking health data.
eReferral Health Information Exchange Project: Developed and
run by JSI, this automated process was a part of a stop-smoking
program for the Massachusetts Department of Public Health that
ran through January 2015. The Health-e-link health information
exchange network supports data exchange between physician
EMR systems and the QuitLine database application enabling
doctors to make referrals in a timely, efficient manner. The system
provides security features that meet federal HIPAA requirements
as well as meaningful use requirements associated with the
Affordable Care Act.
Title X Region I Family Planning Data System: Since 2002, JSI has
supported Region I’s (New England) Family Planning Data System
(which JSI also developed and implemented), a region-wide
information system that supports both electronic and paper-based
collection of Title X family planning encounter data, as well as on-
line transactional and management reporting for FPAR and other
grantee-determined requirements.
Since 2010, JSI has provided programmatic and user support for
the Massachusetts Immunization Information System (MIIS)
Program, used by healthcare professionals including (but not
limited to) healthcare providers, local boards of health, school
nurses, and staff at state agencies involved in immunizationto
report all administered immunizations to the MIIS.
AHRQ National Resource Center (NRC) Domain 2 Health IT
Projects Monitoring and Reporting: From 2009-2014, JSI provided
monitoring and reporting support for the grantees and contractors
funded through the Health IT Portfolio, and is tasked with the
overall monitoring of the funded projects.

Center for Health
Information, Monitoring,
and Evaluation
For more information, please
contact the center at
chime@jsi.com
Follow
@jsihealth

Portfolio Summary: JSI's Work in Research, Monitoring, Evaluation & HIS

Recommended

Recommended

More Related Content

What's hot

What's hot (16)

Viewers also liked

Viewers also liked (20)

Similar to Portfolio Summary: JSI's Work in Research, Monitoring, Evaluation & HIS

Similar to Portfolio Summary: JSI's Work in Research, Monitoring, Evaluation & HIS (20)

More from JSI

More from JSI (20)

Recently uploaded

Recently uploaded (20)

Portfolio Summary: JSI's Work in Research, Monitoring, Evaluation & HIS