Overview of the Partners In Health Liberia Community Health Worker (CHW) baseline training evaluation, justification, findings and recommendations for next-steps.

1. 1
Evaluation Justification: Why is evaluating CHW trainings
and learning important?
The UN Secretary-General’s Special Envoy for Financing the Health
Millennium Development Goals released a report titled “Strengthening
Primary Health Care through Community Health Workers: Investment
Case and Financing Recommendations.” The Ministry of Health of
Liberia, Last Mile Health, UNICEF, PIH and other partners contributed to
this report which highlighted that investing in the training of CHWs and
evaluation of CHW programs is among the ten core principles of sound
program implementation.
Overview of PIH/CHT CHW Training Program
• In May 2015 the Community Health Worker program began in
Maryland and Grand Gedeh Counties
• All newly recruited CHWs and Supervisors received an intensive,
six-day, 11-Module baseline training of the core competencies in HIV/
AIDS, Tuberculosis, Leprosy prevention, control, stigma reduction, and
Voluntary Counseling and Testing
• All supervisors received a three-day Supervisor training
• Curricula was drawn from existing NACP and NTLCP HIV/AIDS, TB, Leprosy
material as well as PIH’s ‘best practice’ CHW/Accompagnateur materials
Celebrating Learning through Evaluation
Overview of the Community Health Worker (CHW) Baseline Training Evaluation, Justification, Findings and
Recommendations for Next-Steps
Evaluation Highlights
• 63 CHWs and 19 CHW Peer and Clinical
Supervisors participated in an evaluation to
assess baseline knowledge retention at a 1 or
4 month follow-up
• 54 female, 28 males participated across
two counties, seven Health Care Facilities
(clinics, health centers, hospitals)
• Knowledge Assessment (written
questionnaire) and Practical Assessment
Tool (checklist) were used in the assessment
• CHWs were assessed on a pre-determined
set of “Key Health Education and Health
Promotion” messages around TB, HIV/AIDS
and Leprosy Control, Treatment, Prevention
and stigma reduction
• All CHWs and CHW Supervisors received
a “Personal Evaluation Results Summary”
highlighting top three strength areas and
three areas for improvement
• Learning points included: evaluation
methods that allow CHWs to demonstrate
their competencies rather than write
their competencies seem to be a better
assessment of knowledge and abilities
among Grand Gedeh and Maryland
County CHWs
• Average Written Assessment Score: 81.7%
(Maryland County) and 78.9% (Grand
Gedeh County)
• Average Practical Assessment Test score:
80.4% (Maryland County) and 86.3%
(Grand Gedeh)
“While each country’s context will be
different when strengthening CHW
programs, countries should strive to
adhere to ten core principals, including
establishing sufficient health worker
training and building monitoring and
evaluation capacity and structured
supervision, to ensure Return on
Investment (ROI).”
Grand Gedeh CHW Supervisors conducting the Practical Observation Checklist
Assessment, October 2015

2. 2
Objectives of the CHW Training Evaluation
• Among the CHWs who received the Baseline, Phase 1 training, what percentage of the CHWs achieved a 65% pass
rate on their written Knowledge Assessment questionnaire?
• Among the CHWs who received the Baseline, Phase 1 training, what percentage of the CHWs achieved a 65% pass
rate on their demonstrated Practical Checklist Assessment in the field?
• What can we learn about the best format for assessing CHW knowledge and how evaluation findings can inform
new subject and refresher-training subject areas and program improvement decisions moving forward?
Summary Statistics
The chart below highlights the summary statistics of the CHW and Supervisor Training Evaluation. Data was analyzed
across sub groups (male/female, by county, by CHW and Supervisor and across health care facility). Differences
found across sub-groups were not significant. Differences between CHW performances on the baseline knowledge
assessment as compared to the post-test at the 1-4 month follow-up was low. Between the baseline knowledge and
post-knowledge assessment the measured knowledge assessment never fell below 2.8% percentage points for any
sub-group as shown below.
Number Baseline
Written
Assement
(KA)
Post-test
Written
Assesment
(KA)
Practical
Assesment
(KAA)
Baseline Post
Difference
Compostie of
KA & KAA
Female 54 81.5% 79.7% 82.2% -1.8% 80.9%
Male 28 82.2% 82.4% 82.2% 0.2% 82.3%
Grand Gedeh 29 NA 78.9% 86.3% NA 82.8%
Maryland 53 81.7% 81.7% 80.4% 0.0% 81.2%
CHWs 63 81.7% 79.6% 82.2% -1.1% 81.0%
Supervisors
(Peer &
Clinical)
19 81.7% 84.5% NA -2.8% 84.5%
Total 82 79.6% 80.7% 82.2% -1.4% 81.8%
Summary of Accomplishments by County

3. 3
Evaluation Strengths
• CHT Participation: County Health Team/MOH representatives introduced each training
• Strong cross-team collaboration: PIH Liberia Clinical, Program/Logistics and Community Health Team, Boston and
Liberia MEQ team, Global Learning & Training Team
• Preliminary Program Results: Estimates for LTFU/Default patients returning to Care and Treatment by CHWs
between June-October 2015 include 17 ART patients, 40 TB patients, 16 Leprosy patients
• Celebrated Learning through Evaluation: This assessment helped to professionalize CHW training and helped to
provide CHW Supervisors with a clear follow-up plan for refresher trainings
• Established Health Promotion Messages were evaluated: This evaluation was able to assess and demonstrate the
specific health education messages 82 CHWs and CHW Supervisors use during community awareness related to TB,
HIV, Leprosy and stigma reduction.
Areas for Improvement
Challenges Suggestions for Improvement
1. Some questions were worded poorly and were not
understood clearly; this may have contributed to lower test
scores for some participants
• Allow adequate time to ‘field test’ all assessment instruments
to ensure wording and context is clearly understood
• C&T Specialist can coach CH Officer and CH Specialist
to design and draft their own evaluation methodology
and assessment tools to ensure timeliness and
context-appropriate questions
2. Evaluation results highlighted areas in which CHWs and
Supervisors could benefit from additional trainings from
the Clinical team. These areas include: recognizing urgent
and non-urgent side effects among patients on TB, ART and
leprosy medication and recognizing signs and symptoms of
opportunistic infections among TB and/or ART patients
• Community Health Officers (CHO) /Community Health
Specialists (CHS) are encouraged to work together to create
a calendar schedule for ‘mini refresher trainings.’ These mini
refresher trainings can be conducted at the monthly CHW
meeting to reinforce lower-scoring learning areas
• CHO/CHSs are encouraged to work with PIH and CHT
Clinical leaders to plan for a short series of ‘follow up refresher
trainings’ to reinforce some of the learning area that had the
lowest scores across CHW and Supervisor participants
3. CHWs and CHW supervisors reported some anxiety around
the evaluation from the participants. Some were concerned
they would be penalized for incorrect answers
• CHO and CHSs are encouraged to emphasize with their
CHW team that this evaluation is intended to help improve
the training and programming for the CHWs in Maryland
and Grand Gedeh County. All professional health workers
in the Health Care System are required to participate in
on-going learning.
4. PIH Liberia has spent a good amount of money on trainings;
among the trainings in FY15 MEQ reports many of those
trainings lacked baseline and post-test assessments. For
trainings conducted without any evaluation component the
quality of the training is difficult to capture, monitor and report
• PIH Liberia leadership is encouraged to appoint a “Training
Coordinator” (either a dedicated Training Coordinator or
one person who can be the designated “point of contact”) to
improve training coordination, establish a training calendar,
checklist of requirements for training such as defining learning
objectives, timeline, budget, pre and post-test requirements,
printing and material development plan and budget.
5. Linkages between training outcomes and program outcomes
are not causal in this evaluation and can only be inferred
• Strengthen M&E design capacity to begin to collect data
on the patient side, to understand the influence of CHW
messaging and work on patient behavior or knowledge
attitudes and beliefs around the outcome of interest
6. Missing reference point data for Grand Gedeh (Baseline,
Phase 1 Knowledge Assessment was administered to Grand
Gedeh CHWs, but their names were not recorded.
• CH Specialists and Officers are encouraged to remind
their CHWs and Supervisors to write their names on their
assessment tests

4. 4
Key Findings: What did we learn?
• On average, participants scored higher on the Practical
Field Checklist (KAA) than on the Written Knowledge
Assessment (KA)
• When given the opportunity to demonstrate their
knowledge CHWs achieve higher scores on their
knowledge assessments
• Results across sub groups (Male/Female, by county, by
clinic or health center or hospital, or among CHWs and
Supervisors) were not significantly different
• Adequate review and editing of questions is essential
to ensure wording is culturally-appropriate and
language-appropriate
• Multiple choice answers that require the participant
to circle “All the above” and “Both B, D” questions are
confusing and should not be used in future assessments
• Highest performing knowledge areas: ability to describe
disease transmission and prevention of TB, HIV,
Leprosy, ability to use good communication skills with
their families, ability to describe ways to be a ‘buddy’ to
a patient
• Lower performing knowledge areas: describing signs
of opportunistic infections among TB or HIV or co-
infected patients, side effects of TB and Leprosy
medication and early signs of leprosy
Key Recommendations: What can we apply to
sustain training program success?
• Community Health Officers and Specialists are
encouraged to share findings with their clinical and
county health team leaders to celebrate learning and
advocate for support in on-going refresher and new
subject-area trainings
• Community Health Officers should encourage CHW
Supervisors to review the CHW Personal Evaluation
Results plan to help clarify misunderstandings and
ensure on-going study and knowledge refreshment
• Community Health leaders are encouraged to continue
to work closely with CHT partners to further establish
and build consensus around “Key Health Education and
Health Promotion Messages for TB, Leprosy, HIV/AIDS”
• Developing inexpensive, hand-held visual aids with
the agreed upon Key Health Promotion and Health
Education Messaging by the National Community
Health Curriculum Development working group can
serve as a reference for CHWs during Community
Awareness and one-on-one patient education CHW Supervisor Training: Administering the Practical Observation
Checklist, Maryland County
Learning, Next Steps + FY16 Plans
• Evaluation methods that allow CHWs to demonstrate
their competencies rather than write their
competencies seem to be a better assessment of
knowledge and abilities among GG and Maryland
County CHWs/Supervisors
• Clinical and Community Health Team should strive
to incorporate quarterly or half-yearly Field Checklist
Evaluations into CHW monitoring and training
activities to be able to better track knowledge and
messaging used during community awareness
and patient education
• Establishing a ‘training lead’ on the clinical and
community health team would aid in professionalizing
and stream-lining training and evaluation activities
• Evaluation findings can inform monthly refresher
trainings lead by CHW clinical supervisors and
clinical team
• Evaluation findings can create a case for additional
trainings around providing Psychosocial support,
Motivational Interviewing (or another Behavior Change
Communication counseling methodology)
• CHW training program moving towards polyvalence
(CEBS, ANC programming) can employ the checklist as
an evaluation methodology for new CHW programming
• PIH Liberia is currently contributing to the National
Community Health Curricula and core competencies;
aim to establish standard Key Health Education and
Health Promotion messages for TB, Leprosy and HIV