3. Workshop objectives:
1. To launch the first global improvement framework for health worker IST
2. Harvest and share experiences, strategies, and lessons learnt on
addressing key IST challenges for health workers
3. Facilitate networking, shared understanding and learning between
health workers, Ministry of Health representatives, training institutions,
donors and partners to improve IST
Expected outcomes:
1. Participants will learn about the IST improvement framework and how
to use it to improve in-service training systems for health workers
2. Participants will enhance their understanding of how to evaluate inservice training program outcomes and impacts
3. Participants will gain new knowledge and multi-stakeholder
connections to improve in-service training that will enable health
workers to provide better care
4. Participant experiences, lessons learnt and evidence will seed a new
global wiki on in-service training improvement
3
4. Launch of the in-service
training improvement
framework: What, why and
how to apply
Tana Wuliji
Senior Quality Improvement Advisor
USAID ASSIST Project, University Research
Co., LLC (URC)
@TanaWuliji
4
5. The in-service
training improvement
framework
Towards more effective, efficient
and sustainable training
Developed and launched by
International Center for AIDS Care and Treatment Programs (ICAP); International
Pharmaceutical Federation (FIP); IntraHealth; International Training and Education
Center for Health (I-TECH); Jhpiego; Sciences for Health (MSH); Pan-American Health
Organization (PAHO); Pathfinder; World Medical Association (WMA); USAID; University
Research Co. (URC)
5
6. What is the in-service training (IST)
improvement framework?
A set of practice recommendations to improve
in-service training effectiveness, efficiency and
sustainability that were developed and agreed
through international expert consensus
6
7. Global landscape: Need for evidencebased guidance for improving IST
Significant proportion of
country HRH investments
Needed for scale-up of health
services (HIV, TB, MCH)
Growing attention of countries
in reviewing IST portfolios
8. How was the IST improvement framework
developed? Delphi process
4. Rate agreement on
recommendations
1. Develop and revise
recommendations
25 member
consensus group
3. Targeted literature
review on
recommendations
2. Consolidate
recommendations
June – December
2011
9. How was the IST improvement framework
developed?
Delphi process
Development and review of recommendations
through 5 rounds of peer review and ratings
(Delphi process) and literature review
Final consensus:
40 recommendations
across 6 themes by 25
member expert panel
Improvement
framework +
guidance
Expected January
2014
External open consultation: 4th round
• November 3-29 2011, online
• 26 countries
• 86/119 complete responses
10. IST improvement framework themes
1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
10
11. Using the IST improvement framework:
Examples
• Nigeria- IST Assessment of PEPFAR Nigeria Portfolio
(CapacityPlus/IntraHealth)
• Afghanistan- Informed organization of National Training
Standards for the Afghanistan General Directorate of Human
Resources In-Service Department (HSSP/Jhpiego)
• Ethiopia- IST Rapid
Assessment to inform
Ethiopia Federal
Ministry of Health
efforts and strategies
to improve,
institutionalize, and
standardize IST
National IST strategy development workshop,
Oromia, Ethiopia, August 2012. Photo: USAID
(HCI/URC)
11
HCI.
12. In-service training journal series:
Human Resources for Health Journal
Launched October 1st 2013 by
USAID, USAID ASSIST, Jhpiego,
I-TECH and IntraHealth
Open access: www.humanresources-health.com/series/IST
12
13. Design and delivery of training: Recommendation: 4.3
IST should be based on sound, evidence-based learning principles
and methodologies that offer the best opportunity to produce
sustainable performance improvement within the workplace.
Explanatory note:
In order for IST programs to most effectively improve health worker competencies,
they should apply state of the art learning principles and methodologies appropriate
to the context that are based on the latest evidence……
Examples:
Resources:
• For the development of clinical decision
• The Johns Hopkins Evidencemaking skills, a RCT comparing an
Based Practice Center
interactive workshop-based training to
recommends multiple techniques,
simulation training found that simulation
multiple media, interaction and
training resulted in better skill performance.
repetition. Marinopoulos SS et al,
This is reinforced by systematic reviews
2007.
that identify clinical simulations as an
• The Learning for Performance
effective technique for developing
tools provide guidance on
psychomotor and critical thinking skills.
designing training for desired
Daniels et al, 2010; Nestel et al, 2011;
performance
Issenberg et al, 2005
http://www.intrahealth.org/page/le
arning-for-performance
14. Expert Consensus Group
Marita Murrman, International Center for AIDS Care and Treatment
Programs (ICAP); Mike Rouse, International Pharmaceutical
Federation (FIP); Rebecca Bailey, Shaun Noronha, Kate Tulenko,
IntraHealth; Gabrielle O’Malley, Tom Perdue, Frances Petracca,
International Training and Education Center for Health (I-TECH);
Julia Bluestone, Peter Johnson, Edgar Necochea, Jhpiego; Karen
Chio, Gail Naimoli, Management Sciences for Health (MSH); Rosa
Maria Borrell, Pan-American Health Organization (PAHO); Cathy
Solter, Graciela Salvador-Davila, Pathfinder; Julia Seyer, World
Medical Association (WMA); Diana Frymus and Lois Schaefer,
USAID; Tana Wuliji, Akuba Dolphyne, Tisna Veldhuizen Van Zanten
University Research Co. (URC); Habib Benzinan, Alan Lyles, Linda
Ippolito, Hugo Mercer, Independent experts.
14
15. Lightning talks
1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
15
16. 1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
• Engage national authorities, regulatory and professional bodies
• Build capacity and strengthen local infrastructure and trainers
• Support CPD systems
17. Theme 1: Strengthening training institutions and systems
Ethiopia’s experience with inservice training institutionalization
and standardization
Dr Samuel Hailemariam, Health Systems
Strengthening Advisor, USAID Ethiopia
17
18. 1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
• Coordinate IST
• Minimize duplications: Coordination mechanism
• Reduce disruption to health services
• IST tracking mechanism
19. Theme 2: Coordination of training
iHRIS system for tracking training
Rebecca Bailey
Team Lead, Health Workforce Development,
CapacityPlus, IntraHealth
19
20. Coordination of Training
iHRIS system for tracking training
Sam Ngobua, Director, CapacityPlus, Nigeria
Rebecca Bailey, Team Lead, Health Workforce Development, CapacityPlus
21. Nigeria - Context
•
•
•
Substantial US Government
investment in in-service training
(IST) of health workers in
HIV/AIDS-related topics.
A 2012 assessment – based on
the IST Framework – found
limited coordination of IST
among PEPFAR-funded
implementing partners.
Recommended introducing an information system to track IST
across implementing partners in order to decrease duplication,
improve coordination, and ensure that the right health workers
receive the right kinds of training.
22. Objectives of the Training Information System
• Track trainees to avoid double
training of the same health worker
• Coordinate training between
PEPFAR-funded implementing
agencies
• Track availability of Master Trainers
• Ultimately reduce cost and improve
efficiency
23. iHRIS Train Prototype for Nigeria
• Web-based application developed
on the open-source application
platform of iHRIS
• Open-source, user friendly and user
operating system independent,
affordable and customizable
• Sustainable through local capacity
building
• Backed by IntraHealth informatics
experience in over 23 countries
23
30. CONCLUSION
• iHRIS Train Nigeria is a work-in-progress.
• It is scalable to meet the needs of the
PEPFAR-funded implementing agencies and
the Federal Government of Nigeria for an InService Training Information System.
• Affordable and sustainable
• Backed by the multi-national experience of
CapacityPlus/IntraHealth Informatics.
iHRIS Train Nigeria
30
31. Questions for Knowledge Café Discussion
• Is there a need for an electronic IST
tracking system in your country or for your
training programs?
• What characteristics or functions are most
desirable in a training information/tracking
system?
32.
33. 1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
• Synergies between pre-service education and IST systems
• Consistency in approaches and content
34. Theme 3: Continuum of learning from pre-service to inservice
Lessons learnt by professional
associations, regulatory bodies,
nursing schools and Ministries of
Health from strengthening CPD
systems
Carey McCarthy
Health Systems Scientist, CDC supported African
Regulatory Collaborative
34
35. 1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
• Needs based: aligned with plans, understand performance barriers
• Compliance with policies, strategies and laws
• Evidence based methodologies
36. Theme 4: Design and delivery
Findings from an integrative
review of literature on training
design and delivery
Julia Bluestone
Senior Technical Advisor, Jhipego
@juliabluestone
36
37. Effective In-Service Training Techniques,
Frequency, Setting and Media:
Synthesis of an Integrative
Review of the Literature
38. In the context of continuing health
professional education
What evidence is there to
support educational
techniques, Frequency (single
vs. repetitive), setting, and
media used to deliver
instruction?
What evidence exists about
the outcomes (knowledge,
skills, attitudes, provider
behaviors, patient outcomes)
of continued health
professional education?
Julia Bluestone, MS
Peter Johnson, PhD
Judith Fullerton, PhD
Catherine Carr, DrPH
Jessica Alderman, MPH
James BonTempo, MS
39. What surprised us? (Or didn’t!)
Didactic-”low to no” outcomes
Simulation effective
Practice and feedback ‘dosage’
matters
Shorter, but more often
Workplace-setting better
39
40. Question:
Given the current system is
designed for group-based training,
what changes do we need to make
to implement, shorter, repeated,
simulation-heavy, workplace-based
training?
40
41. 1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
• Share resources and materials
• Support trainees post-training
• Life long learning skills
42. Theme 5: Support for learning
Point of care performance support
tools linking standards to
resources needed to support them
Julia Bluestone
Senior Technical Advisor, Jhipego
42
47. SSDI Services
Question:
How do we prepare, support and
incentivize busy health care workers to
provide workplace-based mentoring or
support to others?
48. 1. Strengthening
training institutions
and systems
2. Coordination of
training
3. Continuum of
learning from preservice to in-service
4. Design and delivery
of training
5. Support for
learning
6. Evaluation and
improvement of
training
• Build in evaluation to inform continuous improvement
• Evaluate against defined criteria
• Engage key stakeholders and trainees
49. Theme 6: Evaluation and improvement of training
Training Evaluation Framework
and Tools (TEFT)
Gabrielle O’Malley
Director for Implementation Science, I-TECH
49
50. Is training making a difference?
International Education and Training Center for Health
51. ENVIRONMENT
HEALTH SYSTEM / POPULATION
ORGANIZATION
Organization
systems
improvements
Training
Individual
knowledge,
attitude, skill
outcomes
Individual
performance
outcomes
Individual
patient health
outcomes
Organization
performance
outcomes
Population
level
performance
outcomes
Organization
patient health
outcomes
(impacts)
INDIVIDUAL
Population
level
systems
improvements
Population
patient health
Outcomes
(impacts)
SITUATIONAL FACTORS
INDIVIDUAL
• Trainee background,
knowledge, experience,
education
• Intrinsic motivation
• Family demands
ORGANIZATION
• Management support
• HR – staffing levels, salaries,
burnout
• Available drugs, supplies,
equipment and infrastructure
• Facility systems –
appointments, records, flow,
referrals
• Patient needs
HEALTH SYSTEM/POPULATION
• National, regional, community
systems – labs, supply chain
• National, regional policies
• Partner programs
• Available health workforce,
including informal private,
attrition issues
• Pre-service program
• Retention factors, e.g. pay scales
ENVIRONMENT
• Political instability
• Prevalent disease
• Natural disasters
• Food availability
• Seasonal changes
• Patient access to food,
transportation
• Available community support
resources
Training Evaluation Framework (TEFT)
TRAINING EVALUATION FRAMEWORK
52. 6 planning steps with tools and guidance
Planning steps
Tools
1. Identify anticipated
outcomes
2. Address situational factors
3. Refine the scope of the
evaluation
4. Define evaluation questions,
objectives, and indicators
5. Choose evaluation design
and methods
6. Plan the evaluation
Framework and
template
Worksheet
5 Considerations tool
Worksheet
Resource tables
Planning template
53. Training Evaluation Framework
and Tools
How can evaluation of IST outcomes be strengthened
to inform improvement in training? What are some of
your successes and some of your needs in identifying
results of your training programs?
http://www.go2itech.org/resources/TEFT
54. Knowledge cafe
3 x 10 min discussions on
theme of your choice
1.
Strengthening training
institutions and systems:
Dr Amir Aman, Samuel
Hailemariam
4.
Design and delivery
Julia Bluestone
2.
Coordination of training
Sam Ngobua, Rebecca
Bailey
5.
Support for learning
Lois Schaefer
3.
Continuum of learning
from pre-service to inservice
Carey McCarthy
6.
Evaluation and
improvement of training
Gabrielle O’Malley,
Francie Petracca
54
