The PAG consists of nine modules that guide users through assessing problems and solutions, goals, delivery systems, populations, roles and responsibilities, strengthening interventions, building support systems, monitoring and evaluation, and management. Key features include analyzing delivery systems as social

1. The Program Assessment Guide
Structuring Contextual Knowledge and Experience to
Improve the Design, Delivery and Effectiveness of
Nutrition Interventions
David Pelletier
Division of Nutritional Sciences
Cornell University
CORE Spring Meeting
May 1, 2012
Wilmington, DE

2. Acknowledgments
Collaborators on the MN Tool Project
 Allison Corsi, Lesli Hoey, Suzanne Gervais,
Cornell University
Funders and Facilitators of the MN Tool Project
 The World Bank
 Micronutrient Initiative
 Centers for Disease Control
 UNICEF
 GAIN
 A2Z (USAID and Gates Foundation)

3. The Challenge
?

4. Why Implementation
Science?
p2
 Efficacy-to-Effectiveness
 Knowledge-to-Action
 Evidence-Based Interventions
 Evidence-Based Decisions
 Evidence-Based Policy Making
 Scaling Up
 Etc.

5. Outline
1. Genesis of the project
2. Overview of the PAG
3. Adaptations and next steps for the PAG
4. Q&A

6. Genesis of This Project
 Micronutrient Forum (May 2009)
• From Evidence to Large Scale Programs
 The Innocenti Process (Sept 2008)
• Country implementers, donors, researchers
• Review of evidence for effectiveness at scale
• Identification of overarching issues needing attention
 Country implementers requested stronger
guidance on identifying and resolving
implementation bottlenecks

7. The Micronutrient Program Tools
 Program Assessment Guide (PAG)
 Program Documentation Guide (PDG)

8. The PAG Process
Laying the Three-Day
Follow-
Groundwork Workshop
Through
3-12 months 1-3 years

9. Program Assessment Guide (PAG)
Processes Outputs
1. Action Plan to
Participatory Procedures To Address Barriers
Strengthen: & Enablers
• The Systematic Integration Of 2. Operations
Evidence, Contextual Knowledge & Research
Experience Agenda
• Shared understanding, commitment, 3. Issues for
ownership, motivation & capacity to Inclusion in
M&E
advance the micronutrient agenda
•Links with the broader nutrition and 4. Strategic Plan to
health agendas in the country Build Support,
Capacity &
Sustainability

10. Nine modules:
• Purpose
• Rationale
• Instructions
• Templates
• Examples

11. Program Assessment Guide
Laying the Groundwork
Module 1: Clarifying the Problem and the Solution
Module 2: Goals and Associated Values
Module 3: Delivery Systems
Module 4: Hard to Reach Populations
Module 5: People, Roles and Responsibilities

12. Program Assessment Guide
 Building or Strengthening the Intervention
 Module 6: Needs, Inputs, Activities and System Changes
 Module 7: Action Planning
 Building Support Systems and the Enabling
Environment
 Module 8: Monitoring, Evaluating and Quality Improvement
 Module 9: Organizing, Leading and Managing

13. Some Distinctive Features of the PAG
1. Delivery Systems as Social Systems (Module 3)
2. The Five Needs Tool (Module 6)
3. The Stress-Testing Tool (Module 6)
4. Clear definition of roles and responsibilities (Modules 5,7) and
responsibility for follow-up (Module 9)

14. Module 3: Delivery systems as social systems
Significant Others (barriers and enablers)
A D H
F G J
E
B C I
Household
& Caregiver
Functionaries (barriers and enablers)
National Regional District Clinic Community

15. Module 3: Delivery systems as social systems
• How Will Gulazik Get from the National Level to All Caregivers?
• How Will Gulazik Education Get to All Caregivers?
Caregiver
National Oblast Rayon Clinic Community

16. Module 6: Strengthening the delivery system
Five Universal Needs
2.Awareness
3.Knowledge, Information, Skill
4.Motivation/ Commitment
5.Resources
6.Support from Others

17. Module 6: What does each functionary and significant other NEED in order
to fulfill their role/responsibility of the Sprinkles program (on a daily
basis) and what inputs, activities and system changes are required to
ensure their needs are met?
What does the village
health volunteer need in What is required in order
order to be successful? for these needs to be met?
Village-level
Health Awareness Inputs, Activities
Volunteer & System Changes
Information,
Role/Responsibility: Knowledge, Inputs, Activities
& Skills & System Changes
Provide mothers
with information Commitment/ Inputs, Activities
and education to Motivation & System Changes
encourage proper
use of Gulazik
Resources Inputs, Activities
& System Changes
Support from Inputs, Activities
others & System Changes

18. Systematizing and stress-testing contextual
knowledge and experience (Module 6)
Proposers Challengers
Identifying Functionaries and Significant Others
What does the village
in the Delivery and Support Systems
health volunteer need in What is required in order
order to be successful? for these needs to be met?
Significant Others
Village-level
Health Awareness Inputs, Activities
Volunteer & System Changes
Information,
Role/Responsibility: Knowledge, Inputs, Activities
& Skills & System Changes
Provide mothers
Inputs, Activities A D H
with information Commitment/ F G J
and education to Motivation & System Changes E
encourage proper B C I
use of Gulazik
Resources Inputs, Activities
& System Changes
Support from
Functionaries
others
Inputs, Activities Caregiver
& System Changes
National Oblast Rayon Clinic Community

19. Systematizing and stress-testing contextual
knowledge and experience (Modules 6-8)
Proposers Challengers
Outputs
Actions to address
barriers and
What does the village
health volunteer need in What is required in order
order to be successful? for these needs to be met?
Village-level
Health Awareness Inputs, Activities
Volunteer & System Changes
enablers in the
Information,
Role/Responsibility: Knowledge, Inputs, Activities
& Skills & System Changes
Provide mothers
with information Commitment/ Inputs, Activities
& System Changes
Action Plan
and education to Motivation
encourage proper
use of Gulazik Resources Inputs, Activities
& System Changes
Support from Inputs, Activities
others & System Changes
Identifying Functionaries and Significant Others
in the Delivery and Support Systems
Questions for
Significant Others
Operations Research
Issues to track in
A D H
F G J
B E
C I
monitoring and QA
Functionaries
Caregiver
National Oblast Rayon Clinic Community

20. Module 9: Responsibilities for Follow-Up
to the Workshop
Implementation WGs Monitoring,
Operations
Evaluation
Research WG
and QI WG
Coordination & Support
WG
Relationships Among Working Groups

21. Experiences from Kyrgyzstan,Bolivia and Haiti
1. Sponsorship, Responsibility,
Commitment, Follow-up
2. Timing Before the workshop
3. Preparation
4. Participants
5. Contextuality (selection and sequencing)
During the workshop
6. Time constraints
7. Facilitation

22. Next Steps for the PAG
 Produce additional versions for:
• More complex interventions (ag, livelihood, social protection, etc)
• Integrated (multi-intervention) programs
• Sub-national planning (district, project level)
• Diverse countries
 Partner with NGOs to gain experience applying the PAG
in the above contexts
 Develop and field test the Adoption Guide, Facilitators
Guide and Workshop Planning Guide
 Develop and fiedld test training & coaching approaches

23. Q&A

24. An Example
Breastfeeding BCC programs often improve EBF rates by a
modest amount, e.g., 20% at baseline to 40% at endline.
Using the five needs tools, analyze why this might be?
Five Universal Needs
2.Awareness
3.Knowledge, Information, Skill
4.Motivation/ Commitment
5.Resources
6.Support from Others

25. Module 6: Strengthening the delivery system
Five Universal Needs
2.Awareness
3.Knowledge, Information, Skill
4.Motivation/ Commitment
5.Resources
6.Support from Others

27. PAG in the Context of Other
Frameworks and Models for Scaling Up
High Level Frameworks for Scaling Up
General
(WHO/RH, WHO/ExpandNet, MSI, CMH, CORE, etc)
Guidance
Results Frameworks , Logic Models
(USAID, PIP, etc)
Behavior-centered design of
interventions and delivery
and support systems (PAG)
Detailed
Analysis &
Planning