The Program Assessment GuideStructuring Contextual Knowledge and Experience to Improve the Design, Delivery and Effectiven...
Acknowledgments<br />Collaborators on the MN Tool Project<br />Allison Corsi, LesliHoey Cornell University<br />Robin Hous...
Genesis of This Project<br />Micronutrient Forum (May 2009) <br />From Evidence to Large Scale Programs<br />The Innocenti...
Innocenti: Overarching Issues<br />Key stakeholders share common MN goals but lack the leadership to coordinate and for pr...
Innocenti: Overarching Issues<br />Country teams lack guidance and authority to assess, analyze and take actions needed to...
Relevance to the Current Interest in Nutrition-Ag Linkages<br />Many of the overarching issues are the same<br />There is ...
Locating Evidence, Knowledge and Experience Within the Policy Process<br />Detailed intervention design and  planning (pol...
Program Assessment Guide (PAG)<br />Outputs<br />Processes<br />1. Action Plan to<br />    Address Barriers<br />    & Ena...
 Shared understanding, commitment, ownership, motivation & capacity to advance the micronutrient agenda
Links with the broader nutrition and health agendas in the country</li></ul>2. Operations<br />     Research<br />     Age...
Program Assessment Guide<br />Laying the Groundwork<br />Module 1: Clarifying the Problem and the Solution    <br />Module...
Program Assessment Guide<br />Building or Strengthening the Intervention<br />Module 6: Needs, Inputs, Activities and Syst...
Some Distinctive Features of the PAG Approach<br />Link nutrition interventions to the broader health (and agriculture and...
Creating Local Program Theory<br />
A Common Example: <br />Training Health Workers to Counsel Mothers<br />The Goal:<br />Improved<br />Feeding<br />Practice...
Nutrition Example: BCC IYCF Component<br />Improved<br />Feeding<br />Practices for <br />Young Children<br />Training of<...
Caretakers<br />Understand & Internalize the Knowledge <br />& skills<br />Training of<br />Health workers<br />To do nutr...
“Inhibiting Factors”<br /><ul><li>Limited contact</li></ul>   time with mothers<br /><ul><li> Materials not </li></ul>   a...
The PAG Approach to Creating <br />Local Program Theory<br /><ul><li>Delivery Systems as Social Systems (Module 3)
The Five Needs Tool (Module 5)</li></ul>A simpler, more practical way to build a program model<br />Embraces but simplifie...
Module 3: Delivery systems as social systems <br />Significant Others (barriers and enablers)<br />A<br />D<br />H<br />J<...
The PAG Approach to Creating <br />Local Program Theory<br />Five Universal Needs<br />Awareness<br />Knowledge, Informati...
Module 6: What does each functionary and significant other NEED in order to fulfill their role/responsibility of the Sprin...
Systematizing and stress-testing contextual knowledge and experience (Module 6)<br />Proposers<br />Challengers<br />
Systematizing and stress-testing contextual knowledge and experience (Module 6)<br />Outputs<br />Proposers<br />Challenge...
Experiences from Kyrgyzstan and Bolivia<br />1. Sponsorship, Responsibility, <br />     Commitment, Follow-up<br />2. Timi...
Adaptation to Ag-Nutrition Projects<br />The initial focus is a multi-dimensional problem rather than a single solution<br...
Also,<br />Consider how changes in one domain may affect the other domains, e.g.,<br /><ul><li>Women’s time in ag versus t...
Women’s versus men’s control of income
Effect of crop choice on men’s versus women’s control
Effect of crop choice on men’s vs women’s labor in ag
Effect of crop choice on sales vs retention for own consumption
Acceptability of foods for feeding young children
Efficacy of foods for meeting nutritional needs of young children
Requirements for adopting and growing
Requirements for preparing, using and feeding
Upcoming SlideShare
Loading in …5
×

Pelletier program assessment_guide

1,545 views

Published on

CORE Group Fall Meeting 2010. The Program Assessment Guide, Structuring Contextual Knowledge and Experience to Improve the Design, Delivery and Effectiveness of Nutrition Interventions.

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,545
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
28
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Pelletier program assessment_guide

  1. 1. The Program Assessment GuideStructuring Contextual Knowledge and Experience to Improve the Design, Delivery and Effectiveness of Nutrition Interventions<br />David Pelletier<br />Division of Nutritional Sciences<br />Cornell University<br />CORE Group Fall Meeting<br />September 13, 2010<br />Washington, DC<br />
  2. 2. Acknowledgments<br />Collaborators on the MN Tool Project<br />Allison Corsi, LesliHoey Cornell University<br />Robin Houston, Global Nutrition Consultant<br />Funders and Facilitators of the MN Tool Project<br />The World Bank<br />Micronutrient Initiative<br />Centers for Disease Control<br />UNICEF<br />GAIN<br />A2Z (USAID and Gates Foundation)<br />
  3. 3. Genesis of This Project<br />Micronutrient Forum (May 2009) <br />From Evidence to Large Scale Programs<br />The Innocenti Process (Sept 2008)<br />Country implementers, donors, researchers<br />Review of evidence for effectiveness at scale<br />Identification of overarching issues needing attention<br />
  4. 4. Innocenti: Overarching Issues<br />Key stakeholders share common MN goals but lack the leadership to coordinate and for priority setting, advocacy and action<br />Stakeholder groups within the MN community do not communicate effectively with one another;<br />Stakeholders have misaligned and often competing priorities and approaches at both global and country levels. <br />The MN community has not adequately engaged with broader nutrition, health, or development initiatives;<br />The MN community has not harnessed the full potential of private sector resources, expertise, and delivery mechanisms <br />(continued)<br />Source: Micronutrient Programs: What Works and What Needs More Work? A Report of the 2008 Innocenti Process. Micronutrient Forum, Wash. DC.<br />
  5. 5. Innocenti: Overarching Issues<br />Country teams lack guidance and authority to assess, analyze and take actions needed to fit local contexts<br />Weak program monitoring, evaluation and documentation hinders program effectiveness, learning, advocacy and accountability<br />Limited funding for programs<br />Limited funding for implementation research hinders our efforts to strengthen the design, management, implementation, evaluation, and financing of MN programs at scale<br />
  6. 6. Relevance to the Current Interest in Nutrition-Ag Linkages<br />Many of the overarching issues are the same<br />There is even greater difference in disciplinary lenses, availability and standards of evidence, communication barriers, intervention complexity, implementation systems, etc. are greater<br />The narrative ‘we know what works’ has different evidence beneath it, but we have similar pressures and incentives to pretend we do know, implement at scale and show results quickly<br />Therefore, we have the same need to:<br />adapt interventions, evidence and experience to new contexts <br />question assumptions during intervention design<br />identify and address bottlenecks during implementation <br />assess and document progress and impacts<br />
  7. 7.
  8. 8. Locating Evidence, Knowledge and Experience Within the Policy Process<br />Detailed intervention design and planning (policy implementation)<br />Identifying problems (problem definition)<br />Choosing solutions (policy formulation)<br />Surveys<br />Surveillance<br />Reconnaissance, etc<br />Efficacy<br />Effectiveness<br />Cost<br />Knowledge of and experience in the local<br />context *<br />Evaluation<br />Implementation<br />Operations research, <br />monitoring and<br /> QA during implementation<br />Outcomes, impacts and related factors <br />* Context includes epidemiology, implementation capacities, social acceptability, resources, competing priorities, policy preferences of actors, etc.<br />
  9. 9.
  10. 10. Program Assessment Guide (PAG)<br />Outputs<br />Processes<br />1. Action Plan to<br /> Address Barriers<br /> & Enablers<br />Participatory Procedures To Strengthen:<br /><ul><li> The Systematic Integration Of Evidence, Contextual Knowledge & Experience
  11. 11. Shared understanding, commitment, ownership, motivation & capacity to advance the micronutrient agenda
  12. 12. Links with the broader nutrition and health agendas in the country</li></ul>2. Operations<br /> Research<br /> Agenda<br />3. Issues for<br /> Inclusion in<br /> M&E<br />4. Strategic Plan to<br /> Build Support,<br /> Capacity &<br /> Sustainability<br />
  13. 13. Program Assessment Guide<br />Laying the Groundwork<br />Module 1: Clarifying the Problem and the Solution <br />Module 2: Goals and Associated Values<br />Module 3: Delivery Systems<br />Module 4: Hard to Reach Populations<br />Module 5: People, Roles and Responsibilities<br />
  14. 14. Program Assessment Guide<br />Building or Strengthening the Intervention<br />Module 6: Needs, Inputs, Activities and System Changes<br />Module 7: Action Planning<br />Building Support Systems and the Enabling Environment<br />Module 8: Monitoring, Evaluating and Quality Improvement <br />Module 9: Organizing, Leading and Managing<br />
  15. 15. Some Distinctive Features of the PAG Approach<br />Link nutrition interventions to the broader health (and agriculture and other) agendas (Modules 1,2)<br />Delivery systems as social systems (Module 3)<br />Five universal needs and local program theory (Module 6)<br />Clear definition of roles and responsibilities (Modules 5,7) and responsibility for follow-up (Module 9)<br />Systematize and stress-test contextual knowledge and experience (Module 6)<br />Build capacity for intervention planning, assessment and improvement<br />
  16. 16. Creating Local Program Theory<br />
  17. 17. A Common Example: <br />Training Health Workers to Counsel Mothers<br />The Goal:<br />Improved<br />Feeding<br />Practices<br />Train<br />Workers<br />Improved<br />Counseling<br />The Common Experience:<br />Little Change<br />In Feeding<br />Practices<br />Train<br />Workers<br />Little Change<br />In Counseling<br />Why?<br />
  18. 18. Nutrition Example: BCC IYCF Component<br />Improved<br />Feeding<br />Practices for <br />Young Children<br />Training of<br />Health workers<br />To do nutrition <br />counseling<br />“Desired<br />Outcome”<br />“Input”<br />Building a Program Model: <br />Specify Inputs and Desired Outcomes for Each Component<br />
  19. 19. Caretakers<br />Understand & Internalize the Knowledge <br />& skills<br />Training of<br />Health workers<br />To do nutrition <br />counseling<br />Caretakers <br />Improve<br />Feeding<br />Practices<br />Health workers<br />Internalize the<br />Knowledge <br />& skills<br />Health workers<br />Conduct good<br />Quality <br />counseling<br />“Desired<br />Outcome”<br />“Causal Pathway”<br />(often assumed and not made explicit)<br />“Input”<br />Building a Program Model: Specify the Causal Pathway<br />
  20. 20. “Inhibiting Factors”<br /><ul><li>Limited contact</li></ul> time with mothers<br /><ul><li> Materials not </li></ul> available<br /><ul><li> Disorganized </li></ul> counseling<br /> environments<br /><ul><li>Outdated training </li></ul> materials<br /><ul><li> Rapid turnover</li></ul> in health workers<br /><ul><li> Trainers not </li></ul> well-trained<br /><ul><li> Poor</li></ul> comprehension<br /><ul><li> No hands-on</li></ul> experience<br /><ul><li>Limited</li></ul> food<br /> diversity<br /><ul><li> No social</li></ul> support<br /><ul><li> Didactic training </li></ul> methods<br /><ul><li> Crowded </li></ul> curriculum<br />Caretakers <br />Improve<br />Feeding<br />Practices<br />Caretakers<br />Understand & Internalize the Knowledge & skills<br />Training of<br />Health workers<br />To do nutrition <br />counseling<br />Health workers<br />Internalize the<br />Knowledge & skills<br />Health workers<br />Conduct good<br />Quality counseling<br />“Causal Pathway”<br />(often assumed and not made explicit)<br />“Desired<br />Outcome”<br />“Input”<br />Building a Program Impact Model: Specify Inhibiting Factors<br />
  21. 21. The PAG Approach to Creating <br />Local Program Theory<br /><ul><li>Delivery Systems as Social Systems (Module 3)
  22. 22. The Five Needs Tool (Module 5)</li></ul>A simpler, more practical way to build a program model<br />Embraces but simplifies extensive social and behavioral theory<br />Considers all actors in the delivery system, not just caretakers and households<br />Reveals the inputs, activities and system changes needed for all links in the delivery chain to function well<br /><ul><li>The Stress-Testing Tool (Module 5)</li></ul>Is used when considering input, activities or systems changes to meet the five needs of various actors<br />Reveals hidden assumptions and oversights<br />Avoids group think in the planning process<br />
  23. 23. Module 3: Delivery systems as social systems <br />Significant Others (barriers and enablers)<br />A<br />D<br />H<br />J<br />G<br />F<br />E<br />B<br />C<br />I<br />Household<br />& Caregiver<br />Functionaries (barriers and enablers)<br />Clinic<br />Community<br />District<br />Regional<br />National<br />
  24. 24. The PAG Approach to Creating <br />Local Program Theory<br />Five Universal Needs<br />Awareness<br />Knowledge, Information, Skill<br />Motivation/ Commitment<br />Resources<br />Support from Others<br />
  25. 25. Module 6: What does each functionary and significant other NEED in order to fulfill their role/responsibility of the Sprinkles program (on a daily basis) and what inputs, activities and system changes are required to ensure their needs are met?<br />
  26. 26. Systematizing and stress-testing contextual knowledge and experience (Module 6)<br />Proposers<br />Challengers<br />
  27. 27. Systematizing and stress-testing contextual knowledge and experience (Module 6)<br />Outputs<br />Proposers<br />Challengers<br />Actions to address barriers and enablers in the Action Plan<br />Questions for Operations Research<br />Issues to track in monitoring and QA<br />
  28. 28. Experiences from Kyrgyzstan and Bolivia<br />1. Sponsorship, Responsibility, <br /> Commitment, Follow-up<br />2. Timing<br />3. Preparation<br />4. Participants<br />5. Contextuality (selection and sequencing)<br />6. Time constraints<br />7. Facilitation<br />Before the workshop<br />During the workshop<br />
  29. 29. Adaptation to Ag-Nutrition Projects<br />The initial focus is a multi-dimensional problem rather than a single solution<br />Use the UNICEF Conceptual Framework as a guide<br />Consider food, health, care simultaneously - determinants and solutions<br />The primal need is to ensure that all three are addressed<br />Conduct the five needs analysis on the “delivery systems” for food and health and care interventions<br />
  30. 30. Also,<br />Consider how changes in one domain may affect the other domains, e.g.,<br /><ul><li>Women’s time in ag versus time in child care
  31. 31. Women’s versus men’s control of income
  32. 32. Effect of crop choice on men’s versus women’s control
  33. 33. Effect of crop choice on men’s vs women’s labor in ag
  34. 34. Effect of crop choice on sales vs retention for own consumption
  35. 35. Acceptability of foods for feeding young children
  36. 36. Efficacy of foods for meeting nutritional needs of young children
  37. 37. Requirements for adopting and growing
  38. 38. Requirements for preparing, using and feeding
  39. 39. Requirements for sustaining over time (seasons and years)
  40. 40. etc</li></li></ul><li>Simplified Schematic of Delivery Systems for Ag and Health<br />Hlth<br />Ag<br />national<br />Ag<br />Hlth<br />region<br />Ag<br />Hlth<br />district<br />VHW<br />Ag<br />community<br />household<br />Food<br />Health<br />Care<br />
  41. 41. Thank You<br />
  42. 42. The Five Needs in Relation to the Broad Categories <br />of Behavioral Determinants<br />
  43. 43. Importance of an Explicit Program Model and Assumptions: Results of a Meta-Analysis<br />Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005, <br />National Implementation Research Network: Tampa, FL.<br />
  44. 44. Importance of an Explicit Program Model and Assumptions: Results of a Meta-Analysis<br />Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005, <br />National Implementation Research Network: Tampa, FL.<br />
  45. 45. Importance of an Explicit Program Model and Assumptions: Results of a Meta-Analysis<br />Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005, <br />National Implementation Research Network: Tampa, FL.<br />
  46. 46. Simple (Faulty) Program Model<br />Lectures &<br />Discussion &<br />Demonstrations<br />Teacher’s<br />Practices<br />More Complete (Effective) Program Model<br />Teachers’<br />Knowledge<br />Teachers’<br />Practices<br />Teachers’<br />Skill<br />Lectures &<br />Discussion &<br />Demonstrations<br />Coaching in<br />Classroom<br />Practice &<br />Feedback<br />
  47. 47. (An Even) More Complete Program Model<br />Teachers’<br />Knowledge<br />Teachers’<br />Practices<br />Teachers’<br />Skill<br />Lectures &<br />Discussion &<br />Demonstrations<br />Coaching in<br />Classroom<br />Practice &<br />Feedback<br /><ul><li>Disorganized classroom
  48. 48. No materials
  49. 49. Inattentive (hungry)
  50. 50. Student absenteeism
  51. 51. Teacher absenteeism
  52. 52. Teacher morale</li></ul>Student<br />Learning<br />
  53. 53. 3.3.2 Improved Nutritional Status<br />Prevent undernutrition through community-based programs that increase household resilience and are built on the proven effectiveness of community workers, who are often the best change agents to deliver nutrition education. These programs should be based on an analysis of the specific social and behavioral context in each location and promote nutrition messages aimed at improving key evidence-based practices including proper maternal nutrition, exclusive breastfeeding of infants under six months, appropriate introduction of complementary foods, and promotion of adequate water, sanitation, and hygiene practices. <br />» » Improve diet quality and diversity to prevent nutritional deficiencies and vulnerability to disease caused by low-quality diets. This can be achieved by strengthening agriculture and nutrition linkages through multiple approaches, such as increasing household production of nutritious foods, empowering women with access to income, and educating women about food utilization and nutrition as a means to improving the health and wellbeing of children in the household. We will invest in improving nutrition throughout the value chain, including research to improve the nutrient value of staple food sources and post-harvest preservation and processing techniques that increase the nutrient content of processed staple foods. <br />
  54. 54. 3.3.2 Improved Nutritional Status<br />» Improve delivery of nutrition services by health systems linked to community-based programs. Key nutrition services include community management of acute malnutrition (CMAM) and micronutrient supplementation. The CMAM approach ensures timely detection and referral of cases of severe acute undernutrition in the community to medical treatment by making services available at decentralized treatment points and through community outreach and mobilization. To save lives and reduce morbidity in the short- and medium-term, food-based approaches and prevention programs should be coupled with targeted micronutrient supplementation programs, including vitamin A supplementation for young children and iron folic acid supplementation for pregnant women. Our investments will include strengthening and integrating national and decentralized health and social systems responsible for planning, budgeting, and cost analysis of nutrition service delivery.<br />
  55. 55. The preceding slides were presented at the<br />CORE Group 2010 Fall Meeting<br />Washington, DC<br />To see similar presentations, please visit:<br />www.coregroup.org/resources/meetingreports<br />

×