Community and Social Change in ASRH Programs Strategies For Measuring Change

2,677 views
2,571 views

Published on

This presentation was given at the International Family Planning conference in Kampala, Uganda in November 2009 by IRH Georgetown and the Extending Service Delivery (ESD) Project.

Published in: Health & Medicine, Business
1 Comment
2 Likes
Statistics
Notes
  • true so true loved your slide and health info! http://www.fithuman.net/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,677
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Community and Social Change in ASRH Programs Strategies For Measuring Change

  1. 1. 1<br />Community and Social Change in ASRH Programs: Strategies for Measuring Change<br />Uganda International Family Planning Conference, <br />Kampala, November 2009<br />Facilitated by <br />Cate Lane, Pathfinder International/ESD Project, <br />Susan Igras, Institute for Reproductive Health/ Georgetown University<br />
  2. 2. 2<br />Workshop Objectives<br />By the end of the session, participants will be familiar with: <br />Language (and concepts) to articulate more precisely the role of community involvement in improving ASRH<br />Key issues about monitoring and evaluation of youth reproductive health and HIV prevention programs, and <br />Issues relating specifically to evaluating community involvement programs for youth. <br />
  3. 3. 3<br />Workshop Agenda<br />Block 1<br />Introductions (15 min)<br />Community involvement and ASRH and<br />Monitoring and evaluation issues in ASRH and CI programs (30 min.)<br />Block 2<br />Application and reflection 1: Case studies from Haiti (Robin Anthony Kouyate, ACCESS-FP) and Kenya (Pam Ondusu, Pathfinder International)(60 min.)<br />Block 3<br />Application and reflection 2: Case study from northern Uganda (AbejaApunyo, Pathfinder International) and small group work (30 min.)<br />Block 4 Final discussion and summary (30 min.)<br />
  4. 4. 4<br />Importance of Community Involvement in ASRH Programs<br />IAWG on Community Involvement in RH and HIV Prevention Programs<br />
  5. 5. 5<br />Why is it Important to Involve Communities in ASRH Programs?<br />Within your ASRH programs, do you set goals and objectives to define and measure success in involving communities (whether individuals, groups, or institutions)?<br />
  6. 6. 6<br />Why is a Framework Needed?<br /> Challenges in articulating added value of community-driven interventions<br />Good development practice or a unique program intervention?<br />Process or outcome?<br />Current definitions of success of ASRH programs often limited to service uptake and knowledge gains<br />
  7. 7. 7<br />Why is a Framework Needed?<br />Lack of written goals and objectives against which to measure successful community involvement in ASRH/RH programs<br />Need more research and program evaluation to provide evidence of unique value of community participation<br />Need more systematic knowledge of better program practices to guide resource allocations<br />
  8. 8. 8<br />A First Step<br />Develop a conceptual framework<br />To guide program planners in thinking about and ‘objectifying’ realm of outcomes of community processes in ASRH programs<br />To help evaluators in thinking about monitoring and evaluation of community involvement in ASRH<br />
  9. 9. 9<br />IAWG Members<br />Advocates for Youth <br />CARE International (US) *<br />CEDPA<br />PathFinder International<br />Save the Children Federation (US)<br />UNFPA *<br />Several independent consultants<br />* Co-hosts of the IAWG<br />
  10. 10. 10<br />Nepal Operations Research Study – Is Participation the Answer?<br />Tested effectiveness of participatory approaches to define and address RH concerns of youth.<br />CONTROL SITES<br />Standard intervention set<br />No participant input<br />Didactic training approach<br />Program and training limited<br />Focus on RH info & services<br />STUDY SITES<br />Community-determined interventions<br />Participant input in design and training<br />Interactive program<br />Participatory training approach<br />Holistic programming<br />
  11. 11. 11<br />Results from NepalSource: Mathur et al. YRH in Nepal: Is Paraticiaption the Answer, NY: EngenderHealth and ICRW, 2004<br />Participatory approaches yielded more positive results than traditional RH interventions<br /> - Improvement in KAP in both sites - no significant difference in KAP between sites<br /> - Substantial difference in broad contextual factors that influence YRH – <br />norms, empowerment of youth and adults, development of stronger community institutions.<br />Participatory approaches enable incremental progress that would not be achieved with a more ‘top-down’ approach.<br />
  12. 12. 12<br />CATALYST<br />COMMUNITY<br />CAPACITY<br />Expressed through<br /><ul><li> Individuals,
  13. 13. Institutional structures,
  14. 14. Social systems</li></ul>Influenced by<br /><ul><li> Supporting and constraining factors</li></ul>(e.g., Laws and policies. Social norms. Distribution of power and resources)<br />BROADER BASE OF COMMUNITY SUPPORT<br />Favorable attitudes. Greater resource commitment. Institutionalized youth serving institutions<br />AND<br />IMPROVED<br />ADOLESCENT SEXUAL & REPRODUCTIVE HEALTH<br />COMMUNITY INVOLVEMENT PROCESSES<br />Dialog and public debate leading to<br />SOCIAL CHANGES<br />(Leadership. Participation. Info equity. Collective self-efficacy. Sense of ownership. Social norms.)<br />Community Engagement<br />STRUCTURAL CHANGES<br />(Health services organization. Ability to access resources. Commitment to ASRH issues.<br />ASRH ISSUE or CONCERN<br />Collective Actions<br />INDIVIDUAL CHANGES<br />of those involved<br />(Skills, Ideation, Intention. Behavior)<br />Results or<br />Outcomes<br />Impact<br />Existing state or<br />Baseline<br />Processes or <br />Outputs<br />
  15. 15. 13<br />Role of Catalysts in Engaging Communities<br />Provide spark to begin engaging community publicly around sensitive ASRH issues<br />Internal sparks, eg, a women’s rights group<br />External sparks, eg, a program supported by an NGO<br />
  16. 16. 14<br />Community Involvement Processes<br />Process by which community engages and acts collectively to address a condition or issue affecting the community.<br />Basis is dialogue and public debate (communication for social change precept)<br />Principles of involvement by external catalysts or ASRH programs: <br />Taking a ‘back seat’ / build organizational / social capacity <br />Supporting ASRH/RH activities defined by community<br />Acknowledge and try to minimize power & resource differentials<br />
  17. 17. 15<br />Results Across a Spectrum<br />Structural Change<br />• Well-functioning services <br />• Effective inter-organizational networks <br />• Inclusiveness of social networks<br />• Org’al commitment to ASRH<br />• Access to resources<br />
  18. 18. 16<br />Looking Closer at Individual Change: Adolescents and Adults<br />Individual Change<br />ADULT GATEKEEPERS<br />SKILLS: Improved parent-child communication on SRH-related issues<br />IDEATION: Level of awareness of pregnancy risk of adolescents<br />INTENTION: Intention to grant adolescents rights to SRH information and services <br />BEHAVIOR: Improved family support for adolescent seeking SRH services<br />
  19. 19. 17<br />Illustrative Outcome Indicators of ASRH Programs<br />Structural Change<br />ASSET BASE: Existence of laws and policies supportive of ASRH rights<br />SYSTEMS’ EFFECTIVE FUNCTIONING: Health service have institutionalized ASRH-friendly services<br />INTER-ORGANIZATIONAL CONNECTIONS: Existence of inter-organization collaboration on specific ASRH issues<br />RESOURCE ACCESS: Increased resources for ASRH-related activities in an organizations’ health program<br />
  20. 20. 18<br />Looking Closer at Social Changes<br />Important to foster same level of interest, attention, technical support and resources to social change results.<br />Not as easy as measuring <br />individual or structural changes,<br />But critically important to <br />achieving broad-based and sustained <br />individual changes.<br />
  21. 21. 19<br />Measurement of Results: Illustrative Outcome Indicators of ASRH Programs<br />
  22. 22. 20<br />Monitoring and Evaluation of ASRH/CI Programs<br />
  23. 23. 21<br />CATALYST<br />COMMUNITY<br />CAPACITY<br />Expressed through<br /><ul><li> Individuals,
  24. 24. Institutional structures,
  25. 25. Social systems</li></ul>Influenced by<br /><ul><li> Supporting and constraining factors</li></ul>(e.g., Laws and policies. Social norms. Distribution of power and resources)<br />BROADER BASE OF COMMUNITY SUPPORT<br />Favorable attitudes. Greater resource commitment. Institutionalized youth serving institutions<br />AND<br />IMPROVED<br />ADOLESCENT SEXUAL & REPRODUCTIVE HEALTH<br />COMMUNITY INVOLVEMENT PROCESSES<br />Dialog and public debate leading to<br />SOCIAL CHANGES<br />(Leadership. Participation. Info equity. Collective self-efficacy. Sense of ownership. Social norms.)<br />Community Engagement<br />STRUCTURAL CHANGES<br />(Health services organization. Ability to access resources. Commitment to ASRH issues.<br />ASRH ISSUE or CONCERN<br />Collective Actions<br />INDIVIDUAL CHANGES<br />of those involved<br />(Skills, Ideation, Intention. Behavior)<br />Results or<br />Outcomes<br />Impact<br />Existing state or<br />Baseline.<br />Processes or <br />Outputs.<br />
  26. 26. 22<br />Purpose of Monitoring and Evaluation (M&E):<br />Identify factors (individual, community, programmatic) that influence health outcomes<br />Strengthen the design and implementation of programs<br />Improve use of existing resources, thereby increasing cost-effectiveness<br />Ultimately, to better the human condition<br />
  27. 27. 23<br />Programs Vary Widely in Scope and Content, but Have Similar Structure for M&E:<br />Objectives (implicit or explicit)<br />Intended outcomes<br />Target population<br />Mechanisms to deliver services<br />A “theory of change” <br />Implicit, or<br />Explicit – a conceptual framework that provides rationale for program existence<br />Very important for M&E<br />
  28. 28. 24<br />Standard Logic Model of a Community Involvement Program<br />Population<br />level<br />Program level<br />Outcomes<br />Inputs<br />Processes<br />Outputs<br />Resources:<br />Staff<br />Curricula, <br />Supplies<br />Equipment<br />Activities:<br />Training<br />Logistics<br />IEC<br />Service:<br />% PE trained offering<br />service<br /># community/youth/YSO<br /> members:<br /><ul><li>Engaging in dialogue
  29. 29. Acting on issues
  30. 30. Advocating</li></ul>Intermediate:<br />Social changes<br />Structural changes<br />Individual changes<br />Final:<br />Infection rate<br />Mortality<br />Unintended preg.<br />
  31. 31. 25<br />Quantitative & Qualitative Methodologies<br />Quantitative data describe WHAT people do/think<br />Useful for tracking trends and highlighting differences<br />Qualitative data explore WHY people behave and think as they do<br />Useful for understanding the context in which the trends/differences occur and interpreting quantitative data<br />
  32. 32. 26<br />Why CI Programs Are Often Not Evaluated<br />Programs are multi-sectorial <br />Programs are targeting communities and not necessarily individuals<br />Indicators difficult to measure <br />Process – community engagement; collective action – no good indicators of quality/success<br />Output – numbers served, with what activities; possibility for double/triple counting<br />Outcome - Social changes, structural changes, individual changes – but these are affected by so many other things<br />
  33. 33. 27<br />Evaluation Challenges of Most Youth Programs Including CI Programs<br />Limited evidence of effectiveness <br />Hard to separate contribution of multiple strategies<br />Non-standardized definitions and indicators<br />Climate of urgency around results but behavior change is long-term<br />
  34. 34. 28<br />Challenges Evaluating Youth Programs Generally<br />Measuring influences on behaviors that didn’t occur is difficult<br />Measuring behaviors at a variety of developmental levels can be problematic<br />Showing links with outcomes is not obvious<br />Some changes may not be measurable for a long time; others are hard to measure<br />Attributing changes to a particular strategy is difficult<br />
  35. 35. 29<br />Other Challenges - Generally<br />Programs and measurement often equate age with sexual experience<br />Survey administration<br />Interviewer administered vs. self-administered<br />Literacy matters<br />Location might matter<br />Consent/Assent issues<br />May depend on location of study (clinic, school, household)<br />
  36. 36. 30<br />Additional Challenges Evaluating Youth Programs that are CI<br />Participatory approaches (evaluation) may end up being part of the intervention<br />Lack of tested/accepted indicators<br />By nature, the programs are complicated with numerous levels of programming<br />Consider strategies to measure program ‘participation’ and ‘reach’<br />
  37. 37. 31<br />Don’t Give Up – Evaluation Data/Findings are Crucial<br />Development of standard framework (IAWG on CI) and proposed indicators<br />Need to validate indicators; provide depth on how to measure<br />Consider a variety of data sources including KAP surveys (at community level) and qualitative data among community members (youth, adults)<br />
  38. 38. 32<br />Don’t Give Up – Evaluation Data/Findings are Crucial, cont.<br />Choose an evaluation design that is feasible relative to evaluation questions, program resources, and timeframe<br />Consider complicated ethical issues because of focus on youth and communities<br />Parental consent; youth assent<br />Small communities – risk of deductive disclosure<br />Get input from evaluation experts and M&E resources!<br />
  39. 39. 33<br />Measuring Social Change in ASRH programs – Where are we?<br />
  40. 40. 34<br />Social Change Indicators & Data Sources<br />Indicators<br />Development of indicators still evolving <br /><ul><li>Eventual standardized social change indicators (eg, leadership, collective self efficacy)
  41. 41. Little work yet on social change indicators relating to ASRH specifically</li></ul>Most indicators are qualitative<br />Some work on quantitative indicators<br />eg, individual psychological self-efficacy indices adapted for collective self-efficacy, <br />Likert scales on social change, eg, perceptions of leadership and collective ownership of an issue<br />
  42. 42. 35<br />Social Change Indicators & Data Sources<br />Data sources<br />Information collection is KI interviews, FGDs, structured observations (limited use of surveys)<br />Need to define ‘target populations’ in which to assess changes – individuals (youth / adult), social groups/networks, institutions<br />Preparing to expect the unexpected <br />Use of qualitative methodologies such as Most Significant Change to capture unexpected change<br />
  43. 43. 36<br />Community Evaluation of their Involvement Processes and Outcomes<br />Important evaluation issue if respect community empowerment approach to addressing ASRH issues<br />What happens inside the ‘conceptual box’ of community involvement?<br />Use of qualitative methodologies such as Most Significant Change to capture changes observed by communities<br />Use of participatory, qualitative methodologies to facilitate community reflections <br />Guidelines from Communication for Social Change Consortium<br />
  44. 44. 37<br />Useful resources:<br />“Community Pathways to Improved Adolescent Sexual and Reproductive Health: A Conceptual Framework and Suggested Outcome Indicators”<br />http://www.unfpa.org/publications/detail.cfm?ID=377&filterListType=5<br />“Communities Measure Change” (sourcebook on facilitating community measurement of social change processes) <br />http://www.communicationforsocialchange.org/publications-resources.php?id=281<br />“Communication for Social Change: An integrated Model for Measuring the Process and its Outcomes” (presentation of the communication for social change model with suggested output and outcome indicators) <br />http://www.communicationforsocialchange.org/publications-resources.php?id=107<br />“Using Narrative Methods to Link Program Evaluation and Organizational Development” (Evaluation Exchange discussion paper on application of Most Significant Change methodology)<br />http://www.gse.harvard.edu/hfrp/eval/issue24/pp3.html<br />
  45. 45. Case study questions:<br />What community involvement approaches were used? <br />How could the project strengthen its CI approaches?<br />How could you measure changes in CI that lead to improved ASRH? (Think about indicators and data sources)<br />38<br />

×