• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Strengthening and Measuring Community Capacity for Sustained Health Impact_Snetro_5.4.12

Strengthening and Measuring Community Capacity for Sustained Health Impact_Snetro_5.4.12






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Good morning! Very much wishing that I were in the same room as you all, however being based in Africa makes it difficult to participate in some meetings. I guess you could say, I’m limiting my carbon footprint!! Save the Children has a long history and reputation for building and strengthening communites capacity across the sectors of health, HIV and AIDs, education and EO. In particular we have refined over time our CM approaches to deeply engage communities in getting organized, analyze of their issues, planning and monitoring their own collective action. Over the past 4 years, in particular, we have begun explore and practice methods which will allow us to measure success in community capacity building. What I will share during this brief presentation will be some of the background to our efforts, as well as share a recent effort in Zambia which quantitatively measured statistically significant change in CC in intervention districts over nonintervention districts.
  • I’d like to share with you what SC feels it does and does not know.
  • I think we can say that we have done well over the years measuring individual behaivour change outcomes. And we feel that sustaining these changes over time is of equal importance. But we have not paid enough attention to measuring the efforts which we go to to create the enabling social environment which is vital for supporting the individual change to be sustained.
  • What is the state of the art on MCC?
  • Demonstrates the difference mobilized communities (or capacitated communities) make to health and social outcomes (as opposed to un-capacitated ones). The development of abilities to solve problems, a generalized ability that transcends any or one concern Community capacity as a determinant of health Directly contributes to community change efforts
  • It is the IR 4 Social and Policy Environment within which we have been able to couch our work in MCC.
  • Just to be clear that we have a common understanding of termonology. SC uses it’s CM approach as a process to build and strengthen community capacity. For us, CM is not a campaign or a series of campaigns, - it is an ongoing capacity building process through which community individuals, groups, or organizations plan, carry out and evaluate their activities on a participatory and sustained bases to improved health, HIV and AID and other needs – either on their own initiative or stimulated by others.
  • The SC CAC is the community empowerning process which we facilitate with our communtiy partners. And on the right is the ROCK-HOP model (JHU) which illustrates an analysis of the inputs (and catalyst) required to achieve individual and social change outcomes
  • In 2006 SC with its HCP partners (JHU; AED, and the In’t HIV/AIDS) held a Partners Learning Forum in Zambia to share describe CC, share MCC experience and the state of the art. We developed a commitment to carry out further OR in order to apply what was learned during our forum, and developed a number of tools subsequently to move this work forward. SC, in particular developed MCC research protocol and carried out studies in Uganda, Nicaragua and Vietnam. In addition, along with our JHU partners we carried out MCC research in Zambia through our HCP partners. A number of MCC tools developed over the past four years have been posted on the AIDSTAR website. Included in these are: MCC Literature Review; a Meta-Analysis of CC Domains and sub-domains based on Social Science Literature; a short list of CC indicators based on the SS literature and and range of programs; a Qualitative tool for illiciting community generated capacity domains and indicators; and a quantitative endline measurement tools with community-generated indicators which had been validated.
  • OK –here in Africa (as the world over) sometimes thing get a bit rough. During our Learning Forum there was a bit of lively debate concerning who should measure what? In particular questions were asked: Shouldn’t it be communities who need to measure their own capacity – full stop. Or should it be outsiders for way of proving achievement and furthering donor funding. At SC we feel that MCC is important at multiple levels: by communities; by program implementers; and by donors…… I hope some of you are not vegetarians!
  • I wanted to share with you the HCP Program Framework, because it highlights for me what was key to the project’s success. That is from its conception mobliizng and empowering communities was a key intervention – which was needed to create the enabling environment for change. Specific approaches in the enabling environment were outlined, including increased CC capacity for greater social cohension; collective efficiacy; conflict management; leadership, etc. This greatly helped to focus project efforts and interventions as well as frame the endline results we hoped for.
  • The endline evaluation assessed specifically:
  • The MCC component of the endline was developed over a period of two years. Phase 1 focused on illiciting community-generated domains/subdomains and indicators for MCC. We adapted the Significant Change Technique (by Dart and Davies) to help communities look at what was significant change and what key skills/factors helped this change to happen. Also, how would they measure whether they were strong or weak in this particular element or factor. In phase II these indicators were validated using a survey instrument, and identifying those indicators that illicited consent responses along domains. These validated indicators were then placed into the endline study
  • I’d like to share with you the key community-generated domains and indicators used in the endline. Community explination of the key factor/skills were used to organize the domains. We have used the Social Science termonology for the definition here. They did indicate the domain of social cohesion as an important factor – that is……
  • There is a hidden message here related to the non-intensive districts versus the intensive. In the intensive districts we focus a great amount of effort on capacity building training for NHCs over a 5 year period. For the non-intensive areas we expanded our reach in the last three years through community-to-community sharing and horizontal learning between NHCs – which amounted to very similar results.
  • Some conclusions we can take away include that is it possible to use community generated indicators to measure community capacity.
  • Here are some details on the endline survey design. We had a sample size of 4000. It was a population based, household survey using a probablility sample, stratified into intervention and comparision (non-intervention) district.

Strengthening and Measuring Community Capacity for Sustained Health Impact_Snetro_5.4.12 Strengthening and Measuring Community Capacity for Sustained Health Impact_Snetro_5.4.12 Presentation Transcript

  • Community Capacity Inventory 1• Gifts of the head: (things I know something about and would enjoy talking about or teaching other about, e.g., birds, local history, music).• Gifts of the hands (things I know how to do and enjoy doing, e.g., carpentry, sports, planting, cooking, – be specific).• Gifts of the heart (things I care deeply about, e.g., children, older people, community history, environment).Adapted from World Learning, 1989
  • CORE Group Spring MeetingMay, 2012 Strengthening and Measuring Community Capacity for Sustained Health Impact Gail Snetro-Plewman, Save the Children gsnetro@savechildren.org
  • BackgroundContributors:David Marsh, MD, Senior Child Survival Advisor, Save the ChildrenCarol Underwood, PhD, Johns Hopkins University, Center for Communication Programs (CCP)Mark Boulay, PhD, Johns Hopkins University, CCP
  • Definitions of Community Capacity • “the characteristics of communities that affect their ability to identify, mobilize, and address social and public health problems (Goodman, et.al, 1998; Poole, 1997) • “the set of assets or strengths that residents individually and collectively bring to the cause of improving the quality of life: (Easterling, Gallagher, Drisko, & Johnson, 1998.) • “the ability of people and communities to do the work needed in order to address the determinants of health for those people in that place” (Bopp, GermAnn, et.al, 2000)
  • Why measure community capacity?• We predict that increased capacity helps improve health & social change outcomes.• We predict that this capacity will help sustain positive health & social change outcomes in the future.• We know that SC projects increase community capacity (CC) through its community mobilization (and other) approaches - but how much?
  • Why measure community capacity?• SC devotes a lot of effort to strengthening CC.• SC thinks we are good at strengthening CC.• But we cannot prove it because we lack systematic way to measure CC.• Yet, it seems reasonable that strengthened CC: – Helps increase the use of interventions. – Sustains this use. – Promotes “development” more broadly.
  • State of the Art -Measuring Community Capacity• Limited research undertaken globally to date testing impact of capacity building on social change outcomes – but field is growing.*• No agreed upon theoretical framework from social scientists.• Few indicators from literature have ever been tested for validity and reliability.• Myriad, unsystematic, project-specific indicators from programmers.• ‘Who’ should develop & measure? institutional vs. community?*Sources: “What is the evidence on effectiveness of empowerment to improve health.”, Feb. 2006, WHO/HENS; Johns Hopkins, Communication for Social Change, World Bank
  • Community-capacity interventionsas a means and as an end SOCIAL CHANGE Participation Social capitalCOMMUNITY Social cohesionCONTEXT Empowerment Collective Efficacy- Material resources- Social COMMUNITY Health resources CAPACITY Status- Disease burden- Experience HEALTH- etc. COMPETENCE Consistent & appropriate health behavior Community capacity designed to improve health behavior and health status via social change
  • SC/US Generic Results Framework GOAL: Status Improved SO: Use of Key Practices and Services Improved IR 1 IR 2 IR 3 IR 4 Access and Quality Knowledge and Social andAvailability of Key of Key Acceptance of Key Policy Services and Services Practices and Environment Supplies Increased Services Enabled Increased Increased
  • SC/US Generic Results Framework GOAL: Status Improved SO: Use of Key Practices and Services Improved IR 1 IR 2 IR 3 IR 4 Access and Quality Knowledge and Social andAvailability of Key of Key Acceptance of Key Policy Services and Services Practices and Environment Supplies Increased Services Enabled Increased Increased
  • CM Terminology Sustained Behavior Change Community Sustained Social Change, Mobilization Including Enhanced Community Capacity Sustained Societal Change
  • SC Community Action Cycle and “Rock-Hop” JHU Integrated Model of Communication for Social Change Catalyst Internal Change Mass Agent Innovation Policies Technology Media Stimulus Explore & Prioritize Community Dialogue Recognition of Identification and Clarification Expression of Vision a Problem Involvement of of Individual of Leaders and Perceptions and Shared the Future Stakeholders Interests Conflict-Dissatisfaction DisagreementPrepare Organize Plan Action Plan Consensus Options for Setting Assessment of to on Action Action Objectives Current StatusMobilize Collectiv e Action m C V p n o e u a v r I t f Assignment of i l Mobilization of C S E Participator y p u d o a n e s x Responsibilities Organizations Implementation Outcomes r t i l Evaluation • Individuals • Existing Community • Outcomes vs. • Media Groups Objectives • Health • New Community • Education Task Forces • ReligiousPrepare Evaluate Act • Others • Otherfor Scale Indiv idual Change Social Change • Ski ll s • Leadership • Ideation • Degree and Equity of Participation Knowledge, Attitudes, Perceived • Information Equity Risk, Subjective Norms, Self -Image, • Collective Self-Efficacy Emotion, Self -Efficacy, Social • Sense of Ownership Influence, and Personal Advocacy • Social Cohesion • Intention • Social Norms • Behavior SOCIETAL IMPACT Figueroa and Kincaid, 6/2001
  • Community Action Cycle • Implemented by the community • Fosters equity through participation of those most interested and affected by health issue • Instills ownership and works towards sustainability • Iterative – not linear • Builds community capacity for managing health and development
  • Measuring Community Capacity –Efforts to Date • Measuring Community Capacity - Partners Learning Forum: Lusaka, Aug ‘06 - HCP– learning, change, planning • Literature Review of measuring community capacity research efforts to date • Meta-Analysis of CC domains and indicators to validate • Hypothesis-testing research testing the effect of CC on achieving health outcomes in Uganda, Zambia, Vietnam, Nicaragua • Quantitative and Qualitative measurement tools
  • Literature Review: 30 C x 210 R matrixDomains of Community Capacity by Researcher**L=Laverack; W =World Bank; H=Hastings (proxy of Bopp); G =Goodman; F=Figueroa; R=Rifkin; C =Chaskin; B=Bopp;E =Easterling; S1= Study 1; S 2=Study 2; S3= Study 3; S4=Study 4; S5=Study 5; S6=Study6; S7=Study 7; S8=Study8;S9=Study 9; S10=Study10; S11=Study11; T12= Tool12; T13=Tool 13; T14=Tool 14; T15= Tool 15; S16= Study 16; T17=Tool 17; T18= Tool 18 Researcher # Domain Sub-Domain L W H G F R C B E S1 S2 S3 S4 S5 S6 S7 S8 S 1 Participation small groups, larger organizations X 2 Participation strong participant base X 3 Participation diverse network to enable different interests to act X X 4 Participation benefits over-ride costs of participation X X 5 Participation citizen involvement in defining and resolving needs X X X 6 Participation Citizen participation and control 7 Participation Environement 8 Participation Attracting and keeping volunteers 9 Participation broad, representative range of community X 10 Participation engage diverse members of community in process of consultation, collective analysis & decision-making X 11 Participation, degree & equity of access to participation X 12 Participation, degree & equity of extent and level of participation X 13 Culture of Openness & Learning openess between indiv; comfort to listen & speak X 14 Sense of ownership importance of issue or program to participants X 15 Sense of ownership sense of responsibility for program X 16 Sense of ownership contribution to the program X 17 Sense of ownership benefit from program X 18 Sense of ownership sense of ownership of "credit/blame" for outcome X X 19 Sense of ownership personal identification with program X 20 Leadership organizes groups X 21 Organizational level context Commitment to community Organizational culture and climate X 22 Organizational level context Organizational structure, procedures and authority X 23 Organizational level context Organizational effectiveness and/or sustainability X
  • Long story short CC Outcomes:10 domains with 54 sub-domains
  • Long story short CC Outcomes:10 domains with 54 sub-domains 1. Community history 2. Networks 3. Participation 4. Leadership 5. Social cohesion 6. Ownership 7. Collective efficacy 8. Resource mobilization 9. Information equity 10. Critical thinking
  • Long story short CC Outcomes:10 domains with 54 sub-domains 1. Community history 2. Networks1. Extent 3. Participation2. Equity and diversity 4. Leadership3. Flexibility 5. Social cohesion4. Skills to maintain dialogue 6. Ownership5. Vision and innovation 7. Collective efficacy6. Trustworthiness 8. Resource mobilization7. Exercise of power 9. Information equity 10. Critical thinking
  • # (%) groups effectively* formed aroundSuggested SC Community Capacity health/development issue. *60/40 rule to includeIndicators – “Quantitative” marginalized populations, including women; clear roles/responsibilities of Explore & members; rotating leadership… Prioritize # (%) community members exploring the health/development issue and setting prioritiesPrepare Organize Plan # (%) written community to action plans in placeMobilize # (%) communities completing 70% of action plan on timePrepare Evaluate Actfor Scale # (%) communities using data for decision through community bulletin boards, or health records
  • Who should measure what?Community or externally measured indicators? “Top-down” vs. “Bottom-up”
  • Measuring Community Capacity in Zambia - Context• Strengthening Community Capacity and Engendering Behaviour Change - Health Communication Partnership (HCP) -Zambia• Project Goal: To use strategic communication approaches at scale to support households and communities to take positive health action.• 5 Year – 2004-2009, $31 million Cooperative Agreement with Johns Hopkins University – Save the Children Lead in Zambia - In’tl HIV/AIDS Alliance – sub.• HCP focused on hard to reach districts; low health progress; inactive Neighbourhood Health Committees, weak community capacity Community systems strengthened to focus on health priorities they identified through community-level dialogue, and application of Community Action Cycle as a mobilizing tool for collective action Interventions integrated across health areas (Malaria, RH, Child health, HIV and AIDS, Maternal Health)
  • CM at Scale - ContextContinued… Total Population Covered: 2,848,520 • 22 Districts out of 71 country-wide (presence in all 9 provinces) • 22 District Level Health Center Partners • 1800 Community Core Groups’ -Neighborhood Health Committees (NHCs) • Application of the Community Action Cycle • 1341 with Community Action Plans • 1063 communities completed at least one activity from their action plan • 65 Safe Motherhood Action Groups formed as part of NHC’s.
  • HCP Program Framework ( 2005 – 2009) HCP Enabling Intermediate HealthInterventions Behaviour Environment Effects• Mobilising and • Increased community • Increased knowledge Illustrative: empowering • Reduced high risk • FP use capacity individuals/families Social cohesion behaviours • Birth planning communities Collective efficacy • Increased individual • Delivery at a health• Engaging leaders Conflict management /collective efficacy for facility• Empowering youth Leadership health action • Exclusive breast-• Harmonising health Effective leadership • More equitable feeding messages Individual efficacy gender and social • ITN use • Increased exposure to norms • HIV protective health messages behaviours Drama TV/video Print materials
  • Endline Evaluation assessed:• The effect of the project activities to foster improvements in the community capacities required for social dialogue and action• Increases in individuals’ knowledge, attitudes, and behaviors• Associations between program exposure & knowledge, attitudes, and behaviors• The reach of program messages through community- based and mass-media channels
  • Survey Design Con’tPhase I - Measuring Community Capacity Study• Qualitative approach for community-generated domains and sub-domains• Most Significant Change technique*Phase II – Measuring Community Capacity Study• Survey to test and validate indicators• Principal Components Analysis (PCA) & Cronback’s alpha (a) tests used to develop and assess the scales to measure community capacity domains (validate indicators)Phase III – Quantitative Population Based Endline EvaluationSignificant Change Technique, Dart and Davies, 2003
  • Description of Community CapacityDomains and Indicators for Endline: (1) Social Cohesion - Description of Domain: Seeks to measure the extent to which target communities were able to work together towards a perceived common good. Indicators: • Repay debts to others • Did not help each other in times of need (reversed) • Did not trust one another (reversed) • Strong relationships • Able to discuss problems
  • Domains and Indicators… (2) Collective Efficacy- Description of Domain Seeks to measure the extent to which target groups shared belief in its conjoint capabilities to attain their goals and accomplish desired task. It involves the “belief or perception that an effective collective action is possible to address a social or public health problem”. Indicators: • Work hard to accomplish a project • Confidence in community problem solving • Committed to the same collective goals • Solutions to problems
  • Domains and Indicators…(3) Conflict Management – Description of DomainSeeks to measure the extent to which target communities were able to handle conflicts in a way that was fair and allowed for continued participation of its members towards positive health actionIndicators:• Quick resolution to conflict• Trouble dealing with conflict (reversed)• Feuding for a long time (reversed)• Getting involved to resolve issue
  • Domains and Indicators… (4) Leadership – Description of Domain Seeks to measure the extent to which target communities had leaders with the capacity to engage the diversity of sectors and levels within community life in processes of learning and action for health. Indicators: • Women leaders • Leaders treat people equally • Leaders listen • Leaders lead by example • Leaders are good at resolving disagreements
  • Domains and Indicators….(5) Effective Leadership – Description of DomainSeeks to measure the extent to which the community has the capacity to engage the diversity of sectors and levels within community life in processes of learning and action for health.Indicators:• Participation in meetings• Setting goals & objectives• Developing a plan• Assigning tasks fairly• Obtain money from outside
  • Domains and Indicators… (6) Participation – Definition of Domain Seeks to measure the extent to which target communities can engage its own diverse membership in constructive processes of consultation, collective analysis and decision making. Indicators: • Skills and knowledge • Confidence to solve it • I can participate
  • % reporting that community worked together in past year to solve a health problem - by number of capacities
  • Percentage of NHCs reporting having 50% or more female members by intevention and comparison districts (N=89) 80 73 70 60 54 50Percentage 40 30 20 10 0 Intervention District Comparison District
  • Percentage of females in NHC leadership positions by intervention and comparison districts 50 43 42 40Percentage 30 Intervention District 20 18 17Comparison Districts 11 10 4 0 Chairperson V.Chaireperson Secretary Key Positions
  • Table 4: Adjusted odds ratios from logistic regression models predicting selected health behaviors Adjusted Odds Ratios Current use of a Received HIV test in Youngest child under 5 contraceptive past year and know years slept under a bed method 1 results1 net the past night2Community worked to address healthproblem in past year 1.00 1.00 1.00No 2.14*** 1.76*** 1.54***YesCapacity ScoreFirst quintile (lowest) 1.00 1.00 1.00Second quintile 1.17 1.17 2.49***Third quintile 0.95 1.06 2.58***Fourth quintile 1.31* 1.17 2.35***Fifth quintile (highest) 1.09 1.26 2.21***Community TypeComparison 1.00 1.00 1.00Non-intensive 0.97 0.87 0.93Intensive activity 1.05 0.92 1.07GenderFemale 1.00 1.00Male 1.08 0.46***Age15-24 1.00 1.00 1.0025-34 1.39*** 1.20* 1.71***35 and over 1.38*** 0.99 2.05***EducationPrimary or less 1.00 1.00 1.00Secondary or more 1.16 1.31*** 0.96Use media weeklyNo 1.00 1.00 1.00Yes 1.42*** 1.27** 1.39*Type of residenceRural 1.00 1.00 1.00Urban 1.09 1.18* 1.05ProvinceCentral 1.00 1.00
  • Results! Community Capacity was measured through six domains: - Participation - Collective Efficacy - Conflict Management - Leadership - Effective Leadership - Social Cohesion Significant change in 6 domains of community capacity found in all intervention districts compared to comparison districts Respondents living in an HCP Intervention District scored higher on more capacity scales than respondents in Comparison Districts. Community capacity score was directly related to the community-led collective action. Respondents living in communities with a greater level of community capacity were more likely to indicate that their community had worked together in the past year to solve a problem. Communities that ‘worked together’ were: - 2 x use a modern contraceptive method - 1.8 times more likely to have received a HIV test and know results - 1.5 x more likely to have their youngest child sleep under a ITN
  • Results Con’t• Overall 30% of community action was mediated by increases in community capacity (controlling for age; ed; media use. – Baron and Kenny, 1986).• Increases in community capacity mediated the effect on health behaviors: - 63% of contraceptive use was mediated by community action - 11% of bed net use among young children was mediated by community action - No direct or indirect effect on HIV testing
  • Conclusions:  Community-generated capacity indicators were identified, validated and used to measure improved capacity  First time community capacity index validated and applied to a population based endline survey  Significant changes in community capacity measured in intervention areas over comparison areas  Changes in community capacity directly attributed to increased community collective action for health  Greater community capacity was significantly related to an increase in specific health outcomes!  Strengthening community capacity, in this instance, was both a means to an end – improved health behaviors and reported collective action for health – and an end-in-itself, both of which are vital to social development.
  • Still work to be done! • Test these measures in other settings • Simplify measurement tools and approaches • Standardizing the integration of measuring community capacity into practice Please join us in the challenge!
  • Thank you!
  • Web & Other Recent CMReferences • How to Mobilize Communities for Health and Social Change – http://www.hcpartnership.org/Publications/Field_Guides/Mobilize/htmlDocs/cac.ht • “Rock Hop” – http://www.comminit.com/strategicthinking/stcfscindicators/sld-1500.html • Effects of a participatory interventions with women’s groups on birth outcomes (India), Lancet, April 3, 2010, T. Tripathy, et. al. • Community Participation: Lessons Learned for Maternal and Child Health, M. Rosato, G. Laverack, L Howard-Grabman, et.al, Lancet, 2008: 372: 962-71 • An Evaluation of the Community Action Cycle, T.Kabore, HIV/AIDS Care and Support Program, Ethiopia, May, 2009.
  • More References: • Measuring Community Capacity for Better Health and Social Outcomes - paper by Save the Children. Detailed overview of initiative to measure community capacity in Uganda, Nicaragua, Vietnam and Zambia. •   Zambia Phase 1 PR Discussion Outline - qualitative tool to elicit community generated capacity indicators, Save the Children/HCP • HCP/ZambiaEndlineSurvey - WomenQuestionnaire - quantitative endline household instrument. Section 1 A: Perception of Community includes the community generated indicators that had been validated, Save the Children/HCP • Community Observation Checklist – to validate CC findings, Save the Children/HCP
  • •What experience has your organization on measuring CC?•What tools/materials do you use? What do you think?•Would you be interested in further measuring CC? If yes, what support/materials/tool would be helpful? 1. What experience does your organization have on strengthening CC? 2. What tools/materials do you use? 3. What experience has your organization on measuring CC? 4. What tools/materials do you use? 5. Would you be interested in further measuring CC? 6. If yes, what support/materials/tool would be helpful?
  • Endline Survey Design – Baseline (2005); Endline (2009) – Sample Size: Baseline (3,000); Endline (4,000) – Endline fielded in 21 intervention districts (Kalabo dropped) 13 comparison districts – External evaluation conducted by DCDM (Baseline); Glow Consultancy (Endline) – Probability Sample – One male & female respondent from each sample household; over 80% sample coverage rate – Stratified into intervention and comparison districts; urban/ rural – Intervention districts refer to where HCP directly implemented programs