1. Vibrant Communities Canada: Measuring Impact Donna Jean Forster-GillTamarack-An Institute for Community Engagement
2. Why Vibrant Communities?
3. The Impetus
4. The Complex Nature of Poverty“Poverty is a complex issue. There is no singlecause and no one solution. Its successfulreduction, and ideally its eradication, require aset of linked interventions undertaken by allorders of government working in collaborationwith communities.”Poverty PolicySherri Torjman,Caledon Institute of Social PolicyOctober 2008
5. What isVibrant Communities?
6. A Pan-Canadian initiative exploring comprehensive, multi-sector approaches to poverty reductionLaunched in 2002 by three national partners •  Tamarack: An Institute for Community Engagement •  The Caledon Institute of Social Policy •  The J.W. McConnell Family Foundation
7. Vibrant CommunitiesAn experiment designed to test a specific way to address the complex realities of poverty through local level action. Theory of Change:Guided by 5 principles & assisted by extra supports provided by national sponsors – local organizations and leaders could revitalize poverty reduction efforts in their communities and generate significantly improved outcomes.
8. We believed that using a comprehensive,multi-sector approach communities can … •  Raise the local and national profile of poverty •  Build a constituency for change •  Encourage collaborative ways of working •  Begin to shift the systems underlying poverty •  Generate substantial changes for a large number of people living in poverty.
9. Who are theVibrant Communities?
10. 13 Vibrant CommunitiesPart One – Exploring Principles + Niagara Region
11. How doesVibrant Communities work?
12. What are the 5 components of Vibrant Communities?
14. Comprehensive Thinking & Action address the interrelated root causes of poverty rather than its various symptoms Create Community Knowledge
15. Community Asset Building building on community strengths rather than focusing on deficitsDevelop a Community Aspiration
16. Community Learning and Change embracing a long-term process of learning and change rather than simply undertaking a series of specific interventionsCreate aFrameworkfor Change
17. Poverty Reduction a focus on reducing poverty as opposed to alleviating the hardships of living in povertyMove towards Systemic Change
18. Impact ofVibrant Communities?
19. Collective Impact •  Common agenda •  Shared measurement systems •  Mutually reinforcing activities •  Continuous communication •  Backbone support organization –  John Kania and Mark Kramer, Winter 2011
20. Measuring the impact ofVibrant Communities – Shared Measurement
21. Original Approach•  Logic Models, Outcomes and Outcome Tracking. (Traditional Evaluation methods)•  Used for first 2-3 years of projectSPEED BUMP:•  Approach did not fit with what was happening on the local level•  Shifted to developmental evaluation approach
22. Developmental Evaluation in Vibrant CommunitiesStream One –each local community to articulate the ‘theory of change’guiding its work and to reflect annually on how those ideas were playingout in practice.Stream Two –communities to prepare brief stories describing theirspecific poverty reduction strategies: the challenge addressed, thestrategy employed and the outcomes anticipated. These stories helpedguide their outcome tracking efforts.Stream Three –communities complete semi-annual statistical reportsfocussed on two main targets: number of partners participating in theirwork and number of low-income households benefitting from theirefforts. It also asked for an annual narrative report elaborating on theoverall development of each community’s initiative, including its efforts tobuild community capacity and impact wider systems that contribute topoverty.
23. What was captured:•  Stories and Lessons Learned•  Expected and unexpected outcomes•  Most significant change•  Statistical Reporting and Narrative reporting focused on 5 types of outcomes initially.•  Used sustainable livelihoods model with 5 outcomes areas, 13 indicators and many sub- indicators.•  After 5 years, this was revised to add outcomes around community capacity building and policy and systems change
24. Indicators of Community Change Policy and •  Changes in public policy •  Changes in service and support systems Systems •  Changes in material resources Change •  Changes in community-level assets Community •  Convening capacity •  Multisectoral leadership Capacity •  Collaboration Building •  Community awarenessIndividual and •  Personal assets •  Physical assets Household •  Social assets •  Human assets Assets •  Financial assets
25. Inner resources Basic material goods & services • Self-awareness • Self-esteem and self- •  Emergency supports confidence •  Food, Housing • Hope and motivation •  Transportation •  Dependent care Income, Savings Financial Social Relationships•  Employment income Assets Assets and Networks•  Non-employment •  Civic participation income Human •  Support networks•  Savings and financial Assets assets•  Reduced debt/costs Skills, knowledge, education & health •  Health, Life skills •  Financial literacy •  Education •  Employment Skills
26. Why Developmental EvaluationWorked for VCAt the local level: required reflecting on thetheory of change and upgrading it as requiredto better achieve desired outcomes, rapidresponse to a changing environment, andability to capture the emerging insights andquestions of participants.At the national level: it is about mining the on-the-ground experience of communities forpatterns and themes that helped us understandthe value of this approach to reducing poverty.
27. Challenges in measuring the impact of the Vibrant Communities Initiative
28. Overall Challenges:•  The sheer scope of the work: Lots of time and energy required•  ‘attribution issues’: whose efforts are responsible for outcomes achieved or roles that different partners play.•  The guiding ideas are often left implicit making it hard to assess their validity.•  Evaluation processes need to be highly flexible due to emergent nature of the process.•  Outcomes are integral, but are often difficult to define and measure.•  The long-term focus must be reconciled with the need to track progress in the near- and mid-term.•  appropriate adjustments must be made along the way
29. Most significant challenge:The evaluation needed to accommodatesubstantial differences across the sites whilecontinuing to meet collaborative objectives.National Level:•  Managing the volume of data•  Getting materials submitted on time•  Lack of evaluation advisory group to guide the work•  More collaboration needed between evaluators and local level•  Takes lots of time and concentrated effort
30. Local Level Challenges:•  Keeping up with evaluation demands•  Balance between hard numbers and story•  National Level questions did not always work well at the community level•  Staff turnover, inadequate evaluation training for the communities•  Often off the side of the desk rather than designated staff person•  Takes lots of time and concentrated effort•  Not single organization but collaborations
31. What have we learned?
32. How much more information do we needto know that a hungry child will not dowell in school? Stop admiring theproblem and get on with the work. Mark Chamberlain CEO, Trivaris
33. Evaluating a comprehensive, multi-sector approach to poverty reduction is … 1.  Hard work, messy and time consuming 2.  Fluid, flexible and requires adapting to constantly changing conditions 3.  Requires outside eyes to hold the evaluation pieces. 4.  Orgs can benefit from training in evaluation 5.  Everyone has to develop the evaluation framework
34. Common Success Factors•  Influential and credible convener(s)•  Cross-sector, connected leadership table•  Challenging community aspiration•  Clearly articulated purpose and approach•  High degree of resident mobilization•  Research which informs the work and captures shared impact
35. Reporting to the Community – Continuous Communication
36. Vibrant Communities (2002 - 2010) Evaluation Report Reflecting on Vibrant Communities: 2002-2006Understanding the Potential & Practice of Comprehensive,Multi-sector Efforts to Reduce Poverty - The PreliminaryExperiences of the Vibrant Communities Trail BuildersTo learn about the background of collecting the VC bythe Numbers reports:http://tamarackcommunity.ca/downloads/vc/VC_By_the_Numbers_FAQs_032511.pdfIn From the Field - Exploring the First PovertyReduction Strategies Undertaken by Trail Builders in theVibrant Communities Initiative
37. Where are we headed?The Landscape has Changed…since 13 cities began to experiment…•  Municipally: 84 collaborative poverty reduction roundtables have connected to Vibrant Communities•  Provincially: 11 provinces and territories have or are developing poverty reduction strategies•  Federally: A new all-party Roundtable has been formed to focus on poverty, the Government of Canada – HUMA committee, Senate Roundtable on Cities and Federation of Canadian Municipalities have identified poverty as a critical issue
38. Our Aspiration: Imagine…100 cities reducing poverty TOGETHER