Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Use - Measuring Impact to Improve Performance
Similar to Use - Measuring Impact to Improve Performance (20)
Recently uploaded
Recently uploaded (20)
Use - Measuring Impact to Improve Performance
- 1. “THE USE” – IMPACT MEASUREMENT TO DRIVE IMPROVED PERFORMANCE 10 DECEMBER 2013 Harry Hoare, Associate harry.hoare@socialfinance.org.uk Social Finance is authorised and regulated by the Financial Service Authority FCA No: 497568
- 2. USING IMPACT MEASUREMENT TO SAVE THE WORLD Use: more resources to tackle society’s problems Before Service During Service Use: building an evidence base ©Social Finance 2013 After Service Use: better performance, better results 2
- 3. TODAY’S WORKSHOP This workshop aims to explore some of the (many) uses of impact measurement within a social investment context. It will focus on: • Introduction: who is Social Finance and what is a Social Impact Bond (SIB)? • Uses of Impact Measurement: • • • To raise social investment To improve performance To build an evidence base • What is beyond measurement? FOLLOWING THE PRESENTATION WE WOULD LIKE TO CARRY OUT WORKSHOP ACTIVITIES TO THINK THROUGH HOW THIS RELATES TO BEYOND MEASUREMENT ©Social Finance 2013
- 4. SOCIAL FINANCE – TWO WORLDS TALKING ©Social Finance 2013 4
- 5. ABOUT SOCIAL FINANCE 5 GOVERNMENT SOCIAL ORGANISATIONS INVESTORS Social Issues DELIVER SUSTAINABLE AND SCALABLE SOCIAL CHANGE ©Social Finance 2013
- 6. PETERBOROUGH SOCIAL IMPACT BOND EXISTING PROBLEMS Housing Drugs Alcohol Mental Health Poverty Work Training Education Relationships Attitude ©Social Finance 2013 6
- 7. WHAT IS A SOCIAL IMPACT BOND (SIB)? 7 INVESTORS £5 million Return depends on success SOCIAL IMPACT PARTNERSHIP Payment based on reduced convictions MINISTRY OF JUSTICE/ BIG LOTTERY FUND Reduction in re-offending St. Giles Trust SOVA MIND Ormiston Trust Other Interventions Providing specialised support pre- and postrelease to high/medium risk clients Providing volunteer support pre- and postrelease to low risk clients Low level mental health support to prisoners while they are in prison and post release Support to prisoners’ families while they are in prison and post release Support needed by the prisoner, in prison and the community. Funded as the need is identified 3,000 male prisoners sentenced to less than 12 months ©Social Finance 2013
- 8. HOW DO YOU MEASURE IMPACT TO RAISE SOCIAL INVESTMENT? 8 SIB contracts require government to pay for improved social outcomes. We call the ways of measuring these outcomes “metrics” – they need to capture financial and social improvements. Metrics • Reduction in reoffending Offenders Vulnerable Children NEETs Entrenched Rough Sleepers ©Social Finance 2013 • Reduced time spent in state care • Improved behaviour at school • Educational attendance/attainment • Employment • • • • Stable accommodation Reduced street nights Employment Health & wellbeing Investment raised The Peterborough SIB raised £5million in 2010 and will work with 3,000 offenders over 6 years £3mil SIB to fund work with children and their families at the edge of care or custody in Essex £6.9mil for 2 SIBs supporting 2,500 at risk teenagers in Thames Valley and Lancashire £5mil for a SIB to provide innovative new services to London’s entrenched rough sleepers
- 9. MEASURING IMPACT TO IMPROVE PERFORMANCE – PERFORMANCE MANAGEMENT 9 Service delivery Review meetings •Monthly contract review meetings and weekly calls with service providers to discuss dashboards, expenditure and service improvement •Performance manager meetings with local stakeholders to discuss progress. Data •Daily data entry from caseworkers onto a shared IT system Analysis •Analyst support to collate and analyse data in order to produce data dashboards – snapshots of programme implementation and outcomes SOCIAL FINANCE SUPPORTS VCS ORGANISATIONS IN TRANSITIONING TO THIS NEW WAY OF WORKING ©Social Finance 2013
- 10. EXAMPLE DATA DASHBOARD TO MONITOR OUTCOMES ©Social Finance 2013
- 11. HOW PERFORMANCE MANAGEMENT IMPROVES DELIVERY (1/2) Year One – 5 basic services YEAR ONE SERVICE DELIVERY Family Relationships Addiction Health and Wellbeing Benefits Housing ©Social Finance 2013
- 12. HOW PERFORMANCE MANAGEMENT IMPROVES DELIVERY (2/2) YEAR TWO SERVICE DELIVERY ©Social Finance 2013
- 13. MEASURING IMPACT TO BUILD AN EVIDENCE BASE 13 Example: SIB in Essex to support vulnerable children Social Issue Target Population What was the problem Essex identified? Which group of children would most benefit? Poor outcomes and high costs associated with children entering the care system 11-16 year olds with behavioural problems or family breakdown ©Social Finance 2013 Intervention What services could improve outcomes for this group? Multi-Systemic Therapy Outcomes Metrics How should success be measured and paid for? Total number care placement days saved
- 14. WHAT IS BEYOND MEASUREMENT? SAVING THE WORLD Germany Scotland • Canada • • 14 • Manifesto commitment from government Bertelsmann Foundation supporting early stage market analysis Israel Federal Government (HSRDC) call for concepts • • Exploring applications in criminal justice, homelessness and aboriginal affairs Government interest Plan emerging around employment for orthodox communities Pakistan • US • Massachusetts developing concepts in youth justice and homelessness • Connecticut, New York State and Minnesota developing SIB projects • New York City (with Goldman Sachs / Bloomberg Foundation) launched youth justice SIB 2012 Low cost private schools case study ©Social Finance 2013 Ireland Swaziland Uganda • Manifesto commitment • • • Advisory group and crossgovernment steering group exploring several areas Antiretroviral treatment as prevention of HIV and TB case study • Rhodesian sleeping sickness case study Access to quality secondary education case study Australia • New South Wales codevelopment for applications around reoffending and out of home care • Federal government interest
- 15. THANK YOU FOR YOUR TIME NOW WORKSHOP – LED BY YOU! ©Social Finance 2013 15
Editor's Notes
- History of SF – set up in 2007 Difficult to describe to people at parties – are we finance people or social sector people? I enjoyed last year’s SIAA conference because “Ich bin ein sozialer Wirkungsanalyst” made me feel amongst kindered spirits!Mix of (reformed) bankers and people with social sector experienceInitially financial and social worlds don’t seem to mix well“Two worlds talking” = sometimes people don’t understand each other in the officeStolen this slide from our brand consultants: logo is supposed to illustrate 2 worlds (although a friend helpfully pointed out that the letter F rotated through 360* makes a circle (as does any object))
- What does Social Finance do? We work with 3 sets of stakeholders. Most famous for a product called the social impact bond (which we’ll come on to later)What is it like working with these 3 stakeholders – what have we learnt about their different needs?Government wants: to fund recovery services if they work/save moneySIB provides: a way of testing recovery investment at low riskWhat is a social investor – risk/return/social impactExamples of social impact: reduced crime / more people in recovery / increased employment / less isolated old peopleInvestors want: to invest in systemic changeSIB provides: sustainable model with returns if successfulSocial organisations want: to develop and scale effective services, contract that allows flexibilitySIB provides: adaptive “black box” focus on what works with risk at least partially taken by investors
- Specific problem that Peterborough was developed in response to: No services for short sentence offenders – resulting in majority locked back up within a year£47 discharge grant but not supportBeing inside makes employment, housing, family relationships harderStory of Gareth: hadn’t spend longer than 6 months out of prison in adult life – “frequent flyer”. Complex mental health and alcohol needs
- Specific services delivered: through the gates work, mentoring, low level mental health support, work with families, better signposting into existing government services£5mil to test the hypothesis that working with these guys will reduce reoffending, reduce crime, save money.SIBs designed to address:Complexity – not knowing what service would work bestCoordination failure – getting charities to work together and coordinate with statutory provisionInsufficient evidence base – SIBs need to have an area where services aren’t too risky e.g. Words4Weapons (addressing knife crime through poetry) or too much of a safe bet (vaccination) as government should use its own money for thisInadequate funding – investment brings new resources to bear
- 1st USE IF IMPACT MEASUREMENT – BEFORE SERVICENeed to raise resources to make something happen – how do you do this?SIB is one way to attract EXTRA resources to help address social problems. Which social problems?Offending / vulnerable children / NEETs (explain = not in education employment or training) / Rough sleepersExplain what each of these SIBs means…How do you persuade investors and government that this is a good deal for both of them?At the heart of the SIB contract is the outcome metrics.Good metrics characteristics:Improve outcomes for society and end user – better life chancesSustainable – measure over long termProviders work together – collective goal / shared definition of success – the ONE serviceAvoid perverse incentives – example of waiting in ambulance for A&E or Hong Kong paying for rats’ head during infestation OR avoiding care in Vulnerable Children exampleWhat we learned:Government want metrics that link to cost savings & favour complexityInvestors favour simplicity Creates tension that needs to be worked out
- 2ND USE OF IMPACT MEASUREMENTOne Service manages 4 charities to deliver servicesPerformance management about measuring needs of clients and what activities they have undertakene.g. KPI = % engaged with the service% met at the gateGoals of performance managementInform allocation of resourcesBuild understanding of what is effectiveIdentify service failures as soon as they developProvide basis for re-structuring programme while operatingChallenges for social organisations:Additional reporting and measurement – lots of extra time logging on and entering info e.g. one person who was not computer literate spent 2hrs trying to log in to systemWorking with investors – different worlds talkingAdapting your model – some things may not be working!Commitment to evidence-based procedure – a new way of working
- Data dashboard developed with delivery organisations to show data that is useful for themData fields:Three levels of data recorded: individual, activity and outcomes.Frontline staff input data onto the system. Client needs analysis and needs met/unmetActivities deliveredSubsequent reoffendingThe first challenge – how do you decide what to measure?Warren Buffett found it 'extraordinary' that academics studied what was measurable, rather than what was meaningful. As a friend said, to a man with a hammer, everything looks like a nailThe second challenge – actually using the data system:Meganexus not bespoke database therefore some irrelevant fields and some relevant ones missing – taking a long time to get exactly what we want We set the headings etc with the first manager then tweaked it for the second and staff now have a mixed bag of relevant and irrelevant fields. Staff sometimes not sure which headings to use/not useInconsistent recording and limitations of the databaseFeedback from caseworkers
- How performance management helps evolve the service
- Through analysis of needs of clients. Benefits of PM:Informed commissioning of additional servicesBetter use of existing service providers Improved case worker ownership of services
- 3RD USE OF IMPACT MEASUREMENTLack of evidence is the main barrier that prevents successful interventions going to scale. In order to deliver impact at a national scale, social organisations need a robust evidence base to show their programme works.Here is an example of how we are helping build an evidence base in the UK for MST (= multi-systemic therapy)MST too risky for government but not too risky for investor (somewhere between vaccination and Words 4 Weapons)Aim is to build an evidence base of this intervention so that it can be scaledAll about implementation risk = risk that services don’t deliver good outcomesBetter evidence base means lower implementation risk
- See map for countries interested in Social Impact Bonds…Important how SIBs aren’t the solution to every problemBUT, SIBs are a practical example of how important impact measurement is:Before a new service to get the resources – the people, money and momentum to make it happenDuring a service – to make sure it works and meets the needs of the people that you’re trying to helpAfter a service – to build an evidence base for successful programmes so that they can reach large scale and start to make a differenceTo return to Warren Buffet’s quote – academics focus on what is measureable rather than what is meaningful. As people working to achieve impact – to reduce reoffending, promote employment, get homeless people into stable housing – we are concerned with an essentially practical (not academic) endeavour. We want to bring the best of the academic rigour of measurement to what is meaningful. Does there need to be a distinction between what is measureable and what is meaningful? I believe that to go beyond measurement we need to bring these two together. By measuring what is meaningful – and acting upon it – we can make the world a better place!