Pro bono economics


Published on

Presentation given at Peninsula LIST masterclasses held by South West Forum

Published in: Economy & Finance, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Pro bono economics

  1. 1. 2© Pro Bono Economics PBE aims to match professional economists with charities; providing pro bono help to measure performance and understand charity impact, and fostering a culture of volunteering within the economics profession
  2. 2. 3© Pro Bono Economics History 2009 - incorporated and registered with Charity Commission 11 projects – no management – none completed 2010 – 2 members of staff from September Managing 24 projects – 1 completed 2012 – + part-time economist Managing ± 40 projects – 41 completed to date
  3. 3. 4© Pro Bono Economics People Supported by over 260 volunteers from the public sector (Government Economic Service, Bank of England and others), private consultancy firms (Charles Rivers Associates, Europe Economics, Frontier Economics, FTI Consulting, and Oxera), academics and individual economists Trustees Martin Brookes (Chair), Andy Haldane (Treasurer), Kate Barker,Lynne Berry, Lucy Heady,Nicola Pollock, Dave Ramsden Patrons include Gus O‟Donnell and Adair Turner
  4. 4. 5© Pro Bono Economics Projects 160 charities have registered 168 projects Where: 127 London & South East, 25 North & East, 5 in Scotland,1 Northern Ireland and 2 Wales Who for: 13 with homeless people,13 with ex-offenders, 9 with disable people, 4 with the elderly, 8 with other charities What: 12 employment support, 15 using arts or sport, 20 looking for analysis to support campaigning 88 proposals have been withdrawn or rejected 79 projects have had 162 economists allocated to them 41 reports completed to date 16 projects in pipeline, 14 being scoped, 9 being worked on
  5. 5. © Pro Bono Economics Vfm of interventions UK analysis Data collection advice Other economic advice Some charities PBE has helped
  6. 6. The counterfactual
  7. 7. 8© Pro Bono Economics Types of evaluation Process: how and how well is the project working? Impact: what change occurred as a result of the project? Economic: were the benefits worth the costs? requires an impact evaluation
  8. 8. 9© Pro Bono Economics Measuring outcomes Measure a characteristic before the intervention e.g. number of people with a cold Deliver the intervention e.g. cold and flu medicine Measure the same characteristic after the intervention Difference = impact? Most people get better from a cold with or without medicine
  9. 9. 10© Pro Bono Economics Measuring outcomes Measure a characteristic before the intervention e.g. school attendance Deliver the intervention e.g. mentoring scheme to improve attendance Measure the same characteristic after the intervention Difference = impact? What other things might be going on at the same time? New breakfast club New attendance officer
  10. 10. 11© Pro Bono Economics What would have happened anyway? i.e. without the intervention N.B. the intervention did take place, so we can never know for certain Can we measure changes in outcomes for a credible comparison group, who didn‟t get the intervention? credible = no reason to think that there may be differences (bias) which affect the outcomes e.g. employment programme for women cf. national employment outcomes – unbiased?
  11. 11. 12© Pro Bono Economics Control Groups National average PBE: Foundation Training Company Before PBE: MEAM pilots – adults with multiple needs Matched records in admin dataset Peterborough Social Impact Bond/MoJ datalab Natural experiment in charity data PBE: Barnardos RCT Education Endowment Foundation/Social Research Unit
  12. 12. 13© Pro Bono Economics Estimating impact Measure outcomes Best estimate of what would have happened anyway (the counterfactual) Outcomes minus counterfactual = impact A measurement minus an estimate = an estimate
  13. 13. 14© Pro Bono Economics Discuss What are you using/might you use as a control group?
  14. 14. Data: methods & issues
  15. 15. 16© Pro Bono Economics Data 1: project admin Can provide data on activities and outputs (monitoring) Rarely useful for outcomes (evaluation) Tends to be less reliable over time at an individual level Often essential for identifiers to link to large data sets need to ensure this is gathered from the start
  16. 16. 17© Pro Bono Economics Data 2: survey Produce your own survey Pros: can tailor to your outcomes Cons: time consuming (design, trialing, collection) usually need specialist input biases data collection from control groups Sometimes possible to link to other large surveys
  17. 17. 18© Pro Bono Economics Data 3: large data sets Educational attainment National Pupil Database (NPD) Re-offending PNC, prison, probation, MoJ Data lab Pros: good measures of specific outcomes can „match‟ / control – „synthetic‟ counterfactual Cons: access (security) use….needs v high level of expertise bias (unrecorded variable bias)
  18. 18. 19© Pro Bono Economics Data 4: RCTs Randomised control trials Randomly allocating participants between „treatment / project‟ and „control‟ groups & compare outcomes. Produces the best counterfactual with least bias → best estimation of impact
  19. 19. 20© Pro Bono Economics Issues 1: bias Response bias especially to surveys Unrecorded variable bias self selection attitude personality RCTs are the best way to overcome bias
  20. 20. 21© Pro Bono Economics Issues 2: sample size Only relevant once bias is eliminated or known Needs to be large enough to be representative, but too large is wasteful What is large enough? Ask a statistician
  21. 21. 22© Pro Bono Economics Issues 3: timing Outcomes happen as a result of activity Therefore can only be measured after the activity not during it! Can be difficult for funders who want reports at a project‟s close
  22. 22. 23© Pro Bono Economics Discuss Sample evaluations: What level of evidence? Strengths and weaknesses?
  23. 23. 24© Pro Bono Economics Levels of Evidence
  24. 24. Valuation
  25. 25. 26© Pro Bono Economics Definitions Social value monetary worth of something to society or Social value act “how what is proposed to be procured might improve the economic, social and environmental well-being of the relevant area” Social cost benefit analysis CBA including wider benefits Social return on investment particular methodology for social CBA emphasising role and input of stakeholders
  26. 26. 27© Pro Bono Economics Whose perspective? Value to society won‟t include transfer payments e.g. tax and benefits Value to individuals change in income including benefits (+) Value to Exchequer Increase in tax receipts (+) or increase in benefit payments (-) Potential costs avoided Cashable savings
  27. 27. 28© Pro Bono Economics Which benefits? Hard outcomes Qualifications → employment → lifetime earnings Costs avoided by public services Soft outcomes Early stages of work in this area Related to income, so only for those who are earning But still important so No numbers without stories, no stories without numbers
  28. 28. 29© Pro Bono Economics Which costs and values? Input costs Programme costs Volunteer input Other activities Benefit values Average costs Marginal cost
  29. 29. 30© Pro Bono Economics Discuss What type of valuation might be most relevant to you?