Slides for Derrick Jones' talk at the Wellcome Trust's Engaging Science conference 20 April 2016 as part of a session about evaluation.

1. Evaluation
Engaging Science Conference
April 2016
Derrick Jones, Economist

2. Contents
About Pro Bono Economics
How we work
Evaluation: typical issues and things to think about
Some references/examples
2© Pro Bono Economics

3. Our objectives
PBE matches professional economists with charities;
providing pro bono help to measure performance and
understand charity impact; and
fostering a culture of volunteering within the economics
profession
3© Pro Bono Economics

4. © Pro Bono Economics
Scope
Analyse
Quality Assure
Publish
Data on outcomes
Control group
Monetary values
Transparent Conservative Rigorous Economic analysis
Assess & Match
PBE team
Volunteers
HOW WHO WHAT

5. Typical questions from funders to charities…
1) Who are you aiming to help? (target group)
2) What is your project trying to achieve for your clients? (the outcomes question)
3) What is your project doing with the clients? (activities)
4) What outcomes are you achieving for your clients? (monitoring)
5) How do you know it’s your project rather than other factors causing those
outcomes? (the attribution issue)
6) What’s the economic value/benefit of your impact?
7) What does the project cost?
8) Is it value for money..?
5

6. Some key issues we have come across…
• Lack of clarity about Objectives, Outputs and Outcomes
• Lack of a clear Theory of Change, linking Activities and Resources to
Outputs and Outcomes
• (Plans often focus on monitoring throughput of projects)
• Lack of reasonable data on
• Size of target group (and hence take-up)
• Inputs/resources (£ and in-kind)
• Effect-size evidence (to predict outputs and outcomes)
• Need for a control group or counter-factual
6© Pro Bono Economics

7. Some References…
• Pro Bono Economics website http://www.probonoeconomics.com
http://www.probonoeconomics.com/content/5-tips-charities
Some PBE evaluation projects with charities…
http://www.britishscienceassociation.org/crestsilver-report
http://www.probonoeconomics.com/news/publications/looking-viewer-
engagement-royal-institutions-online-science-videos
http://www.probonoeconomics.com/news/publications/music-education-and-academic-
achievement-london-music-masters-bridge-project
7© Pro Bono Economics