NPC

1. v
MEASURING SOCIAL IMPACT
Cecilie Hestbaek, NPC
The Big Connect, Tuesday 7th June 2015

2. NPC: TRANSFORMING THE CHARITY SECTOR
2
NPC works at the
nexus between
charities and
funders
Charity
SectorFunder
Increasing the
impact of charities
eg, impact-focused
theories of change
Strengthening the
partnership
Eg, collaboration
towards shared
goals
Increasing the
impact of funders
eg, effective
commissioning
Consultancy & Think tank

3. SOCIAL IMPACT
‘The effect of an activity on the social fabric of the community and well-being of the
individuals and families.’
http://www.businessdictionary.com
3

4. a) articulate how you think a service or intervention will have an
impact;
b) and test your thinking against the evidence you have available.
4
THE ESSENCE OF IMPACT MEASUREMENT:
Exercise:
Take 10 minutes to discuss with your neighbour why it’s important to
measure social impact and what the challenges are. Write key words on
different colour post-its.

5. WHY IS MEASURING IMPACT IMPORTANT?
Saves staff
time
Influences the
debate on
“what works”
Improved
services
Raises profile
Motivates
staff /
volunteers
Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater
Impact. New Philanthropy Capital.
Helps secure
funding
5

6. CHALLENGES IN MEASUREMENT
6
Causality, consistency
and predictability.
• What worked before might not work again because the
external environment has changed
Time frame
• Change can happen quickly; or it might take two
decades to come about
Contribution and
attribution
• A variety of external factors are beyond your
organisation’s control and you will often have no
counterfactuals
Data collection
• If you are targeting top-level decision makers, it can be
difficult to get an answer from them about why they did
(or did not) change their mind on a particular issue.
…and ensuring that you measure outcomes, not outputs.

7. THE FOUR PILLARS APPROACH TO
MEASURING IMPACT
7
Map your
theory of
change
Prioritise
what you
measure
Choose
your level
of evidence
Select your
sources and
tools
Effective measurement
framework developed
Strategic vision
Leadership
Case for impact measurement

8. DEVELOPING AND USING
A THEORY OF CHANGE

9. WHAT IS A THEORY OF CHANGE?
Links activities intermediate outcomes  final outcomes
A description of how activities lead to outcomes
9
- Clarifies what the activities aim to
achieve and how
- Provides the case for why achieving
intermediate outcomes is important
- Provides a structure for identifying
what can be measured

10. HOW TO REPRESENT A THEORY OF CHANGE
10
Planning Triangle Logic Model Outcomes Chain
However you represent your theory of change, it should be supported by a written description.

11. 11
Children’s emotional
resourcefulness
improves
Counselling
Clients’ ability to support
their children's healthy
development improves
Clients’ emotional or
psychological
difficulties decrease
Mother / Child
interaction
improves
Clients’ capacity
for self care
increases
EXAMPLE (SIMPLIFIED) THEORY OF CHANGE
MOTHERS’ COUNSELLING SERVICE
Activities Intermediate Outcomes Final outcome

12. EXERCISE
12
• Take 2 minutes to think about 1-2 social outcomes that are central to
what you do.
• Write them down on a piece of paper.
• As we go through the rest of the presentation, try to reflect on the
measurement of that/those outcomes. How would you apply the theory
and measure those?
Categories of outcomes include: ‘soft’ outcomes, such as attitudes,
knowledge, skills, behaviour; and ‘hard’ outcomes, such as employment,
educational attainment, reduced reoffending etc.

13. PRIORITISING OUTCOMES

14. PRIORITISE OUTCOMES THAT:
• are directly influenced (rather than indirectly supported)
• are important / material to the mission
• are not too costly to measure
• will produce credible data
14

15. CHOOSING THE RIGHT
LEVEL OF EVIDENCE

16. COUNTERFACTUAL
• Comparing the world with your
organisation in it with what the
world would be like without it.
• Control group
• Attribution
16

17. LEVELS OF EVIDENCE
17
Randomised
control trial
Anecdotes /
quotes
Before and
after survey
Self-reported
change
Case
studies
Control
groups
Credibility
Basic Advanced

18. SELECTING DATA
SOURCES AND TOOLS

19. • Quantitative data (numbers)
• Statistical estimates
• Prevalence of views,
attitudes and experiences
• Admin data or questionnaires
(paper, web, etc.)
• Qualitative data (words)
• Detailed understanding
• In-depth interviewing
(telephone or face-to-face)
• Observation
• Stakeholders’ views
Proportion of beneficiaries
whose outcomes have
improved, and by how
much.
What did beneficiaries
think, did it make a
difference to them? How?
How could it have been
better?
DIFFERENT TYPES OF EVIDENCE

20. EVALUATION TRAPS
20
Collect data that matters, and work together
Don’t force
squares into circles
& don’t collect
arbitrary data

21. INSPIRING IMPACT: MEASURING UP! AND
THE IMPACT HUB
Measuring Up!
• online, step-by-step self-assessment tool
• looks at the way you plan, evidence, communicate and
learn from the difference your work makes
The Inspiring Impact Hub
• pulls together resources relevant to improving impact
practice
• enables users to search and filter results according to their
needs
inspiringimpact.org
21

22. QUESTIONS AND COMMENTS
22