This presentation provides an introduction to planning a small-scale project evaluation. It's aimed at small organisations, charities, purposeful businesses.

1. Introduction to Evaluation
Live Projects
www.liveprojectsolutions.com
We are a small and agile London-based research bureau providing impactful and creative research and storytelling projects for creative and community
placeshapers, cultural districts, social and community enterprises, purposeful businesses and the creative industries. We fuse research and consultancy
methods with storytelling and impact journalism to bring about positive change.

2. A process of exploring/understanding…
 how a project or activity is working/has worked
 what difference it has made for who – what’s changed for
those involved (aka impact evaluation)
 how and why… (aka process or learning evaluation)
… for practical purposes – e.g. sharing learning, making
decisions, evidencing impact, applying for funding.
Evaluation…

3. What did we do?
Who did we reach?
How much did we do?
How well did we do it?
What difference did we make?
What’s changed?
What made this possible?
Outcomes: the changes, benefits, learning or other
effects that happen as a result of a project or
organisation’s work. Outcomes can be positive or
negative, expected or unexpected (NCVO)
Outputs: The tangible things that happen as
a result of your project or organisation’s
work. E.g. Sessions, events, group sizes,
attendance, audience sizes.

4. Capture the Stats and the Stories

5. Theory of Change
 Theory of Change….
 Aka Story of Change
 Aka Logical Framework / Logic Model / Log Frames
 A planning tool to plan and explain what you do and why.
 Clarify your focus
 Helps your prioritise
 Leads to better evaluation by measuring what matters
 Explains the positive change you hope to bring about (outcomes) for your beneficiaries
and how you intend to do this (activities)
 Shows where your resources are going
 Flexible/versatile

6. £££
Vision / Goals
Outcomes
Activities &
Participants
Inputs / Resources
Outcome
Indicators…
What would we see /
hear that indicates
this outcome has
happened?
Outcome
Indicators…
What would we see /
hear that indicates
this outcome has
happened?
Outcome
Indicators…
What would we see /
hear that indicates
this outcome has
happened?
Outputs
How much did we do
(Quantity)?
How well did we do it
(Quality)?
Outputs
How much did we do
(Quantity)?
How well did we do it
(Quality)?
Outputs
How much did we do
(Quantity)?
How well did we do it
(Quality)?
Outputs
How much did we do
(Quantity)?
How well did we do it
(Quality)?
Does this stack-up?
If we do these things does it lead to these
outcomes?
If we achieve these outcomes, does that move
us closer to our longer-term goals?
The ‘How’ or Mechanism
for Change
Situation
Statement
Needs /
Problems
Assumptions / Enabling Factors Challenges / Constraining Factors
Intermediary Outcomes
Source: Happy Museum Video (Mandy Barnett)

8. Priority Outcome(s) Indicator(s)
What will we actually see
about as we move
achieve this outcome?
What would ‘indicate’
has happened or is
Methods of evaluation:
What type of data and
information will we
this indicator to tell me
change is happening or
happened?
What methods will we
collect this information
Practicalities
Who will collect the
information and
When? How? How
How will we ensure
information and data
collected ethically?
Targets / Goals
What does success
like?
What is realistic?
would be a stretch?
1.
2.
3.
Planning your evaluation activity

9. No. Project Priority Outcome(s) Indicator(s)
What will we see / hear as
move towards and
achieve this? What tells
us the change has
happened?
Methods of evaluation
and data collection
Practicalities –
How will we
collect data? How
often?
Targets / Goals
Wellbeing: Local residents
involved in projects feel reduced
stress about current
circumstances and increased
hopefulness about the future.
Local residents sharing
about their wellbeing and
sharing that they feel
more hopeful about the
future.
We see/hear about
examples of the NHS 5
steps to wellbeing in the
project work – connecting
to others, being physical
active, learning new skills,
giving to others,
mindfulness/presence.
Storytelling / narrative
research to explore and
listen out for the 5
steps to wellbeing in
people’s projects.
Pre and post-project
survey with subjective
wellbeing questions
Post Project Story
Collection
Pre and Post
Project Wellbeing
Survey
We hear improved
wellbeing/reduced
stress as a consistent
theme in the stories of
12-14 local residents
collected yearly/bi-
yearly. We see evidence
of the 5 steps to
wellbeing in the stories.
80%+ of surveyed
project participants
report improvements in
wellbeing
Planning your evaluation activity… think about your indicators, methods, what data you will collect,
who will collect and when

10. School Project 1 Creative Project 1
At the start of
the project…
• Project lead to hold a reflective conversation with the teacher to
understand and capture:
• Where the pupils are at and their starting point: heritage knowledge
and skills.
• Have the pupils done anything like this before?
• Hopes and expectations for the project: why is the project useful?
What are the desired outcomes?
• What are the pupils looking forward to and what do they hope to
learn?
• Any worries/concerns?
• Project lead to facilitate a pre-project ‘hopes and expectations’
conversation with the 4 x participants.
• What are you hoping to get out of the programme?
• What are you bringing to the programme?
• What are you excited about?
• Is there anything you're worried/anxious about?
• Participants complete baseline skills and confidence survey: creative
skills, heritage, employability.
In the middle of
the project…
• Keep a track and record of participation and engagement
• Document the production of creative outputs (e.g. film and photography),
asking for consent.
• Capture quotes/comments as they arise.
• Keep track of participation and engagement
• Document creative outputs
• Capture quotes/comments/reflections
At the end of the
project…
• Project lead facilitates a post-project reflection session with a sample of
the students and the teacher.
• Head / Heart / Bag / Bin exercise – what did you learn, what did you
experiences, what are you taking away, what could be improved?
• Very short end of project survey to be completed by students:
experience/enjoyment, participation in heritage, skills, reflections.
• Post-project reflection session
• Post project survey
Share session outputs with evaluator ahead of interim and final reporting.
Evaluator to interview project/teacher leads and creative partner ahead of interim final reporting
Planning your evaluation activity… think about what needs to happen at the start of your
project, during and at the end of your project. Good practice to have a ‘baseline’ understanding
of the things you hope will change (e.g. skills or confidence levels), so you can explore and
compare changes as the project progresses.

11. Ingredients of good evaluation…
 Plan your evaluation collaboratively to measure what matters
 What questions/themes/topics matter to us?
 What will we do at the start, during your project/event, and at the end?
 Involve people in the design and delivery of evaluation
 Use a mixture of approaches if you can
 Conduct research ethically:
 be transparent about the purpose and use,
 let people know they don’t have to take part or answer questions and can withdraw at anytime
 let people know how their data will be collected and used
 collect information and data anonymously as far as possible
 think about participants wellbeing and how you will safeguard them against harm
 Include a range of voices and perspectives
 Document the real-world creative outputs (film, photography, events, artwork,
commissions)
 Use the artforms and methods you are working with already
 Keep it proportionate to your project / budget / resources

12. Ethics
 Transparency: Inform people about the purpose and use of the research, who it is being
carried out by, and who to contact for more detail.
 Data protection:
 Inform people about how their information and data will be collected and used, for what
purpose, how it will be stored/process, how long it will be held for.
 Inform people about their data rights – to access, rectify or erase their comments at any time,
and who they should contact if they want to do that.
 Confidentiality: Collect information and data anonymously as far as possible and let
people know when that isn’t possible or when they might be identified
 Voluntary participation and consent:
 Let people know that there is no obligation to take part, that they don’t have to answer any of
the questions.
 Parental consent is required for children under 16
 Wellbeing and preventing harm: plan for the wellbeing of research/evaluation
participants. Put things in place to support people with specific needs (E.g.
interpretation/assistance)

13. Measuring Change – Common Research Methods
Quantitative methods
 Pre and Post Project Surveys – measuring
changes in attitudes, behaviours, skills
 Self-reporting surveys
 ‘Skills Progress – e.g. Skills Builder
 Outcome Stars’
 Analysis of demographic, participation,
ethnicity data.
 Analysis of economic data – no. training
placements/volunteering roles/jobs.
Qualitative research methods
 Interviews / Story Collection
 Observation and Journaling
 Group Discussions / Workshops (e.g.
Head, Heart, Bag, Bin).
 Case studies
 Panels / Alumni Networks
Good to have a
mix of quant and
qual for
robustness.

14. Generic Social Outcomes – a starting point to think about what you
want to explore and measure
https://www.artscouncil.org.uk/sites/defa
ult/files/S3D29_GSO_Indicator_Bank.pdf
https://www.museumoflondon.org.uk/application
/files/6014/5624/7481/generic-learning-
outcomes-chart.pdf

15. https://www.skillsbuilder.org/universal-framework/listening

16. Creative Research Methods
 Reflections
 Writing
 Diaries and Journals
 Interviews
 Videos
 Online methods
 Draw and Write
https://helenkara.com/2015/03/26/creative-research-methods
https://sweetmachinepublishing.bandcamp.com/merch/docu
ment-your-culture-pamphlet
https://thevinylfactory.com/news/how-to-document-your-
culture-emma-warren-pamphlet/

17. Story Collecting
Intro
1. What was your involvement?
2. What changed for you personally?
3. Why was this change important to you?
4. What made this possible?
Close
https://oldfirestation.org.uk/our-work/storytelling-evaluation-
methodology/

18. https://whatworkswellbeing.org/

19. https://www.theaudienceagency.org/

20. https://www.outcomesstar.org.uk/

21. At the start…
• Head: What do you hope to learn?
• Heart: What do you hope to
experience or get out of the project?
What are you looking forward to?
• Bag/CV: What do you hope to take
away with you?
• Head: Anything you’re worried or
unsure about?
At the end…
• Head: What have you learned?
• Heart: What have you felt or
experienced? What did you enjoy?
Feel most proud of?
• Bag/CV: What will you take away?
• Bin: What didn’t you find useful?
https://www.hisengage.scot/equipping-professionals/participation-
toolkit/head-heart-bag-and-bin/

22. Source:
Ali Eisa

23. Ethics
 Transparency: Inform people about the purpose and use of the research, who it is being
carried out by, and who to contact for more detail.
 Data protection:
 Inform people about how their information and data will be collected and used, for what
purpose, how it will be stored/process, how long it will be held for.
 Inform people about their data rights – to access, rectify or erase their comments at any time,
and who they should contact if they want to do that.
 Confidentiality: Collect information and data anonymously as far as possible and let
people know when that isn’t possible or when they might be identified
 Voluntary participation and consent:
 Let people know that there is no obligation to take part, that they don’t have to answer any of
the questions.
 Parental consent is required for children under 16
 Wellbeing and preventing harm: plan for the wellbeing of research/evaluation
participants. Put things in place to support people with specific needs (E.g.
interpretation/assistance)

24. Ingredients of good evaluation…
 Theory of Change – helps you plan and measure what matters
 Involve people in the design and delivery of evaluation
 Use a mixture of approaches
 Include a range of voices and perspectives
 Document the real-world creative outputs (film, photography, events, artwork,
commissions)

25. 1. Background information about organisation
and activities
2. What the need is and why addressing that is
important and urgent
3. How you will meet the needs of your
beneficiaries
4. Evidence of your work with examples of
recent successes
5. What will happen (or not happen) if your
work does not go ahead and the impact of this.
6. How much money you need to raise +
timescales
See also: NCVO Video Guidance on Writing
an effective Funding Bid.
https://knowhow.ncvo.org.uk/studyzone/writ
e-a-winning-funding-bid
Case for Support Template Headings, Adapted from Charity Excellence
Good practice to include your evaluation findings/evidence of impact as part of your ‘Case
for Support’ and fundraising bids…

27. Contact Details
Very happy to answer any questions or queries.
You can also set up a 1:1 / support session with me
 Chris.hayes@liveprojectsolutions.com
 07736 042753

28. Further links and reading
 Arts Council England – Evaluation Toolkit: https://www.artscouncil.org.uk/self-
evaluation-toolkit
 Heritage Fund – Evaluation Toolkit: https://www.heritagefund.org.uk/funding/good-
practice-guidance/evaluation-guidance
 Happy Museum – Story of Change: http://happymuseumproject.org/story-of-change/
 Arts at the Old Fire Station – Storytelling: https://oldfirestation.org.uk/our-
work/storytelling-evaluation-methodology/
 Oral History Society / British Library – Popular training in oral history collection:
https://www.ohs.org.uk/
 Social Research Association – https://the-sra.org.uk/ - has 1 day training and CPD on
evaluation.
 Helen Kara – Creative Research Methods in the Social Sciences -
https://helenkara.com/2015/03/26/creative-research-methods/