1
Evidencing Impact and Value
Alison O’Hara
Chief Executive
Audiences North East
2
Impact
Image: courtesy of Hales Gallery, London
and I-20 Gallery, New York
3
Definitions
• Impact: the effect or impression of one thing against
another
• Economic impact: Radich(1987), the economi...
4
Value
Image: courtesy of BePak
5
Definitions
• Value: worth in usefulness or importance, utility
or merit
• Matarasso (1996), which value systems are
use...
6
Current drivers for measuring
impact and value
• Economic downturn
– Local and national governments invest public
money ...
7
Cuts in public investment
• DCMS – 25% cut for 2011/12 to 2014/15
• ACE cuts – 29% cut, 6.9% cut for majority
of RFOs fo...
8
The Big Society
• www.cabinetoffice.gov.uk Building the Big
Society
• Five priorities:
– Give communities more power
– E...
9
Measuring Intrinsic Value
• Measuring Intrinsic
Value: How to stop
worrying and love
economics (2009) by
Hasan Bakhshi, ...
10
Measuring Intrinsic Value
• Recommends cultural sector embracing
the need to measure economic impact
• Cultural world w...
11
Intrinsic v instrumental
• Intrinsic value (own merits) v instrumental value
(non artistic side effects eg social inclu...
12
Culture and Sport Evidence
Programme (CASE)
• Vision is to influence culture and sports’
policy development and policy ...
13
CASE
• Publication
– Understanding the drivers,
impact and value of
engagement in culture and
sport, July 2010
www.cult...
14
Benefits of engaging in culture and sport
Individual engager
Achievement
Continuity with the past
Creativity
Diversion
...
15
CASE publications
• Measuring short-term private benefit of
engagement
– Subjective well-being measures, two-step appro...
16
CASE publications
• Measuring long-term public benefit of
engagement
– Due to data and evidence constraints work
was fo...
17
CASE database
18
Impact and value frameworks
• Arts Council England - self evaluation
framework
• MLA - Inspiring Learning
• Local Gover...
19
Arts Council England – self
evaluation framework
• Online flexible
development tool to
support arts organisations
in ev...
20
Arts Council England – self
evaluation framework
• Six key areas:
– Vision
– External environment
– Artistic aspiration...
21
MLA - Inspiring Learning
• A self-help improvement framework for
museums, libraries and archives, 2008
• Inspiring Lear...
22
MLA - generic learning outcomes
• The generic learning
outcomes are
underpinned by a broad
definition of learning
which...
23
MLA - generic learning outcomes
• GLO checklist
• Recording and
analysing
qualitative and
quantitative data
• GLO codin...
24
MLA - generic social outcomes
• Case studies, guidance and
tools to support museums,
libraries and archives in
planning...
25
MLA - generic social outcomes
• Three outcome areas that
relate to government policy
priorities
• Look at organisation’...
26
Local Government Improvement
and Development
• Formerly IDeA new web resource on how
to create a local outcomes framewo...
27
Local outcomes framework -
process
28
Local outcomes framework -
resources
• Guidance on what a local culture and sport
outcomes framework can help you do
• ...
29
Audiences London – festivals and
outdoor events
• Information sheets
available to download,
September 2010
• Key points...
30
Audiences London – festivals and
outdoor events
• Information sheets:
– Evidencing success
– Methods
– Using questionna...
31
New Economics Foundation -
theatre
• Capturing the audience
experience – a handbook for
the theatre
• Presents a new mo...
32
The Audience Experience
Framework
33
Film Council - impact of local
cinemas
• Impact of local cinema –
five case studies, 2005
• Suggested tools for use
in ...
34
Film Council - case studies
• Desk research
– No. of screens and seats
– Type of programme
– Annual ticket sales
– Reve...
35
Film Council - case studies report
• The social, cultural and environmental
impact of local cinema
• The impact of loca...
36
Film Council - suggested tools for
further research
• Pack of research materials:
– Templates for data collection
– Top...
37
Impact and value frameworks – the
highlights
• Arts Council England - self evaluation
framework
• MLA - Inspiring Learn...
Upcoming SlideShare
Loading in …5
×

Impact and value presentation

493 views
418 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
493
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Give communities more power – empowering local communities and giving them a voice so that they can have public money invested in the services they want.
    Encourage people to take an active role in their communities – increasing participation in community based activities as well as increasing the contribution from local people to supporting the services they use and value.
    Transfer power from central to local government – devolving power and finance to the most appropriate spatial level.
    Support co-ops, mutuals, charities and social enterprises – providing support to Third Sector organsations, including helping them to become more enterprising and self-sufficient.
  • Estimated impact on SWB on doing sport, going to the cinema and going to live arts eg concerts and theatre demonstrates that engagement in culture and sport has a positive effect on SWB. The exception is doing sport once a year or less for which no effect was identified. Further, a higher frequency of engagement is generally associated with a higher level of SWB.
    The SWB measures used in the analysis are responses to the question: ‘How dissatisfied or satisfied with your life overall?’
  • Estimated impact on SWB on doing sport, going to the cinema and going to live arts eg concerts and theatre demonstrates that engagement in culture and sport has a positive effect on SWB. The exception is doing sport once a year or less for which no effect was identified. Further, a higher frequency of engagement is generally associated with a higher level of SWB.
    The SWB measures used in the analysis are responses to the question: ‘How dissatisfied or satisfied with your life overall?’
  • Impact and value presentation

    1. 1. 1 Evidencing Impact and Value Alison O’Hara Chief Executive Audiences North East
    2. 2. 2 Impact Image: courtesy of Hales Gallery, London and I-20 Gallery, New York
    3. 3. 3 Definitions • Impact: the effect or impression of one thing against another • Economic impact: Radich(1987), the economic impact of a given phenomenon can be defined as ‘the effect of that phenomenon on such economic factors as the economic behaviour of consumers, businesses, the market, industry, the economy as a whole, national wealth or income, employment and capital’ • Social impact: Landry et al (1993), ‘those effects that go beyond the artefacts and the enactment of the event or performance itself and have a continuing influence upon, and directly touch, people’s lives’
    4. 4. 4 Value Image: courtesy of BePak
    5. 5. 5 Definitions • Value: worth in usefulness or importance, utility or merit • Matarasso (1996), which value systems are used to provide benchmarks against which work will be measured, and about whom defines quality, value and meaning • Social and economic rationales for the arts, with their emphasis on the arts as a means to other ends will serve to devalue arts for its own sake eg John Tusa
    6. 6. 6 Current drivers for measuring impact and value • Economic downturn – Local and national governments invest public money in cultural opportunities. How do we know if the money is well spent or wasted? Does a £1 invested by government deliver more than a £ in benefit? • The Big Society
    7. 7. 7 Cuts in public investment • DCMS – 25% cut for 2011/12 to 2014/15 • ACE cuts – 29% cut, 6.9% cut for majority of RFOs for 2011/12 and reduction of 14.9% to overall budget available for RFOs for 4-year settlement period • National museums – 15% cut and remain free to enter • English Heritage – 32% cut • Visit Britain – 34% cut
    8. 8. 8 The Big Society • www.cabinetoffice.gov.uk Building the Big Society • Five priorities: – Give communities more power – Encourage people to take an active role in their communities – Transfer power from central to local government – Support co-ops, mutuals, charities and social enterprises – Publish government data
    9. 9. 9 Measuring Intrinsic Value • Measuring Intrinsic Value: How to stop worrying and love economics (2009) by Hasan Bakhshi, Alan Freeman and Graham Hitchen Image: Jaipur Kawa Brass Band, Durham BRASS Fest
    10. 10. 10 Measuring Intrinsic Value • Recommends cultural sector embracing the need to measure economic impact • Cultural world will back itself into a corner if it claims immunity from measuring economic impact when money could be spent on other things • Even though it’s difficult to measure, it’s not really optional – governments choose between alternative expenditures
    11. 11. 11 Intrinsic v instrumental • Intrinsic value (own merits) v instrumental value (non artistic side effects eg social inclusion, crime prevention and learning) • Proper assessment of intrinsic value • Public’s own valuation of the arts – contingent value (CV) and willingness to pay (WTP) • Ask the public what they would be prepared to pay, faced with a choice of spending the money on something else
    12. 12. 12 Culture and Sport Evidence Programme (CASE) • Vision is to influence culture and sports’ policy development and policy agenda through the development of strategic, policy-relevant, high-quality, cross-cutting, social and economic evidence base for culture and sport sectors – Current research – Research needs to provide – Tools to collect research
    13. 13. 13 CASE • Publication – Understanding the drivers, impact and value of engagement in culture and sport, July 2010 www.culture.gov.uk • Database – www.impact.arts.gla.ac.uk/ c5,800 individual studies or reviews on the drivers, impact and value of engagement in culture and sport, advanced search terms, links to reports Image: The Alnwick Garden
    14. 14. 14 Benefits of engaging in culture and sport Individual engager Achievement Continuity with the past Creativity Diversion Enjoyment Escape Expression Health Income Inspiration Knowledge of culture Self-esteem Self-identify Skills/competency Solace/consolation Community Bequest value Community cohesion Community identity Creativity Employment Existence value Innovation Option to use Productivity Reduced crime Shared experience Social capital National Citizenship International reputation National pride
    15. 15. 15 CASE publications • Measuring short-term private benefit of engagement – Subjective well-being measures, two-step approach • First survey data is used to estimate how a person’s SWB changes when they engage in culture and sport • This change in SWB is valued monetarily using the ‘income compensation approach’ ie the analysis estimates the increase in SWB generated by an increase in income – Deriving economic values from SWB • The income compensation approach can be used to convert estimates of the SWB effect of policy outcomes such as engagement in culture and sport into estimates of monetary value of these policy outcomes.
    16. 16. 16 CASE publications • Measuring long-term public benefit of engagement – Due to data and evidence constraints work was focused on the health gains associated with doing sport • However, CASE regional insights have been developed to provide partners with data and evidence on the position and role of culture and sport within key local and regional agendas
    17. 17. 17 CASE database
    18. 18. 18 Impact and value frameworks • Arts Council England - self evaluation framework • MLA - Inspiring Learning • Local Government Improvement and Development (formerly IDeA) • Audiences London – festivals and outdoor events • New Economics Foundation – theatre • Film Council – impact of local cinemas
    19. 19. 19 Arts Council England – self evaluation framework • Online flexible development tool to support arts organisations in evaluating their own performance and to help them inform their future planning, July 2010 • www.artscouncil.org.uk/s elfevaluation/background- self-evaluation/ Image: Constellation by Kiki Smith, National Glass Centre
    20. 20. 20 Arts Council England – self evaluation framework • Six key areas: – Vision – External environment – Artistic aspirations and programme – Participation and engagement – Organisational capacity and capability – Business model • Each area has three strands: – Topic – a breakdown of the key area into smaller areas of focus – What success looks like – a high-level description of an organisation that is delivering each topic effectively – Questions you might ask yourselves – types of questions you might ask to evaluate your strengths and weaknesses relating to that topic – Resources – website links
    21. 21. 21 MLA - Inspiring Learning • A self-help improvement framework for museums, libraries and archives, 2008 • Inspiring Learning supports organisations to: – Assess their strengths and plan improvements – Provide evidence of the impact of their activities through the generic learning and generic social outcomes – Improve their strategic and operational performance • www.inspiringlearningforall.gov.uk/
    22. 22. 22 MLA - generic learning outcomes • The generic learning outcomes are underpinned by a broad definition of learning which identifies benefits that people gain from interacting with museums, libraries and archives • www.inspiringlearningfora ll.gov.uk/toolstemplates/g enericlearning/ Image: Killhope
    23. 23. 23 MLA - generic learning outcomes • GLO checklist • Recording and analysing qualitative and quantitative data • GLO coding • Templates
    24. 24. 24 MLA - generic social outcomes • Case studies, guidance and tools to support museums, libraries and archives in planning how they deliver their services and in measuring their contribution to social outcomes • Step-by-step approach to using the guidance, sample questionnaires and case studies • www.inspiringlearningforall.gov .uk/toolstemplates/genericsoci al/index.html Image: Sunderland Museum and Winter Gardens
    25. 25. 25 MLA - generic social outcomes • Three outcome areas that relate to government policy priorities • Look at organisation’s high level aims • Identify which audiences are priorities to target • Consider how your project/service contribute to one or more of the GSOs • Decide what evidence you need to gather
    26. 26. 26 Local Government Improvement and Development • Formerly IDeA new web resource on how to create a local outcomes framework for culture and sport, September 2010 • Local outcomes framework will help you measure and evidence the contribution culture and sport provision makes to better outcomes for your area • www.idea.gov.uk/idk/core/page.do? pageId=21649171
    27. 27. 27 Local outcomes framework - process
    28. 28. 28 Local outcomes framework - resources • Guidance on what a local culture and sport outcomes framework can help you do • What should an outcomes framework include: – Examples of outcomes triangles: children and young people, economy, environment, health and well-being, older people, safer communities, stronger communities – Logic model templates – Evidence template – Performance indicators template
    29. 29. 29 Audiences London – festivals and outdoor events • Information sheets available to download, September 2010 • Key points to consider when trying to measure and evidence success or impact of a festival • www.audienceslondon.or g/1891/our- resources/advice-for- audience-research-at- festivals-and-outdoor- events.html Image: Stockton International Riverside Festival
    30. 30. 30 Audiences London – festivals and outdoor events • Information sheets: – Evidencing success – Methods – Using questionnaires – Sampling – Working with volunteers – Research guidelines and data protection – Measuring economic impact
    31. 31. 31 New Economics Foundation - theatre • Capturing the audience experience – a handbook for the theatre • Presents a new model for describing the audience experience with standard survey templates and guidance on how to use them • Value – explores what it is about a good theatre experience that makes it ‘worth coming out for’ • www.itc- arts.org/uploaded/documents/ Theatre%20handbook.pdf Image: Theatre Royal
    32. 32. 32 The Audience Experience Framework
    33. 33. 33 Film Council - impact of local cinemas • Impact of local cinema – five case studies, 2005 • Suggested tools for use in further cinema impact studies • Methodological notes • www.ukfilmcouncil.org.uk/ 10022? page=1&step=10&viewby =category&value=17004 Image: Tyneside Cinema
    34. 34. 34 Film Council - case studies • Desk research – No. of screens and seats – Type of programme – Annual ticket sales – Revenue from food/drink/merchandising, advertising, project funding, other income – Annual expenditure – Type of location and town – Population of town • Site visit • Depth interviews with cinema manager and staff • Focus group with cinema audience • Phone interviews with cinema’s local suppliers, community groups and educational organisations, local council and local press
    35. 35. 35 Film Council - case studies report • The social, cultural and environmental impact of local cinema • The impact of local cinema on the local economy • Appendix – tables which analyse information (quantitative and qualitative) by impact eg economic, social, cultural, environmental
    36. 36. 36 Film Council - suggested tools for further research • Pack of research materials: – Templates for data collection – Topic guides for interviews with stakeholders – Briefing notes for cinema staff – Monitoring form – Draft letters – Forms for measuring spending
    37. 37. 37 Impact and value frameworks – the highlights • Arts Council England - self evaluation framework • MLA - Inspiring Learning • Local Government Improvement and Development (formerly IDeA) • Audiences London – festivals and outdoor events • New Economics Foundation – theatre • Film Council – impact of local cinemas

    ×