Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Nvf slides 4 may 16 final

1,446 views

Published on

National Volunteering Forum - 4 May 2016

Published in: Government & Nonprofit
  • Login to see the comments

  • Be the first to like this

Nvf slides 4 may 16 final

  1. 1. NATIONAL VOLUNTEERING FORUM BIGGER IMPACT AND BETTER VALUE: DEMONSTRATING THE IMPACT OF VOLUNTEERING 4 MAY 2016
  2. 2. NICK OCKENDEN HEAD OF RESEARCH NCVO
  3. 3. AGENDA MORNING SESSION Time Agenda Item Speaker 10.00am Tea and coffee available 10.30am Welcome and introduction Nick Ockenden Head of Research, NCVO 10:35am ‘Why is measuring impact important now?’ Karl Wilding, Director of Public Policy and Volunteering, NCVO 11:05am ‘Planning your impact’ Georgina Anstey, Consultant, NCVO 11.35am Roundtable discussions and feedback Chair 12.15pm Lunch
  4. 4. AGENDA AFTERNOON SESSION Time Agenda item Speaker 1.15pm ‘What difference do your volunteers make? Top tips on how to undertake a volunteering impact assessment’ Joanna Stuart, Senior Research Officer, NCVO 1.45pm ‘Challenges & tools’ Claire Knight, Head of Volunteering, Breast Cancer Care Rebecca Stewart, Head of Volunteering, Age UK Representing the National Network of Volunteer Involving Agencies (NNVIA) 2.05pm Q & A Chair 2.20pm ‘Volunteering in care homes’ Matt Hill, Senior Research Officer, NCVO 2.40pm ‘Research into impact on health & wellbeing’ Daiga Kamerade, Work and employment researcher, University of Birmingham 3.00pm Q & A Chair 3.15pm Closing remarks Chair
  5. 5. JOIN THE DISCUSSION #volforum @NCVOvolunteers
  6. 6. VOLUNTEERING: WHY IS MEASURING IMPACT IMPORTANT NOW? @KARLWILDING NCVO PUBLIC POLICY & VOLUNTEERING MAY 2016
  7. 7. 1. WHERE ARE WE NOW? 2. TERMS OF TRADE 3. DRIVERS FOR CHANGE 4. CHALLENGES 5. WHERE NEXT? 7
  8. 8. 1. WHERE ARE WE NOW? 8
  9. 9. VOLUNTEERING
  10. 10. 2. TERMS OF TRADE 11
  11. 11. WHAT IS IMPACT? • Impact is overall change we seek to achieve. • What we do – our outputs – designed to deliver impact. • Impact significantly bigger and more important than evaluating, or monitoring, or measuring.
  12. 12. 3. DRIVERS FOR CHANGE 13
  13. 13. THE BRAVE NEW WORLD OF DOING GOOD
  14. 14. 16
  15. 15. 18
  16. 16. 4. CHALLENGES 19
  17. 17. 22
  18. 18. WHAT WORKS, EVIDENCE OF HOW CHANGE HAPPENS
  19. 19. 25
  20. 20. 26
  21. 21. 5. WHERE NEXT? 27
  22. 22. PLANNING YOUR IMPACT GEORGINA ANSTEY CHARITIES EVALUATION SERVICE NCVO
  23. 23. PLANNING YOUR IMPACT
  24. 24. PLAN DOASSESS REVIEW IMPACT PRACTICE CYCLE
  25. 25. WHY PLAN YOUR IMPACT?  Strategy  Communication  Fundraising  Measurement
  26. 26. LANGUAGE AND LOGIC Resources used to create the services offered What is done with the inputs, to enable it to develop and deliver its outputs Products, services or facilities that result from activities The changes, benefits, learning or other effects that result from what the project or organisation makes, offers or provides Broader or longer-term effects of a project’s or organisation’s activities, outputs and outcomes     Input Process Output Outcome Impact
  27. 27. More clients get and sustain jobs Improved work experience Skills workshops Improved basic skills More appropriate behaviour at work Greater confidence Increased knowledge of job market Work placements Peer mentoring Reduction in youth unemployment regionally YOUTH EMPLOYMENT PROJECT
  28. 28. HOW DO YOU DO IT? Describe WHAT change needs to happen first Show HOW changes will be made by delivering outputs Long-term impact Necessary pre-condition Necessary pre-condition Necessary pre-condition Necessary pre-condition Necessary pre-condition Outcomes must be achieved BEFORE long- term impact Output Output
  29. 29. MONITORING AND EVALUATION FRAMEWORK 3 7 CORNERSTONE OF (SELF) EVALUATION: Outcomes Outcome indicators Information collection methods When and by whom How to report and use Outputs Output indicators Information collection methods When and by whom How to report and use
  30. 30. “NCVO CES offers consultancy, training and publications to the voluntary sector and its funders. This increases knowledge and skills in evaluation, and helps people collect better data to inform their decision making. This helps them improve their services which improves the lives of vulnerable people at the frontline.” SUPER QUICK THEORY OF CHANGE
  31. 31. EXERCISE: You have 3 minutes to create your own mini theory of change, in three sentences: 1. Describe what you do 2. The changes you bring about 3. The long-term, wider change that your work contributes to Max 20 words a sentence! Share it with your neighbour See if you can help improve each others’
  32. 32. FIND OUT MORE Theory of change: • Making Connections: Using a theory of change to develop planning and evaluation Free download at www.ces-vol.org.uk • Creating your theory of change: NPC’s practical guide Free download at www.thinknpc.org Impact practice: • Inspiring Impact - http://inspiringimpact.org/ Email: ces@ncvo.org.uk Tel: 020 7520 3193 Twitter: @CESOnline
  33. 33. ROUNDTABLE DISCUSSIONS
  34. 34. LUNCH
  35. 35. WHAT DIFFERENCE DO YOUR VOLUNTEERS MAKE? TOP TIPS ON HOW TO UNDERTAKE A VOLUNTEERING IMPACT ASSESSMENT JOANNA STUART, IVR
  36. 36. VOLUNTEERING IMPACT ASSESSMENT TOOLKIT Guidance and tools to help you assess the impact of volunteering on:  Your volunteers  Your organisation  Service users  Wider community
  37. 37. PLAN DOASSESS REVIEW
  38. 38. IDENTIFY WHAT YOU ARE TRYING TO ACHIEVE/USE YOUR THEORY OF CHANGE INVOLVE OTHERS AND WORK TOGETHER IDENTIFY CLEARLY WHY YOU WANT TO ASSESS IMPACT AND WHO IS IT FOR? BE REALISTIC AND PROPORTIONATE PLAN
  39. 39. MAKE USE OF AVAILABLE TOOLS AND FRAMEWORKS BE PROPORTIONATE – WHAT IS NECESSARY? WHAT IS MOST USEFUL? USE EXISITING INFORMATION AND EVIDENCE DO
  40. 40. VIAT FRAMEWORK
  41. 41. BUILD IN ENOUGH TIME FOR ANALYSIS AND MAKING SENSE OF DATA THINK ABOUT POSITIVES AND NEGATIVES – BE HONEST AND OPEN ASSESS
  42. 42. BE CREATIVE IN HOW YOU SHARE AND USE YOUR FINDINGS USE THE LEARNING – BE WILLING TO CHANGE YOUR VOLUNTEER PROGRAMME/PROJECT AND ACT ON WHAT YOU FIND REVIEW FEEDBACK TO THOSE INVOLVED CELEBRATE!
  43. 43. IVR undertakes research, consultancy and evaluation on volunteering and is part of NCVO. www.ivr.org.uk For further information on the toolkit: www. ncvo.org.uk/practical- support/publications/volunteering Next toolkit training will be on July 11th in London www.ncvo.org.uk/training-and-events 5
  44. 44. Impact – Challenges and Tools Claire Knight – Breast Cancer Care Rebecca Stewart – Age UK May 2016
  45. 45. What difference can we make? NNVIA and NCVO members
  46. 46. Challenges  Third Sector purpose is to make social progress  £43Bn income – means that we can (and do!) change millions of lives  Donations, and funding, means scrutiny  Every project should ask: ‘what difference is being made ?’  Impact, and not simply delivery
  47. 47. Standardising impact assessment  Umbrella bodies are well-placed to drive standardisation  Rationale – help the sector make more informed and visible comparisons  In order to do more that works
  48. 48. So how do we put this into practice?
  49. 49. The power of a peer group
  50. 50. Background • Who got involved? • What are we trying to achieve? • Why do we think this is important? • Why so many dogs?
  51. 51. Who got involved?
  52. 52. Where did we start? • Designed a process for capturing our views • Created a dummy project to test each tool • Established criteria to measure against • Created a list of tools to consider • Invited Sally Cupitt to talk to us
  53. 53. Which tools & resources would we review? • VITA / VIAT • VODG / NCF • Better Impact • Volunteers Count • Inspiring impact • Social Return on Investment • Balanced scorecard • Theory of change
  54. 54. What did we learn? • Information & guidance is easily available • There are plenty of tools to choose from • It’s important to pick the right tool for the right job • Plan ahead to ensure the right data is collected • Beware the common pitfall: measuring outcomes of projects, instead of impact
  55. 55. Next steps • Complete review of all tools • Share with all NNVIA members (September 2016) • Asked NNVIA members if anything missing? • Decided to invite Sally Cupitt to speak to all NNVIA members (June 2016) • Create a “Which Guide” to share
  56. 56. QUESTIONS
  57. 57. ‘VOLUNTEERING IN CARE HOMES’ MATTHEW HILL NVF MAY 2016
  58. 58. Volunteering in care homes pilot project • 3-year Department of Health funded project • Place volunteers in befriending and activity- based roles in care homes • Partnership model between NCVO, Volunteer Centres (5) and Care Homes (15) • Over 259 volunteers, almost 10,000 hours over two years • Evaluation aimed to draw out good practice learning and assess impact
  59. 59. The context • Burdens on care homes • Exceptionally high time pressure • Many new to volunteering • One-step removed from evaluation • Particular user group - especially dementia • 80% of residents dementia or severe memory problems • Capacity to consent • Has to be face-to-face
  60. 60. Changing and flexible pilot • Not a set ‘intervention’ • Uncertainty around focus of impacts • Two stage process • Interim evaluation – explore impacts and reflect • Project completion – assess impact • Further research
  61. 61. Social & emotional vs mental & physical Social & emotional vs mental & physical • Take a broad approach initially • Draw on existing frameworks • VIAT • ASCOF, Senses and CQC • Combining can invalidate the tools and reduce comparability • Try to explore all types of impact
  62. 62. Subjective vs objective • Practical challenges for objective measures • care home burden / non-disclosure • residents • Subjective reliable for wellbeing • Subjective but systematic • Link with existing evidence • … but does not satisfy everyone
  63. 63. Staff perception of impact on residents (n=29 of 38) (%) Major positive Moderate or minor positive No impact Major, moderate or minor negative Emotional wellbeing 62 31 3 3 Social wellbeing 59 38 3 0 Mental wellbeing 48 45 7 0 Physical wellbeing 41 38 21 0
  64. 64. Return on investment • Volunteer Investment and Value Audit • Investment – staff time, training, expenses • Value – equivalent wage rate per volunteer hour • Advantages – very simple, total output, gives a monetary value, comparability • Limitations – only measures output, monetising a non-monetary phenomenon, adopts market conception of value
  65. 65. Volunteer Investment and Value Audit Quarter Investment Value Return Oct-Dec 13 £10,388 £0 -£10,388 Jan-Mar 14 £17,021 £2,813 -£14,208 Apr-Jun 14 £15,927 £8,199 -£7,728 Jul-Sept 14 £15,334 £15,558 £224 Oct-Dec 14 £18,203 £16,484 -£1,719 Jan-Mar 15 £17,412 £15,010 -£2,401 Apr-Jun 15 £16,947 £17,834 £887 Jul-Sept 15 £19,488 £23,516 £4,028 Oct-Dec 15 £16,919 £23,980 £7,060
  66. 66. Quant vs qual (stats vs stories) • Not an either or • … but triangulation of data • Qualitative case studies and in-depth interviews allows deep understanding of impacts • …and the factors that underpin them • Involved residents and relatives
  67. 67. "If I didn't have the volunteer my morale would be very low. I'd be shouting at everybody - all of us in here have some form of dementia. I ring this bell and nobody comes and nothing happens and then I scream at the matron and say you haven't got enough staff and she gets very annoyed when I say that" [Resident]
  68. 68. “instead of sitting in their rooms…. keep people thinking the whole time and keeping the brain alert instead of being a dodo” [Resident]
  69. 69. “The communication has broken down a little bit.. we seem a little bit out of the loop…it’s a bit disappointing really. Cos we felt like we were building a rapport with the volunteers – it seems to have been a bit of a breakdown really… We need to go back down to basics …. and start from scratch” [Home Manager]
  70. 70. Care home cake Where are the volunteers?
  71. 71. Communicating impact • Reports and presentations • Video: • Need to convince people to engage with research outputs • Helps give the context for understanding impact • Promotional video but underpinned by evaluation findings…
  72. 72. https://www.youtube.co m/watch?v=alORoCYwK a0
  73. 73. References: Cabinet Office (2015) Community Life Survey 2014-15, Cabinet Office Naylor, C., Mundle, C., Weaks, L., & Buck, D. (2013) Volunteering in health and care: Securing a sustainable future, King’s Fund Alzheimer’s Society (2014) Statistics. Available at <http://www.alzheimers.org.uk/site/scripts/documents_info.php?documentID=341> See details of the project. Available at https://www.ncvo.org.uk/practical-support/volunteering/volunteering-in-care- homes Davis Smith, J., Gaskin, K., Ellis, A. & Howlett., S. (2015) Volunteering Impact Assessment Toolkit: A practical guide for assessing the difference that volunteering makes, Institute for Volunteering Research. Available at https://www.ncvo.org.uk/component/redshop/1-publications/P78-volunteering-impact-assessment-toolkit Nolan, M. R., Brown, J., Davies, S., Nolan, J. And Keady, J. (2006) The Senses Framework: improving care for older people through a relationship-centred approach. Getting Research into Practice (GRiP) Report No 2. Sheffield Hallam University Gardiner, C. & Barnes, S. (2016) The impact of volunteer befriending services for older people at the end of life: Mechanisms supporting wellbeing Progress in Palliative Care Fujiwara, D., Oroyemi, P. & McKinnon, E. (2013) Wellbeing and civil society Estimating the value of volunteering using subjective wellbeing data Cabinet Office Kirkman, E., Sanders, M. & Emanuel, N. (2015) Evaluating Youth Social Action, An Interim Report: Does participating in social action boost the skills young people need to succeed in adult life? Cabinet Office Ellis Paine, A., McKay, S. & Moro, D. (2013) Does volunteering improve employability? Evidence from the British Household Panel Survey Third Sector Research Centre
  74. 74. Hostedby: Impact of volunteering on wellbeing and mental health Dr. Daiga Kamerāde, Third Sector Research Centre, University of Birmingham d.kamerade@bham.ac.uk
  75. 75. Subjective wellbeing and mental health More: Diener et al. (2009).
  76. 76. Evidence Systematic reviews and meta analyses Randomised controlled trials (experiments) Cohort and longitudinal panel designs Coho Cross-sectional surveys Cro Qualitative interviewsSee reviews: (Kamerāde, 2015, Jenkinson et al. 2013).
  77. 77. Evidence summarised – The impact of regular volunteering on well-being is positive, more so for a sustained regular volunteering and for people with lower levels of wellbeing to start with ( Binder, M., & Freytag, A. ,2013).
  78. 78. How to measure impact of volunteering on wellbeing and mental health? • Subjective wellbeing (see Dolan, P., Layard, R., & Metcalfe, R. 2011) (Scale 0-10): – Life satisfaction: • Overall, how satisfied are you with your life nowadays? • Overall, how satisfied are you with the area you live, relationships , employment prospects etc – Happiness: • Overall, how happy did you feel yesterday? – Anxiety: • Overall, how worried did you feel yesterday? • Overall, how worried are you about your financial situation, environmental problems etc. – Worthwhileness • Overall, how worthwhile are the things that you are doing in life? (e.g. Volunteering) • Mental health (presence/absence of depression) – The 5-item World Health Organization Well-Being Index (WHO-5) (see Topp, et al. 2015). • Please indicate for each of the 5 statements which is closest to how you have been feeling over the past 2 weeks • Over the past 2 weeks ... I have felt active and vigorous, ... my daily life has been filled with things that interest me etc (Scale 5 (all the time) to 0 (at no time)
  79. 79. A basic experimental design for estimating the effects of volunteering on wellbeing. Peoplewhohave expressedan interestto volunteer Experimental group (volunteers) Control group (waitinglist) Pre-test Pre-test Volunteering Not volunteering Post-test Post-test are randomly assigned to =
  80. 80. Implications for policy and practice • Volunteering as a public wellbeing and mental health intervention? – Not enough robust evidence – Robust pilots and their evaluation is needed. • Claims about volunteering: – ‘Volunteering may improve your wellbeing and mental health’ [not ‘boosts, improves, or increases’]. – ‘Nearly 60% of 70 volunteers in this volunteering programme reported improved wellbeing’. • Design&Evaluation: – Lenght, frequency and type of volunteering: any realistic chances and mechanisms for improving wellbeing? – If wellbeing and mental health are the key expected outcomes&there is a relatively large number of volunteers: consider pragmatic randomised controlled trials. – If improved wellbeing and mental health are not the key expected outcomes& there are very few volunteers: different research questions and variety of designs. – Specific wellbeing measurements more informative than global/general ones.
  81. 81. More information • Kamerāde, D. (2015) Third sector impact on human resources and community: a critical review. TSRC Working Paper Series No.134. Seventh Framework Programme (grant agreement 613034), European Union. Brussels: Third Sector Impact • Jenkinson, C., Dickens, A., Jones, K., Thompson-Coon, J., Taylor, R., Rogers, M., et al. (2013). Is volunteering a public health intervention? A systematic review and meta-analysis of the health and survival of volunteers. BMC Public Health, 13(1), 773.
  82. 82. References Binder, M., & Freytag, A. (2013). Volunteering, subjective well-being and public policy. Journal of Economic Psychology, 34, 97-119. Diener, E., Oishi, S., & Lucas, R. E. (2009). Subjective Well-Being: The Science of Happiness and Life Satisfaction. In Oxford handbook of positive psychology (pp. 187) http://homepage.psy.utexas.edu/homepage/Class/Psy418/Josephs/Wynne %20Folder/5-Subjective%20Well-Being.pdf Dolan, P., Layard, R., & Metcalfe, R. (2011). Measuring subjective well-being for public policy.http://eprints.lse.ac.uk/35420/1/measuring-subjective- wellbeing-for-public-policy.pdf Musick, M. A., & Wilson, J. (2003). Volunteering and Depression: the Role of Psychological and Social Resources in Different Age Groups. Social Science and Medicine, 56(2), 259-269. Topp, C. W., Østergaard, S. D., Søndergaard, S., & Bech, P. (2015). The WHO-5 Well-Being Index: A Systematic Review of the Literature. Psychotherapy and psychosomatics, 84(3), 167- 176.http://www.karger.com/Article/Pdf/376585
  83. 83. QUESTIONS
  84. 84. CLOSING REMARKS
  85. 85. GET IN TOUCH Volunteering@ncvo.org.uk @NCVOvolunteers https://www.ncvo.org.uk/ncvo-volunteering

×