#CHYPS2013
Measurement and Evaluation
Eibhlin Ni Ogain
New Philanthropy Capital
CHYPS presentation.pptx
SHARED APPROACHES TO
MEASURING IMPACT
Confederation of Heads of Young People’s Services Summer Seminar
Manchester, 4th Jul...
NPC: TRANSFORMING THE
CHARITY SECTOR
3
NPC works at the
nexus between
charities and funders
Charity
SectorFunder
Increasin...
4
Charities and social enterprises under pressure to
demonstrate impact
New funding mechanisms, e.g. PbR and SIBS’s requir...
WHAT IS SHARED
MEASUREMENT?
5
• Shared measurement is about using common tools to track outcomes
across similar organisati...
THE JOURNEY TO EMPLOYMENT
FRAMEWORK
A shared approach to measuring young people’s
employability
6
STEP TO SHARING
7
Understanding
your sector
Developing
shared
outcomes
Using
common tools
Using
common
methods
Sharing and...
ORGANISATIONS CONSULTED
8
• Big Lottery Fund
• Birmingham City Council
• Careers Development Group / Shaw
Trust
• Centre f...
+ +
+
++
+
Intrinsic
factors
Extrinsic
factors
External factor: The labour market
Employment
+ Sustainable
+ Quality
Perso...
JET—THE RESPONSE
• ‘really good job at capturing the key points that we as
a sector need to consider when assessing the jo...
STEP TO SHARING
11
Understanding
your sector
Developing
shared
outcomes
Using
common tools
Using
common
methods
Sharing an...
NPC’S WELL
BEING MEASURE
NPC’S WELL-BEING MEASURE
• An online survey-based tool for 11 to 16 year olds
• Measures eight aspects of well-being
• Val...
SUBJECTIVE WELL-BEING IS WHAT WE THINK AND FEEL
ABOUT OUR LIVES
14
Aspect of well-
being
Description
Self-esteem A child’s...
NPC’S WELL-BEING MEASURE -
RESULTS
15
WHERE WE ARE
TRYING TO GET
TO?
CAADA: SHARED MEASUREMENT
FOR DOMESTIC VIOLENCE
17Embedded shared measurement magnifies the impact of funders and grantees...
CAADA: SHARED MEASUREMENT
FOR DOMESTIC VIOLENCE
18 18
0%
10%
20%
30%
40%
50%
Disclosure of sexual violence
Disclosure of s...
TABLE DISCUSSIONS
Reactions to JET?
How do you currently measure impact?
How can you use the JET framework?
How important ...
Upcoming SlideShare
Loading in …5
×

New philanthropycapitalmeasurement&evaluation

177 views

Published on

Published in: Health & Medicine, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
177
On SlideShare
0
From Embeds
0
Number of Embeds
23
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Definition isn’t controversial – we’ve mad a judgment call having looked at the literature, as to what aspects or dimensions of well-being are most useful for organisations working with children and young people.Self esteem, emotional health and resilience are aspects related to how a young person feels about themselves. Inner circle on previous pageFriends, family, school and community are aspects related to feelings associated with experiences. The measure of life satisfaction looks at an overall rating of views on life. Interesting because this simple question is used in many studies and allow you to check your survey is broadly consistent, ie the way you ask questions isn’t skewing answers.ASK – hwo far dio group agree on this cluster of dimansions beign representative of well-being?? (2 minute discussion)Is there anything missing? What would you take out?But why is well-being important
  • Go to live site.Results – what do they tell you?Once logged in: here is where you can view all of the surveys that you have. You can get a overview of your results whilst a survey is still open and once closed (when everyone has completed it) you can access a full report of your results.Filters – compare differences between subgroups – those where there are two bars show that there has been significant change in results.
  • New philanthropycapitalmeasurement&evaluation

    1. 1. #CHYPS2013 Measurement and Evaluation Eibhlin Ni Ogain New Philanthropy Capital CHYPS presentation.pptx
    2. 2. SHARED APPROACHES TO MEASURING IMPACT Confederation of Heads of Young People’s Services Summer Seminar Manchester, 4th July 2013
    3. 3. NPC: TRANSFORMING THE CHARITY SECTOR 3 NPC works at the nexus between charities and funders Charity SectorFunder Increasing the impact of charities eg, impact-focused theories of change Strengthening the partnership Eg, collaboration towards shared goals Increasing the impact of funders eg, effective commissioning Consultancy Think tank
    4. 4. 4 Charities and social enterprises under pressure to demonstrate impact New funding mechanisms, e.g. PbR and SIBS’s require more robust and comparable measurement But: impact information often of low quality, not embedded, rarely standardised→ impedes learning and improvement • What should they do to improve? • Driven by funders or for own benefit? • What’s proportionate for them? • Working in isolation, reinventing the wheel BACKGROUND
    5. 5. WHAT IS SHARED MEASUREMENT? 5 • Shared measurement is about using common tools to track outcomes across similar organisations and settings. • Shared measurement is a process: understanding a sector’s shared outcomes • Shared measurement is also a product: any tool used by more than one organisation to measure impact.
    6. 6. THE JOURNEY TO EMPLOYMENT FRAMEWORK A shared approach to measuring young people’s employability 6
    7. 7. STEP TO SHARING 7 Understanding your sector Developing shared outcomes Using common tools Using common methods Sharing and comparing results
    8. 8. ORGANISATIONS CONSULTED 8 • Big Lottery Fund • Birmingham City Council • Careers Development Group / Shaw Trust • Centre for Economic and Social Inclusion (CESI) • City Gateway • Department for Work and Pensions • Edge • Employment Pathways • Impetus • London Youth • Participle • Private Equity Foundation • Resurgo • Sheffield Hallam University • Street League • Teens and Toddlers • Tomorrow’s People • Young Foundation • Barry Sheerman MP • Clare Coghill, Councillor for Waltham Forest • Employment Related Services Association (ERSA) • University of Portsmouth • University of Bath • Chartered Institute of Personnel and Development (CIPD) • nef (the new economics foundation) • ASDAN • London Employment and Skills Policy Network
    9. 9. + + + ++ + Intrinsic factors Extrinsic factors External factor: The labour market Employment + Sustainable + Quality Personal circumstances Emotional capabilities Attitudes Employability skills Qualifications, education + training Experience + involvement Career management Journey to EmploymenT framework (JET) EMOTIONAL CAPABILITIES • Self-esteem • Autonomy and control • Grit and determination • Empathy ATTITUDES • Aspirations • Attitudes to work EMPLOYABILITY SKILLS • Teamwork • Communication • Problem solving • Self-management • Leadership QUALIFICATIONS, EDUCATION + TRAINING • Basic skills • Achieving qualifications • Attendance and behaviour EXPERIENCES + INVOLVEMENT • Work experience • Perception of value of work experience • Networks • Community involvement CAREER MANAGEMENT • Career direction • Job search skills • Presentation to employers PERSONAL CIRCUMSTANCES • Access to transport • Access to the internet • Access to childcare • Access to support for young carers • Access to support for disabled people • Reduced substance abuse • Reduced offending/anti-social behaviour EMPLOYMENT • Entry into employment • Sustaining employment • Quality of employment • Satisfaction with employment
    10. 10. JET—THE RESPONSE • ‘really good job at capturing the key points that we as a sector need to consider when assessing the journey to employment.’ —Tomorrow’s People • ‘…my immediate response is that this is an excellent framework for all of us working in EET with young people. The contents resonate with all the aspects of EET work with vulnerable young people here at NHYC. I have shared this with our Youth Work Manager who agrees with my estimation and is going to use it at a review taking place tomorrow of EET provision at the 10
    11. 11. STEP TO SHARING 11 Understanding your sector Developing shared outcomes Using common tools Using common methods Sharing and comparing results
    12. 12. NPC’S WELL BEING MEASURE
    13. 13. NPC’S WELL-BEING MEASURE • An online survey-based tool for 11 to 16 year olds • Measures eight aspects of well-being • Valid for a group of 10 or more • Survey includes the ‘best bits’ from academic well- being scales • Rigorously tested, with a variety of groups of young people 13 INITIAL SURVEY INTERVENTION FOLLOW-UP SURVEY
    14. 14. SUBJECTIVE WELL-BEING IS WHAT WE THINK AND FEEL ABOUT OUR LIVES 14 Aspect of well- being Description Self-esteem A child’s feeling of his or her own worth Emotional health Whether the child experiences worries and feelings of stress Resilience Whether the child is good at dealing with difficult events and setbacks Friends Quality of relationships with peers Family Quality of relationships within the child’s home School Satisfaction or enjoyment of school Community Satisfaction or enjoyment with where the child lives Life satisfaction Overall rating of views on life These are measures associated with feelings about self These are measures associated with feelings about experiences This is a measure of overall life satisfaction Source: NPC’s Well-being Measure
    15. 15. NPC’S WELL-BEING MEASURE - RESULTS 15
    16. 16. WHERE WE ARE TRYING TO GET TO?
    17. 17. CAADA: SHARED MEASUREMENT FOR DOMESTIC VIOLENCE 17Embedded shared measurement magnifies the impact of funders and grantees 17 SHARED MEASUREMENT SYSTEM Commissioners and policy makers Beneficiaries Practitioners Service managers Funders My experience influences the services I receive I know how to improve the services we provide, I can evidence our impact Data gathering doesn’t get in the way of my job, I can see what a difference I make My grantees have the capacity to deliver better services and become sustainable, I can work more effectively with fellow funders I know which interventions work, I have evidence of need Better services Better outcomes
    18. 18. CAADA: SHARED MEASUREMENT FOR DOMESTIC VIOLENCE 18 18 0% 10% 20% 30% 40% 50% Disclosure of sexual violence Disclosure of sexual violence Average 0% 10% 20% 30% 40% 50% 60% Disclosure of mental health issues Disclosure Average 18 Impact of training: 50% increase in disclosure Impact of co-location 50% increase in disclosure 0% 5% 10% 15% 20% 25% 30% Disclosure of alcohol misuse Disclosure of alcohol misuse Average
    19. 19. TABLE DISCUSSIONS Reactions to JET? How do you currently measure impact? How can you use the JET framework? How important is it for you to measure well-being? How would using a tool like this work for you? How can you get buy-in from partners? What other challenges with measurement and evaluation do you face? What do you need support with? 19

    ×