Your SlideShare is downloading. ×
0
The Importance of outcomes and evaluation in service delivery           Yfoundations           10th July 2012       Prof I...
Will cover   Why outcomes are important   Evaluating for outcomes   Monitoring and evaluation   Types of evaluation  ...
Why are outcomes important Most services focus on outputs:   Numbers of clients seen   Numbers of service events   Uni...
How does evaluation improveservices Important to know what impact the service is  actually making on clients and why. Be...
Outcomes and accountability Accountability increasingly focused on  outcomes rather than outputs. Funding decisions now ...
Evaluation and monitoring Monitoring – regular collection of data which  is used for quality assurance, adherence to  sta...
Theory of change or program logic Links inputs processes, outputs and  outcomes. Explains why activities should lead to ...
Process evaluation Is service doing the things it is supposed to  do in the most efficient way possible? Has the program...
Ultimate Outcome:                                  Young people find stable accommodation and are able to participate acti...
Methodologies Administrative data (employment, health,  housing) Worker assessments, case files   Use of standard valid...
Outcomes Immediate   Satisfaction, appropriate referral Intermediate   Improved wellbeing, skills, housing Long term ...
Challenges of measuringoutcomes When?   Should you follow up clients? How?   Questionnaire, feedback, admin data What...
Who should do the evaluation? In house   Advantages     • Cheaper and more tailored to agency needs.   Disadvantages   ...
Ilan KatzSocial Policy Research Centre        Matthew Gray        CAEPR, ANU    Ilan.katz@unsw.edu.au    www.sprc.unsw.edu...
Upcoming SlideShare
Loading in...5
×

The Importance of Measuring Outcomes

259

Published on

Professor Ilan Katz

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
259
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "The Importance of Measuring Outcomes "

  1. 1. The Importance of outcomes and evaluation in service delivery Yfoundations 10th July 2012 Prof Ilan Katz (SPRC)
  2. 2. Will cover Why outcomes are important Evaluating for outcomes Monitoring and evaluation Types of evaluation Some key methods Conclusions
  3. 3. Why are outcomes important Most services focus on outputs:  Numbers of clients seen  Numbers of service events  Unit costs However this does not measure impacts ie whether the service makes a difference  can create perverse incentives. Outcome measurement can improve service delivery.
  4. 4. How does evaluation improveservices Important to know what impact the service is actually making on clients and why. Better understanding of who benefits most from the service and which aspect of the service is most beneficial. Results are fed back into the service so that lessons can be learned about elements of effective practice.
  5. 5. Outcomes and accountability Accountability increasingly focused on outcomes rather than outputs. Funding decisions now rely on cost effectiveness as well as assessment of need. Outcomes and evaluation can also increase accountability to service users, as participants and recipients.
  6. 6. Evaluation and monitoring Monitoring – regular collection of data which is used for quality assurance, adherence to standards and compliance. Evaluation – one off or episodic activity focusing on effectiveness of implementation, processes, outcomes and/or cost effectiveness. Both are important and can build on each other.
  7. 7. Theory of change or program logic Links inputs processes, outputs and outcomes. Explains why activities should lead to particular outcomes. Needs to be backed up by theory and empirical evidence and be specific. Explains the link between inputs, processes and outcomes.
  8. 8. Process evaluation Is service doing the things it is supposed to do in the most efficient way possible? Has the program been implemented as planned? Is it targeting the right people? What are the barriers to accessing the service? How well is it working with other services? What are clients’ and other stakeholders’ views of the service?
  9. 9. Ultimate Outcome: Young people find stable accommodation and are able to participate actively in society to their maximum potentioal Intermediate Outcome: Young people at risk of homelessness are provided with safe accommodation and given the opportunity to address issues leading to homelessness Impacts For children and adolescents: Improvement in housing situation Outputs SafetySituation What services will provide: Improvements in educational/vocational counselling participation Limited access to initial assessments Improvements in social functioningCommunity specialist Inputs Temporary accommodation Improvements in financial stabilityservices brief interventions $X put into youth Reduction in wait time for service Benefits advice High demand for service homelessness Better interaction with family Peer support Staffing resources Improvement in health and mental health providing education and support to Young people unable to Policies – referrers For homeless Service System:access mainstreamhousing services Commonwealth and Lower numbers of homeless youth NSW Who the services will interact with: Young people with acute housing Better inter-agency collaboration Young people leaving issues Longer term engagement with vulnerable Other servicesfamilies and becoming families and carers clientsvulnerable to exploitation other members of the broader youth Client Group: Youngand abuse services For Referrers: people at risk of homelessness primary referrers and other service Clear referral pathway partners Improvement in collaborative care 9
  10. 10. Methodologies Administrative data (employment, health, housing) Worker assessments, case files  Use of standard validated surveys Self completed surveys  pre and post questionnaires  Satisfaction, wellbeing, circumstances Qualitative methodologies/action research
  11. 11. Outcomes Immediate  Satisfaction, appropriate referral Intermediate  Improved wellbeing, skills, housing Long term  Home ownership/stability, employment, social networks
  12. 12. Challenges of measuringoutcomes When?  Should you follow up clients? How?  Questionnaire, feedback, admin data What?  How to define a good outcome Comparison and benchmarking Attribution  How do you know it was your service making the difference?
  13. 13. Who should do the evaluation? In house  Advantages • Cheaper and more tailored to agency needs.  Disadvantages • Lack of credibility and independence. • Need organisational expertise External  Advantages • Independent, authoritative – provides feedback  Disadvantages • Expensive and potentially burdensome, potential embarrasment
  14. 14. Ilan KatzSocial Policy Research Centre Matthew Gray CAEPR, ANU Ilan.katz@unsw.edu.au www.sprc.unsw.edu.au www.anu.edu.au/caepr G2 Western Campus University of New South Wales Kensington 2052 NSW, Australia +61 2 9385 7810
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×