Creating an outcomes framework for your organisation

  • 1,595 views
Uploaded on

Key steps in creating a client outcomes measurement framework for a welfare / human service organisation. Particular focus on homelessness assistance services.

Key steps in creating a client outcomes measurement framework for a welfare / human service organisation. Particular focus on homelessness assistance services.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,595
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
50
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Homelessness assistance sector, Australia, November 2010 Mark Planigale Research & Consultancy results by design
  • 2. 2 Performance measurement Compliance reporting Program logic
  • 3. 3Performance measurement Bigger picture of organisational performance measurement  Compliance reporting currently focuses on effort  Many organisations monitor inputs and client satisfaction  But are we making a difference? National frameworks: National Affordable Housing Agreement, National Partnership Agreement on Homelessness ‘Manufacturing-based’ performance models need to be enhanced
  • 4. 4Program logic Mission Vision Environment • Problem or need • Barriers • EnablersInputs Activities Outputs Outcomes ImpactsPeople and What gets What clients Changes in Changes inresources done receive clients’ lives society• Clients • Functions • Client • Short term • Social• Human • Processes participation • Long term outcomes resources • Actions benefits • Interim• Other • Client direct resources benefits Objectives Aims
  • 5. 5 Useful information is produced throughwell-planned systems.
  • 6. 6 Purpose and scope Defined outcomes Defined measures and tools Data collection and storage processes Analysis and reporting processes Use of outcomes information Strategies for sustaining the system
  • 7. 7Clarify purpose and scopeWhy measure outcomes? Questions to consider Compliance  Which programs? Individual assessment and  Which populations? planning  What do we mean by Oversight outcomes? Quality improvement Advocacy
  • 8. 8Define key outcome areasOutcome: Questions to consider“... a change or absence of  If the program works change that results (at least in really well, how are part) from actions of staff of clients’ lives better? the organisation”  If things go wrong Desired vs. undesired for our clients, where do we see this in Short term, long term, interim their lives? Domains vs. locus of change  What is our core business?
  • 9. 9Example outcomes1. Person obtains appropriate, affordable long-term housing2. Person takes medication more regularly3. Person has a better understanding of tenancy rights and responsibilities4. Person is placed on the OOH Early Housing waiting list5. Person gains part-time employment6. Person becomes homeless
  • 10. 10Select measures and toolsMeasure: Questions to consider“...an observable, measurable  If our key outcomes characteristic of a person or occurred, how would their situation, which is linked we know? with a state or condition of interest to us”  Which measures have the greatest Base measures, derived communicative measures and KPIs power? Varied vantage points  What data do we Use modules for flexibility and already have? consistency
  • 11. 11Example measuresStatus change/maintenance scale:1. % of clients who were homeless at the end of the periodLevel of functioning scale:2. % of clients who rate their parenting skills better at exit than they did at entryGoal attainment scaling:3. % of clients who achieved better than expected outcomes in majority of goals reviewed this period
  • 12. 12Example tool: Outcomes StarTriangle Consulting / London Housing Foundation. http://www.homelessoutcomes.org.uk/The_Outcomes_Star.aspx
  • 13. 13Collect dataData collection and entry: Questions to consider: Has a huge impact on data  Who will collect the quality data? What training will they require? Should be integrated as far as  All clients or a sub- possible with service delivery sample? Requires client consent  How and when will May need to respond flexibly data be collected? to circumstances of client  How will data be stored?
  • 14. 14Analyse and reportOutcomes reporting enablers: Questions to consider: A knowledge of stakeholder  Who will use the requirements reports? Data analysis skills  What levels of Standardised: aggregation are  calculation procedures useful?  report templates  How can complexity  database queries be acknowledged? Access to data!
  • 15. 15Example: control graphProportion of service episodes with housing situation improved at completion Quarterly proportion Cumulative proportion 41.3% 40% 38.4% 37.4%Percentage ofcompleted service 34.5% 35.2% 34.3% 34.4% 33.3% 34.9%episodes 31.8%(current quartern = 165, 28.5% 30%cumulative 28.6%n = 1097) 20% Jun Sep Dec Mar Jun Sep Dec Mar Jun 2009 2009 2009 2010 2010 2010 2010 2011 2011
  • 16. 16Use outcomes informationContexts for use: Questions to consider: Individual casework:  How can outcomes assessment, planning and information be review shared with clients? Service review and reflection  How can staff play a Strategic planning: “turning the part in “giving curve” meaning” to Advocacy outcomes information?
  • 17. 17Caution! Outcomes monitoring data by itself cannot “prove” service effectiveness To build your case, triangulate with other evidence
  • 18. 18Sustain the systemWhat helps? Using the information – complete the loop Strong commitment to and endorsement of outcomes measurement by senior management Performance expectations – staff to collect outcomes data Ongoing resourcing Regular review of framework
  • 19. 19 Risks Implementation process The bottom line
  • 20. 20Risks Poor value for money  Outcome measurement can be resource-intensive and time-consuming  Information produced may not be high quality Distortion of service delivery Staff opposition  Data collection burden  Feeling scrutinised Negative results
  • 21. 21Process tips Involve stakeholders early and ongoingly  Board, Executive, management, service delivery staff, clients Combine top-down and bottom-up elements Pilot locally, implement sequentially Typically 6 – 24 months to end of pilots  Varies with size of organisation, complexity of services, and level of resourcing
  • 22. 22The bottom line Client wellbeing comes first Client rights are respected Information produced is useful  Relevant  Reliable  Valid System is affordable  Data collection burden on staff is minimised
  • 23. 23Useful resourcesResults-Based Accountability  www.raguide.org  Friedman (2009) Trying Hard Isn’t Good EnoughUK Approaches (incl. Outcomes Star)  www.homelessoutcomes.org.uk  Burns & Cupitt (2003) Managing Outcomes: A Guide for Homelessness OrganisationsUS: National Alliance to End Homelessness  Spellman & Abbenante (2008) What Gets Measured, Gets Done: A Toolkit on Performance Measurement for Ending HomelessnessReference – technical and implementation issues  Planigale (2010) Literature Review: Measurement of Client Outcomes in Homelessness Services. http://www.homeground.org.au/assets/literature-review-measurement-of- client-outcomes-in-homelessness-services.pdf
  • 24. Mark PlanigaleResearch & Consultancyresults by designPhone: 0429 136 596Mail: PO Box 754, Macleod VIC 3085Email: results@planigale.comWeb: www.planigale.com