Creating an outcomes framework for your organisation


Published on

Key steps in creating a client outcomes measurement framework for a welfare / human service organisation. Particular focus on homelessness assistance services.

Published in: Business, Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Creating an outcomes framework for your organisation

  1. 1. Homelessness assistance sector, Australia, November 2010 Mark Planigale Research & Consultancy results by design
  2. 2. 2 Performance measurement Compliance reporting Program logic
  3. 3. 3Performance measurement Bigger picture of organisational performance measurement  Compliance reporting currently focuses on effort  Many organisations monitor inputs and client satisfaction  But are we making a difference? National frameworks: National Affordable Housing Agreement, National Partnership Agreement on Homelessness ‘Manufacturing-based’ performance models need to be enhanced
  4. 4. 4Program logic Mission Vision Environment • Problem or need • Barriers • EnablersInputs Activities Outputs Outcomes ImpactsPeople and What gets What clients Changes in Changes inresources done receive clients’ lives society• Clients • Functions • Client • Short term • Social• Human • Processes participation • Long term outcomes resources • Actions benefits • Interim• Other • Client direct resources benefits Objectives Aims
  5. 5. 5 Useful information is produced throughwell-planned systems.
  6. 6. 6 Purpose and scope Defined outcomes Defined measures and tools Data collection and storage processes Analysis and reporting processes Use of outcomes information Strategies for sustaining the system
  7. 7. 7Clarify purpose and scopeWhy measure outcomes? Questions to consider Compliance  Which programs? Individual assessment and  Which populations? planning  What do we mean by Oversight outcomes? Quality improvement Advocacy
  8. 8. 8Define key outcome areasOutcome: Questions to consider“... a change or absence of  If the program works change that results (at least in really well, how are part) from actions of staff of clients’ lives better? the organisation”  If things go wrong Desired vs. undesired for our clients, where do we see this in Short term, long term, interim their lives? Domains vs. locus of change  What is our core business?
  9. 9. 9Example outcomes1. Person obtains appropriate, affordable long-term housing2. Person takes medication more regularly3. Person has a better understanding of tenancy rights and responsibilities4. Person is placed on the OOH Early Housing waiting list5. Person gains part-time employment6. Person becomes homeless
  10. 10. 10Select measures and toolsMeasure: Questions to consider“ observable, measurable  If our key outcomes characteristic of a person or occurred, how would their situation, which is linked we know? with a state or condition of interest to us”  Which measures have the greatest Base measures, derived communicative measures and KPIs power? Varied vantage points  What data do we Use modules for flexibility and already have? consistency
  11. 11. 11Example measuresStatus change/maintenance scale:1. % of clients who were homeless at the end of the periodLevel of functioning scale:2. % of clients who rate their parenting skills better at exit than they did at entryGoal attainment scaling:3. % of clients who achieved better than expected outcomes in majority of goals reviewed this period
  12. 12. 12Example tool: Outcomes StarTriangle Consulting / London Housing Foundation.
  13. 13. 13Collect dataData collection and entry: Questions to consider: Has a huge impact on data  Who will collect the quality data? What training will they require? Should be integrated as far as  All clients or a sub- possible with service delivery sample? Requires client consent  How and when will May need to respond flexibly data be collected? to circumstances of client  How will data be stored?
  14. 14. 14Analyse and reportOutcomes reporting enablers: Questions to consider: A knowledge of stakeholder  Who will use the requirements reports? Data analysis skills  What levels of Standardised: aggregation are  calculation procedures useful?  report templates  How can complexity  database queries be acknowledged? Access to data!
  15. 15. 15Example: control graphProportion of service episodes with housing situation improved at completion Quarterly proportion Cumulative proportion 41.3% 40% 38.4% 37.4%Percentage ofcompleted service 34.5% 35.2% 34.3% 34.4% 33.3% 34.9%episodes 31.8%(current quartern = 165, 28.5% 30%cumulative 28.6%n = 1097) 20% Jun Sep Dec Mar Jun Sep Dec Mar Jun 2009 2009 2009 2010 2010 2010 2010 2011 2011
  16. 16. 16Use outcomes informationContexts for use: Questions to consider: Individual casework:  How can outcomes assessment, planning and information be review shared with clients? Service review and reflection  How can staff play a Strategic planning: “turning the part in “giving curve” meaning” to Advocacy outcomes information?
  17. 17. 17Caution! Outcomes monitoring data by itself cannot “prove” service effectiveness To build your case, triangulate with other evidence
  18. 18. 18Sustain the systemWhat helps? Using the information – complete the loop Strong commitment to and endorsement of outcomes measurement by senior management Performance expectations – staff to collect outcomes data Ongoing resourcing Regular review of framework
  19. 19. 19 Risks Implementation process The bottom line
  20. 20. 20Risks Poor value for money  Outcome measurement can be resource-intensive and time-consuming  Information produced may not be high quality Distortion of service delivery Staff opposition  Data collection burden  Feeling scrutinised Negative results
  21. 21. 21Process tips Involve stakeholders early and ongoingly  Board, Executive, management, service delivery staff, clients Combine top-down and bottom-up elements Pilot locally, implement sequentially Typically 6 – 24 months to end of pilots  Varies with size of organisation, complexity of services, and level of resourcing
  22. 22. 22The bottom line Client wellbeing comes first Client rights are respected Information produced is useful  Relevant  Reliable  Valid System is affordable  Data collection burden on staff is minimised
  23. 23. 23Useful resourcesResults-Based Accountability   Friedman (2009) Trying Hard Isn’t Good EnoughUK Approaches (incl. Outcomes Star)   Burns & Cupitt (2003) Managing Outcomes: A Guide for Homelessness OrganisationsUS: National Alliance to End Homelessness  Spellman & Abbenante (2008) What Gets Measured, Gets Done: A Toolkit on Performance Measurement for Ending HomelessnessReference – technical and implementation issues  Planigale (2010) Literature Review: Measurement of Client Outcomes in Homelessness Services. client-outcomes-in-homelessness-services.pdf
  24. 24. Mark PlanigaleResearch & Consultancyresults by designPhone: 0429 136 596Mail: PO Box 754, Macleod VIC 3085Email: results@planigale.comWeb: