Mission Australia conducted a pilot study to measure client wellbeing outcomes in two of its programs. The study found that clients in the PHaMs program, which provides intensive wrap-around support, experienced significant improvements in personal wellbeing, life satisfaction, and reduced poor outcomes over time. In contrast, clients in the Job Services Australia program, which has a narrower focus on employment, did not see similar improvements. Key lessons included developing a sound framework, gaining management support, using repeat measures to track progress, and reflecting on results to inform practice improvements.
This is a summary of all of the QI projects reported by KDHE staff in 2019. For questions about projects, please contact the team leader or other team members listed for the project.
Consumer participation @ ReGen: Transforming an organisation (11/05/16)Uniting ReGen
#iAOD16 presentation on the development of Consumer Participation practice at ReGen. For more event details see: http://www.regen.org.au/more-events/724-2016-innovation-seminar-11-may
Consumer Participation in Drug Treatment Services: Overview of Australian Res...Uniting ReGen
Keynote #iAOD16 presentation by Prof Carla Treloar. For more event information, see http://www.regen.org.au/more-events/724-2016-innovation-seminar-11-may.
Impact practice in the third sector for public health practitionersCatherine A. Greaves
Sharing Impact Practice (outcomes measurement) from third sector (community & voluntary sector) wellbeing projects for public health mental health & alcohol interventions
This is a summary of all of the QI projects reported by KDHE staff in 2019. For questions about projects, please contact the team leader or other team members listed for the project.
Consumer participation @ ReGen: Transforming an organisation (11/05/16)Uniting ReGen
#iAOD16 presentation on the development of Consumer Participation practice at ReGen. For more event details see: http://www.regen.org.au/more-events/724-2016-innovation-seminar-11-may
Consumer Participation in Drug Treatment Services: Overview of Australian Res...Uniting ReGen
Keynote #iAOD16 presentation by Prof Carla Treloar. For more event information, see http://www.regen.org.au/more-events/724-2016-innovation-seminar-11-may.
Impact practice in the third sector for public health practitionersCatherine A. Greaves
Sharing Impact Practice (outcomes measurement) from third sector (community & voluntary sector) wellbeing projects for public health mental health & alcohol interventions
Workplace Wellness in Flux – Nicolaas Pronk at Modern Healthcare's 8th annual Workplace of the Future Conference on Wednesday, October 14, 2015 at the Omni Hotel in Nashville Tennessee.
Tailoring programs and services to methamphetamine (Nov 15)Uniting ReGen
Presentation by Laurence Alvis & Rose McCrohan on ReGen's work developing a range of treatment responses to people seeking treatment for methamphetamine dependence.
NatCon 2018 Presentation: Pay for Success and Social Impact Bondscommteam
Faith Richie, SVP of Development at Telecare, as well as Shannon Mong, Director of Innovation Initiatives at Telecare, provided an insider’s perspective on the planning and implementation of our “pay for success” behavioral health program, Partners in Wellness. This community-based program in San Jose, California, establishes performance incentives that align clinical goals and social impact regarding reducing expensive utilization of wraparound services.
ROI of wellness programs - Optimity webinar series Dec 2016Jane Wang
ROI on Wellness Programs
OVERVIEW OF HOW WELLNESS PROGRAMS ARE BECOMING MORE OUTCOMES DRIVEN
"Traditional wellness programs, with engagement hovering around 30%, just don't cut it any more. We share 3 tips on how you can drive ROI for your wellness initiatives and dive deep into industry best practices. The benefit of a "one stop shop" platform like Optimity can help easily cross-polinate engagement from different initiatives and bring in targeted coaching support to move the needle on your health outcomes. "
Expert: Jane Wang, CEO
Host: Trista Chan, Advisor - Wellness Strategies, Optimity
The Leadership Pipeline: Cultivating Your Organization’s High Potential Emplo...Modern Healthcare
The Leadership Pipeline: Cultivating Your Organization’s High Potential Employees – Joseph Cabral at Modern Healthcare's 8th annual Workplace of the Future Conference on Wednesday, October 14, 2015 at the Omni Hotel in Nashville Tennessee.
The Clinician's Role in Developing a Patient Experience StrategyRenown Health
Learn how clinicians and marketing consultants can work together to develop a patient experience strategy that enables team to work at the highest levels and achieve outstanding results.
Reaching the Unreachable: Engaging People with SUDS in Pre-Contemplation Phasecommteam
Learn about Co-Occurring Education Groups, identify strategies to assist mental health and substance use staff, and learn about the qualitative outcomes of a non-judgmental, educational approach.
State of the Independent Practice: What it Means For YouKareo
Join Kareo as we discuss the results from our recent nation-wide survey of 1,000 medical practices. We'll discuss our findings from the survey and how these insights will shape your billing company's future growth.
Jo Partington, AHP Lead at the Department of Health, Transforming Community Services, addresses AHP Leadership Challenges, Referral to Treatment Project, Service Improvement Project and Transforming Community Services. COT Annual Conference 2010 (22-25 June 2010)
Happy at work 4 do the drills because corporate wellbeing programs lead to em...Salt & Light Ventures
Gerry Plana's talk presentation at the Happy at Work Conference 2014 titled, "Do the Drills: Because Corporate Well-Being Programs Lead to Employee Happiness"
Workplace Wellness in Flux – Nicolaas Pronk at Modern Healthcare's 8th annual Workplace of the Future Conference on Wednesday, October 14, 2015 at the Omni Hotel in Nashville Tennessee.
Tailoring programs and services to methamphetamine (Nov 15)Uniting ReGen
Presentation by Laurence Alvis & Rose McCrohan on ReGen's work developing a range of treatment responses to people seeking treatment for methamphetamine dependence.
NatCon 2018 Presentation: Pay for Success and Social Impact Bondscommteam
Faith Richie, SVP of Development at Telecare, as well as Shannon Mong, Director of Innovation Initiatives at Telecare, provided an insider’s perspective on the planning and implementation of our “pay for success” behavioral health program, Partners in Wellness. This community-based program in San Jose, California, establishes performance incentives that align clinical goals and social impact regarding reducing expensive utilization of wraparound services.
ROI of wellness programs - Optimity webinar series Dec 2016Jane Wang
ROI on Wellness Programs
OVERVIEW OF HOW WELLNESS PROGRAMS ARE BECOMING MORE OUTCOMES DRIVEN
"Traditional wellness programs, with engagement hovering around 30%, just don't cut it any more. We share 3 tips on how you can drive ROI for your wellness initiatives and dive deep into industry best practices. The benefit of a "one stop shop" platform like Optimity can help easily cross-polinate engagement from different initiatives and bring in targeted coaching support to move the needle on your health outcomes. "
Expert: Jane Wang, CEO
Host: Trista Chan, Advisor - Wellness Strategies, Optimity
The Leadership Pipeline: Cultivating Your Organization’s High Potential Emplo...Modern Healthcare
The Leadership Pipeline: Cultivating Your Organization’s High Potential Employees – Joseph Cabral at Modern Healthcare's 8th annual Workplace of the Future Conference on Wednesday, October 14, 2015 at the Omni Hotel in Nashville Tennessee.
The Clinician's Role in Developing a Patient Experience StrategyRenown Health
Learn how clinicians and marketing consultants can work together to develop a patient experience strategy that enables team to work at the highest levels and achieve outstanding results.
Reaching the Unreachable: Engaging People with SUDS in Pre-Contemplation Phasecommteam
Learn about Co-Occurring Education Groups, identify strategies to assist mental health and substance use staff, and learn about the qualitative outcomes of a non-judgmental, educational approach.
State of the Independent Practice: What it Means For YouKareo
Join Kareo as we discuss the results from our recent nation-wide survey of 1,000 medical practices. We'll discuss our findings from the survey and how these insights will shape your billing company's future growth.
Jo Partington, AHP Lead at the Department of Health, Transforming Community Services, addresses AHP Leadership Challenges, Referral to Treatment Project, Service Improvement Project and Transforming Community Services. COT Annual Conference 2010 (22-25 June 2010)
Happy at work 4 do the drills because corporate wellbeing programs lead to em...Salt & Light Ventures
Gerry Plana's talk presentation at the Happy at Work Conference 2014 titled, "Do the Drills: Because Corporate Well-Being Programs Lead to Employee Happiness"
Bertus Van Niekerk: Unlocking the True Potential of Integrated Occupational H...SAMTRAC International
This presentation argues that the value of occupational health and safety, and corporate wellness programmes, can be increased exponentially through an integrated information system. This is accomplished by integrating data collected from a host of standalone safety technologies with an electronic health record, corporate wellness and ERP systems.
As part of the Go Green Business Breakfast - Happy & Healthy during Bristol's first Healthy City Week Nicola Rich and Anne Sheekey explained the development of the Workplace Wellbeing toolkit, packed full of ideas for workplaces to run fun activities that engage their staff on wellbeing issues. The full pack can be downloaded for free from http://www.workplacewellbeingbristol.co.uk/
Worksite Wellness Toolkit for Community Based Organizationsbeccapurnell
The Worksite Wellness toolkit was designed to serve as a resource guide for community-based organizations in order to implement worksite wellness activities and programs at little to no cost. Worksite wellness is the promotion of employee health at the worksite.
How to create better wellbeing outcomes for students through designAtkins
Dr Caroline Paradise, Atkins’ UK head of design research, gave a presentation on applying lessons learned from post occupancy evaluation to create better wellbeing outcomes for students at Education Estates Conference 2016.
Find out how Atkins put wellbeing at the heart of design at www.wellbriefing.com
What should you know about Employee Wellness Programs in 2014 and Beyond? We've released an infographic, which provides an overview of the different types of wellness programs and their benefits, as well as a look at the trends among employers across the country regarding employee health and wellness.
Report: What Employees Want In Corporate Wellness ProgramsShapeUp
ShapeUp has completed its survey of one thousand employees, and the surprising results show what employees think about corporate wellness programs.
View the webinar slides here and download the complete report to discover what employees are saying about wellness, including:
* Their top wellness goals
* Most popular wellness program components
* Why they join corporate wellness programs
* How you can pique their interest in wellness
* What programs they are willing to pay for out-of-pocket
* Strategies to keep them constantly engaged
And much more! Download the full report at http://www.shapeup.com/employee-wellness-survey/
This presentation outlines my personal understanding of what happiness is, based on 4 years of learning about it, and finding my own path in the world.
This presentation was given by Mark Williamson, Director of Action for Happiness on Tues 23 Aug 2011.
It was part of a British Council workshop on the theme of Wellbeing involving young social and political activists from the UK and the Middle East.
It gives an overview of why wellbeing is important and the political context, summarises some important recent scientific findings relating to happiness and suggests some actions that governments and individuals can take to help create a happier society.
Your business is important, and so are the employees who make it great. When they're healthier, they're happier and when they're happier, they're more engaged at work. That's why having a wellness program that works is the key.
Your business is important, and so are the employees who make it great. When they're healthier, they're happier and when they're happier, they're more engaged at work. That's why having a wellness program that works is the key.
What You'll Learn:
- The direct & indirect costs of unhappy & unhealthy employees
- How the highest rated employers have a relentless focus on wellness
- The New Corporate Strategy Pyramid
- How the weight of your employees impacts your bottom line
Companies are under attack! Not from rising energy costs, foreign competition, or regulatory pressure, but from a steady decrease in employee health & happiness. The "weight" of this problem is falling on the bottom lines of companies in the form of rapidly increasing costs and rapidly decreasing employee productivity and engagement. Once thought to be an involuntary benefit, Employee Wellness is now being discussed in corporate board rooms all over the world as a critical element of business strategy. HR is expected to understand and manage the risk associated with this problem and create a sustainability strategy that includes health & happiness. This eBook discussed the economics of this fight and how the war talent is now happening in the gym and cafeteria.
A birds eye view of the mental health of the people who are in the corporate Field. How they become vulnerable and what can be done to improve their overall health and life.
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...OECD CFE
Presentation by Jonathan Potter, OECD LEED Senior Policy Analyst, and Stuart Thompson, OECD LEED Policy Analys, tat the seminar organised by the OECD LEED Trento Centre for the Officers of the Autonomous Province of Trento on 13 November 2015.
https://www.trento.oecd.org
Running Head INTEGRATED QUALITY AND RISK MANAGEMENT PLAN 1 .docxwlynn1
Running Head: INTEGRATED QUALITY AND RISK MANAGEMENT PLAN 1
INTEGRATED QUALITY AND RISK MANAGEMENT PLAN 30
MPM357 Project Performance and Quality Assurance
Quality Dimensions
Charles Williams
3/4/2019
Table of Contents
Project outline 4
Purpose of the project 4
Structure of the project 4
Goals and objectives of the project 6
Project deliverables 7
Report about patient’s response 7
Organizational Readiness for Quality Management 7
Organizational quality management program readiness 7
Quality management project readiness 7
Quality Systems Analysis 8
Current Quality system 8
Organizational readiness to incorporate IQRMP 8
Pros and Cons of ISO 9000 8
Pros and cons of Six Sigma 10
Pros and cons of Capability Maturity Model Integration 10
The combination most appropriate for this project 11
Quality dimension and criteria 12
Quality Process Improvement Tools and Techniques 17
Quality Performance Monitoring and Control 23
Management's Role in Quality Management 28
Quality Performance Communication Plan 29
References 30
Project outlinePurpose of the project
The goal of this plan is to establish a coordinated approach that will address the superiority assessment and course enhancement within the Patient Care Section of the Bureau of HIV/AIDS, North Carolina Department of Health. The Patient Care Section is dedicated to ensuring the highest quality of HIV medical care and support services provided to HIV/AIDS clients throughout the state of North Carolina.Structure of the project
Framework: Ryan Act 200 demands that all Ryan White agendas need to create a quality management program. This program will, therefore, support providers in ensuring that supportive services give access and adherence, ensuring adherence to PHS guidelines and lastly ensure that clinical, demographic and consumption information is accessible when monitoring and evaluation of the native endemic are needed.
Legislative requirements of this project are categorized into six themes.
i. Enhanced access
ii. Eminence management
iii. Aptitude improvement
iv. Embattled resources
v. Synchronization and associations
vi. Contribution and collaboration of other agencies.
The state of North Carolina in conjunction with the unit of health has embraced the sterling criteria of organizational brilliance. This criterion was founded on a set of interrelated core values, behaviors and beliefs that are present in accomplishment organizations. The basic framework of quality assurance is based on the Sterling criteria because this criterion is a foundation for integrity key business requirement in a result-oriented context (Kerzner, 2018).
The senior management team in the patients care section is responsible for planning, directing and coordinating health services related to the States HIV programs. The leadership of this team approves and reviews the activities of the plan when they carry out their activities. A committee has been established to evaluate the plan's objectiv.
If inspection is the enemy of improvement, someone's not doing it right.Rami Okasha
What is the relationship between scrutiny and improvement? How can modern forms of scrutiny which focus on outcomes support improvement and innovation in social care? This paper describes some emergent approaches and results in Scotland being pioneered by the Care Inspectorate.
ZGB - The Role of Generative AI in Government transformation.pdfSaeed Al Dhaheri
This keynote was presented during the the 7th edition of the UAE Hackathon 2024. It highlights the role of AI and Generative AI in addressing government transformation to achieve zero government bureaucracy
Russian anarchist and anti-war movement in the third year of full-scale warAntti Rautiainen
Anarchist group ANA Regensburg hosted my online-presentation on 16th of May 2024, in which I discussed tactics of anti-war activism in Russia, and reasons why the anti-war movement has not been able to make an impact to change the course of events yet. Cases of anarchists repressed for anti-war activities are presented, as well as strategies of support for political prisoners, and modest successes in supporting their struggles.
Thumbnail picture is by MediaZona, you may read their report on anti-war arson attacks in Russia here: https://en.zona.media/article/2022/10/13/burn-map
Links:
Autonomous Action
http://Avtonom.org
Anarchist Black Cross Moscow
http://Avtonom.org/abc
Solidarity Zone
https://t.me/solidarity_zone
Memorial
https://memopzk.org/, https://t.me/pzk_memorial
OVD-Info
https://en.ovdinfo.org/antiwar-ovd-info-guide
RosUznik
https://rosuznik.org/
Uznik Online
http://uznikonline.tilda.ws/
Russian Reader
https://therussianreader.com/
ABC Irkutsk
https://abc38.noblogs.org/
Send mail to prisoners from abroad:
http://Prisonmail.online
YouTube: https://youtu.be/c5nSOdU48O8
Spotify: https://podcasters.spotify.com/pod/show/libertarianlifecoach/episodes/Russian-anarchist-and-anti-war-movement-in-the-third-year-of-full-scale-war-e2k8ai4
This session provides a comprehensive overview of the latest updates to the Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards (commonly known as the Uniform Guidance) outlined in the 2 CFR 200.
With a focus on the 2024 revisions issued by the Office of Management and Budget (OMB), participants will gain insight into the key changes affecting federal grant recipients. The session will delve into critical regulatory updates, providing attendees with the knowledge and tools necessary to navigate and comply with the evolving landscape of federal grant management.
Learning Objectives:
- Understand the rationale behind the 2024 updates to the Uniform Guidance outlined in 2 CFR 200, and their implications for federal grant recipients.
- Identify the key changes and revisions introduced by the Office of Management and Budget (OMB) in the 2024 edition of 2 CFR 200.
- Gain proficiency in applying the updated regulations to ensure compliance with federal grant requirements and avoid potential audit findings.
- Develop strategies for effectively implementing the new guidelines within the grant management processes of their respective organizations, fostering efficiency and accountability in federal grant administration.
A process server is a authorized person for delivering legal documents, such as summons, complaints, subpoenas, and other court papers, to peoples involved in legal proceedings.
Many ways to support street children.pptxSERUDS INDIA
By raising awareness, providing support, advocating for change, and offering assistance to children in need, individuals can play a crucial role in improving the lives of street children and helping them realize their full potential
Donate Us
https://serudsindia.org/how-individuals-can-support-street-children-in-india/
#donatefororphan, #donateforhomelesschildren, #childeducation, #ngochildeducation, #donateforeducation, #donationforchildeducation, #sponsorforpoorchild, #sponsororphanage #sponsororphanchild, #donation, #education, #charity, #educationforchild, #seruds, #kurnool, #joyhome
2. Mission Australia is
client focused.
We strive for client informed
services that achieve positive
outcomes with individuals
and communities.
3. “How do we know what
we’re doing is helping
people?” and
“how do we measure that impact
and aggregate the results to
show how much difference the
organisation is making?”
4. Outputs are the products, services or facilities that result from an
organisation’s or project’s activities.
Outcomes are the changes, benefits, learning or other effects that result
from what the project or organisation makes, offers or provides.
Impact is the broader or longer-term effects of projects or organisation’s
outputs, outcomes and activities.
Outputs, outcomes and impact
5. Our approach
Inputs, Outputs
& Activities
Outcomes Impact
Organisational
Strategies
• Early intervention
and prevention
• Advocacy and client
voice
• Evidence based best
practice and case
management
• Self-efficacy and optimism
• Confidence and fulfillment
• Participation and inclusion
• Independence
Evaluation
Methods
• Input and output
measurement using
existing
administrative
systems
• Development of indicators.
• Consultation with service
and organisational staff
• Lessons from previous
pilots
• MA Client Wellbeing Survey
pilot
• Data analysis and
reflection of the
Theory of Change
and Practice
• Reporting back to
stakeholders and
practitioners
7. Outcome measures must:
• Be externally credible
• Be used with the external population (to provide comparison data)
• Have communication power, internally and externally
• Demonstrate program effectiveness to funders and donors
Principles
8. 1. Outcome hierarchies developed and implemented
2. Indicators and collection methodologies developed
3. Consultation and approval with staff across the organisation
4. MA Client Wellbeing Survey pilot developed and piloted in two services:
PHaMs and JSA Stream 4:
• The Personal Helpers and Mentors service (PHaMs) is a non‐clinical
recovery‐focussed support service for people experiencing
mental health issues.
• Jobs Services Australia, now JobActive, was the Australian
Government’s employment program. Stream 4 clients were those with
multiple and complex needs.
Stages
9. Stages of the pilot:
1. Processes for data collection developed
2. Ethics, consent and protocols developed
3. Sampling framework – all new clients in a set period
4. Training – face to face and webinars
5. Repeat measures data collection implemented
6. Data analysis and reporting
Implementing impact measurement
11. Mission Australia Client Wellbeing Survey
Captures subjective wellbeing through a
combination of measures
• Personal Wellbeing Index (PWI)
• Single item measures on:
• Housing
• Financial security
• Health
• Self-esteem and control
• Demographics:
• Collected via MA systems
• Age, gender, ATSI status
• Service interventions
12. PWI is the foundation of the survey.
It measures subjective wellbeing by asking
about satisfaction in:
• Standard or living
• Health
• Achieving
• Relationships
• Safety
• Community
• Future security
• Life as a whole
Mission Australia Client Wellbeing Survey
14. PHaMs JSA
Participants at intake
Many of our clients have very low levels of personal wellbeing
High Risk
PWI 0-50
Challenged
PWI 51-69
Normal Range
PWI 70-100
94%
6%
50%
25%
25%
15. PHaMs participants
MA saw significant improvements for the PHaMs clients across a range of
domains and self-reported measures after eight months in the program
50
48
35
0 20 40 60 80 100
Personal Wellbeing Index
Wave 1 Wave 2 Wave 3
17. PHaMs participants
82
63
77
47
59
25
47
18
41
19
35
18
0 20 40 60 80 100
Control over own life: has less
than mixed control
Housing adequacy: is not
adequate
Coping: Not at all or a little
Not enough money to meet needs
%
Proportion self-reporting poor outcomes
Wave 1
Wave 2
Wave 3
19. JSA participants
Significant improvements were not seen in wellbeing of
JSA Stream 4 job seekers
54
51
55
0 20 40 60 80 100
Personal Wellbeing Index
Wave 1 Wave 2 Wave 3
21. Mission Australia’s Client Wellbeing Pilot has been
validated as a useful tool to measure and inform
client journeys to independence in programs where
clients receive wrap-around intensive supports, such
as in the PHaMs program.
The findings of the pilot indicated that where a
program has a narrow focus and a constrained delivery
model such as in the Job Services Australia model, the
tool is not well adapted to collect the outcomes.
22. 1. Consider what is already known to
develop a sound framework
2. Gain management endorsement and
collaborate
3. Use repeat measures
Key lessons about measuring impact
23. Informing policy & practice
• Continuous quality improvement in our services.
• Where site results were reported and reflective practice undertaken,
service improvements have been identified and implemented.
• The pilot will be continued in selected sites to inform the design of a
model for the roll out of the NDIS.
• To inform a review of an outcomes measurement framework across MA
for reducing homelessness and strengthening communities.
• As an iterative process with government in tender and program design.
How Mission Australia uses outcome measurement
24. How Mission Australia uses outcome measurement
Reflecting on practice
PHaMs sites reflected on their practice. They were provided with site specific
outcome measures and questions to help them reflect on what the outcome
measures mean for practice:
• Are our clients better off?
• If not, what could be the reasons and are any of them reasons that we
can influence with our practice?
25. Feedback from the field:
“By having regular discussions the pilot has enabled us to identify:
• How we can continue to increase support for participants,
carers and families,
• How we can improve on group activities, external factors
which may be influencing participants’ daily lives etc.
As a team our purpose is to assist participants in pursuing the
best outcomes possible towards their recovery, as such being
able to participate in the Client Impact Pilot has been a valuable
experience.”
PHaMs Manager
26. • There is no common set of indicators being used across
Commonwealth and State Governments for outcomes-based funding
and program delivery.
• The modified PWI approach was adopted to measure our impact on
client independence. Future impact evaluation needs to measure
outcomes against MA’s current goals of reducing homelessness and
strengthening communities across our diverse programs and
services.
• Developing outcomes measurement systems and requisite IT
platforms requires staff capacity and resources.
Challenges
27. Outcome measurement is
important. It shows us and our
stakeholders how we are
achieving client outcomes.
It assists us to continually
reflect and improve.
Mission Australia is a national community service organisation.
We have been helping people to regain their independence for over 155 years.
Our goal is to reduce homelessness and strengthen communities.
We have a client focus when doing this.
MA believes a good quality of life is closely associated with independence, which includes the ability to achieve individual potential and the opportunity for social and economic participation.
“How do we know if what we’re doing is helping people?”
It’s the question community organisations seeking to help clients change their lives have to ask themselves every day. This is quickly followed by, “how do we measure that impact and aggregate the results to show how much difference the whole organisation is making?”
As part of grappling with these questions Mission Australia commenced an impact measurement pilot to better understand how our services contributed to clients’ journeys towards independence. The pilot was designed to inform funders and donors on program outcomes and MA on the effectiveness of its services.
The aim of the pilot was to enhance our understanding of the role of MA’s interventions (which are our outputs)
in improving self-efficacy and optimism, confidence and fulfilment, and participation and inclusion (which are the outcomes we aim to achieve)
and ultimately to measure clients’ journey towards independence (which is the impact we hope to have)
So we wanted to better understand the role of interventions in improving wellbeing, and ultimately to measure clients’ journeys towards independence.
To develop a stronger understanding of what independence means for clients, we reflected on MA’s strategic goals, program logics and service models.
We found that self-efficacy and optimism, confidence and fulfilment, and participation and inclusion were common themes in MA’s rationales for what we do.
These contribute to our theory of change, which articulates the desired impact we hope our clients achieve through our services. It sets out the projected outcomes, while spelling out the underlying assumptions of how we work towards helping people get back on their feet.
Having identified these assumptions, we then work out how change will be manifested and articulate short, medium and long term outcomes.
Outcomes are achieved through our inputs and program outputs, which are, broadly speaking: early intervention and prevention, advocacy and client voice, evidence based practice and a case management approach.
From the theory of change and outcomes hierarchies, measurable indicators were developed that can be used to quantify client progress.
MA undertook an extensive literature review of both national and international indicators and measures which mapped against program specific outcomes.
During 2011/12 MA piloted over 100 outcome measures from sources such as the Australian Bureau of Statistics (ABS), Household Income and Labour Dynamics in Australia Survey (HILDA) and World Health Organisation (WHO), and the Personal Wellbeing Index (PWI) to get an understanding of which measures were effective in capturing change and which could be easily implemented and analysed.
From this experience we learnt that we needed to keep the process of capturing the data simple for both staff and clients: the administration had to be easy for the staff to understand; the tool easy for clients to complete; and the measurements meaningful for the service providers.
It was important that normative and comparative data be available for these measures to help service delivery staff and stakeholders ‘make sense’ of results.
This is particularly important when demonstrating program effectiveness to funders and donors.
Mission Australia developed a logical framework approach to our understanding of outcomes, as well a methodology to measure these. Our services use program logics which map expected behavioural change at the program level.
Measurable indicators were developed to quantify client progress. This included international and national review; matching to outcomes frameworks; piloting with staff and clients; review and revision.
Consultation with teams across MA including: program managers, case workers, service design and delivery staff, business intelligence, and clients.
Approval by the Board’s Service Impact committee and executive leaders.
MA Client Wellbeing Survey developed and piloted in two services – PHaMs and JSA.
Processes included using online and paper surveys (which took no more than 10 minutes to be self completed by the client) as well as existing administrative systems.
The pilot underwent an organisational ethical review
Information sheet and consent forms for clients to provide informed consent
Client participation was voluntary
Data was de-identified at collection and no identifiable data was made available.
The sample included:
PHaMs: All new clients entering services between May and July 2014
JSA : all new Stream 4 clients, or newly placed (employment/education/training) clients during the same time period.
Training was provided to all staff responsible for the data collection.
PHaMs: consultation occurred at the site level via program staff that validated measures and sought input from clients.
JSA: comprehensive data already collected for service reporting was deemed sufficient.
Collection:
PHaMs: Program managers were responsible for implementation. Data was collected by Program Managers and Case Workers
JSA: Employment Advisors offered the survey to eligible jobseekers.
Data captured from the client surveys, caseworker surveys and administrative sources were combined on the basis of service client ID. Wave-on-wave comparisons were made on data gathered from the same matched set of clients.
MA used the Personal Wellbeing Index (PWI) as the foundation of measuring client outcomes (more about this on next slide).
The survey tool also included some single item questions to ensure that all domains in MA’s outcomes hierarchy were properly covered. To achieve this we added questions from the HILDA survey and Longitudinal Survey of Australian Children (LSAC) and the WHO Quality of Live (WHOQOL) to monitor the progress of our clients.
PHaMs program managers took part in defining appropriate performance measures as part of a training workshop. The measures developed were shared with other PHaMs staff to validate and shortlist. As well as being validated by case workers, the measures were validated by peer workers and clients. A questionnaire for case workers was developed to capture data on any remaining measures.
Administrative systems were used to match the responses with the demographic profile of the clients, and JSA performance measures.
The PWI was produced by Deakin University’s Australian Centre on Quality of Life and Australian Unity.
It has been used in Australia for the past 12 years and has been used internationally in over 50 countries. Over this time it has been shown to been a valid and reliable measure of subjective wellbeing.
Subjective wellbeing doesn’t fluctuate or change much over time. This is because people tend to ‘self-regulate’ their wellbeing: a solid level of wellbeing is something we all tend to seek. In other words, we try to manage it so that it is stable.
If circumstances are particularly difficult (i.e. if someone doesn’t have housing, is being abused, etc.) then that self-regulation is more difficult and wellbeing may decrease.
The PWI is particularly sensitive, so if we address such circumstances we could reasonably expect it to register changes in wellbeing.
Survey participants rate their satisfaction with these domains on a scale of 0-10. The scores on these seven domains are averaged into a 0-100 scale, where 0 is completely dissatisfied and 100 is completely satisfied.
The following guidelines are given by the developers of the PWI:
70+ points: ‘Normal’: A person is likely to be experiencing a normal level of wellbeing.
51-69 points: ‘Challenged’: Personal wellbeing is likely to be challenged / compromised.
<50 points: ‘High-risk’: Very low personal wellbeing / strong likelihood of depression.
In Australia the average PWI is around 75
Almost all PHaMs participants had a PWI score which placed them in the ‘high risk’ category.
Half of JSA participants had a PWI which placed them in the ‘high risk’ category, and a further quarter had a PWI score which placed them in the ‘challenged’ category.
Clients also reported very low levels across the other measures of self-esteem, control and coping.
Health, housing and finances were also all poorly reported against
Domains where PHaMs clients showed the most improvement were:
Standard of Living
Future Security
Health
Achieving in Life
Generally, the greatest improvements in domains of wellbeing occurred in the first four months. In the following four months wellbeing generally remained at improved levels or continued to increase but less sharply.
Note: Any ‘decreases’ between Wave 2 and Wave 3 were not statistically significant. This means that we interpret these results as having remained stable.
The pilot results suggest that the PHaMs service model delivers outcomes consistent with MA’s theory of change and that the service model and its delivery contribute to the client journey to independence.
The PHaMs program, which offers a recovery service rooted in a strengths based approach, is effective in supporting client wellbeing and paths to independence. It is expected that similar programs will also support client wellbeing and paths to independence.
Other areas which showed significant improvements.
Note: These are drops in the proportion of clients with each negative outcome. Although these outcomes look larger than the PWI outcomes on the previous slide, the scales are different and cannot be compared. We cannot compare a change in PWI to a change in per cent, just as we cannot compare a change in weight to a change in temperature.
Caseworkers assessed that over the eight months more participants were accessing appropriate services.
Generally JSA participants remained at the same levels of wellbeing.
It could be argued that providing opportunities for employment, education and training is a mechanism for improving wellbeing.
MA did not see significant improvements in the wellbeing of job seekers who were placed in employment, education or training.
However, MA was not able to follow-up with job seekers after they had exited the program to determine if employment, education and training had an impact on their wellbeing in the longer-term.
We also need to be careful when directly comparing the PHaMs and JSA results. The participants of the two programs were different people with different initial levels of subjective wellbeing and different barriers to independence.
Similarly – we did not see significant changes in the domains of the PWI, as we did for PHaMs.
Note: The increase in future security was not statistically significant. Statistical tests take into consideration how variable a measure is within a group. In this case there was a lot of variance in how much JSA participants rating of Future security changes. When there is a lot of variance we need larger sample sizes to tell if a result is statistically significant.
The pilot results suggest that the PHaMs Service model delivers outcomes consistent with MA’s theory of change and that the service model and its delivery contribute to client journey to independence.
The PHaMs program, which offers a recovery service that takes a strengths based approach, is effective in supporting client wellbeing and paths to independence. It is expected that similar programs will also support client wellbeing and paths to independence.
Significant improvements were not seen in wellbeing of JSA Stream 4 jobseekers. While measuring wellbeing in this context offers an opportunity to understand the jobseeker and their needs better, the outcomes that the service model demands and the large case load means that employment advisers are not able to provide the sort of intensive, flexible and holistic support available in a program such as PHaMs.
Note:
Caution needs to be exercised in attributing the positive outcomes exclusively to MA interventions.
Proving a causal link is difficult when there are multifaceted external factors in play in each client’s circumstances which can influence the outcomes observed.
Before developing outcomes indicators, MA developed sound outcomes hierarchies and a theory of change. It is on these frameworks that the outcome indicators and measures rest. This highlights the importance of allocating time and resources upfront to comprehensively understand high level organisational goals, and those of the programs.
Management endorsement is critical to success in outcome measurement. The pilot also required internal resources from a number of teams and across different business units at all stages of development and implementation. Many staff were trained and supported to undertake this pilot and there has generally been excellent feedback from different levels of services staff.
MA utilised a repeat measures design for measuring outcomes, in which the same cohort of participants were measured over three waves. This type of research design allowed us to look at average improvement over time, as well as explore individual differences between participants in their journey towards independence.
MA staff reported significant value in understanding and reflecting on client outcomes. (More about this on following slides).
Mission Australia’s Strategic Plan commits the organisation to delivering evidence-based integrated services to achieve the goals of reducing homelessness and strengthen communities. The Plan also commits MA to delivering more relevant and effective services and integrating client outcomes across core services.
The Client Impact Pilot will be used to inform a review of an outcomes measurement framework across MA for reducing homelessness and strengthening communities. The framework to be used will include assessment of a number of measurement tools appropriate to funder requirements and service type.
As Commonwealth and State Governments move to outcomes based payments and measurement for program delivery, there is no common set of indicators being used. One purpose of the pilot has been to use this as an iterative process with government in tender and program design.
For the reflection exercise staff were asked:
What does the data tell us- how are we doing with on these measures?
Pick one OUTCOME measure/result that they feel is important to focus on.
How can we optimise this outcome for our client?
Who are the partners who can help us?
What works or could work to do better (low cost and no cost ideas)?
What are we going to do?
• As Commonwealth and State Governments move to outcomes based payments and measurement for program delivery, there is no common set of indicators being used. One purpose of the pilot has been to use this as an iterative process with government in tender and program design.
• Mission Australia has a diverse range of programs and services and the modified PWI approach was adopted to measure how effective our case management interventions have been in improving client independence. MA’s current strategic plan has set organisational goals of reducing homelessness and strengthening communities and our further work on impact measurement will need to include additional indicators to measure outcomes against these goals.
• Governments and donors expect service providers to allocate program funding to front line service delivery and Mission Australia and other providers have limited resources to develop outcomes measurement systems with the requisite IT platforms. MA has therefore funded the pilot from existing resources.
Quote from the field:
“the experience has been very motivating very motivating for staff here who know that they are doing a good job but have never really seen hard evidence.”
For more information…..
Please see Mission Australia’s Client Wellbeing Pilot Final Report
Or contact the Research and Evaluation team by emailing research@missionaustralia.com.au