Spread methodology presentation - Innovation Forum, 19 November 2015
Memory Matters Evaluation _ Report
1. 1
Evaluationof
changes to Home
Help as part of
Memory Matters
By Debra O Neill, Project Consultant and Researcher, LinkAge Consultancy.
This evaluation report identifies the distinctive features of the
community element of the Carlow/Kilkenny Memory Matters
project in relation to person centered home help support. It
explores the findings and learnings from the project and their
influence on sustainability planning.
2.
3. 2
Evaluation of
changes to Home
Help as part of
Memory Matters
What has worked really well about the Memory Matters project in relation
to changing Home Help services to make them more person-centered?
Howhavepracticesandpolicieschanged?
The evaluation workshops were conducted in two sessions in January and
April 2016. The workshop participants are listed in Appendix 1.
One to one interviews were also conducted and meetings observed.
This report identifies:
The most significant Observations about the project
What Insight was gained about what worked well and what
have we learned?
Reflecting on these insights how has this changed work
practices?
What Decisions need to be made to ensure sustainability?
4. 3
Content
1. Background to Memory Matters Project
2. Change Map and Outline
3. Evaluation Framing
4. Focus Group Feedback
4.1 Observations and Achievements
4.2 Insights on Design and Delivery
4.3 Reflections on Barriers and Challenges
4.4 Decisions on Actions and Sustainability
5. Vox Pops – What has changed?
6. Summary Learnings, & Next Steps
Appendix & Notes:
1 Evaluation Participants
2 Post Project Care Pathway
3 Post Project Home Help Application Form
Acknowledgements
About the Author
3
Content
1. Background to Memory Matters Project
2. Change Map and Outline
3. Evaluation Framing
4. Focus Group Feedback
4.1 Observations and Achievements
4.2 Insights on Design and Delivery
4.3 Reflections on Barriers and Challenges
4.4 Decisions on Actions and Sustainability
5. Vox Pops – What has changed?
6. Summary Learnings, & Next Steps
Appendix & Notes:
1 Evaluation Participants
2 Post Project Care Pathway
3 Post Project Home Help Application Form
Acknowledgements
About the Author
4
6
7
9
9
10
13
15
17
17
20
21
22
25
25
6. Conclusions and Learning
5. 4
1.Background to Memory Matters Project
Memory Matters is a two year community based project for Carlow & Kilkenny
which is co-funded by the Health Service Executive, Older Persons Services,
Carlow and Kilkenny and Genio Trust, a philanthropic organisation, which
supports social change and projects in the area of dementia and mental health.
The project, which is an initiative by Kilkenny Age Friendly County Programme,
commenced in mid 2014 and will conclude in Autumn 2016.
The main objective of the research project was to design and test new ways to
support those living with dementia to remain at home in their own communities.
While this may seem straight forward, the scope of the project was ambitious,
stretching across two counties and affecting hundreds of people.
The project incorporated a number of individual initiatives which are both
interlinked and interdependent, including training, telecare and home care
packages. While the HSE designed and tested alternative service delivery models
and reablement programmes under the banner of the programme, the voluntary
and other statutory partners prepared, through training and engagement, to
support a whole system sustainable change. This change project was focused on
repositioning the person at the centre of service delivery while ensuring that all
six elements (Figure 1) of the Memory Matters project were both inclusive and
sustainable.
One of the main activities of the project was providing flexible person centered
support for the individual and their carers and family, through Home Help
Supports, including alternative respite, Home Care Packages and other
multidisciplinary multiagency supports. This element of the project was primarily
led by the Older Person’s Services team at the HSE, supported by other non-
profit service providers.
7. 6
2. Change Map & Outline
Using the Memory Matters Community Dementia Project as a spring board, HSE
Older Persons Services were able to review how Home Help is accessed and
supports those to remain living at home. This Genio research project was an
opportunity to track changes, discuss alternatives and explore options which
would not have been possible during the normal delivery of services.
The provision of a dedicated resource, mainly the independent project
consultant, ensured that data was collected and analysed consistently and
independently. Processes and assumptions were robustly tested, by designing,
documenting, testing and revisiting. The process was not without it’s
communications challenges as the formative evaluation took place in the lead up
to the project, as well as during the project in order to improve the process
design as it was being implemented. This continual improvement has provided a
more qualitative method of evaluation or enquiry which has delivered evidence-
based feedback and action planning that is designed to keep staff motivated.
Continual Improvement Intervention Model:
Design Process
and
Communicate
Test Design
and Map
Analysis
Feedback &
Evaluation
Action Plan
Implementation
8. 7
3. Evaluation Framework
Using the ORID Evaluation Framework (Objective, Reflective, Interpretive, and
Decisional) coupled with strategic questioning; two Focus Groups were
conducted with key service providers in January and April 2016. A number of
one-to-one interviews were also conducted and a multidisciplinary meeting was
observed in May 2016.
The ORID framework is a specific facilitation tool which enables a focused
conversation with groups of people in order to reach some point of agreement or
clarification in relation to how a project or process has worked. It was developed
by the Institute of Cultural Affairs (ICA) in Canada and involves the facilitator
asking people four levels of questions in relation to their experience of the
process. It's based on the theory that people need access to actual data and
process maps in order to deal with their emotional responses to the process in
order to undertake better analysis, decision making and sustainability as well as
up scaling.
With a project of this size and scope it was also important to establish;
What exact element is being evaluated in this report;
What is the purpose and criteria for the evaluation;
What are the key evaluation questions;
How data will be collected, analysed, and reported.
The “change process” is the part of the project which is being evaluated. This will
explore the changed service for those living with dementia in the community.
The impact of this change on the person’s quality of life and well being is not
being evaluated in this report and will be explored in the overall project review
report which is due for publication in July 2016.
9. 8
The purpose of this micro-evaluation is to develop an understanding of the
change process from the service provider’s point of view. In order to better
understand the sustainability of these changes the report will explore what has
worked well and what could have worked better. Through personal reflection the
evaluation explores what real change looks like from a sample of people who
were engaged in the project either through training or through their service
provision.
The evaluation will look to explore and understand the staffing impact on the
programme and also the programmes impact on staffing. The evaluation will also
track useful information for later use by stakeholders in service design.
10. 9
4. Focus Group Feedback
4.1 Observations and Achievements
One of the main observations identified by the second focus group in April 2016
was the “expanded criteria for accessing home help hours”. The new, more
flexible, Home Help application form enabled Public Health Nurses (PHNs) to
request re-ablement hours and longer periods of support outside the normal 30-
45 min allocated for “personal care”. The criteria were more inclusive and not
purely focused on physical functionality (i.e. Bartel Score). The Home Help
application form was now considered to be more holistic and included those
living with dementia which would previously have been excluded from standard
clinical needs assessment. The Public Health Nurses had observed the “new
process working and were now reviewing care needs in a different way”. This was
possible because they had witnessed a change in process through the project
and were now more inclined to request “non-standard interventions” for their
patients. This was viewed as the ultimate test for the sustainability of the project
although there were some concerns about continued budgets and resources.
The group felt there was a real change in relation to the attitude toward
dementia by health care professionals in general. Elements of the Memory
Matters application form were now included in the standard home help
application form. These important elements included a circle of care prompt,
areas for narrative comments about carer’s wellbeing and other non-medical
information which were invaluable when understanding the social community
connect and support alternatives.
This group also observed that Home Helps had a real “in-depth knowledge” and
understanding of the very personal likes and needs of their clients. They had
developed relationships and had a practical understanding of personal
preferences including the ability to notice subtle changes in behaviours and
11. 10
abilities. This was also confirmed by the Home Help focus group themselves in
January 2016. This was an invaluable support to PHN’s and other disciplines
when reviewing services.
Home Helps reported in their focus group that they;
“Often found the activities prescribed by the reablement care plan were too
specific for the project participants”
Home Helps said it was the ordinary everyday activities like watching TV or
walking to the shop for ice cream which were of greatest benefit to the person.
The workshop participants in April 2016 also commented on the nature of the
care plans activities provided by the Occupational Therapists and questioned if
this was indeed the most person-centered way to provide a reablement
assessment, given the advanced stage of the project participants. This
questioning demonstrates a new way of critical thinking in relation to the
“person before the process and the practice”. Further observations were
captured through additional questioning.
4.2 Insights on design and delivery
Exploring the observations the group indicated that the language on the new
Home Help Application Form actually helped to facilitated the change.
Terminology like “sitting service” and “time to task” were no longer commonly
used, and were considered by some to be the opposite of their understanding of
person-centered. However some longer serving senior team members were
observed using such terms at subsequent meetings.
Home Helps themselves felt empowered with the changes, feeling they no longer
needed to “justify” leisure activities like knitting or singing with their client.
There was considerable discussion about the content of the Occupational
Therapist’s care plan and reablement activities being influenced by a family
12. 11
member’s recollection of favoured activities and chores. Home Helps indicated
that in their experience previously enjoyed activities were not necessarily an
indication of current preferences.
The group in April explored what additionality was provided by the Occupational
Therapy assessment, considering the wait for the assessment at such an
advanced stage of the illness. This critical insight demonstrates a more flexible
approach to problem solving. The Home Helps reported that it took a number of
weeks to progress activities and a great degree of flexibility from day to day was
needed. It was acknowledged that the expert knowledge acquired by Home
Helps was built up over weeks and months and it was not always possible for
occupational therapists to gain this insight in a few hours during assessment. It
was also acknowledged that because each person experienced dementia in their
own way, each person, each response and each day was different. The degree of
flexibility required made it difficult to manage the process and ensure
“standardised responses”.
Public Health Nurses were also reconfirmed as the person who had clinical over
sight and were best positioned in relation to family, local and personal
knowledge as a result of the process. The Public Health Nurse acted as the
gatekeeper in a positive way and was fully aware of the limitations of the “quick
fix” request for additional home help hours. Public Health Nurses were best
positioned to direct services locally and explore the nature of additional support
hours and what need it filled for the person living with dementia. The role the
Primary Care Team, and in particular its multidisciplinary nature, played in
decision making was also acknowledged. It was also recommended that the
decision making should be made as close as possible to the person living in the
community. This element of the design and delivery was further explored at the
multidisciplinary team meeting in May 2016.
13. 12
While exploring the Home Help service delivery the question of quality was also
discussed. This was a difficult area to manage and was very subjective. The
provision of services to those living with dementia could be challenging for some
home helps but like all relationships some people enjoyed each other’s company
more than others. The Home Help coordinators played an important role in this
area “naturally matching” home helps with clients where possible. There was no
evidence of any “mismatch” at the Home Help focus group and all participants
displayed a desire and understanding to improve their knowledge.
Since the conclusion of the Memory Matters project recruitment a new “team
approach” is being tested in relation to Home Help services. Initially the new
model of service was questioned by family in the test area of Callan; however
this now appears to be providing variety and a social element to clients. The
service is now provided by a “Home Help Team” instead of one individual, this
also provides a great support for Home Helps. This is another example of new
processes being tested and explored since the introduction of Memory Matters.
Home Helps themselves acknowledge the invaluable support provided by their
peers in what can be viewed as a “field role”, outside the main structure of the
HSE.
Senior participants at the April focus group felt that the service and contribution
of Home Helps was often undervalued in general in the organisation. This was
viewed as a legacy cultural issue. This sentiment was echoed by the Home Helps
themselves at their session in January 2016.
The inclusion of the Home Care Package Manager at the weekly Carlow
allocation meeting ensured consistency across both counties and this was viewed
as a positive development as a result of Memory Matters. This county
comparison was further evidenced at additional meetings where processes were
challenged.
14. 13
4.3 Reflections on Barriers and Challenges
On reflection the policy within the HSE of relocating PHN’s based on services
requirements and staff shortages was a challenge, both from a communication
point of view, and also from a patient knowledge point of view. This was
discussed and was a resource issue in general and also identified by the
Occupational Therapy Manager as a growing concern for the OT Department.
The rotation of Occupational Therapists presented a particular communication
challenge at the early stages of the project, which did create a process challenge.
Reflecting back to reablement care plans it was difficult to ascertain to what
extent these plans were influenced by family members, due to the advanced
stage of the project participants. Home Helps indicated that they did not truly
reflect current personal preferences of the person living with dementia. While
this was a challenge for project participants who in the main were unable to
express their own wishes, outside the project this would be an important factor
to consider. The participants at the Home Help focus group reconfirmed this.
The group also indicated that very often family members indicated what the
persons preferred activities were historically. In practice Home Helps found that
it took time and dedication to establish suitable activities and a degree of
flexibility was also needed as the preference changed.
While the inclusion of an Occupational Therapists assessment was acknowledged
as best practice, the additionality of the report and the value it provided in
relation to project participants was debatable, considering the delay it caused in
relation to accessing services. This delay and resource issues were viewed as a
real barrier to continuing this element of the project. At the multidisciplinary
review meeting in May 2016 eighteen months was cited as the waiting time for a
P1 assessment by Occupational Therapists. In a few cases in Carlow a number of
months had passed waiting for an Occupational Therapist assessment report.
15. 14
The value of reablement plans was viewed as being of mixed benefits depending
on the person’s stage of dementia. Occupational Therapists viewed this as a
matter of clinical option. What was noted was the time needed to complete OT
Assessments, which sometimes involved a number of visits. While best practice
was always viewed as optimal, the practicality of delivering OT reports on a
standard basis would stretch the service beyond its capacity.
It was noted that weekly Allocations meetings were longer than previous, due to
the level of detailed discussions which took place in relation to clients. This was a
positive result and provided an atmosphere to explore creative and alternative
solutions. However while this outcome was very positive, it did create an added
pressure to busy schedules. Where a consensus was not found, service decisions
could be deferred, pending further investigation, this could result in further
delays.
It was also suggested that the contribution of Home Helps would be an added
advantage at Multidisciplinary Team meetings; however this would not be
possible in all cases, due to resource issues. However in complex cases the
attendance of the client’s Home Help would be very useful. Home Helps also felt
that they should be included in complex case discussions, as they sometimes
visited a home 3-4 times per day and were best positioned to comment on
needs. This was especially the case where there was no family available locally to
advocate for the person.
The group also acknowledged the challenges providing transport in rural areas in
relation to day care. The stigma that was attached to Day Care by many older
people themselves was also explored as a barrier to service delivery. While the
preferred and most economic option for HSE health care professional was day
care, which ensured access to social activities, good meals and company, the
group recognised the individual’s right to decline Day Centre service.
16. 15
This social perception of Day Care was explored and the group felt that in general
few day centres provided creative “club style” settings, which were attractive
and inviting for older people. The group advised older people themselves felt day
centres were the “last resort”, akin to “psychiatric day wards” and they were
neither inspired nor encouraged to attend.
4.4 Decisions on Actions and Sustainability
A new proposed “check list” will be introduced to support earlier intervention
especially in the area of community supports like Day Care and Meals on Wheels.
This will complement and act as a precursor to the formal Home Help application
submission for Public Health Nurses. This is currently being finalised by the
Assistant Director of Public Health Nursing for introduction.
The role of the Public Health Nurse as “case expert” needs to be reinforced. The
Public Health Nurse is the person who formally requisitions services and as such
all reports, OT assessments and any other supplementary information must be
returned to the Public Health Nurse for presentation to the weekly Allocations
Meeting. The OT assessment should not be submitted directly to the Allocations
Meeting. This will enable the Public Health Nurse to review and explore
occupational therapists recommendations. The Public Health Nurse has a more
holistic view of local services including other community support services which
may support reablement.
To support the number of Public Health Nurses on extended leave, a back to
work interview will be conducted to update on new services and also brief
returning Public Health Nurses about changes in their area. Community
Registered Nurses should also be provided with this type of induction process.
This is will be explored also.
17. 16
Clarification is needed in relation to specific hours being requested following
occupational therapist assessment. While the groups acknowledged the
importance of the carer’s wellbeing, it is not always possible to service “very
specific” requests at peak times i.e. Knitting from 9-11a.m. Both focus groups felt
that the activities could be more effectively delivered without a specific time slot
request.
Exploring these types of requests with Occupational Therapists (OT) they did in
fact confirm that the time of day could be a critical element to the reablement
delivery, dependent on energy levels, medication and other factors.
It was discussed that in future the OT will be requested to liaise with Public
Health Nurse in this regard and the assessments provided should not specific day
and duration of service, unless discussed with Public Health Nurse.
Communication is key in this area and while it worked very well in some areas
where office spaces were shared, in some areas communication was poor, across
disciplines.
Moving forward both groups felt the inclusion of client’s Home Help at complex
case planning meetings would be very beneficial. Understanding the role of
Home Helps and valuing their daily engagement could be better acknowledged
by the HSE in general. This is an ongoing cultural development challenge.
18. 17
6. “Vox Pops” – What has changed?
6. Conclusions and Learnings
The focus groups and subsequent team meetings provided very useful insights
into what had worked well and what remained a challenge for the post Memory
Matter changed community care pathway. The overarching challenge was
communication across disciplines, this sometimes still lead to misunderstanding
about eligibility for services, and where lines of responsibilities were draw in
relation to accessing services for client.
In a multidisciplinary team meeting on 9th
May 2016 it was recognised that the
decision making process was more effective and efficient for everyone including
the client when made as close to the client as possible.
“I am much more aware of
carer’s stress”,
Assistant Director of Public
Health Nursing
“I feel I have more
empathy with family
members now when I’m
on the phone”.
Home Help Coordinator
“In the future I feel we should
include Home Help’s in
complex care planning
meetings especially for those
living with dementia”
Assistant Director of Public
Health Nursing
“I’m much more aware of pain
now which I understand can be
undiagnosed, I’m changing the
way I approach my
assessment”.
Occupational Therapist
“I understand now how important team
work is, especially including family”
Home Help
5.
19. 18
This would ensure that local knowledge and a more holistic view of services
needed was taken. There was also some discussion in relation to the role of
primary care teams and the involvement of GP in the decision making discussion.
While the ideal situation would involve a client GP this was not always possible.
Some primary care teams were “over medicalised”, were doctors outnumbering
other care professionals at meetings.
The group indicated that the “capturing of a baseline” was needed to map
existing services; this was seen as a critical element of monitoring changes. This
would ensure that all team members truly understand the service in practice and
their role. It was also the only way to review process and progress. This had been
a very beneficial element of the Memory Matters project.
This mapping exercise has also been used in May 2016 to track and evaluate the
Telecare element of the Memory Matters project as it is mainstreamed into
services being provided by the Occupational Therapists post Memory Matters.
Earlier intervention was also identified as a critical component in order for
services to provide a more person-centered service. The Memory Matters
project participations were at an advanced stage of their illness and the
expectations of the project were not always deliverable. To this end an
assessment by an Occupational Therapist at the Memory Clinic and/or Primary
Care level would ensure long term maximization of services. This earlier
intervention of the occupational therapy services would provide real
additionality to services in the future. When the application for additional home
help hours is made, this is too late for occupational therapy support in many
cases.
The service which was previously more reactive has changed and is now more
responsive. Group participants felt they had a greater understanding and
empathy with families when communicating with them in relation to services.
20. 19
Both group participants felt there were a genuine move towards and
understanding of promoting and valuing independence at all levels. This
extended to independence of choice in services and was not simple restricted to
independent living.
Some Occupational Therapists also expressed the view that it was carer’s stress
which necessitates additional block hours of care, which would not necessarily
benefit the client’s reablement due to the stage of their illness or capacity.
Occupational Therapists also expressed the view that activities of daily living
were in fact meaningful activities especially for those living with dementia
Information and education is an important element of any change. Reviewing
and reinforcing is critical. Leading by example can continue to endorse real
change which is a long term investment by everyone concerned. The group felt
that language, attitudes and societal changes are supportive of the cultural
change which is current well advanced in the Health Service Executive in Carlow
and Kilkenny.
Communication was good but was sometimes dependent on relationships and
geographical locations of teams. This could be improved across all areas of
operation.
21. 20
Appendix 1 Focus Group Participants
Venue
Kilkenny Age Friendly County Office 16th
January 2016 2-5pm
Kilkenny Age Friendly County Office 26th
April 2016 10-1pm
Consultation Room, HSE Community Services, James Green
Kilkenny, 9th
May 2016.
Facilitator Debra O Neill, Project Consultant, Memory Matters
Mary Ahern Home Help Coordinator
Jackie Bradley Home Help Coordinator
Paula Brophy Home Care package Manager
Kathleen Bulman Home Help Coordinator
Hazel Byrne HSE Home Help
Siobhan Coady Home Help Coordinator
Katherine Connolly Home Help Coordinator
Majella Cunningham Assist. Director Public Health Nursing
Theresa Griffin Assist. Director Public Health Nursing
Mary Israel HSE Home Help
Teresa Kinsella HSE Home Help
Rita Langton HSE Home Help
Bernadette Lawrence HSE Home Help
Patricia McEvoy Manager Older Person's Services
Michelle Murphy HSE Home Help
Marie O Sullivan HSE Home Help
Sandra Watkins HSE Home Help
Tracey Woods HSE Home Help
22. New Service Delivery Model
Home Help Services Application
Step 1
Public Health Nurse identifies
service need
Request Multi-disciplinary
assessment as necessary
(via GP and other)
• Memory Clinic
• Geriatric Clinic
• POLL Referral
• SAL Referral
• Physiotherapy Referral
Step 3
Completed Home Help Service
Application forwarded to OT by
PHN requesting assessment.
Step 5
Home Help Allocations Meeting
Consider all information and assign services
if approved.
Troubleshooting
1. What is working?
2. What is not working?
3. What can you/others do?
Can the concern been resolved?
Step 4a URGENT CASE
Home Help Application Form submitted to
Allocations meeting for consideration for
urgent services.
When OT report returned to PHN outlining
functional difficulties and care plan the PHN
makes further request for services if
required.
Is care plan
providing
expected
results?
NO YES
Step 2
Complete Home Help Application
PHN completes Home Help Service
Application form
Step 4b NON URGENT CASE
OT Assessment Conducted
Home Help Application form submitted to
Allocations meeting by PHN with OT report
which outlines the functional difficulties
and plan of care.
Step6
Public Health Nurse continues to review care
plan and service.
Version 7/16th October 2015
Appendix 2: Post Project Care Pathway
21
23. 21
Appendix 2: New Home Help Application Form (April 2016)
Home
Assessed for
the first
time.
Carers difficulties
also documented
on application,
since Memory
Matters
Including all
formal support
provides a clearer
picture of unmet
needs
Appendix 3:
22
24. 22
Change in pattern
of behaviour
noted and for the
first time
Responsive
Behaviours.
Circle of Social
Care prompts
now part of
Home Help
Application
23
26. 24
Acknowledgements
Special thanks are due to all those who have participated in the focus groups
and especially to Patricia McEvoy who has supported every element of the
Memory Matters project since its launch in 2014. Majella Cunningham,
Assistant Director of Public Health Nursing, has provided advice, direction
and support to the change process, liaising with her colleagues and providing
invaluable feedback as pathways and process were designed, tested and
redesigned.
The Home Help coordinators and Home Care Package Manager have,
without exception, continued to support the essence of the Memory
Matters project throughout its design, testing and implementation stage,
and continue to champion the learning on a daily basis post project.
To the Home Helps who contributed to this evaluation, thank you sincerely,
for your initial advice in our first focus group in October 2014, and for your
continued feedback. Your commitment was invaluable at the early stage of
the project design and your feedback is greatly appreciated.
A big thank you to the Teresa Griffin, Assistant Director of Public Health
Nurses, all the Public Health Nurses, Occupational Therapists, administrative
support staff and also to all those who are involved in the delivery of
services.
About the Author
Debra O’ Neill is an independent consultant and change practitioner who has
provided services to the Memory Matters project since September 2014. In
her capacity as consultant she has acted as animator, facilitator, change
practitioner and researcher. This report is one of a number of reports due to
be published in 2016. (www.linkage.ie)
25
