Lessons Learnt from
   the Evaluation of
    Reminding Aids
C Nugent, S O’Neill, M Donnelly, L Galway, M Beattie, S McClean, B Scotney
Dementia
•   Dementia is a progressive, disabling, chronic
    disease.

•   28 million people suffer from dementia worldwide.
•   By 2030 the numbers are expected to double and
    by 2050 they are expected to triple (Prince 2010).
•   The costs of care associated with those suffering
    from dementia were estimated at £400bn in 2010.

•   Typical symptoms include impairments of
    memory, thought, perception, speech and
    reasoning.

•   It is also common to witness changes in personality
    and behaviour in addition to exhibit symptoms of
    depression, apathy and aggression.
Medication
                      Management       Technology Aids




                              Location tracking
                                                      Robot assisted
                                                         activity



Activity Assistance
                 Electronic
               Memory Aids
                                                  Item locators
Mobile Phone Video Reminding

 Support persons with mild Dementia
    to improve independence and QoL.
    to reduce caregiver burden.

 Focused on using a truly ‘everyday’ technology:
    Use a familiar face to offer the reminders
    Provide a ‘virtual Caregiver’ throughout the day.
Evaluation
            Methodology




                                                   Range (Days)
                  # Participants   Days of Usage

   Initial             21               79             2-5
    ABA                9                18              2
Longitudinal I         6               205            6-61

Longitudinal II         4              101            12-34

    Total              40              403
Evolution of Reminder
       Interface




               EVOLUTION
Evolution of Carer’s interface




            EVOLUTION
User Recruitment
•   Participants were recruited by clinical research staff at the Memory
    Clinic, Belfast City Hospital.

•   Inclusion criteria:
      – recorded as having mild dementia (MMSE >18),
      – who lived alone and/or with a wiling carer able to record video
         messages.

•   Both patient and carer were involved in the informed consent
    procedure.

•   Baseline measurements of cognition were made based on MMSE and
    the Disability Assessment for Dementia was used for functional ability.

•   Pre and post evaluation questionnaires were administered by a research
    nurse via face-to-face interviews.
User Engagement


Patient   Age   Gender Length of   MMSE   DAD   Carer    Carer Relation         Living
  ID                   condition                 Age    Gender   ship       Arrangement


 C11      75    Female   2 years    26    78     45     Female   Daughter    Together
                                                                  in law

 C12      73    Female   2 years    18    68     77      Male    Spouse      Together

 C14      61     Male    4 years    27    60     58     Female   Spouse      Together

 C15      71    Female   2 years    25    63     73      Male    Spouse      Together


   Details of participants and carers engaged in evaluations
Identified Needs

                                                                  Mobile       Other
                   Carer       Carer     Post-it or   Diary or                           Don't
                                                                 phone or   Technology
                 personally   calls to   reminder     Calendar                           need
                                                                    TV        (please
                  reminds     remind       note        entry                              it
                                                                 reminder     specify)
   Meals                                                                                  4
    Drink                                                                                 4

Entertainment        2                       1           1                                1

Appointments         2                       2           3

 Medication          2                       1                                            1
 Housework                                                                                4
 Make calls          2                       2           2
   Other


              Summary of identified user needs based on pre-evaluation
                                  questionnaires.
In-situ measurements



        1




        0




        1




        0
Results
                                                            The majority of reminders were
 Numbers of missed
                                                             acknowledged the first time it
reminders were low.
                                                                    was received


  Participant   Duration     No. of       No. of      No. of       No. of      No. of       No. of
                           reminders    reminders   reminders    reminders   reminders    reminders
                            recorded      missed      ack. 1st    ack. 2nd     ack. 3rd     ack. 4th
                               and                   attempt      attempt     attempt      attempt
                            delivered
     C11        34 days       145           8          108          17           7            5
     C12        12 days        /            /           /            /           /            /
     C14        27 days       22            0          19           2            1            0
     C15        28 days       49           10          34           2            0            3


                Analysis of Results following evaluations
Analysis of post evaluation
                  questionnaires
•    Overall the feedback from the participants and carers (based on set of 30
     questions post evaluation) was positive:
    – It supported reminders for their necessary tasks.
    – The phone was easy to use.
    – The phone interface and button size was very usable.

•    Only 1 participant wore the phone around their neck others felt it was too
     heavy.

•    There was reported embarrassment about the reminder being delivered in
     public.

•    When asked about WTP, all dyads agreed that they would pay for the
     service.
Analysis of post evaluation
              questionnaires
•   Negative Comments

    “It didn't do anything, I didn't hear the messages.”

    “It was too much she couldn't manage it and got frustrated, she kept
    trying to listen like a phone to her ear” (Carer)

•   Positive Comments

    “Reminders very good especially re medication in middle of day.”

    “useful prompt, took time to get used to it, only getting real hang of it
    now”
Challenges
• Internet access (Broadband / Dongle)
• Personalisation
  – Level of detail shown on screen and on
    documentation
  – Mode of reminder delivery (text, video)
• Data Management
  – Heartbeat
  – Activity / Acknowledgments
  – Battery level
• Compliance
  – Compliance Assessment (Sensorised Environment)
  – Feedback Management
Conclusions
• Although data was limited with 4 participants, a
  useful insight has been gained into the overall
  utility of the system. (Long duration and recording
  of video messages.)
• Varying levels of personalisation should be offered.
• Value the importance of technical framework to
  support deployments.
• Support by carer is correlated with success of using
  the device.
• Consider factors affecting technology adoption.
Acknowledgements
•   Kyle Boyd
•   David Craig
•   Dorian Dixon
•   Mark P. Donnelly
•   Leo Galway
•   Sarah Mason
•   Sally McClean
•   Brian Meenan
•   Chris D. Nugent
•   Sonja A. O’Neill
•   Guido Parente
•   Bryan Scotney

Lessons Learnt From The Evaluation Of Reminding Aids - Chris Nugent

  • 1.
    Lessons Learnt from the Evaluation of Reminding Aids C Nugent, S O’Neill, M Donnelly, L Galway, M Beattie, S McClean, B Scotney
  • 2.
    Dementia • Dementia is a progressive, disabling, chronic disease. • 28 million people suffer from dementia worldwide. • By 2030 the numbers are expected to double and by 2050 they are expected to triple (Prince 2010). • The costs of care associated with those suffering from dementia were estimated at £400bn in 2010. • Typical symptoms include impairments of memory, thought, perception, speech and reasoning. • It is also common to witness changes in personality and behaviour in addition to exhibit symptoms of depression, apathy and aggression.
  • 3.
    Medication Management Technology Aids Location tracking Robot assisted activity Activity Assistance Electronic Memory Aids Item locators
  • 4.
    Mobile Phone VideoReminding  Support persons with mild Dementia to improve independence and QoL. to reduce caregiver burden.  Focused on using a truly ‘everyday’ technology: Use a familiar face to offer the reminders Provide a ‘virtual Caregiver’ throughout the day.
  • 5.
    Evaluation Methodology Range (Days) # Participants Days of Usage Initial 21 79 2-5 ABA 9 18 2 Longitudinal I 6 205 6-61 Longitudinal II 4 101 12-34 Total 40 403
  • 6.
    Evolution of Reminder Interface EVOLUTION
  • 7.
    Evolution of Carer’sinterface EVOLUTION
  • 8.
    User Recruitment • Participants were recruited by clinical research staff at the Memory Clinic, Belfast City Hospital. • Inclusion criteria: – recorded as having mild dementia (MMSE >18), – who lived alone and/or with a wiling carer able to record video messages. • Both patient and carer were involved in the informed consent procedure. • Baseline measurements of cognition were made based on MMSE and the Disability Assessment for Dementia was used for functional ability. • Pre and post evaluation questionnaires were administered by a research nurse via face-to-face interviews.
  • 9.
    User Engagement Patient Age Gender Length of MMSE DAD Carer Carer Relation Living ID condition Age Gender ship Arrangement C11 75 Female 2 years 26 78 45 Female Daughter Together in law C12 73 Female 2 years 18 68 77 Male Spouse Together C14 61 Male 4 years 27 60 58 Female Spouse Together C15 71 Female 2 years 25 63 73 Male Spouse Together Details of participants and carers engaged in evaluations
  • 10.
    Identified Needs Mobile Other Carer Carer Post-it or Diary or Don't phone or Technology personally calls to reminder Calendar need TV (please reminds remind note entry it reminder specify) Meals 4 Drink 4 Entertainment 2 1 1 1 Appointments 2 2 3 Medication 2 1 1 Housework 4 Make calls 2 2 2 Other Summary of identified user needs based on pre-evaluation questionnaires.
  • 11.
  • 12.
    Results The majority of reminders were Numbers of missed acknowledged the first time it reminders were low. was received Participant Duration No. of No. of No. of No. of No. of No. of reminders reminders reminders reminders reminders reminders recorded missed ack. 1st ack. 2nd ack. 3rd ack. 4th and attempt attempt attempt attempt delivered C11 34 days 145 8 108 17 7 5 C12 12 days / / / / / / C14 27 days 22 0 19 2 1 0 C15 28 days 49 10 34 2 0 3 Analysis of Results following evaluations
  • 13.
    Analysis of postevaluation questionnaires • Overall the feedback from the participants and carers (based on set of 30 questions post evaluation) was positive: – It supported reminders for their necessary tasks. – The phone was easy to use. – The phone interface and button size was very usable. • Only 1 participant wore the phone around their neck others felt it was too heavy. • There was reported embarrassment about the reminder being delivered in public. • When asked about WTP, all dyads agreed that they would pay for the service.
  • 14.
    Analysis of postevaluation questionnaires • Negative Comments “It didn't do anything, I didn't hear the messages.” “It was too much she couldn't manage it and got frustrated, she kept trying to listen like a phone to her ear” (Carer) • Positive Comments “Reminders very good especially re medication in middle of day.” “useful prompt, took time to get used to it, only getting real hang of it now”
  • 15.
    Challenges • Internet access(Broadband / Dongle) • Personalisation – Level of detail shown on screen and on documentation – Mode of reminder delivery (text, video) • Data Management – Heartbeat – Activity / Acknowledgments – Battery level • Compliance – Compliance Assessment (Sensorised Environment) – Feedback Management
  • 16.
    Conclusions • Although datawas limited with 4 participants, a useful insight has been gained into the overall utility of the system. (Long duration and recording of video messages.) • Varying levels of personalisation should be offered. • Value the importance of technical framework to support deployments. • Support by carer is correlated with success of using the device. • Consider factors affecting technology adoption.
  • 17.
    Acknowledgements • Kyle Boyd • David Craig • Dorian Dixon • Mark P. Donnelly • Leo Galway • Sarah Mason • Sally McClean • Brian Meenan • Chris D. Nugent • Sonja A. O’Neill • Guido Parente • Bryan Scotney