Aphasia and Technology: The GReAT Project Abi Roper and Jane Marshall On behalf of the GReAT Project Team Division of Language and Communication ScienceDepartment of Human Computer Interaction Design City University London
Presentation Outline•The Project•Designing and Refining a Computer GestureTherapy - Gest•Gest Demonstration•Delivering a Computer Therapy•Gest Pilot Study•Preliminary Outcomes
Project Aims• To develop an affordable, computer-based technology that can be used in therapy at home to help people with severe aphasia to gesture.• To establish how to design effective/engaging interactions for people with aphasia.• To evaluate the efficacy of the technology within a pilot therapy study
Project Structure• Phase 1: Designing a prototype gesture therapy using participatory design methods.• Phase 2: Testing and piloting the prototype
Project TeamHuman Computer Interaction Design & Language and Communication Science Stephanie Wilson Sam Muscroft Julia Galliers Jane Marshall Naomi Cocks Tim Pring Abi Roper
Phase 1• Designing a prototype gesture therapy using participatory design methods.
ConsultantsJustine Everson Gerald Hartup Carol Watson Philip Pepper Emma Buswell
Consultants•Role: to test and feedback about relevant technology.•Person Specifications: –Expressive aphasia language difficulties. –Able to attend university once or twice a month for participatory design sessions.•Recruited through in house clinic and through links withthe Stroke Association Communication SupportCo-ordinators.•Employed by City University London as Casual Staffmembers.
Methods: Participatory Design Sessions•Participatory design – engaging end users in designprocess•Sessions explored offline gesture therapy, computergesture recognition, interaction within 3D worlds andcomputer interfaces.•Consultants took part in 9 sessions each•Project team involved in each session - 1 HCID Researcher - 1 HCID Developer - 1 Speech and Language Therapist Researcher - 2 or 3 Consultants
Session Structure1. Introduction to scheduled activities2. Round table gesture activity3. Demonstration of Technology4. Trial use of technology by one consultant - followed by interview at computer5. Tea break6. Trial use of technology by remaining consultant(s)
Participatory Design• Watch a video excerpt of the design process in action on the next slide
What did we learn from the Sessions?1. Consistency2. Simplicity3. Pace4. Reliability5. Rewards6. Individual Differences7. Potential of ‘gaming’.
Demonstration•Watch a video Demo of theGest prototype in action onthe next slide
Using the Therapy at home•How does this work at home?Key differences between lab andhome –User practisingindependently, User intending topractise daily. User practising innon-lab conditions.
Things to consider when setting up•Lighting conditions•Safety and permanence(negotiate!)•User comfort and access
Things to consider when training•Develop the user’s confidence in the system.(Be confident yourself)Demonstrate:1. Allow user to observe entirely2. Allow user to observe and operate interaction buttons3. Allow user to operate alone but with support as needed (confidence)
Things to consider when training•Reinforce how to switch the computeron and off several times.•Make an appointment to come back inone week to review.•At review appointment, observe and re-train difficult procedures.
Questions• Will practice with Gest improve participants’ production of gestures &/or spoken words?• Will improvements be specific to items that feature in the programme?• Will gains occur when Gest is used without ongoing therapist support?• Will gains be maintained after Gest is withdrawn?• What are participants’ views about Gest?• What are carers’ views about Gest? (where relevant)• Is Gest easy and enjoyable to use?
Participants• 10 people with severe aphasia – Consent to take part – Fluent pre-stroke users of English – Naming score <20% – Able to recognise pictures – No known dementia or other cognitive impairment
Consent Screening Phase 1 with weekly Tests (1) visits from therapist 3 Weeks Practice Phase 2 with no weekly Tests (2) visits from therapist 3 Weeks Practice Tests (3) 3 weeks no tool Total time commitment: about 14 weeks Tests (4)
Practice Phases• Each last 3 weeks• Each practise 15 gestures with the tool• Phase 1: Weekly visits from therapist• Phase 2: Initial but no weekly visits
Tests• 60 items – Gesture from picture – Name from picture What is the name of this? How would you gesture Items: this? 30 practised with Gest 15 familiarised only 15 controls
Scoring Gestures• Gesture tests are filmed• 4 Scoring videos created• Each video contains 60 gestures in random order: – 15 from test 1 – 15 from test 2 – 15 from test 3 – 15 from test 4
Scoring Gestures• Scores – Recognition Score – Rating Score• Scorers are ‘blind’ to the time of assessment
Usability Evaluations• Observe participants using the tool• Interview participants• Interview carers (if relevant) – When technology is installed – After each practice phase
Usage Logs• Record – Number of sessions – Length of sessions – Levels of programme accessed – Number of gestures recognised
Mean Usage: 7 Participants 60 50 40 30 20 10 0 Days No of Time used Time per available sessions (hrs) session (mins)
Individual Usage: 3 participants80706050403020100 Days available No of Time used Time per sessions (hrs) session (mins)
Usage x Recognition80 30070 2506050 20040 1503020 10010 0 50 Days No of Time used Time per 0 available sessions (hrs) session Recognition score (mins)
Mean Usage over Phases28 40027 35026 300 25025 Supported 20024 Independent 15023 10022 5021 0 No of sessions Time spent (mins)
Usage: Levels• Three participants use level 1 more than 2 & 3• Two participants use all 3 levels and rate them equally highly• Two participants rate levels 2 & 3 more highly than 1• Possibly contingent on navigation abilities
Usage Observations: Challenges• Set up – Lighting – Positioning (e.g. wheelchairs) – Security• Glove – Putting glove on the wrong hand – Using the peg board (although often not necessary)
Usage Observations: Challenges• Starting and stopping – Pressing key board buttons before menu has appeared – Not always pressing ‘off’ at end of session
Usage Observations: Challenges• Navigation – Variable use of OK, forward, back & menu buttons – Variable navigation between levels – Some unprincipled button pushing Speed and competence may relate to prior computer usage
Usage Observations: Challenges• Gesture production – Knowing when to gesture; waiting for 321 ping – Knowing when the gesture has been recognised – Variable use of cues; e.g. some adjust handshape in response to glove image others do not
Usage Observations: Enjoyment• All signal high enjoyment levels – Thumbs up sign – Drawn smiley face• Positive reactions to level 2 – Game format – Narrative context – Environments
Usage Observations: Enjoyment• Positive reactions to level 3 – Humour (spider, dentures) – Stroke survivors as actors – Presence of children
Other Observations• Some target spoken words produced during Gest use• Spontaneous uses of practised gestures (‘umbrella’ gestured when participant noticed that it was raining outside; ‘child’ gesture when talking about grandchild)
Independence of Use• ‘She uses it all on her own, I don’t know how to operate it’• The first session I stayed with L, after that I’ve helped only if she’s found something particularly frustrating’• All comment that the participant initiated use of Gest
Enjoyment• All say that the participant enjoyed Gest• ‘he likes it when they clapped’• ‘some of the gestures are particularly fitting and she enjoyed rainbow’
Views about Technology• ‘I was a technophobe and when they said ‘computer’ I thought it was going to cause problems. I thought I wouldn’t understand and he wouldn’t understand it. But it’s so ‘easy’
Reservations• Carry over to real life (1 carer):• ‘while she works on it here (points to computer) it doesn’t necessarily translate’• She wanted a hankie last night and didn’t make a gesture’
Conclusions• Gest was created through participative design involving people with aphasia – It offers 6 packages of hierarchical practice on 30 gestures – It is accessible even to people with severe strokes – It can be used successfully in diverse home settings – It allows for flexible, self directed practice and is typically intensively used – It is enjoyable to use, with no reports of increased ‘carer burden’
Conclusions• But we do not know if – Gest improves gesture production – Gest improves spoken naming – Effects generalise to unpractised targets – Effects are maintained• The results of the pilot study will give us answers to these questions
AcknowledgementsThe Research Councils UK Digital Economy Programme The Stroke Association Consultants and their families Participants and their families Thank You GReAT@city.ac.uk www.soi.city.ac.uk/great