People with Communication Disability Striving, Thriving,.pptx
•
0 likes•838 views
- Professor Bronwyn Hemsley gave a keynote presentation on how people with communication disabilities are striving, thriving and surviving as technology advances. - She discussed various technologies including smart speakers, augmented and virtual reality, and 3D food printing that have potential applications for rehabilitation but also have usability and accessibility issues. - Hemsley argued for an inclusive biopsychosocial-tech model and virtuous tornado approach to design that recognizes human diversity and uses co-design processes to develop technologies that can truly benefit those with communication disabilities.
Recommended
Recommended
More Related Content
Similar to People with Communication Disability Striving, Thriving,.pptx
Similar to People with Communication Disability Striving, Thriving,.pptx (20)
More from Bronwyn Hemsley
More from Bronwyn Hemsley (20)
Recently uploaded
Recently uploaded (20)
People with Communication Disability Striving, Thriving,.pptx
- 1. People with Communication Disability Striving, Thriving, and Surviving as Technology Advances Professor Bronwyn Hemsley Bronwyn.Hemsley@uts.edu.au @BronwynHemsley Keynote at Australian Society for the Study of Brain Impairment Conference “Getting on with the Business of Living” 2022
- 2. The Girra Maa wattle seed artwork signifies our unity and diversity. Produced by Wiradjuri artist Nathan Peckham.
- 3. Declaration of Interests • Funding - National Health and Medical Research Council for purchase through Equipment Grants for the Hololens 2 and the Foodini 3D Food printer • Funding - Australian Research Council for funding on the use of 3D food printers for people with swallowing disability • Honorarium from ASSBI for my presentations at ASSBI 2022 • Funding - University of Technology Sydney for research using VR/AR at UTS
- 4. Theoretical Model for Using Technology in Cognitive- Communication Rehabilitation with People after TBI • Drivers of use • Individual and environmental factors, impairment, training and supports • Technological factors Brunner M, Hemsley B, Togher L, Palmer S. Technology and its role in rehabilitation for people with cognitive-communication disability following a traumatic brain injury (TBI). Brain Inj. 2017;31(8):1028-1043.
- 5. Devices not designed with people with brain impairment in mind: ‘Rehab Repurposing’ ‘off-the-shelf’ not dedicated design
- 6. Do you use one of these? https://redcap.link/Smart_Speakers Google Nest Amazon Echo Cortana Siri Apple Home Pod
- 7. Smart speakers & communication disability Bérubé et al (2021) Chronic and Mental Health Conditions 12 studies (only 1 relevant) Greuter, Balandin & Watson (2019) 9 adults with ID For learning social skills through interactive storytelling Google Home Allen, Shane & Schlosser (2017) The Echo as a speaker- independent speech recognition device (autism) Amazon Echo + iPad Yu, Shane, Schlosser, O’Brien, Allen, Abramson & Flynn (2018) SLPs using Echo Show to deliver visual supports. 5 SLPs Amazon Echo Show Smith, Sumner, Hedge & Powell (2021) Speech intelligibility in adults with ID 21 adults with ID Amazon Echo Google Home Smith, Sumner, Hedge Powell (2020) ID agency and wellbeing 22 adults with ID Amazon Echo Google Home
- 8. Amazon Echo and Disability Repurposed for accessibility, increasing independence in people with disability. “Most humans can not understand me, but Alexia can,” while R318 wrote, “Ordinarily voice programs can’t understand what I am saying due to my speech impairment, but Alexa responds to my commands without fail.” … Alexa has forced her to slow down and enunciate clearly. … something her speech therapists have been working on with her for years. Alexa has gotten these results from her in a few months.” (R185) Pradhan, Mehta & Findlater (2018). “Accessibility came by accident”: … Association for Computing Machinery, doi.org/10.1145/3173574.3174033 Research: 346 Amazon reviews of the Amazon Echo by or about people with cognitive, physical, or sensory disability, communication disability Parkinson’s speech: Duffy, O., Synnot, J., McNaney, R., Zambrano, P.B., & Kernohan, W. G (2021). Attitudes towards the use of voice assisted technologies among people with Parkinson Disease: Findings from a web-based survey. JMIR Rehabilitation Assistive Technology, Jan-Mar 8(1): e23006. doi.10.2196/23006
- 9. Home …. Hospital? …. Rehab? …. https://medcitynews.com/2021/10/amazon-brings-alexa- devices-to-hospitals-assisted-living/
- 10. Do you use one of these?
- 12. Aida, Chau & Dunn (2018) Immersive virtual reality in traumatic brain injury rehabilitation: A literature review. 11 studies - TBI Bryant, Brunner & Hemsley (2020) A review of virtual reality technologies in the field of communication disability. 17 studies – any population ABI or developmental conditions Brassel, Power, Campbell, Brunner & Togher (2021) Recommendations for the design and implementation of virtual reality for acquired brain injury rehabilitation: Systematic review Part 1 – 14 studies ABI Part 2 - 5 studies TBI Bailey, Bryant & Hemsley (2020) Systematic Review VR and AR Children with Developmental 69 studies (only 1 immersive VR with headset and 5 AR studies) Montoya-Rodriguez et al (2022) Virtual reality and augmented reality as strategies for teaching social skills to individuals with intellectual disability: A systematic review 6 studies 5 VR/AR reviews guide future co-design research
- 13. Views of experts on applications of VR in communication rehabilitation Bryant, L., Sedlarevic N., Stubbs, P., Bailey, B., Nguyen V., Bluff, A., Barnett D., Estela M., Hayes, C., Jacobs C., Kneebone, I., Lucas, C., Mehta, P., Power E., Hemsley, B., (In press ). Collaborative co-design and evaluation of an immersive virtual reality application prototype for communication rehabilitation (DISCOVR Prototype). Disability and Rehabilitation: Assistive Technology. Cognitive fatigue after TBI and co-design of VR Nunnerley, J., King, M., Hodge, K., Hopkins, P., Stockwell, R., Thorne, N., Snell, D., & Gozdzikowska, K. (In press). Co-design of a therapeutic virtual reality tool to increase awareness and self-management of cognitive fatigue after traumatic brain injury. Disability and Rehabilitation: Assistive Technology.
- 14. 3D printed food shapes … useful?
- 15. ? Evidence: 3D Food Printing & Dysphagia Hemsley, Palmer, Kouzani, Adams, & Balandin (2019) Review Informing the Design of 3D Food Printing for People with Swallowing Disorders: Constructive, Conceptual, and Empirical Problems 16 papers (constructive problems) Hemsley, Dann, Balandin et al., (2020) A device looking for a purpose and user- centred co-design: 3D food printing not yet delivering on expectations of benefit for people with swallowing disability. International Conference on Space Food Engineering and 3D Food Printing Smith, Bryant & Hemsley (In press) A review of the impact of food design on the mealtimes of people with swallowing disability who require texture-modified food. International Journal of Food Design. 35 studies Smith, Bryant & Hemsley (In press). Allied health professionals’ views on the use of 3D food printing to improve the mealtime quality of life for people with dysphagia: Impact, cost, practicality, and potential. American Journal of Speech-Language Pathology.
- 16. There are (several) digital autonomy and usability issues with these devices Looks like we’ll put up with the digital autonomy and usability issues ??? Will we
- 17. Is it time for the bio-psycho-social-tech model? Scherer, M. J. (2020) It is time for the biopsychosocialtech model. Disability and Rehabilitation: Assistive Technology, Issue 15(4), p.363-364. “The four interrelated constructs (biological, psychological, social and technological) and how they change over time and reconstitute one another need to be considered as a unit.”
- 18. The Virtuous Tornado https://creativecommons.org/licenses/by-nc/4.0/ https://medium.com/@jutta.trevira/the-three-dimensions-of-inclusive-design-part-two-7cacd12b79f1 Jutta Trevinarus Director, Inclusive Design Research Centre at OCAD University “1. Recognize, respect, and design with human uniqueness and variability 2. Use inclusive, open & transparent processes, and co-design with people who have a diversity of perspectives, including people that can’t use or have difficulty using the current designs 3. Realize that you are designing in a complex adaptive system”
- 19. Better speech recognition / Inclusive speech recognition Digital autonomy in disability and use of smart devices Improved design of the hardware & software – headsets and controls, more attention to aesthetics Inclusive design: hardware and software, for people with physical, sensory or cognitive disability Smarter Artificial Intelligence being applied to more activities, more reliable, digital autonomy Dedicated design for rehabilitation (not only repurposing) What will move inclusion forward?
- 20. Conclusion • Incremental innovations promise much but deliver little in terms of usability or accessibility by people with disability. • Inclusive design principles should guide strategies for increasing the application and usability of innovative technologies to improve participation and inclusion for people with communication disability. • Collaboration across disciplines, diverse teams, co-design.
- 21. To interact … @BronwynHemsley on Twitter Bronwyn.Hemsley@uts.edu.au
Editor's Notes
- Good morning, and thank you to Michelle Kelly and Janet Wagland, Co Conference Convenors and Program Chairs of the ASSBI conference, it is a great honour to be invited to speak about digital inclusion and technology design. Thanks also to the Chair of this session Prof Leanne Togher.
- I acknowledge the Gadigal People of the Eora Nation and their elders past present and emerging as true custodians of knowledge and of the land sea and air surrounding UTS, from where I am presenting today. This Girra Maa (UTS Indigenous Health Discipline) wattle seed artwork by Nathan Peckahm signifies our unity and diversity.
- I’d also like to acknowledge the funding bodies supporting our research and my co-investigator on 3D food printing Prof Susan Balandin and Dr Stephen Dann who is at ANU; also Dr Lucy Bryant and Rebecca Smith whose research at UTS has informed this presentation.
- I also acknowledge the prior research which has informed this presentation, and research colleagues across many teams.
- Today you will hear about three incrementally innovative digital technologies. Think about applying the Theoretical model for using technology in cognitive-communication rehabilitation with people after TBI, a paper in Dr Liss Brunner’s PhD, reviewing 95 studies on all types of communication technologies. Think about the DRIVERS – the motivations and purposes of using the technologies; the INDIVIDUAL and ENVIRONMENTAL factors, IMPAIRMENTS and training SUPPORTS that you know would impact upon use, and the TECHNOLOGIES.
- Advanced communication technologies, such as smart speakers (also called personal digital assistants, or intelligent personal assistants), and virtual and augmented reality devices, are becoming more available and cheaper in price, within reach for more people, stimulating greater recognition when it comes to usage. The same cannot be said for 3D food printers, which are rarely seen and expensive, beyond the reach of domestic households who would want to print pureed food. How do you think they look? The design and aesthetics of the smart speakers is typically of a round, attractive orb of wisdom, sitting in a household, listening and talking – sometimes all by itself. For this, the user might be quite willing to put up with the device having mediocre voice recognition, and unreliable connections to the wifi or Internet of Things in the household devices the user wishes to control. The headsets of the Virtual and Augmented reality are jarring, appearing otherworldly, awkward, and uncomfortable. Their weight could even be too heavy for a person with neck weakness or poor head control. For the downside on aesthetics, the wearer would need to experience a much greater benefit on balance to warrant the effort of use. The design and aesthetics of 3D food printers is typically of a box-shaped machine, like a large microwave before microwaves became a more polite size. The 3D food printer just cannot escape being a machine for printing pureed food, which you can probably predict, is going to be messy and involve a lot of human-food-interaction. So, let’s hear more about the evidence and device trials that have gone on, in the repurposing of these devices to rehabilitation purposes.
- First up is the Smart Speaker. Do you own one of these? Myself, Dr Lucy Bryant and Associate Professor Phillippa Carnemolla (Design school at UTS) would love to hear from you – our research is currently investigating how the general population are using these and what they think of the interaction. Does it help your safety, participation, engagement, conversation, or enjoyment of life, access to information, reminders or games? So what does the literature tell us about their use with populations with communication disability associated with any health condition?
- A recent systematic review by Berube and team in 2021 checked on the use of ‘conversational agents’ (which could include chat bots) with people with any chronic health or mental health condition. They included 11 studies, 4 related to smart speakers, but only one of these included adults with communication disability. Stefan Greuter, Sue Balandin and Jo Watson used a smart speaker with 9 adults with Intellectual Disability, to teach social skills using interactive storytelling. Although the device did not always parse the commands correctly, participants were not frustrated when they like others using these systems were not understood. The authors concluded and I quote: the smart speaker “has clear benefits in terms of accessibility. Human Computer Interaction based technology once set up can be accessed independently by users, placing less demand on paid personnel and increasing the user’s sense of independence and control along with providing meaningful activity.” The Amazon Echo has appeared in studies from the leaders of the pack when it comes to visual supports in autism research, Children’s Hospital Boston, who tried using the Amazon Echo to remotely control the appearance of visual supports on an iPad app. This was not reliable. So they then used the Amazon Echo Show (which has a touchscreen visual display as well) and first checked whether five SLPs at the hospital can reliably speak commands, using carrier phrases with instruction, to the Echo Show in order to display visual supports – photos – on the screen. Results were better, but the device needed one carrier phrase re-worded to go from 0% to 100% success (close the door versus shut the door) and did not recognise speech of a participant wearing a retainer (who also anecdotally reported feeling quite nervous). Smith et al., in 2020 and 2021, investigated the use of Smart Speakers in group homes of 21 adults with Intellectual Disability. It’s not known whether these adults had poor intelligibility prior to the study, but intelligibility on the researchers’ speech task (using the Smart Speaker carrier phrases, reading words or saying them after the researcher) improved significantly compared to controls, simply by having a smart speaker in the home for 8-12 weeks. Now, these participants did not also have speech and language therapy at the time, and the authors concluded that they did not know why the clarity of speech in the recordings had improved pre and post having the smart speakers. But this is supported by anecdotal reports.
- The Amazon Echo has understandably received a lot of reviews on Amazon. In 2018, Pradham and team collected up 346 reviews by or about people with cognitive, physical or sensory disability, and coded them for content. The reviews indicated that some people with speech impairment found Alexa understood them better than other people. One review reflected that the person with speech impairment had to slow down her speech to be understood, and this had helped her speech intelligibility overall. Changes in speech with use of a smart speaker have also been reported in populations with Parkinson’s, although not universally and like Smith the mechanism is not clear. Duffy and colleagues ran an online survey asking people with Parkinson’s about their use of smart speakers. Some said their device gradually asked them to repeat less times. They wondered if the presence of the speaker as a non-judgemental communication partner giving endless attempts at voice commands helped some people, or perhaps it was just the person learning which commands were more effective. Pradham et al 2018 – examined 346 reviews for content themes re Amazon Echo, Echo Dot and Tap that described use by people with cognitive, sensory or physical disability (One third by people with disability, two thirds by another party about use by a person with disability). Coded the data qualitatively along tone (positive/negative0, uses, accessibility challenges. Voice controlled digital assistants offer “tremendous potential for inclusive, accessible interaction.” Included use by people with hearing loss and speech impairments. Reviews were “overwhelmingly positive” – eas of use, ability to independently complete everyday tasks. Areas for future work – limited functionality of the device, unexpected use for speech therapy, learning and memory support. 31 reviews re users with speech impairment and comments about speech recognition. comments were positive (23/31; 74.2%). For example, R144 stated, “Most humans can not understand me, but Alexia can,” while R318 wrote, “Ordinarily voice programs can’t understand what I am saying due to my speech impairment, but Alexa responds to my commands without fail.” Another review (R126) mentioned using their AAC device to give commands to Amazon Echo. These findings may reflect the views of users who are early adopters and those with the digital connectivity to leave reviews – may not apply to others. To what extent are off-the-shelf IPAs, which were not necessarily designed with accessibility in mind, accessible? How are people with disabilities making use of them? What design opportunities do these devices offer to further support everyday activities for users with disabilities? collecting and analyzing online customer reviews of the Amazon Echo, a popular IPA, and its offshoots, the Echo Dot and Tap. We identified 346 reviews that described use of the device by a person with a cognitive, sensory, or physical disability, written either from a first- or third-person perspective. We conducted a content analysis of the reviews, qualitatively coding them along dimensions such as overall tone (positive/negative), uses of the device, and accessibility challenges. To complement these findings, we then conducted a second study to offer a more in-depth understanding of one specific subset of users: those with visual impairments. Here, we interviewed 16 participants with visual impairments who owned an Amazon Echo or Google Home device. The interview covered similar themes to the analysis of reviews. Findings from both studies show that the new paradigm offered by voice-controlled IPAs offers tremendous potential for inclusive, accessible interaction. Although some accessibility challenges arose, Study 1 shows that users with a broad range of disabilities, even some with hearing loss and speech impairments, are making use of IPAs. Reviews were overwhelmingly positive, mentioning impacts such as ease of use compared to existing technology and the ability to more independently complete everyday tasks. At the same time, the currently limited functionality of the device and unexpected use cases of speech therapy, learning support, and memory support point to potentially fruitful avenues of future work. Study 2 findings confirm many of the conclusions from Study 1, albeit specifically with blind and visually impaired users.
- There are increasing media reports of the use of Smart Speakers in home, hospital and rehabilitation environments. It looks like this Echo Show is coming in handy at the hospital bedside. Has anyone ever seen one being used in a rehabilitation setting? If a smart speaker in a room can give reminders, instructions, prompts and play music, maybe it would be great in a lot of places.
- So let’s move on to Augmented Reality, and Virtual Reality. VR and AR are considered immersive reality technologies. You will also hear the term ‘Mixed Reality’ which recognises the blend of physical and digital worlds. Presentations yesterday indicated that VR is beginning to be used in rehabilitation; primarily by physiotherapists. Understandable perhaps, as it is hard to ‘picture’ communication, and recreate Avatars who might be interactants, as communication is co-constructed and exists between two people as a mutual establishment of meaning.
- As there might be less familiarity with AR, the picture on the left is augmenting the environment, by superimposing a moving dinosaur onto an office carpet. The middle photo is Lucy wearing the Hololens 2, an augmentative reality headset, you can see it is like a large pair of glasses over her eye-glasses, there’s the powerpack and wifi components at the back, it’s hands free and wireless, no tethering to a computer, and she can control it with hand gestures and eye gaze, and voice commands. So it’s a hands free device, that puts the augmentation onto her field of view, as if it is actually appearing in the real world. The picture on the right has a virtual dog that Lucy can pinch and resize, the lower picture has a ‘guide’ showing how to turn the page of a book to the left. Having worn the device, I would say it is an unusual user experience but you do get used to it without forgetting you have it on. The user instructions say it is only to be worn indoors. Most software applications available for it have been designed for industrial settings and worker orientation or training, architecture and design, and so on. So for communication disability, you will need access to more co-designed apps relating to cognitive-communication disability specifically.
- Relevant to populations with communication disability including TBI or ABI, there are at least 3 literature reviews of VR, showing paucity of research but promising potential in terms of purposes in interdisciplinary, collaborative research and providing useful guidance on the co-design of VR. There are also two systematic reviews, both of VR and AR, one in relation to children with developmental disorders, and another relating to people with intellectual disability. Our review of the 69 studies on VR and AR revealed most were at that point desktop non-immersive virtual reality, only 1 related to immersive VR headset, and 5 to Augmented Reality.
- So, now there are good reviews available, it is good to see more original research coming through; there is quite a bit to work from in VR in terms of foundational knowledge. Here are just two which are in press at present.
- It’s great to see Dr Clare Burns here announcing on Twitter being awarded funding for developing a VR tool for communication rehabilitation. One more time I can promote the use of Twitter for staying on top of what is going on in the world of VR and brain injury.
- Here are some images of printed shapes. The first is pink icing, the second is guacamole, and the third is a sweet potato chicken leg. I know of two other sites also implementing 3D food printing – one is Bathurst High School, and the other is in NSW. I will be following those two sites to see what their experiences are, as well.
- With engineering literature making broad claims that 3D food printing will change the world for older people with swallowing disorders, we have reviewed the literature according to the design issue being addressed, for any study or review of 3D food printing which mentioned benefit for people with dysphagia. Most studies focused on the engineering aspects of the food that could be printed, and the devices. So we purchased a device with NHMRC funding, and asked a diverse interdisciplinary expert reference group and a person with dysphagia for their views. The results show that at best it is early days and the feasibility of use is low for populations with dysphagia and their supporters. It’s hard to see the ‘cost-benefit’ in terms of time and ease of use – it’s fiddly, requires a lot of human-food-computer interaction, and the device we have used, the Foodini, involves little interactive feedback to the user to help in this new process. Overall, the relative advantage of the 3D food printer over food moulds and piping bags is not convincing. In her PhD at UTS, Rebecca Smith has looked more broadly at the impact of food shaping on quality of life, finding it to be of great importance, but little research actually seeking the views of people with dysphagia or their supporters. She is finding that food shaping is worth the trouble it takes to make the shape attractive on the plate, but it must look like the food it is made of.
- So what is the upshot of these three incrementally innovative technologies? The smart speaker stands out from the other two, being readily available, becoming common and ubiquitous in use, and being of high aesthetics and design fitting into our everyday worlds. This one is likely to be useful for people with brain injury, pending success on the interactions with the device. Virtual and augmented reality is emerging and growing in use with a likely trajectory of adoption more broadly and hence greater recognition and acceptance in populations with disability. 3D food printing remains a research and development exercise, and we have an Engineering capstone project student Mackenzie Brown now moving to co-design a 3D printer that can print more easily and also deliver the Minced and Moist texture of food to see if we can expand possibilities there. These devices all raise several issues relating to digital autonomy and usability that are yet to be resolved and would need to be for use in populations with communication disability.
- So let’s look at this with an eye on inclusive design and development of conceptual frameworks that guide incremental innovation. The Editor of the flagship journal in this field, Disability and Rehabilitation: Assistive Technology, Marcia Scherer is proposing that, since COVID-19 and massive uptake of technology impacting all aspects of life, it is time for a bio-psycho-social-tech model, that recognises the inherent blending and influence of technology in all domains of life.
- Bird and colleagues, in writing about research involvement and engagement with end users, is proposing a ‘Generative Co-design Framework’ for healthcare innovation, that emphasis a non-linear approach to co-design, with user-centred involvement in pre-design, co-design, and post-design (or re-design) in an iterative process.
- The field of inclusive design is a broad and deep one, particularly in relation to disability and inclusion across education, employment, and health. Professor Jutta Trevinarus is director of the Inclusive Design Research Centre at OCAD University in Toronto Canada and has proposed a model of inclusive design in assistive technologies, from a design discipline perspective. She writes about three steps, recognising and respecting human uniqueness and variability or diversity; using inclusive, open and transparent processes and co-design with people who have diverse perspectives (including those who cannot use the current design), and realising that this is design in a complex, adaptive system. She talks about Vicious Cycles of digital exclusion, and Virtuous Cycles of digital inclusion. Ultimately, what we don’t see enough of yet but is emergent, is the Virtuous Tornado of Design, which is not a flat model, but rather the circular and growing evolution of cycles, with each cycle of design involving co-design, develop, implement, evaluate and refine at the base, with the injection of user needs and characteristics at each cycle, leading to more adaptable designs, more innovative, more resilient, more inclusive, and more accessible designs.
- What will help the affordances of these devices overall? Better speech recognition – even the 3D food printer needs this to be added. We should not have to raise our voice and repeat ourselves to ask a smart device to do something that is meant to save us time and effort. But perhaps doing so will improve our speech? (Over-articulation!). Inclusive speech recognition – to understand imperfect speech, or softer less intense voices. Graphic cues and video models for people with communication disability Better user-centred design of the hardware and software – headsets and controls, and more attention to aesthetics. Digital autonomy – all of these devices go to the cloud where data is stored, for functionality of the device and purposes around reminders. The tracking of our data and usage of it against us in a court of law is one example of the current limitations of data privacy, and little research examines digital autonomy for people with disability who rely on others for assistance in logging in and using these ‘smart’ systems connected to third party cloud servers. Digital autonomy also relates to how much choice and control we have over the settings in our devices. Inclusive design – of both the hardware and software, for people with physical or cognitive disability. Being smarter! Able to do more things, and have more uses, and work more reliably Dedicated design for rehabilitation (not only repurposing) will also recognise the 15% of the population who have a disability, and the remainder may benefit from the tech designed for them, and repurposed for the general population.
- So promising much, we need to see more collaboration with industry that is interdisciplinary and inclusive of the users, which includes the person with communication disability, their families, and their health or disability support providers. Let’s do this together as a community across populations, recognising the diversity and variability of the human experience in using digital technologies to help all of us get on with the business of living.
- Keep in touch and collaborate with us at UTS as we go forward in our cycles of development.