This presentation will give an overview of my EngD research project, "Investigating Augmented Reality as a Cognitive Prosthesis for People Living with Dementia". Learn about example projects from the CDE, followed by a description of augmented reality and how it could be used to help people living with mild-to-moderate dementia. This presentation was giving at the Assisted Living Action Network event in Bath on 18th September 2018.

1. Investigating Augmented Reality
as a Cognitive Prosthesis for
People Living with Dementia
Thomas Williams
EngD Digital Entertainment Student
Centre for Digital Entertainment (CDE) | University of Bath
#ARforDementia
1

2.  Centre for Digital Entertainment (CDE)
 Research Project
 Overview
 What is Augmented Reality (AR)?
 Living with Dementia
 Electronic Prompting (Designability)
 Research Questions
 Research Blog: blogs.bath.ac.uk/ar-for-
dementia
 Close and Questions
18/09/201
8
#ARforDementia 2
Presentation Overview

3. Centre for Digital Entertainment
(CDE)
18/09/201
8
#ARforDementia 3

4. 18/09/201
8
#ARforDementia 4
Example Projects
CDE Website: http://www.digital-entertainment.org/
Twitter: @centre_digi_ent
 Gaze tracking in cultural heritage
Adapated from Williams (2018).

5. 18/09/201
8
#ARforDementia 5
Example Projects
CDE Website: http://www.digital-entertainment.org/
Twitter: @centre_digi_ent
 Gaze tracking in cultural heritage
 Bespoke musical instruments for people with
cognitive, physical, or sensory impairments
and/or disabilities
 Visual effects (motion capture, stop motion)
Ciucanu,A.(2018)

6. 18/09/201
8
#ARforDementia 6
Example Projects
CDE Website: http://www.digital-entertainment.org/
Twitter: @centre_digi_ent
ScreencaptureprovidedbyCiucanu,A.(2018)

7. 18/09/201
8
#ARforDementia 7
Example Projects
CDE Website: http://www.digital-entertainment.org/
Twitter: @centre_digi_ent
 Gaze tracking in cultural heritage
 Bespoke musical instruments for people with
cognitive, physical, or sensory impairments
and/or disabilities
 Visual effects (motion capture, stop motion)
 Virtual reality (VR) for brain injury rehabilitation

8. Research Project
18/09/201
8
#ARforDementia 8

9. 18/09/201
8
#ARforDementia 9
Overview
Research Project
 Potential of AR as an assistive technology not
yet fully explored
 Build on Designability’s experience of task
sequencing with dementia patients
 Can AR be used to help people living with
dementia with daily living tasks?
Investigating Augmented Reality as a
Cognitive Prosthesis for People Living
with Dementia

10.  MR: Any combination of real and virtual content
(Milgram and Kishino, 1994)
 AR: Virtual content displayed in real environment
 Not limited to visual. Can also be audio, touch, even
smell.
(van Krevelen and Poelman, 2010)
 Good, informal explanation:
https://www.realitytechnologies.com/
AdapatedfromMilgramand
Kishino(1994)
18/09/201
8
#ARforDementia 10
What’s AR? | Explanation
Research Project

11. 18/09/201
8
#ARforDementia 11
What’s AR? | Timeline
Research Project

12. 18/09/201
8
#ARforDementia 12
What’s AR? | Examples
Research Project
Source:https://developer.apple.com/arkit/
AdaptedfromBrolletal.(2004)

13. 18/09/201
8
#ARforDementia 13
What’s AR? | Examples
Research Project
AdapatedfromKutteretal.(2008)
Source:
https://techstockinsider.com/micr
osoft-mixed-reality-hololens-
benefits-industries/

14. 18/09/201
8
#ARforDementia 14
Living with Dementia
Research Project
 Symptoms:
 Short term memory loss
 Impaired communication
 Difficulty thinking and problem solving
 Daily activities can become difficult
 With assistance, patients can continue to do
these tasks and live independently at home
Content adapted from Boyd et al.
(2018)

15. 20/06/2018CDE Showcase – Digital Catapult Centre London 15
Electronic Prompting
Research Project
Content adapted from Boyd et al.
(2018)
Boydetal.(2018)

16.  Enables patients to carry out daily tasks
 Personalised
 Audio and text effective for prompting
 Can this work in AR?  New research project…
20/06/2018CDE Showcase – Digital Catapult Centre London 16
Electronic Prompting
Research Project
Content adapted from Boyd et al.
(2018)

17. 18/09/201
8
#ARforDementia 17
Research Questions
 How can AR technology be used in the support of
daily living tasks and activities for those with mild
to moderate dementia and their carers?
 What are the strengths and weaknesses of AR in
the support of those tasks?
 Which requirements need to be addressed for a
system to be successful?
 What design features could support the adoption
of AR as an assistive technology for people living
with dementia?
Research Project

18.  Attitude towards technology
 “How will this help me?”
 Emotional impact
 In terms of technology, headset currently too
clunky for assistive technologies
18/09/201
8
#ARforDementia 18
Potential Barriers
Research Project
Content adapted from Boyd et al.
(2018)

19. Attitude towards technology
“How will this help me?”
Emotional impact
In terms of technology, headset currently too
clunky for assistive technologies
BUT…
 This research will provide the knowledge
required for when technology inevitably
becomes smaller. 18/09/201
8
#ARforDementia 19
Potential Barriers
Research Project
Content adapted from Boyd et al.
(2018)
ByMikepanhu-Ownwork,CCBY-SA3.0,
https://commons.wikimedia.org/w/index.php?curid=323
48020

20. Follow and subscribe!
blogs.bath.ac.uk/ar-for-dementia
18/09/201
8
#ARforDementia 21
Augmented Reality for Dementia
Research Blog
Keep up to date with:
 Research progress
 My EngD journey
 Presentations

21. Thank you.
Any questions?
18/09/201
8
#ARforDementia 22
Follow and subscribe!
blogs.bath.ac.uk/ar-for-dementia

22. Acknowledgements
Many thanks to the EPSRC Centre for Doctoral Training in Digital Entertainment for the funding
that they provide me to carry out my research. Big thanks are also in order for the CDE staff at the
University of Bath and Bournemouth University for the continual support they provide.
Thank you to my supervisors (Prof. Nigel Harris, Dr. Elies Dekoninck, Dr. Simon Jones, and Dr.
Christof Lutteroth) for their input and guidance throughout every stage of the project so far.
18/09/201
8
#ARforDementia 23

23. Links
Blog: blogs.bath.ac.uk/ar-for-dementia/
Twitter: www.twitter.com/tjw_94
LinkedIn: https://www.linkedin.com/in/thomas-williams-uniofbath/
Designability: www.designability.org.uk
CDE: www.digital-entertainment.org
18/09/201
8
#ARforDementia 24

24. References
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4):355–385.
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer
Graphics and Applications, 21(6), 34–47. https://doi.org/10.1109/38.963459
Billinghurst, M., Clark, A., & Lee, G. (2015). A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction,
8(2–3), 73–272. https://doi.org/10.1561/1100000049
Boyd, H., Evans, N., Ridgers, J., Harris, N., Cheston, R., Noonan, K., Jarvis, A., and Ingram, T., (2018). Digital Prompting Technology to
Support People With Early Dementia. Presentation at Assisted Living Action Network, 27/06/18.
Broll, W., Broll, W., Lindt, I., Ohlenburg, J., Wittkämper, M., Yuan, C., … Strothmann, A. (2004). ARTHUR: A Collaborative Augmented
Environment for Architectural Design and Urban Planning. JVRB - Journal of Virtual Reality and Broadcasting, 1(2004)(1).
https://doi.org/10.20385/1860-2037/1.2004.1
Ciucanu, A., (2018). E-StopMotion: Digitizing Stop Motion for Enhanced Animation and Games. To be published in Motion Interaction in
Games, Nov 8-10 2018.
Diversified Internet Holdings LLC. (2018). Reality Technologies - Mixed Reality, Augmented Reality, Virtual Reality. Retrieved September 6,
2018, from https://www.realitytechnologies.com/
Feiner, S., MacIntyre, B., Höllerer, T., & Webster, A. (1997). A touring machine: Prototyping 3D mobile augmented reality systems for
exploring the urban environment. Personal Technologies, 1(4), 208–217. https://doi.org/10.1007/BF01682023
Kutter, O., Aichert, A., Bichlmeier, C., Traub, J., Heining, S. M., Ockert, B., … Navab, N. (2008). Real-time Volume Rendering for High Quality
Visualization in Augmented Reality.
Milgram, P., & Kishino, F. (1994). A Taxonomy of Mixed Reality Visual-Displays. Ieice Transactions on Information and Systems, E77d(12),
1321–1329.
van Krevelen, D., & Poelman, R. (2010). A Survey of Augmented Reality Technologies, Applications and Limitations. The International
Journal of Virtual Reality, 9(2), 1–20. Retrieved from
https://www.researchgate.net/publication/279867852_A_Survey_of_Augmented_Reality_Technologies_Applications_and_Limitation
s
Williams, T. (2018). A Novel Gaze Tracking System to Improve User Experience at Cultural Heritage Sites. Dissertation MSc. University of
Bath, Bath.
18/09/201
8
#ARforDementia 25