The Mobile Augmented Teleguidance-based Safety Navigation Concept for Senior Citizens article published by the Journal of Finnish Universities of Applied Sciences (№2, 2012)

1. 1
Mobile Augmented Teleguidance-based
Safety Navigation Concept for Senior Citizens
Petri Pulli, Zeeshan Asghar, Mika Siitonen, Risto Niskala, Eeva Leinonen, Antti Pitkänen, Jaakko Hyry,
Dept of Information Processing Science, University of Oulu, Oulu, Finland
Jarmo Lehtonen, Scope Associates Oy, Helsinki, Finland
Vadym Kramar and Markku Korhonen, (markku.korhonen@oamk.fi),
Pehr Brahe Center for Industrial and Services ICT (PBOL), Oulu University of Applied Sciences, Finland
Abstaract
Many countries are facing a problem when the age structure of the society is changing. The numbers
of senior citizens are rising rapidly and care-taking personnel numbers cannot match the problems
and needs of these citizens. Using smart, ubiquitous technologies can offer ways in coping with the
need of more nursing staff and the rising costs of taking care of senior citizens for the society.
Helping senior citizens with novel, easy-to-use interfaces that guide and help could improve their
quality of life, and make them participate more in daily activities. This article describes smartphone-
based navigation concept with easy-to-use functionality dedicated to senior citizens and their
relatives. Such a system can be used outside of the home environment to ease and support the lives
of elderly people as society members of full value, and improve their quality of life. A safety area for
outdoor activities of senior citizens may be pre-set by virtual safety borders. The borders may be
determined in advance by people in charge, such as relatives or caretakers. The people in charge as
well as security centres, providing such services, are able to observe and follow up of movements of
senior citizen via convenient interfaces. The interfaces provide users with browsing opportunities,
abilities to see and review pictures and videos taken by cameras of senior citizens’ mobile terminals,
map observation, and communication abilities. The Safety Navigation system is intended to increase
awareness of both, the target users and their relatives or other people in charge, of a situation of the
target users, and thus to ensure a feeling of safety.
Keywords:
Senior citizens; teleguidance, user interface, smart phone; ambient environment; ubiquitous home
environment; memory problems; Alzheimer’s; scenario, Quality of Life; QFD

2. 2
Introduction
Societies in Finland and Japan are aging at an alarming rate [7, 8]. As the structure of the population
is shifting towards the elderly, studies have shown that Finland is facing a crisis when the cost of
supporting the elderly rises and the amount of caretakers is not sufficient enough for support [2].
The numbers of elderly who live alone grows and the older they get, the more assistance they
require [3]. The amount of elderly people, suffering from varying memory impairments, is also going
to more than double during the next 30 years [1]. If we can improve the caretaker productivity by
just 1% and postpone the decline in cognitive and memory functionality of the elderly by just 5
years, the need for more caretakers plummets from near 25% to about 15% [3]. Thus accumulating
vast savings accumulated for the society until the year 2040 [2].
Technological innovations have created a net of wireless information exchange between individuals
with ubiquitous solutions, which could also be used to improve the quality of living for elderly. It is
important to consider the availability and use of technology in different situations and to create
novel solutions that meet the demands of users.
Memory problems are common for elderly and range from simple age-related problems to
Alzheimer’s disease. A collaborative study in Nordic countries was made to individuals with
dementia and the goal was to find out what kinds of aid devices are used, suitability for the users,
and to gather improvement feedback [6]. Conclusion was that introducing aid devices improved
management of daily activities, helped maintain skills and made people socially active. A study by
Sorri et al. [5] has also suggested that a way finding advising technology has the potential to provide
important support for the elderly by similarly motivating and empowering them to perform their
daily activities.
Based on the Nordic study basic requirements for future design can be made: combine
functionalities, smaller number of devices, tailor interfaces and offer tele-presence assistance. Over
40 different aid-devices were used and the median of use was a year and four months. Looking at
different types of aid-devices, e.g. GPS, calendar, portable alarm and a safety camera, some existing
solutions e.g. a mobile phone already combine some functions, but doesn’t offer usability designed
for dementia. Reducing the number of devices is feasible, but care has to be taken in the design of
the devices for the end-users. However, misplacing a device, means losing all the functionalities
associated with it. Based on the feedback data, the need to learn use of new devices was constant.
The degrading nature of the disease presents new problems and existing devices had to be switched
to new ones. In some cases people having same severity level of dementia in similar living
environments didn’t always use same devices or used them differently. Devices with tailored
information would lessen the need to introduce new devices and would be familiar to the user
throughout the process. Implementation of tele-guidance could provide help regardless of place and
time.

3. 3
Sorri et al. [5] way finding prototype was tested with real users in real environment using predefined
routes. The orientation advice was given through three modalities, visual, audio and tactile signals,
two of which were used at a time. Nine subjects, with a median age of 84 years, participated in the
user study. Their severity of dementia ranged between mild and severe, and walking abilities ranged
from “frail to hobby skier”. In addition, two elderly persons were recruited as control subjects.
In most cases, the orientation with the way finding aid on predefined routes succeeded, with a few
misinterpretations. The severity of dementia didn’t seem to foretell success in orientation with the
way finding aid. Finding the right door, straying from the defined route, and the attractions of real-
life context like other people were challenged most the test persons. Also the correct timing of the
way finding advice was found to be crucial but difficult because of the varying walking speed of the
subjects. If the advice appeared too early the subjects could forget them or, alternatively, they
complied with the advice literally and as a result in worst cases, turned against the wall. As against
using “left”, “right” and “go straight on” commands as the way finding advice seems to be more
successful than using landmarks. Confirming subjects were on the correct track turned out to be
beneficial for longer legs of the routes [5].
Methods
The applicability of the approach was first evaluated in a multidisciplinary research project Value
Creation in Smart Living Environment for Senior Citizen (vesc.oulu.fi) workshop. This multi-
disciplinary research group used living lab approach, where technology was designed and evaluated
by and in co-operation with the end-users. Smart living environment technology was designed by
taking into account human-computer interaction, communication, product and services as well as
human interaction. In this study focus was on testing the designed navigation system prototype with
scenario-based video with actors. In the video an actor (represents end-user) tested the navigation
system prototype alone and with the help of security personnel and family. Quality of Life
improvement was assessed using Quality Function Deployment (QFD) framework.
Storyboard
In this study focus was on presenting a new idea of a smart navigation system for people who have
mild cognitive impairment due to Alzheimer’s disease, for instance, and who often require help in
their daily activities. Aim of the whole system design was to make it as simple and illustrative as
possible to accommodate the needs of such people. Specifically the aim was to design the system
for senior citizens. This was achieved by designing the system based on a script written in the form
of a storyboard [4]. The script was first illustrated with four early storyboard designs which were
presented and evaluated by the research team. Such approach was chosen to help the research
team/audience better identify themselves with the main character of the scenario. According to the
feedback the final 30 page storyboard was constructed and it formed the basis for the scenario
video. The feedback also helped to make the video more compact and easier to understand. The
storyboard was useful for the main character and other people to understand the whole concept of
the safety navigation system.

4. 4
Figure 1. Storyboard was used to develop the usage scenario.
Scenario video
The designed scenario video 1 features Marjatta, a 73-years old widower who lives alone. She is
suffering from a mild cognitive impairment due to Alzheimer’s disease. Such characteristics were
chosen for the main actor of the video, since they most likely are those observable in our end-user
population. In the video we picture a typical shopping trip, representing typical activities of daily
living. The trip included planning, walking from home to the shop and back. The journey contained
different problems, such as choosing directions, with the possibility of the main character Marjatta
being disoriented. A narrator leads the story forward. Different informative signs and guides are
presented in the video which are essential for the senior citizen to comprehend the functioning of
the smart navigation system but also help the viewers. The main idea of this short video was to
demonstrate and describe how the possible future smart navigation system could actually work.
1
The Safety Navigation scenario video is available at:
http://finplatform.pbol.org/content/InfoDayMaterials/Smart.navigation.for.senior.citizens.mp4

5. 5
Figure 2. Concept video was used to describe and collect feedback from the users and stakeholders.
Implementation considerations
Safety Navigation Overview
The main purpose of the Safety Navigation system is to help elderly people in their outdoor activities
that they would feel safe and secure, and to increase awareness of the situations of the elderly for
those who are concerned. The objective of the system is to assist the seniors remotely utilising a
variety of behavioural scenarios and modern technologies such as positioning, embedded cameras,
multi-channel and multi-media communications, and laser pointers. The entire community of the
Safety Navigation system consists of MoTe (Mobile Terminal) users and Aware users. Aware users
are in charge or may provide help to the MoTe users.
Main Functionality and User Interfaces
The Mobile Terminal concept is developed on a base of the Android OS smartphones. Such
technologies and connected frameworks were enhanced with personalized features such as
addresses and safety check points. Safe areas, like the way to a bank, hospital or a store, can be also
defined to the user. When the user selects a destination, the application shows the safest route for
them. After departure a predefined countdown timer is activated for the user. If the time limit is
exceeded, the system will notify Aware users via e.g. SMS or email message, including location, date
and time, so they can take appropriate actions in order to help the MoTe user.

6. 6
Ubiquitous Home Environment
Ubiquitous Home Environment (UHE) is a user-centric set of systems through which users can
interact with their living environment and outside world. An essential part of the UHE is a serving
engine. Typically the engine achieves interoperability with the UHE infrastructure through a variety
of generic and dedicated modules. The engine should expose a number of GUIs to terminal devices.
PBOL at Oulu UAS has built an implementation of the serving engine, the UbiHOMESERVER. For
sensing the physical environment, the UbiHOMESERVER interoperates with Wireless Sensor
Networks (WSN) and IPTV, mobile and web channels serve for human interaction. As a core of the
engine, an intelligent module was developed. Knowledge technologies, such as Semantic Web, were
considered in design.
The UbiHOMESERVER offers a set of GUIs through which it is possible to interact with the UHE as
well as to consume and manage the ICT Home Services. Device recognition and content adaptation
technologies were used to reduce end-user GUIs’ dependencies on devices. The end-user GUIs
utilise multimodal information presentation, exposing textual, symbolic, colour, sound, voice and
visual modalities. The UbiHOMESERVER and GUIs are able to handle environmental changes
dynamically. A grid layout was chosen for the UbiHOMESERVER GUI design. Entire interface is
implemented as low-hierarchical and intuitive.
Figure 3. Main view of the UbiHOMESERVER GUI.
Several systems, for example the Safety Navigation System, were made interoperable with the
UbiHOMESERVER and thus became integrated parts of the UHE.

7. 7
Safety Navigation System Architecture
The backbone of the Safety Navigation System is the UHE. The UbiHOMESERVER is running 24/7 and
serve the users of the UHE. For example, family members may share the same living environment of
the MoTe users, or be connected to it remotely. Those of them, who are trusted, may also be
granted access to the MoTe user’s UHE and thus use a dedicated functionality of the Safety
Navigation. Security staff doesn’t not have access to MoTe user’s UHE. Instead, their systems
may interoperate with the UbiHOMESERVER through dedicated interfaces.
Figure 4. Architecture of the Safety Navigation system.
The Mobile Terminal may be implemented in a form of a self-sufficient all-purpose device, or in a
form of a distributed/interconnected set of devices. Among those may be a wearable camera, a key
ring with laser pointer and a smart phone. Due to a specific target group, the Mobile Terminals must
provide end-users with controls and GUIs dedicated to the use by elderly people.
The wearable camera starts to take pictures of the surrounding area when the user is considered
lost. Those pictures supplement location technologies and help the Aware users to identify the
position of the MoTe user more precisely.

8. 8
Figure 5. Safety Navigation system for identifying user position.
Users may launch a live audio/video conversation, and receive guidance from the Aware users at any
time. A laser pointer can be used to display a proper direction with an arrow or other symbols on a
flat surface when the user is not able to use audio/video communication properly.
The door guard generates an event when the user departures from home. Safety Navigation system
keeps track of such state changes by logging them with a time-stamp. The log data may be used for
defining safety areas and time frames more precisely, and in the future such knowledge may bring
an opportunity to develop a proactive behaviour of the system.
Acknowledgments
This work has been partly funded by the Academy of Finland and JSPS (Japan) under Smart Living
Environment for Senior Citizen research projects (VESC, P-SESC) and by Tekes for Well-being
Ecosystem (Ryhti) project.

9. 9
References
[1] Ferri P, Prince M, Brayne C, Brodaty H, Fratiglioni L, Ganguli M, Hall K, Hasegawa K, Hendrie H,
Huang Y, Jorm A, Mathers C, Menezes PR, Rimmer E, Scazufca M (2005) Global prevalence of
dementia: a Delphi consensus study. Lancet 366(9503):2112-7.
[2] Niemelä K, Salminen K (2009) Kansallisten eläkestrategioiden muotoutuminen ja Euroopan
unionin avoin koordinaatiomenetelmä (in Finnish, English summary). Sosiaali- ja terveysturvan
tutkimuksia No. 104. Sastamala: Vammalan Kirjapaino Oy, s. 114-136.
[3] Parkkinen P. (2007) Riittääkö työvoima sosiaali- ja terveyspalveluihin.(in Finnish, English
summary). (VATT discussion papers 433. Government Institute for Economic Research, Oy Nord Print
Ab.
[4] Siitonen M. (2011). Muistihäiriöisten vanhusten turvanavigointi. Oulun yliopisto
Tietojenkäsittelytieteiden laitos, pro gradu -tutkielma. 84s.
[5] Sorri L., Leinonen E. and Ervasti M. (2011) "Wayfinding aid for the elderly with memory
disturbances"; Proceedings of the 19th European Conference on Information Systems (ECIS), ICT and
Sustainable Service Development,
Tuunainen Virpi, Nandhakumar Joe, Rossi Matti and Soliman Wael (eds.), Helsinki, 2011. ISBN 978-
952-60-3574-1.
[6] STAKES (2008) Apuvälineet ja Dementia Pohjoismaissa (in Finnish). [Web-document]
http://www.stakes.fi/verkkojulkaisut/muut/M267.pdf
[7] Statistical Handbook of Japan (2007) Statistical Research and Training Institute, MIC [Web-
document] Available: http://www.stat.go.jp/english/data/handbook/c15cont.htm
[8] Tilastokeskus (2007) Väestöennuste 2007-2040 (in Finnish). [Web-document] Available:
http://www.stat.fi/til/vaenn/2007/vaenn_2007_2007-05-31_tie_001.html