Presentation deck prepared for the paper 'Object Recognition-based Mnemonics Mobile App for Senior Adults Communication' to be presented during ICCCNT'15 conference

1. Object Recognition-based Mnemonics
Mobile App for Senior Adults
Communication
Centre of Social Media Innovations for Communities (COSMIC)
Nanyang Technological University, Singapore

2. Agenda
• Introduction
• Related Work
• UbiCuts System Requirements
– Usage Scenarios
– Roles and Events
• UbiCuts System Design
– Image Recognition and Data Storage Technology
– System Architecture
– User Interface
• Current Status of Work

3. Introduction of Target Domain
Background
• One of the main aims of ICT is in creating and sustaining an all-
inclusive environment for co-existence of different age-groups [1]
• Smartphones revolutionized communication capabilities by
integrating traditional telephony services with broadband internet
services
Problem Area
• There will be more than two billion senior adults by 2050 [2]
• Most of the smartphone apps are not meant for senior adults usage

4. Related Work
Smartphone Apps for Senior Adults Usage
• Protégé [6] -> Android app for facilating effective communuication between
caregivers and seniors
 Features:
 Fall alert,
 No-activity alert
 Medicine alert
 Activity report request
• eCAALYX [7] -> Healthcare-based android app for seniors
 Input data from patient-wearable garments with sensors and GPS (Global
Positioning System)
 Specifically designed for older patients with multiple chronic problems such as
diabetes mellitus and cardiovascular diseases

5. What is UbiCuts (Ubiquitous Shortcuts)?
 Aim: To provide a simple communication solution for older adults and
disabled users with motor impairments and technological illiteracy which
prohibit conventional use of smartphones.
 Enables a series of commands/actions (e.g. making a phone call, sending
out an SMS, posting a tweet…) to be executed by just taking pictures of
objects in the user's environment.
Usage of UbiCuts in its Intended Final Form

6. System Requirements – Usage Scenarios
Scenario 1: Twitter Update
• Elderly person uses the app to indicate his current location to his family
members
• Points the app at his usual seat at the local library
Scenario 2: Making Phone Call
• Elderly person uses the app to make a phone call by pointing it at a photo
Scenario 3: Sending SMS
• Elderly person uses the app to inform family people or caregivers for
completing some task (e.g. buy new packet of rice)
• Points the app at the rice container

7. Events
1. Sending SMS Messages
2. Making Phone Call
3. Posting updates to Twitter
System Requirements – Roles and Events
Senior Adults Family People
Caregivers

8. System Design
• Qualcomm’s Vuforia Augmented Reality (AR) service used for
image recognition
• SQLite used for data storage in smartphones
UbiCuts System Architecture

9. User Interface
• Use-case: Create New Ubicut

10. User Interface
• Use-case: Scan Target Objects

11. User Interface
• Use-case: Edit or Delete Existing Ubicuts

12. Design Limitations
• Plain objects such as walls, bottles/containers with no stickers or
any object are currently not recorded due to lack of adequate
features
• Unnecessary objects in the background when an image of the target
object is captured, can potentially hamper the target scanning
process
• The features present in the particular view of a target object will only
be captured
• If there is a lack of natural lighting, objects will not be recognized
during target scanning.

13. Current Status of Work
 An improved Android app  A workable prototype in the
shape of a bracelet
 User testing in progress with 20 elderly smartphone users
 Field study to find out older adults’ perceptions and preferences of
wearables

14. • Rissanen, M.J., Fernando, O.N.N., Iroshan, H., Vu, S., Pang, N. & Foo, S. (2013) Ubiquitous
shortcuts: mnemonics by just taking photos. In Proceedings of the ACM SIGCHI Conference on
Human Factors in Computing Systems Extended Abstracts (CHI EA), Paris, France, April 27-May
2, 2013. New York, NY: ACM.
• Rissanen, M. J., Vu, S., Fernando, O. N. N., Pang, N., and Foo, S. (2013). Subtle, natural and
socially acceptable interaction techniques for ringterfaces ─finger-ring shaped user interfaces.
In: Proceedings of Distributed, Ambient and Pervasive Interactions (DAPI), July 21-26, 2013. Las
Vegas, NV, Springer.
• Foo, S., Pang, N. and Zhang, X. (2014). UbiCuts: Wearable communication and surveillance aids
for older adults and the disabled. Gerontechnology, 13(2), 196.
• Pang, N., Zhang, X., Foo, S., and Vu, S. (2014). Smartphones Use by older adults in Singapore.
Gerontechnology, 13(2), 270.
• Pang, N., Vu, S., Zhang, X., & Foo, S. (2015). Older adults and the appropriation and
disappropriation of smartphones. To be published in the Proceedings of the 17th International
Conference on Human-Computer Interaction HCII 2015, under Theme: Human Aspects of IT for
the Aged Population.
• Pang, N., Foo, S., Raamkumar, AS., Zhang, X., & Vu, S. (2015, May). UbiCUTS: Ubiquitous
Shortcuts for easy communication. Demo presented at The Seventh International Conference on
Information and Communication Technologies and Development (ICTD 2015), Singapore.
Related Publications

15. THANK YOU