Tetsuo Yamabe, Kiyotaka Takahashi, and Tatsuo Nakajima. 2008. Design issues and an empirical study in mobility oriented service development. In Proceedings of the 1st Workshop on Mobile Middleware: Embracing the Personal Communication Device (MobMid’08, workshop paper)

1. Design Issues and Empirical Study in Mobility Oriented
Service Development
Tetsuo Yamabe
Distributed Computing Laboratory
Waseda University

2. Preamble
¡ How we should design mobile/wearable service interaction, in
order to improve its mobility for nomadic people?
¡ This work aims to explore the service design principles, with developing
practical services.
¡ We are exploring approaches to decrease a user s cognitive load (i.e.
attention to a mobile service).
¡ This work was initiated in Nokia Research Center Tokyo.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

3. Outline
¡ Introduction & Motivation
¡ Design issues in mobile interaction & Our approach
¡ Prototyping & Experimental evaluation
¡ Discussion from findings and comments
¡ Future works
¡ Conclusion
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

4. Introduction
!! Nowadays people are too addicted to mobile services.
!! Using mobile phones while walking, on a bicycle, even driving a car…
!! Current mobile service design is too attention-consuming.
!! From cognitive psychology point of view, attention is a key factor to
improve mobile usability in the wild.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

5. Usability degradation on the move
!! Usability of mobile service becomes degraded while moving.
!! Environmental transition - Situational disabilities forces a user to pay
more attention.
!! Multitasking environment - Fragmentation of attention doesn t allow a
user to concentrate on mobile interaction.
!! Fine usability allows a user to perform tasks with less attention.
!! Flexible and multimodal interaction is required.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

6. Design issues in mobile interaction
¡ Multimodality
¡ Cognitive resource is limited, so mobile services should support multimodal
interaction to approach available sense.
¡ Also wearable devices play important role to change the modality/presentation of
feedback according to the level of its importance.
¡ Simplicity
¡ Direct feedback (stimulus) makes interaction style passive, so that a user doesn t
have to interact with a mobile service proactively (i.e. event-driven interaction).
¡ However, the direct stimulus should be simple in order to decrease cognitive load.
¡ Adaptability
¡ System should change the modality of interaction, since available sense changes
according to mobile context.
¡ Also combination of available device changes according to users.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

7. Multi-layered, Multi-modal and Multi-presentation
approach for mobile/wearable interaction
Pedestrian navigation on a mobile device
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

8. Multi-layered, Multi-modal and Multi-presentation
approach for mobile/wearable interaction
Direct feedback:
Pedestrian navigation on a mobile device symbolic presentation
Simple format indication
on a head-mounted display
Detailed map on a ordinary
mobile phone
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

9. Multi-layered, Multi-modal and Multi-presentation
approach for mobile/wearable interaction
Direct feedback:
Pedestrian navigation on a mobile device symbolic presentation
Simple format indication
on a head-mounted display
LED/LCD indication on a
wrist watch type device
Detailed map on a ordinary
mobile phone
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

10. Multi-layered, Multi-modal and Multi-presentation
approach for mobile/wearable interaction
Direct feedback: Direct feedback:
Pedestrian navigation on a mobile device ambient presentation symbolic presentation
Simple format indication
on a head-mounted display
LED/LCD indication on a
wrist watch type device
Detailed map on a ordinary
mobile phone
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

11. Multi-layered, Multi-modal and Multi-presentation
approach for mobile/wearable interaction
Direct feedback: Direct feedback:
Pedestrian navigation on a mobile device ambient presentation symbolic presentation
Simple format indication
on a head-mounted display
LED/LCD indication on a
wrist watch type device
Tactile indication on a
Detailed map on a ordinary belt type wearable device
mobile phone
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

12. Multi-layered, Multi-modal and Multi-presentation
approach for mobile/wearable interaction
Direct feedback: Direct feedback:
Pedestrian navigation on a mobile device ambient presentation symbolic presentation
Simple format indication
on a head-mounted display
Text
LED/LCD indication on a
wrist watch type device
Tactile indication on a
belt type wearable device
A ticker tape style indication LED indication on an
Audio based indication ordinary mobile phone
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

13. Empirical study on a pedestrian navigation
¡ A pedestrian navigation service
on N800 Nokia internet tablet.
¡ Added some simplified user
interfaces and route navigation
feature to maemo-mapper.
¡ 3 different presentation styles
are supported:
¡ Simple text sentence
¡ Simple voice message
¡ Simple visual indication
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

14. ¡ The event types in the prototype, which makes direct feedback as
an indication to a user.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

15. 1. Determine a user s location with a GPS receiver
2. Identify event types
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

16. 3. Navigate the user according to detected event types
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

17. Preliminary experimentation
¡ As the first step, we mainly focused on the simplicity in this
evaluation.
¡ How the simplified indication works and affects to a user s behavior?
¡ Compared normal interaction mode and simple interaction mode.
¡ Field experiments are performed with using the prototype.
¡ 5 participants from NRC Tokyo joined.
¡ Two tasks are given to each participant.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

18. Tasks
¡ Task A: Participants are instructed to walk Route A only with the normal interaction
mode. The normal interaction mode shows the user's current location and route to
the destination. The user can reach the destination by following the route, but the
shown area on the display is small enough so that the user can not remember the
entire route.
¡ Task B: Participants are instructed to walk Route B with a combination of the
simple interaction mode (i.e. blink indication) and the normal interaction mode. In
this case, the simple interaction mode has to be mainly used and the normal
interaction mode is used auxiliary when the user has lost his/her way. Users can
switch modes by manually pushing a hardware key.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

19. ¡ Number of corners and total distance are almost same.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

20. Examinee observation
¡ Two observers pursued the participant with video cameras to
record a user s behavior and surrounding environment.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

21. Experimental results
!! TT [sec]: Total time spent to reach the destination
!! ST [sec]: Total time of stationary state (i.e. just standing without walking)
!! LT [sec]: Total (loss) time spent for walking incorrect route
!! LD [m]: Total (loss) distance of incorrect route walking
!! SP [km/h]: Average walking speed
!! GR [%]: Proportion of time that the participant watch the display to TT
!! MT [sec]: Total time of the normal interaction mode
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

22. Some findings and discussions
¡ Users would expect the next event and attempt to know when it
will occur.
¡ Time to next event helps to create mental model in a user s mind and
prepare for the event.
¡ Some events and indications seem to be unnecessary.
¡ The blinking pattern was comprehensible, but no indication seems to be
needed while the user is walking on the right route.
¡ Since the user interface on the simple user interaction mode was
not interesting, users started to look the scenery.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

23. Future work #1: Multiple device deployment
!! Advanced version of the prototype with multiple wearable devices.
!! HMD: Nikon UP300x
!! Visual/Audio modality
!! Head gesture with 3D accelerometer
!! Wrist watch: SunSpot based prototype
!! Visual/Haptic modality
!! Gesture with 3D accelerometer
!! Environmental sensors
!! Mobile: iPhone/iPod touch (or maemo platform)
!! Visual/Audio modality
!! Gesture with 3D accelerometer
!! Support gesture/context based adaptation
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

24. Future work #2: System support development
Event
¡ Develop a middleware for multi- desc
device interaction App App
¡ Device (resource) management
¡ Adaptation policy management Context Adaptation
pool policy
¡ I/O management
¡ Context sensing
Event
¡ Gesture input Dispatcher
¡ Simplified event notification
Resource
mgmr
¡ Develop a framework for multi-
layered interaction design
Dev Dev
¡ Multimodality
¡ Simplicity Dev
¡ Adaptability
sensor display I/O
desc
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

25. Conclusion
¡ We have pointed out design issues in mobile interaction design, in
order to improve usability of mobile services on the move.
¡ A pedestrian navigation service was developed and we evaluated
its feasibility through field experiments.
¡ We have successfully decreased a user s cognitive load by adding
a simple interaction mode.
¡ Also, we have proved that the service semantics was successfully kept
even in the simple interaction mode.
¡ We will continue to develop the system based on the comments
and discussions, in order to realize the complete a picture of the
proposed design framework.
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium

26. APPENDIX
Mobile Middleware: Embracing the Personal Communication Device (MobMid'08) , December 2, 2008,
Leuven, Belgium