The document discusses design challenges for mobile input and output. It covers types of mobile devices and input methods like virtual keyboards. Finger input has disadvantages like occlusion but is fast for phone calls. Research aims to improve text entry as it is the most common mobile interaction. Movement and off-screen visualizations can also be used as input. When viewing documents on small screens, overviews are important. Audio and haptic output reduce demands on visual attention by providing information without the screen.

1. EPFL, spring 2012 – week 8!
mobile i/o

2. overview
➝ types of mobile device
➝ design challenges
➝ in: text entry
➝ in: overcoming finger occlusion
➝ in: movement
➝ out: overviews
➝ out: off-screen visualizations
➝ out: audio & haptic output
✱this lecture is based on Will Seager s (UCL) lecture of mobile systems

3. handhelds: three broad categories

4. why touchscreens?
➝ larger screens for
video, maps, websites,
documents etc
➝ easier to point

5. finger vs stylus
advantages of finger…
➝ can’t lose stylus
➝ fast response to alerts e.g.
phone calls
➝ one-handed operation
disadvantages…
➝ low pointing accuracy
➝ finger occlusion
➝ dirty screen

6. design challenge: screen space

7. design challenge: context

8. in: text entry
➝ > 1 billion text messages sent per day
➝ most common type of mobile interaction
➝ companies are looking for improvements to mobile text
entry methods
➝ many methods currently exist

9. Text entry research timeline
Mackenzie 2008

10. three broad categories
Physical Virtual Keyboards
key-based finger-based stylus-based

11. physical vs virtual keyboards
➝ physical keyboards
➝ mobile phone keypad, mobile qwerty (e.g. Blackberry), 5 button
pager, 3-key date stamp, 1 key input etc
➝ virtual keyboards
➝ aka “soft keyboards” or “on-screen keyboards”
➝ similar to clicking buttons in a GUI
➝ used with a stylus or a finger (but also with other input mechanisms
e.g. eye tracking)
➝ design issues
➝ number of keys, key layout, key size, key shape, activation force,
feedback, disambiguation, language modelling, word prediction etc

12. number of keys & layout
QWERTY
➝ both physical & virtual
keyboards vary in number
of keys & layouts
➝ for mobile text input, 26 MOBILE
key qwerty & 9/12 key
ABC are by far the most
common
➝ other 26-key layout
variations include Opti,
Dvorak & Fitaly
➝ other 9/12 key variations
include 9/12 key qwerty

13. number of keys & layout
QWERTY
➝ other layouts have been
shown to lead to better
performance
BUT familiarity a crucial
OPTI
factor
opti outperforms qwerty (faster, fewer
errors) after a few hours practice

14. number of keys continuum
more ambiguity continuum
less

15. ambiguity
7
8
6
6
3
7
?
PQRS
TUV
MNO
MNO
DEF
PQRS
or, is it SUMMER, is it STONES ?
➝ ambiguity occurs if there are fewer keys than symbols in
the language => disambiguation is needed to select the
intended word from the possibilities.
➝ disambiguation methods include multi-tap and T9

16. virtual keyboards
stylus methods
SWYPE
➝ tapping on virtual
keyboards
➝ handwriting recognition
finger methods
➝ tapping on virtual
keyboards
new method for stylus &
finger
➝ sliding stylus/finger
across the screen

17. virtual keyboards: feedback
➝ performance with
virtual keyboards
improves with vibro-
tactile feedback
➝ visual and audio
feedback may also be
useful

18. ➝ finger/thumb occludes
➝ lower precision when
pointing

19. offset cursor

20. “shift” target selection technique
➝ “shift” – a technique for
enabling fine cursor
pointing using fingers

21. “escape” target selection technique
a) the user presses his/her thumb near the desired target
b) the gestures in the direction indicated by the target
c) the target is selected, despite several nearby distractors.

22. behind touch

23. pseudo transparency

24. tilting
(Rekimoto uist 96)

25. “chameleon”
(Fitzmaurice 1993)

26. peephole display
(Yee 2003)

27. camera phone based motion sensing
(Whang, Zhai & Canny 2006)

28. viewing large documents on small displays

29. overviews
(O’Hara et al 1999)
(Woobrock et al 2002)

30. off-screen visualizations: edge radar

31. off-screen visualizations
(Baudisch & Rosenholtz 2003
Gustafson et al 2008)

32. audio & haptic output
➝ non speech audio
output
➝ bleeps, earcons,
auditory icons
➝ haptics
➝ refers to interaction
via sense of touch Example earcons from (Brewster et al 2008)
and/or motor
activity.

33. why use audio and/or haptic output?
➝ attention grabbing
➝ saves screen real
estate
➝ can provide
information without
requiring visual
attention

34. earpod

35. head-mounted displays
➝ user can look at environment
& display at the same time
➝ potentially good for location
based and augmented reality
servies as potential for clear
link between information &
the environment
but…
➝ require separate input device
e.g. trackball or else speech
only input

36. summary: some key points
➝ Key design challenges: small screens & context
➝ mobile text entry research ongoing, in particular for
finger-based input via touch screens
➝ movement as input
➝ importance of overviews when browsing documents on
small screens
➝ off-screen visualizations
➝ audio & haptic output is a way to reduce demand on
visual attention