Tap is the New Click
Dan Saffer, Kicker Studio
DRC 2009 // Dan Saffer, Kicker Studio
DRC 2009 // Dan Saffer, Kicker Studio
We're using bodies
evolved for hunting,
gathering, and gratuitous
violence for information-
age tasks like word
processing and
spreadsheet tweaking.
—David Liddle
We’re in the midst
of an interaction
design revolution.
How do we design
for interactive gestures?
What we’re going
to talk about
Sensors and touchscreen types
Kinesiology and physiology
Touch targets
Communicating
Choosing appropriate gestures
Case study: Canesta Entertainment Center
Gesture: any physical
movement that can be
sensed and responded to
by a digital system
without the aid of a
traditional input device
such as a mouse or stylus.
DRC 2009 // Dan Saffer, Kicker Studio
DRC 2009 // Dan Saffer, Kicker Studio
Two types of
interactive gestures
Touchscreen
aka TUI
Single and multi-touch (MT)
Free-form
Wide variety of forms
Why not to have
a gestural interface
Heavy data input
Relies heavily on the visual (for now)
Can be inappropriate for context
More physically demanding
Why have a gestural
interface?
More flexible
Less visible hardware
Hardware fits context better
More “natural”
More fun
The secret sauce: sensors
Common sensors
Pressure
Light
Proximity
Acoustic
Tilt
Motion
Orientation
Types of touchscreens
Resistive: pressing two layers together creates the
touch event
Surface wave: finger disrupts ultrasonic waves
Capacitive: finger conducts electricity
Infrared: finger breaks grid of infrared beams
Camera-based: looks for “blobs.” Rear- and front-
projectors
Kinesiology & physiology
The ergonomics
of human gestures
Avoid hyperextension or extreme stretches
Avoid repetition
Utilize relaxed, neutral positions
Avoid staying in a static position
No “Gorilla Arm”
Gorilla arm
Humans not designed to hold their arms in front of
their faces, making small gestures
Ok for short-term use, not so much for repeated,
long-term use
Fun Fact: Telegraph operators had “glass arm”
Sorry, Minority Report-style UIs
Gorilla arm
Gorilla arm
Stephan Pheasant’s (via
Rob Tannen) cardinal
rules of anthropometrics
Reach
Clearence
Posture
Strength
The more challenging and
complicated the gesture,
the fewer people who will
be able to perform it.
What about
accessibility?
No good, clear answer
Improving via addition of haptics (and hopefully,
eventually, speech)
Some touchscreen systems much better than
traditional WIMP systems
Special care when designing touch targets
16-20mm
8-10mm
10-14mm
Fingers
Fingernails: Blessing
and curse
Fake fingernails: evil
Finger oil
Fingerprints
(Left) Handedness
Wrist support
Gloves
Inaccurate (when
compared to a cursor)
Attached to a hand aka
Screen Coverage
DRC 2009 // Dan Saffer, Kicker Studio
DRC 2009 // Dan Saffer, Kicker Studio
Avoid putting essential
features or information
like a label below an
interface element that
can be touched, as it may
become hidden by the
user’s own hand.
Touch events and targets
Touch target size
Remember Fitts’ Law! (Time it takes to get to a target
= distance to target / size of target)
As close to the user as possible to avoid users’
covering the screen with their hands
Space between the targets (when possible)
Create reasonably-sized targets: no smaller than
1cm in diameter/square (the size of finger pads)
Touch target size
comparisons
~25mm ~18mm ~13mm ~8mm 5mm
Two touch target tricks
Iceberg tips
Adaptive targets
Two touch target tricks
Iceberg tips
Adaptive targets
Traditional UI elements
to watch out for
Cursors
MouseOvers and hovers
Double-click
Right-click
Selected default buttons
Undo
Touchscreen patterns
Tap to open/activate
Tap to select
Drag to move object
Slide to scroll
Spin to scroll
Flick to nudge
Pinch to shrink &
Spread to enlarge
Ghost fingers
Freeform patterns
Proximity activates/
deactivates
Move body to activate
Point to select/activate
Wave to activate
Rotate to change state
Step to activate
Shake to change state
Prototyping gestures
Low-fidelity:
Paper prototype
Low-fidelity: The “man
behind the curtain”
Low-fidelity:
Environments
High-fidelity: Exact
High-fidelity:
Off-the-Shelf
High-fidelity:
Do It Yourself
Turning gestures
into code
Variables: what are you measuring?
Data: get the data in from the sensor
Computation: determine difference between data
Patterns: what do the sums mean?
Action: if a pattern is matched, do something
Documenting gestures
Dance notation
Annotated wireframes
still work
Architectural
wireframes
“Master UI” “Individual UI”
Run by presenter
Live Touchscreen
Projection Area
Used by show attendees
[floor]
[ showing typical arm’s reach for 6’ tall user ] [ showing typical arm’s reach for 6’ tall user ]
touchscreenoverview
[floor]
Keyframes
Gestural modules
Gestural modules
Storyboards
Swim lanes framework
Animation
Movies
Communicating
interactive gestures
Three zones
of engagement
Attraction
Observation
Interaction
Attraction affordance
Written instruction
Illustration
Demonstration
Symbolic
Determining the
appropriate gesture
Four part equation
1. The task that needs to be performed
2. The available sensors and input devices
3. The physiology of the human body
4. The context
This can be pretty straightforward
Or not
Context
Different behaviors in different locations
Avoiding accidental emotional weight
Cultural issues
Usability issues
Avoid unintentional triggers via everyday actions!
Wide variation in performing gestures: need
requisite variety
Pick one: select then action, or selecting does action
Gestures as command keys: Provide a normal means
of performing the action (buttons, etc.) but have
“advanced” gestures as shortcuts
Case study: Canesta
Entertainment Center
Case study: Canesta
Entertainment Center
Case study: Canesta
Entertainment Center
Case study: Canesta
Entertainment Center
The complexity of the
gesture should match
the complexity of the
task at hand.
The best designs are
those that “dissolve
into behavior.”
(Naoto Fukasawa)
The best, most natural
designs, then, are those
that match the behavior
of the system to the
gesture humans might
already do to enable
that behavior.
Thanks.
http://www.kickerstudio.com
http://www.designinggesturalinterfaces.com
dan@kickerstudio.com
odannyboy on Twitter

Tap is the New Click