Tap is the New Click

Tap is the New Click Dan Saffer, Kicker Studio

DRC 2009 // Dan Saffer, Kicker Studio

We're using bodies evolved for hunting, gathering, and gratuitous violence for information- age tasks like word processing and spreadsheet tweaking. —David Liddle

We’re in the midst of an interaction design revolution.

How do we design for interactive gestures?

What we’re going to talk about Sensors and touchscreen types Kinesiology and physiology Touch targets Communicating Choosing appropriate gestures Case study: Canesta Entertainment Center

Gesture: any physical movement that can be sensed and responded to by a digital system without the aid of a traditional input device such as a mouse or stylus.

Two types of interactive gestures Touchscreen aka TUI Single and multi-touch (MT) Free-form Wide variety of forms

Why not to have a gestural interface Heavy data input Relies heavily on the visual (for now) Can be inappropriate for context More physically demanding

Why have a gestural interface? More ﬂexible Less visible hardware Hardware ﬁts context better More “natural” More fun

The secret sauce: sensors

Common sensors Pressure Light Proximity Acoustic Tilt Motion Orientation

Types of touchscreens Resistive: pressing two layers together creates the touch event Surface wave: finger disrupts ultrasonic waves Capacitive: finger conducts electricity Infrared: finger breaks grid of infrared beams Camera-based: looks for “blobs.” Rear- and front- projectors

Kinesiology & physiology

The ergonomics of human gestures Avoid hyperextension or extreme stretches Avoid repetition Utilize relaxed, neutral positions Avoid staying in a static position No “Gorilla Arm”

Gorilla arm Humans not designed to hold their arms in front of their faces, making small gestures Ok for short-term use, not so much for repeated, long-term use Fun Fact: Telegraph operators had “glass arm” Sorry, Minority Report-style UIs

Gorilla arm

Stephan Pheasant’s (via Rob Tannen) cardinal rules of anthropometrics Reach Clearence Posture Strength

The more challenging and complicated the gesture, the fewer people who will be able to perform it.

What about accessibility? No good, clear answer Improving via addition of haptics (and hopefully, eventually, speech) Some touchscreen systems much better than traditional WIMP systems Special care when designing touch targets

8-10mm 10-14mm 16-20mm

Fingers Fingernails: Blessing Wrist support and curse Gloves Fake ﬁngernails: evil Inaccurate (when Finger oil compared to a cursor) Fingerprints Attached to a hand aka Screen Coverage (Left) Handedness

Avoid putting essential features or information like a label below an interface element that can be touched, as it may become hidden by the user’s own hand.

Touch events and targets

Touch target size Remember Fitts’ Law! (Time it takes to get to a target = distance to target / size of target) As close to the user as possible to avoid users’ covering the screen with their hands Space between the targets (when possible) Create reasonably-sized targets: no smaller than 1cm in diameter/square (the size of ﬁnger pads)

Touch target size comparisons ~25mm ~18mm ~13mm ~8mm 5mm

Two touch target tricks Iceberg tips Adaptive targets

Traditional UI elements to watch out for Cursors MouseOvers and hovers Double-click Right-click Selected default buttons Undo

Touchscreen patterns

Tap to open/activate

Tap to select

Drag to move object

Slide to scroll

Spin to scroll

Flick to nudge

Pinch to shrink & Spread to enlarge

Ghost fingers

Freeform patterns

Proximity activates/ deactivates

Move body to activate

Point to select/activate

Wave to activate

Rotate to change state

Step to activate

Shake to change state

Prototyping gestures

Low-fidelity: Paper prototype

Low-fidelity: The “man behind the curtain”

Low-fidelity: Environments

High-fidelity: Exact

High-fidelity: Off-the-Shelf

High-fidelity: Do It Yourself

Turning gestures into code Variables: what are you measuring? Data: get the data in from the sensor Computation: determine difference between data Patterns: what do the sums mean? Action: if a pattern is matched, do something

Documenting gestures

Dance notation

Annotated wireframes still work

Architectural wireframes touchscreenoverview “Master UI” Run by presenter “Individual UI” Used by show attendees Live Touchscreen Projection Area [floor] [floor] [ showing typical arm’s reach for 6’ tall user ] [ showing typical arm’s reach for 6’ tall user ]

Keyframes

Gestural modules

Storyboards

Swim lanes framework

Animation

Movies

Communicating interactive gestures

Three zones of engagement

Attraction

Observation

Interaction

Attraction affordance

Written instruction

Illustration

Demonstration

Symbolic

Determining the appropriate gesture

Four part equation 1. The task that needs to be performed 2. The available sensors and input devices 3. The physiology of the human body 4. The context This can be pretty straightforward Or not

Context Different behaviors in different locations Avoiding accidental emotional weight Cultural issues

Usability issues Avoid unintentional triggers via everyday actions! Wide variation in performing gestures: need requisite variety Pick one: select then action, or selecting does action Gestures as command keys: Provide a normal means of performing the action (buttons, etc.) but have “advanced” gestures as shortcuts

Case study: Canesta Entertainment Center

The complexity of the gesture should match the complexity of the task at hand.

The best designs are those that “dissolve into behavior.” (Naoto Fukasawa)

The best, most natural designs, then, are those that match the behavior of the system to the gesture humans might already do to enable that behavior.

Thanks. http://www.kickerstudio.com http://www.designinggesturalinterfaces.com dan@kickerstudio.com odannyboy on Twitter

Tap is the New Click

1 comments

Notes on slide 1

12 Favorites