The Mobile Frontier
Rachel Hinman
Senior Research Scientist
Nokia Research Lab
Palo Alto, California USA
Hinman

2
I used to be a web designer, too!

Mobile has arrived…

Mobile = Wild West

Even in situations in which a spirit of
exploration and freedom exist, where faculty
“Hey, I’ve got this great idea for an app…”
are free to experiment to work beyond physical
and social constraints, our cognitive habits
often get in the way.
Marshall McLuhan called it “the rear-view
mirror effect,” noting that “We see the world
through a rear-view mirror. We march
backwards into the future.”!

The Rearview Mirror Effect

Even in in situations in whichaaspirit of
Even situations in which spirit of
exploration and freedom exist, where we are
exploration and freedom exist, where faculty
free to experiment and to work beyond physical
are free to experiment work beyond physical
and social constraints,
and social constraints, our cognitive habits
our cognitive habits often get in the way.
often get in the way.
Marshall McLuhan called called it
Marshall McLuhan it “the rear-view
“the rear-viewnoting that “We noting that
mirror effect,” mirror effect,” see the world
“We see the world through a rear-view mirror.
through a rear-view mirror. We march
We march backwards intofuture.”!
backwards into the the future.”

Mobile presents an
opportunity to invent
new ways for users to
interact with information.
11!

Where to Look?

Emergent Mobile Topics
1 Shapeshifting
2 A Brave NUI World
3 Comfortable Computing

Emergent Mobile Topics
1 Shapeshifting
2 A Brave NUI World
3 Comfortable Computing

Shapeshifting

18!

“We made the web
in the image of
ourselves, and in
the U.S., that
means OBESE.”
~Jason Grigsby

A!
20!

21!

Q:
“I think a lot of web design in the
past five to ten years is actually
Q:!
just a reflection of print again.
We’ve been designing for one
context, which isn’t much better
than a PDF document.”
~ Bryan Rieger
22!

Q: “Our obsession with layout
(carried over from print design)
Q:!
keeps us from seizing much
bigger opportunities on mobile.”
~ Luke Wroblewski
23!

24!

Shapeshifting

Plans & Situated Action
Mutual Reconfiguration

Q: Suchman’s theory of mutual
reconfiguration suggests a person’s
Q:!
capacity to act (their agency) is
reconfigured when it comes into
contact with another thing or person -
that human action is constantly
constructed and reconstructed
from dynamic interactions with the
material and social worlds.

30

Two
Q:
Q:!
Design Implications

Q:
Q:!
1. Shift in contextual assumptions

Q:
Q:!
2. Shift in our sensibilities
around content as a design material

Q:
Q:!
1. Shift in contextual assumptions

A!

A!
Throughout the design process,
our work should be situated in
the contexts where it will be used.

Go to the Gemba

Go to the Gemba

Go to the Gemba

Q:
Q:!
2. Shift in our sensibilities
around content as a design material

Design as the manipulation of materials.

Q:
Q:!
Pages and screens
are not our design material

Content as a design material.

“Right now many designers end up
focusing a lot of energy on the overlying
Q:
view – the whole template and look and
feel. But the smaller the screen gets, the
less that stuff matters.
Q:!
One of the most important things we’ve
started looking at is writing and working
with APIs rather than designing ‘pages’
for one context.”
~Stephanie Rieger

“For so much digital content, there is
no good metaphor to render anymore
– the content is just information, text
and images – so new approaches to
Q:!
interaction and visual UI design are
needed.”
~Mike Kruzeniski

How?

2
steps

1.

A!
Think of ways to integrate context
considerations into your design
process.

Ideate in the wild…
62!
Text entry will never be easy!

Test prototypes with users
early and often

Modifications to your design
process will be necessary.

2.

Lead with the content

?

Emergent Mobile Topics
1 Shapeshifting
2 A Brave NUI World
3 Comfortable Computing

71!

Seated in a relatively predictable environment
Large screen enables multi-tasking
Keyboard and a mouse for input
72!

We’re reaching the edges
of what GUI can do
73!

74!

GUIs become brittle
on a mobile device
75!

78!

79!

Key differences between
NUI and GUI

Defining Attributes of
GUIs…

GUI Mental Model =
Computer as tool
82!

GUI = Recognition
“What you see is what you get”
83!

GUI = Metaphorics, containment and place
84!

GUIs = Heavy Chrome, Icons & Buttons
85!

Defining Attributes of
NUIs…

NUI Mental Model =
Computer as media
87!

NUI = Intuition
“What you do is what you get”
88!

NUI = Fluid, Unmediated, and Organic
89!

NUIs = Content is the Star
90!

GUI = Experiences are anchored
91!

NUI = Experiences Unfold
92!

NUIs Unfold Like a Game
93!

NUIs Can Feel Anchor-less
94!

95

Mobile Experiences Unfold
Patterns for how mobile experiences unfold
and progressively reveal their nature
1 The Nested Doll Pattern
2 The Hub and Spoke Pattern
3 The Bento Box Pattern
4 The Filtered View Pattern

Nested Doll Pattern
97!

Hub and Spoke Pattern
98!

Bento Box Pattern
99!

Filtered View Pattern
100!

Mobile Experiences Unfold
Patterns for how mobile experiences unfold
and progressively reveal their nature
1 The Nested Doll Pattern
2 The Hub and Spoke Pattern
3 The Bento Box Pattern
4 The Filtered View Pattern

A!
Trick to unfolding =
Information boulders to pebbles
102!

A!
103

A!
104

Your Design Challenge!
From GUI to NUI
105!

Your Design Challenge!
From GUI to NUI
1 Identify an interaction sequence you’d like
to recreate using NUI principles.
2 Sketch out the interaction using the
templates provided.
3 Identify how you’d like your mobile
experience to unfold.
106!

107!

108!

REMEMBER!
It’s easy to go crazy and try to do it all.
Instead, focus on applying what we’ve covered…
Understanding the differences between
graphical and natural user interfaces.
Experimenting with how your mobile
experience can unfold and and progressively
reveal its nature.
Play around with the unfolding patterns that
have been presented… or invent some of your
own. 109!

110

Paper In-Screen Prototypes
Following Process documented by Diego Pulido via
UX Magazine – Paper In-Screen Prototypes
Photos courtesy of Diego Pulido and UX Magazine
111!

1. Sketch screen layouts. !
!
!
! !
112
Photo courtesy of Diego Pulido and UX Magazine

1. Sketch screen layouts.
2. Scan or photograph
the sketches.
113
Photo courtesy of Diego Pulido and UX Magazine

1. Sketch screen layouts.
2. Scan or photograph
the sketches.
3. Making sizing
adjustments to the
files.
114
Photo courtesy of Diego Pulido and UX Magazine!

4. Save the resized
images in a file format
supported by the mobile
device. Be mindful of the
sequencing of your
screens and label files
accordingly.
115
Photo courtesy of Diego Pulido and UX Magazine

4. Save the resized
images in a file format
supported by the mobile
device. Be mindful of
the sequencing of your
screens and label files
accordingly.
5. Import the files into
the mobile device’s
photo gallery.
116!
Photo courtesy of Diego Pulido and UX Magazine!

4. Save the resized
images in a file format
supported by the mobile
device. Be mindful of the
sequencing of your
screens and label files
accordingly.
5. Import the files into
the mobile device’s
photo gallery.
6. Click and swipe away.
117!
Photo courtesy of Diego Pulido and UX Magazine
Paper Prototype example!

Q:
Q:!

?

Emergent Mobile Topics
1 Shapeshifting
2 A Brave NUI World
3 Comfortable Computing

Mark Weiser

Q:
Q:!

“The most profound technologies
Q:
are those that disappear.
Q:!
They weave themselves into the
fabric of everyday life until they
are indistinguishable from it.”
~ Mark Weiser
The Computer for the 21st Century

Q:
Q:!
Tab Pad Yard-Sized Board
1991

2010

Mobile devices are
the gateway drug for ubicomp

Following toddlers into the future

“I like cuddling up in bed with
Q:
Netflix on the iPad instead of
sitting in front of the TV. It’s more
Q:!
comfortable to go to bed with a
movie the same way I used to go
to bed with a book”
~ Norbert
iPad Study Participant

“It’s almost like my blankie…
Q:
I curl up in bed with it, or on the
couch. I usually have it with me,
Q:!
regardless of extenuating
circumstances ”
~ Erin
iPad Study Participant

“It’s almost like my blankie…”

Say Goodbye to Done

This should look familiar…
137!

This should look familiar…
The web has evolved around a
task-efficiency model.
Mobile can be different.
138!

Tasks are about completion…
139!

Tasks are about completion…
Possibilities are interactions that
accrue over time…
140!

Tasks are about completion…
Possibilities are interactions that
accrue over time…
… or facilitate
exploration…
141!

Tasks are about completion…
Possibilities are interactions that
accrue over time…
..or are about
… or facilitate SENSING
exploration… INTENT!
142!

143!

144!

145!

146!

147!

148!

Emergent Mobile Topics
1 Shapeshifting
2 A Brave NUI World
3 Comfortable Computing

Mobile presents an
opportunity to invent
new ways for users to
interact with information.
151!

Thank you!
Email:
rachel.hinman@nokia.com
Thanks, and next up…!

Oh! By the way…
I’m writing a book!
Expected Publication:
April 2011
! ! !