Workshop SXSW 2017 - Iskander Smit - The Shape of New Things

ThingsCon Salon SXSW
‘New Type of Things’
@NewDutchWave, March 12, 2017

Introducing ThingsCon (5’)
What is new in the new type of things? (45’)
presentation
discussion
New possibilities (45’)
exploration in groups
share insights
Human centric & responsible IoT (45’)
reflection in groups
share insights
Program
this Salon

Acknowledgements
sources &
inspiration
Research paper Nazli Cila: Products As Agents, Metaphors for designing
the products of the IoT age
PhD research Gijs Huisman on Social Touch Technology
Tom Coates: essay The Shape of Things
Just Things foundation, The IoT Manifesto

We craft digital products
that people love to use

making IoT work
connectable .io

2014. The Internet of Xmas Gifts

2016. Embodiment of new possibilities

A new type of Things and our intelligent future
Hardware as a Platform

An example; a new bbq experience
Products become systems

Master of the party
occasionally

26
(ecosystem of) things
data
activities while using

Input
Interface
Measure height
Well done
Raw
Measure temperature
Computation
Position
Energy =
Time x Heat x Height
Measure temperature
31

Hardware as a
platform
Conversations with
the machines
Beyond screen
interactions
Context aware,
rule-based
New type of things

the software layer is deﬁning the experience
In the product system

Rethink products
Cloudwash, BERG

embodiment allows a dual use in representing the digital model and allowing control of the digital representation
the next section, the model of TUI is introduced in comparison with GUI to illustrate this mechanism.
Basic Model of Tangible User Interface
The interface between people and digital information requires two key components; input and output, or control a
representation. ontrols enable users to manipulate the information, while external representations are perceive
with the human senses. Fig. 1 illustrates this simple model of a user interface consisting of control, representatio
and information.
In the Smalltalk-80 programming language (Burbeck, 1992; Goldberg, 1984), the relationship between these
components is illustrated by the "model-view-controller" or "MVC" archetype – which has become a basic interac
model for GUIs.
Drawing from the MVC approach, we have develope
an interaction model for both GUI and TUI. We carr
over the "control" element from MVC, while dividing
"view" element into two subcomponents: tangible an
intangible representations, and renaming "model" a
"digital information" to generalize this framework to
illustrate the difference between GUI and TUI.
In Computer Science, the term "representation" ofte
relates to the programs and data structures serving
the computer's internal representation (or model) of
information. In this article, the meaning of
"representation" centers upon external representatio
– the external manifestations of information in fashio
directly perceivable by the human senses that includ
visual, hearing and tactile senses.
GUI
In 1981, the Xerox Star workstation set the stage for the first generation of GUI (Johnson, et al., 1989; Smith, 19
establishing the "desktop metaphor" which simulates a desktop on a bit-mapped screen. The Star workstation w
the first commercial system that demonstrated the power of a mouse, windows, icons, property sheets, and
Fig. 1 User Interface
The interface between people and digital information
requires two key components: 1) external representation (or
view) that users can perceive, and 2) control with which
users can manipulate the representation.
digital information
control representation
input output

In the Smalltalk-80 programming language (Burbeck, 1992; Goldberg, 1984), the relationship between these
components is illustrated by the "model-view-controller" or "MVC" archetype – which has become a basic interaction
model for GUIs.
Drawing from the MVC approach, we have developed
an interaction model for both GUI and TUI. We carry
over the "control" element from MVC, while dividing the
"view" element into two subcomponents: tangible and
intangible representations, and renaming "model" as
"digital information" to generalize this framework to
illustrate the difference between GUI and TUI.
In Computer Science, the term "representation" often
relates to the programs and data structures serving as
the computer's internal representation (or model) of
information. In this article, the meaning of
"representation" centers upon external representations
– the external manifestations of information in fashions
directly perceivable by the human senses that include
visual, hearing and tactile senses.
GUI
In 1981, the Xerox Star workstation set the stage for the first generation of GUI (Johnson, et al., 1989; Smith, 1982),
establishing the "desktop metaphor" which simulates a desktop on a bit-mapped screen. The Star workstation was
modeless interaction. The Star also set several important HCI design principles, such as "seeing and pointing vs.
remembering and typing," and "what you see is what you get (WYSIWYG)." The Apple Macintosh brought this new
style of HCI to the public's attention in 1984, creating a new trend in the personal computer industry. Now, the GUI
is widespread, largely through the pervasiveness of Microsoft
Windows, PDAs, and cellular phones.
GUI uses windows, icons, and menus made of pixels on bit-
mapped displays to visualize information. This is an
intangible representation. GUI pixels are made interactive
through general "remote controllers" such as mice, tablets, or
keyboards. In the pursuit of generality, GUI introduced a
deep separation between the digital (intangible)
representation provided by the bit-mapped display, and the
controls provided by the mouse and keyboard.
Figure 2 illustrates the current GUI paradigm in which
generic input devices allow users to remotely interact with
digital information. Using the metaphor of seashore that
separates a sea of bits from the land of atoms, the digital
information is illustrated at the bottom of the water, and
mouse and screen are above sea level in the physical
Fig. 1 User Interface
The interface between people and digital information
requires two key components: 1) external representation (or
view) that users can perceive, and 2) control with which
digital information
control representation
input output
Fig. 2 Graphical User Interface
GUI represents information with intangible pixels on a
bit mapped display and sound General purpose input
digital information
remote
control
input
output
pixels
sound
physical
digital
intangible
representation
GUI
In 1981, the Xerox Star workstation set the stage for the first generation of GUI (Johnson, et al., 1989; Smith,
establishing the "desktop metaphor" which simulates a desktop on a bit-mapped screen. The Star workstation
modeless interaction. The Star also set several important HCI design principles, such as "seeing and pointing
remembering and typing," and "what you see is what you get (WYSIWYG)." The Apple Macintosh brought th
style of HCI to the public's attention in 1984, creating a new trend in the personal computer industry. Now, the
is widespread, largely through the pervasiveness of Mi
GUI uses windows, icons, and menus made of pixels o
intangible representation. GUI pixels are made interac
through general "remote controllers" such as mice, tab
keyboards. In the pursuit of generality, GUI introduced
representation provided by the bit-mapped display, and
generic input devices allow users to remotely interact
digital information. Using the metaphor of seashore tha
separates a sea of bits from the land of atoms, the digi
digital information
remote
control
input
output
pixels
sound
physical
digital
intangible
representation
Graphical User Interface

and computational models. Urp illustrates examples of such couplings, including the binding of graphical geo
(digital data) to the physical building models, and computational simulations (operations) to the physical wind
Instead of using a GUI mouse to change the location and angle graphical representation of a building model b
pointing, selecting handles and keying in control parameters, an Urp user can grab and move the building mo
change both location and angle.
The tangible representation functions as an interactive physical control. TUI attempts to embody the digital
information in physical form, maximizing the directness of information by coupling manipulation to the underly
computation. Through physically manipulating the tangible representations, the digital representation is altere
Urp, changing the position and orientation of the building models influences the shadow simulation, and the
orientation of the "wind tool" adjusts the simulated wind direction.
Intangible Representation
Although the tangible representation allows th
physical embodiment to be directly coupled to
information, it has limited ability to represent c
many material or physical properties. Unlike
malleable pixels on the computer screen, it is
hard to change a physical object in its form, p
or properties (e.g. color, size) in real-time. In
comparison with malleable "bits," "atoms" are
extremely rigid, taking up mass and space.
To complement this limitation of rigid "atoms,"
also utilizes malleable representations such a
projections and sounds to accompany the tan
representations in the same space to give dyn
expression of the underlying digital informatio
computation. In the Urp, the digital shadow th
accompanies the physical building models is s
example.
The success of a TUI often relies on a balanc
strong perceptual coupling between the tangib
intangible representations. It is critical that bo
tangible and intangible representations be
perceptually coupled to achieve a seamless in
that actively mediates interaction with the und
Fig. 3 Tangible User Interface
By giving tangible (physical) representation to the digital
information, TUI makes information directly graspable and
manipulable with haptic feedback. Intangible representation (e.g.
video projection) may complement tangible representation by
synchronizing with it.
digital information
output
physical
digital
e.g. video projection
of digital shadow
e.g. building model
Input/
output
control
tangible
representation
intangible
representation
modeless interaction. The Star also set several important HCI design principles, such as "seeing and pointing vs.
remembering and typing," and "what you see is what you get (WYSIWYG)." The Apple Macintosh brought this new
style of HCI to the public's attention in 1984, creating a new trend in the personal computer industry. Now, the GUI
is widespread, largely through the pervasiveness of Microsoft
GUI uses windows, icons, and menus made of pixels on bit-
intangible representation. GUI pixels are made interactive
through general "remote controllers" such as mice, tablets, or
keyboards. In the pursuit of generality, GUI introduced a
representation provided by the bit-mapped display, and the
generic input devices allow users to remotely interact with
digital information. Using the metaphor of seashore that
separates a sea of bits from the land of atoms, the digital
digital information
remote
control
input
output
pixels
sound
physical
digital
intangible
representation
Tangible User Interface
Tangible User Interface, Hiroshi Ishii, 2006

Invisible design
Clair Rowland, Design for Connected Objects

Push the power of the service layer
far beyond where it is now
Tom Coates, 2016

Dialogues
Talking Trainers, Google + Adidas

Objects that are aware, conscious, stubborn…
Google Home

The Collector The Actor The Creator
THREE METAPHORS OF PRODUCT AGENCY
Products as Agents
deﬁning the roles of
products with their
users
Products As Agents; Nazli Cila, Elisa Guiccardi, Iskander Smit, Ben Kröse; 2016

The collector
Growﬁcient, high precision agriculture

The Actor
Addicted Toaster (Simone Rebaudengo & Usman Haque)

Spimes, objects that nudge the user
Shaping Things, Bruce Sterling

The Creator
Starﬁsh self-modeling robot

Issues of delegation
Uninvited guests, Superﬂux

human computer
integration
human computer
interaction

Interact everywhere, with everything
Hyper-reality, 2016

New interactions
Project Jacquard

Haptics embedded
Apple Taptic Engine

Social Touch Technology
2017 Gijs Huisman

Touch, care and well being
Social Touch Technology, PhD Gijs Huisman

Anger 59* Hitting,
squeezing,Fear 51* Trembling,
squeezing,Happiness 38 Swinging,
shaking, liftingSadness 35 Stroking,
squeezing,Disgust 83* Pushing, lifting,
tappingSurprise 24 Squeezing,
lifting, shakingEmbarrassment 18 Shaking,
tapping,Envy 21 Pulling, lifting,
strokingPride 25 Shaking, lifting,
squeezingLove 62* Stroking, ﬁnger
interlock,Gratitude 66* Shaking, lifting,
squeezingSympathy 57* Patting,
stroking,
Hertenstein et al. (2006)

with haptics embedded
Impuls driven things

A dialgue made tangible
Steward by Felix Ros

Scripting
adaptive
moments 
as the
new
interface

… worship the moment,
design the rules

The dominant
interaction paradigm
will be trigger based dialogues

Haptics enhances these dialogues

Dialogues
James Bridle, Surveillance Spaulder

1
just
things
JUST THINGS FOUNDATION
“… increase the
awareness about ethical
dilemmas in the
development of internet
connected products and
services”

just
things
Peak of inﬂated expectations
Technology trigger
Trough of Disillusionment
Plateau of Productivity
Slope of
enlightenment

just
things
Concluding
We pledge to be skeptical of the cult of the
new — just slapping the Internet onto a
product isn’t the answer. Monetizing only
through connectivity rarely guarantees
sustainable commercial success.
The world is becoming increasingly connected. This
offers opportunities for designers, engineers and
entrepreneurs to create unprecedented products
and services. Yet, a connected world also brings new
questions and challenges to the table.
This manifesto serves as a code of conduct for
everyone involved in developing the Internet of
Things, outlining 10 principles to help create
balanced and honest products in a burgeoning ﬁeld
with many unknowns.
WE DON’T BELIEVE
THE HYPE
I
With connectivity comes the potential for
external security threats executed through
the product itself, which comes with serious
WE KEEP EVERYONE
AND EVERY THING
SECURE
IV
A complex web of stakeholders is forming
around IoT products: from users, to
WE AIM FOR THE
WIN-WIN-WIN
III
Value comes from products that are
purposeful. Our commitment is to design
WE DESIGN
USEFUL THINGS
II
IOT DESIGN MANIFESTOFirst drafted by a number of design professionals,
this manifesto is intended to be a living document
that the larger community of peers working within
the IoT ﬁeld can contribute to and improve upon.
This manifesto is a living document, we seek your
input to help it grow. Please discuss, contribute,
remix, and test the boundaries of these principles.
www.iotmanifesto.org
v1.0 · May 2015

New collabs
BBQ as platform product
+ ?

1.Heinz creates a special line of sauces
that leverage the bbq
2.Masterchef runs an on demand show
around the bbq
3.The city of Austin creates a special
edition of the BBQ
4.‘Elderly care’ runs a program for
alzheimer phase 1 people
Briefs, choose one
+ ?

Introductions!
• Who?
• What?
• Why?
• Favourite BBQ food?

Make team (3 ppl) & define roles
UX Designer
Design a seamless UX and
express the information
needed from the user to
make it possible.
Data scientist
Define what data is
needed to improve this
and future product
performance
Product manager
Define ways to add as much
value as possible at lowest
cost possible.

Debrief: Make it your own (5’)
• How can we create / design / improve
• a …….………..
• for …….………..
• to …….………..
smart waste station
large family household
sell your waste

Expand the concept (5’)
• Take your role’s goal to the furthest extend
• It can be “evil” - there are no no no limits!
• Draw it! (advertisement poster)

Concept
DesignVerify
Implement
IOT DESIGN MANIFESTO CHEATSHEET
This is the IoT Design Manifesto cheatsheet. It aims to make the principles of the manifesto
actionable.
How to use this?The principles of the manifesto work on different abstraction levels. So when you design a
product, you would address conceptual issues ﬁrst, and become more speciﬁc toward
implementation. Then iterate once more to account for interrelated issues.
1. Concept What is the raison-d’être ? Why is it connected? What value do we create?
2. Design How should it work? How would people interact? How would it show…
3. Implementation What do we need to develop? How do we account for privacy?
I. WE DON’T BELIEVE THE HYPEImagine, your product would be advertised
without any mentioning of connectedness,
data, the internet or smartness. What
would it be that would trigger a customer?
II. WE DESIGN USEFUL THINGSThis is basically a challenge of good design.
What kind of untapped potential is there in
this product? What would’ve been
impossible 10 years ago, but would now
suddenly be possible? What needs can the
product cater for that it couldn’t do before?
III. WE AIM FOR THE WIN-WIN-WINIoT products are connected. Not only to the
web or a service, but through that service to
anyone involved in creating it.Who is involved with the product, and what
is to gain from being connected? And if one
stakeholder clearly wins, how’s that of value
to the other stakeholders?
IV. WE KEEP EVERYONE AND EVERY THINGSECUREWhat are the scenarios you can think of,
where security is at stake. And what are the
potential points where security can be
breached? On product level, service level?
Or does your product put other products
around it at risk?V. WE BUILD AND PROMOTE A CULTUREOF PRIVACYThis is an organisational issue. Privacy is a
complex matter and you need to align
everyone working on a product or service
to have a common notion and policy
relevant to the context of your business.
When drafting your policy, try to ‘be’ your
customer, and push for extreme scenarios
like company acquisitions, security
breaches, partner company bankruptcies,
potential outsourcing of processing and
storing data, etc. etc.
VI. WE ARE DELIBERATE ABOUT WHATDATA WE COLLECTWhat is the minimal amount of data we
need to process to make this product work?
And what data could be of use for the
current user, future users, or future
versions of a product? How could a product
become a better version of themselves?
How could other products work better
through data from your product?
If any data stream doesn’t server any of
these purposes, why bother to collect or
store it?
VII. WE MAKE THE PARTIES ASSOCIATED
WITH AN IOT PRODUCT EXPLICITHow will your user know who is involved
with your product? When, in the process of
bringing this product into her/his life, will
she/he understand that the product is an
element of a greater network with more
parties involved?
VIII. WE EMPOWER USERS TO BE THE
MASTERS OF TH

Discuss
• What happened?
• How was thinking on this new type of platform things?
• How did the manifesto help / obstruct / inﬂuence your group?

Thanks!
Iskander Smit, @iskandr

Workshop SXSW 2017 - Iskander Smit - The Shape of New Things

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Workshop SXSW 2017 - Iskander Smit - The Shape of New Things

Similar to Workshop SXSW 2017 - Iskander Smit - The Shape of New Things (20)

More from Iskander Smit

More from Iskander Smit (20)

Recently uploaded

Recently uploaded (20)

Workshop SXSW 2017 - Iskander Smit - The Shape of New Things