In the same way as the web is quickly extending onto the mobile platform, we are starting to see the web moving further into the physical world. Many emerging technologies are beginning to offer physical-world inputs and outputs; multi-touch iPhones, gestural Wii controllers, RFID-driven museum interfaces, QR-coded magazines and GPS-enabled mobile phones.
These technologies have been used to create very useful services that interact with the web such as Plazes, Nokia Sports Tracker, Wattson, Tikitag and Nike Plus. But the technologies themselves often overshadow the user-experience and so far designers haven’t had language or patterns to express new ideas for these interfaces.
This talk will focus on a number of design directions for new physical interfaces. We will discuss various ideas around presence, location, context awareness, peripheral interaction as well as haptics and tangible interfaces. How do these interactions work with the web? What are the potentials and problems, and what kinds of design approaches are needed?
The web in the world
Timo Arnall elasticspace.com
I do design and research, based at the Oslo School of Architecture & Design, where I run a design
research project that looks at emerging technologies.
We work in interaction and industrial design mainly,
so we are in a good position to be thinking about the current shift in technology,
from screen-based interaction towards physical interaction with the world around us.
So what do I mean by The web in the world?
When we think of the mobile web many of us think about small screen rendering of web pages.
And this is clearly happening, the web is moving towards an enormous number of tiny screens.
And it’s leaking out into a lot of other devices.
This is the result of having Twitter sending out SMSes for a few days.
There are of course a lot of subtler ways that the web is ﬁltering out into the world.
The iPhone is also clearly making a platform for the mobile web, with it’s assumed always-on network
Applications are starting to take advantage of this always-on-ness and to make use of some of it’s
But we are working towards a ‘ubiquitous web’ has greater implications than everyone constantly
engaged in tiny-screen interaction.
In fact I think we should set out to NOT create an overload of this interaction.
The ‘internet of things’ may instead increase awareness of the physical and social environment, and to
do more, with less screen-time.
Marc Smith of Microsoft Research said last week at the Social Computing Symposium that:
“Most digital interaction takes place in another interaction order – in another space – not in the
physical interaction order of co-presence.”
“This is about to change...”
So digital interaction takes place, mentally and physically, somewhere else. We go onto the internet...
And what is enabling this to happen?
Spimes (Bruce Sterling)
Data shadows (Mike Kuniavsky)
The Internet of things...
Things & links
So I’ve put together a number of ‘patterns’ that show some issues and directions for the future web in
Then a few design directions that are needed to make these happen.
What is the basic building block of an internet of things?
Well according to many, it is the humble link.
If we go right back to basics, we can see that URLs have been inhabiting the physical world for a few
From gra!ti artists to large companies, the url has become a standard part of public marketing and
And an ever increasing number of objects are used to attempt to link people back into the web.
But the interaction experience for this is not yet well deﬁned.
Shortcodes for instance are quite successful, short text input yields richer information sent back,
usually via a URL.
So many projects and platforms are setting out to ﬁnd other ways of linking the physical to the digital.
Yellow arrow used a kind of shortcode, or unique ID for places, and encouraged storytelling (didn’t
really work on a wide scale)
Thinglink attempts to create unique identiﬁers for physical products. They have a lovely idea about
creating greater visibility for the long tail.
So there are many incompatible barcode systems that have emerged, some like QR coding are
designed for linking, particularly on the mobile phone.
But experiences of barcoding stuquot; beyond grocery goods haven’t gone very well.
This is the Cuecat, the massive fail project that partnered with Wired.
This is one ‘unique identiﬁer’ project that I really like, made by Maya Lotan at Ivrea.
It has unique and decorative patterns that can be converted through image recognition into a unique
ID. She designed a range of products, from clothes, to accessories to paper material that contained the
I really like the integration with fashion.
This is another way of linking, a box that asks you to turn on bluetooth and lets you download
The weirdness with this is that it’s di!cult to tell where the bluetooth begins and ends.
It’s not ‘direct manipulation’.
Near Field Communication is perhaps the most advanced kind of object linking. It uses battery-less
RFID chips that store and transfer data.
The idea is that we’ll use phones for ticketing, payment, and for discovering ‘services’ in the physical
Raﬁ probably showed Mir:ror yesterday.
It uses the same RFID technology
But what is required is a more nuanced vision of the physical link. I think the hyperlink is a ﬂawed
model for physical interaction.
Yes it allows for objects to communicate, for things to be tracked as part of logistics systems.
But does it really make sense from the user perspective?
What is the user-experience? What is the context? What is useful or playful in this interaction?
We have also created products that use physical objects to trigger interactions with media.
These wooden objects have embedded RFID chips, that create reactions when they are placed inside
We really tried to concentrate on objects and interfaces that would ﬁt in the home environment (on the
and would allow for subtle and tangible interactions with media (particularly with kids)
Which leads us to...
What I think we need to be thinking about from a user-perspective is tangible interaction.
This is the use of physical objects and gestures to control interactions.
The aim is to use more senses and more of the body in our interactions.
Eyetoy or sixaxis controller.
GPS in, in lots of things, cars, mobiles.
I really hate GPS, as someone who lives most of their lives walking in dense cities, it doesn’t work
But lets not overlook the humble microphone...
The games produced that include sound and breath input for the DS are astounding.
And of course, tangible interaction with objects is perfect for games and toys, this is where it is
starting to emerge.
I use the word ambient loosely (there is a whole research equot;ort on ‘ambient computing’) but I think the
concept is interesting.
Instead of the focused, attention grabbing interaction with the web, what if we could use more of our
senses to get information:
Peripheral, background, sound, haptic
Perhaps the quintessential ambient displays are made by “Ambient Devices”
This is their orb, that can be set up to display a few variables of data, through brightness and colour.
Their HomeJoule device also oquot;ers background information in speciﬁc places.
To do this they have set up their own network in the US.
Their ‘dashboard’ oquot;ers lovely analogue pointers that display information.
This is the kind of display that can be glanced, or is easily understood without direct attention.
One really lovely thing about this dashboard is the custom plates that allow customisation of the
displays. Not only does this change the functionality, it changes the scale or the data that is presented.
There are a lot more products on the horizon that oquot;er some sense of ambient output. The chumby is
equot;ectively ‘widgets for physical space’.
Photo CC by dalager:
Sound is a critical aspect of interfaces that inhabit the world with us
And ﬁnally, haptics.
The use of vibration to indicate interaction output.
This is a lovely use of haptic feedback, Chris Woebken at the RCA made a simple digital compass that
vibrated whenever it pointed north. People who wore this apparatus gained a new sense of direction.
Part of the animal superpowers project by Chris Woebken
In our Touch project one of my students created ‘Sniquot; the dog’ that sniquot;s RFID tags and gives sound
and vibration feedback.
This is a lovely explorative toy where the sound and vibration adds up to create emotional responses
to the environment.
I use the word context rather than location. It’s more general, and has more interesting connotations.
Context is hard. We also have a whole research domain called ‘context aware computing’ that has been
for decades looking for ways of making devices respond smarter to their surroundings.
Also, contextual design practices will have to be brought to the forefront.
But one deﬁnition of context that is useful right now is:
Devices that are aware of other devices...
Plazes turns the proximity to wiﬁ routers into one way of determining simple context
*The use of ﬁelds to determine context*
Founder says that it was important for Plazes location lookups to be implicit in the use of mobile
devices and laptops, not explicitly chosen by the user (but with user control over privacy)
Simply mapping devices creates a lot of rich data.
This is the ‘Mapping Graz’ project by MIT
And this data can be turned back into interface
Here is ‘Citysense’ allowing people to visualise the number of people (cellphones) around them
The other thing that is clearly happening is visualising the geo-located data from the web, in place.
I’m skeptical about Augmented Reality, particularly the goggles/glasses variety but this is quite a
Here is one piece of evidence, ‘The future of internet search’ by Mac Funamizu
He has successfully visualised ideas about using data in the physical world
An interface that adds to physical experience rather than making us withdraw to small screens.
Visibility is one of the biggest issues for ‘ubiquitous computing’.
And Donald Norman is taking those ideas further.
There is an equot;ort towards embedding computing in everything, and making it ‘disappear’.
But the consequences of that from a user-perspective have not been fully thought through.
Every modern electronic device we carry has some sort of invisible interaciton with the world around it,
through various ‘bubbles of radio’ of varying sizes.
RFID in particular has been a short-fuse of controversy around ubiquitous computing.
This is both because it can be invisible (Embedded)
And because it has features that are not visible (Radio)
These radio bubbles surround us most of the time.
3G coverage in Stavanger, Norway
This stuquot; is starting to have an impact on our everyday lives.
It’s experienced as dropped connections, as unreliability, as slowdowns.
The mapping of this stuquot; is important.
Skyhook wireless is mapping out these ﬁelds, and turning them into material we can use in
Their coverage can be visualised on their website
How will you explain to users that their location lookups depend on a number of bubbles of radio
Or how will they know that they are in a GPS shadow?
This is my student’s work on visualising radio ﬁelds, she imagined ﬁctional ‘species’ of radio waves
that she turned into an encyclopaedia.
This is some work I did with Adam Greenﬁeld towards making a set of symbols for his book
We attempted to explain various elements of ubiquitous experience through simple symbols.
Work in progress.
A lot of these technologies lend themselves to tracking and tracing.
So something has emerged in what Matt Jones and Tom Coates are calling “Personal Informatics”, “the
ability to leverage data about yourself and your friends to your advantage”
The Nike+ is a fascinating example of new forms of entertainment
It is designed, a physical object, using high technology.
But the connection to the web is a critical part of the experience.
The sharing and reﬂection on data is a large part of the experience
Nokia Sports Tracker:
It’s the same with the Wattson from DIYKyoto.
Plugging this thing in for the ﬁrst time is something that gets Flickr’ed over and over.
And the resulting data visualisations are then excellent ‘social objects’ with which to compare and
If the mobile phone and internet is compressing time and space,
any move back towards the interaction with the physical world re-introduces scarcity and distance.
This is what I got when I loaded Geode for the ﬁrst time.
A total lack of content.
Not having an active community near you means that many services will only work in certain
If we have ‘smart’ posters oquot;ering content, is this the only place that it is available, can I take the
content with me and access it again?
These are questions that need to be answered.
Poken is a physical (rﬁd based) object that lets you connect your social networks in physical space, by
touching them together you create a connection. It is open-API based.
underestimating the di!culty of achieving critical mass
or: overestimating their ability to aquot;ect culture
There is an enormous equot;ort to reach the critical point at which cluster equot;ects can take place, and many
users will feel the full brunt of the ‘ﬁrst fax machine’ problem.
Which also leads to adaptability.
The mouse, keyboard and screen is a wonderfully adaptable interface, so much so that it has stuck with
us for over 20 years.
And the web is a fantastically adaptive environment, built out of layers of generalised code (material).
But physical and tangible interfaces tend to be much more specialised.
In particular the economics of adaptability.
This is a toy called Swinxs, it connects to the web and allows lots of games to be played, but the
interactions are limited to touching this console.
Suitable for toys and museums, and that is exactly where we are seeing the ﬁrst of these kinds of
So there are modular approaches to hardware, which should provide some means of getting some of
the adaptable features of the web onto into the physical world.
This is the Olinda radio from Schulze & Webb.
It goes some of the way towards creating an adaptive, social piece of hardware, that mirrors the web.
They even made a hardware API.
Matt Webb says that “Some things are di!cult to do in social software: quot;But these things I get for free
by having them in a physical objectquot; eg quot;Storing my identity in an object, and gifting it to someonequot;.”
So what does this mean for design practice?
The good news is that this stuquot; is already emerging from the web. Web practices are very applicable,
dealing with networks, connections, social media, etc.
Schulze & Webb for example are actively encouraging web-practices in physical product design...
But this stuquot; is di!cult, it needs to be platformed out so that practices, methods and data can be
http://www.pachube.com oquot;ers a wide platform for starting some of these linkages.
This stuquot; is di!cult, even compared with the web.
Arduino and other hardware prototyping platforms make it easier.
This is something that Mike Kuniavsky talks about: Sketching in hardware, the ability to use relatively
simple hardware tools to make and revise quick prototypes.
We could also call it rapid prototyping.
Both this and the last image are the iterations towards a ﬁnished “Sniquot;”.
The challenges come in making physical objects which are robust, which have the qualities of good
products, and work interactively.
This requires lots of skills, from interaction design to software and industrial design.
Some of this is made easier through visualisation.
Visualisation skills become important in actually uncovering the materials that we are working with.
This is a series of sketches that work towards understanding how RFID works.
We wanted to sketch out the aquot;ordances of RFID:
quot;the perceived and actual properties of the thing, primarily those fundamental properties that
determine just how the thing could...”
“A quality of an object, or an environment, that allows an individual to perform an action.”
And we ended up with a diagram that explains the important aspects of RFID for designers.
We can also start to explain some of the invisible aspects of RFID, here we are looking at the ﬁelds that
surround an RFID object.
Evidence: create the stuquot; that we will all talk about, and can either agree or disagree about.
This is from Urban Seeder by Maya Lotan.
Fiction is really important, especially when the
Emerging themes Design approaches
Things linked Sketching
Tangible in and out Visualisation
Ambient and haptics Fiction