How will algorithms and machine learning change interaction design practice? Giles Colborne's slides from Interaction 16 in Helsinki.

1. Image: Akritasa (cc)
Interaction designers
versus algorithms
@gilescolborne

2. No matter how cool
your user interface is,
it would be better
if there were less of it.
Alan Cooper
Algorithms and artificial intelligence give us the power to
simplify interactions.
What does that mean for interaction design practice?

3. Spotify’s Discover
Weekly is one of it’s most
delightful and valuable
features, according to
users I’ve spoken to.
But it’s a playlist. If you
were the interaction
designer, what might
you have contributed?

4. Stuff that the user wants
goes here.
Maybe this sketch?
It looks like the real
work was being done by
the engineer who wrote
the algorithm.
Is that changing the
nature of interaction
design?

5. What about something
like this. I know people
who’ve spent a lot of
time figuring out how to
help users move through
this inofmation.
How might this be
redesigned?

6. Book me an off peak return from
Bath to London for next Tuesday
with a seat reservation on the way
back at 4:30.
Would you like to add a Zone
1-5 Travelcard for £5.80?
Yes plz
That comes to £78.20 including
booking fee. Want to go ahead?
Trainline BookingMenu My tickets
OK
Chatbots can answer the
same questions in a
natural way that feels
familiar to users.
The interaction here is
with the collection of
natural language
algorithms beneath the
hood.

7. Here is exactly
what you need
right now
So now I’m wondering,
how much design will get
displaced by data
scientists and algorithm
engineers.

8. Ariel Luenberger
Let’s imagine we’re
designing for a bus
company.
This chap needs to know
‘where’s my bus?’. How
could an algorithm help?
Well, you need to begin
with data…

9. Imagine we have layers of data. We know when buses were late,
what the weather was like, locations of roadworks, traffic and
so on. We could use it to predict how late your bus will be.

10. Machine learning
is not magic
it’s engineering
Well, if you come up with an idea, you need to know enough
about algorithms to have a sensible conversation with an
engineer. Here are the basics of that conversation…

11. Here’s the task. You have input data (weather, traffic patterns
and so on), an algorithm, and some outputs (is the bus late?).

12. You need to know what kind of output is useful to the user. Is it
enough to say ‘late?’ Or do you need to give a precise delay? More
detailed output means a more complex engineering challenge.

13. The engineer chooses the algorithm and trains it by showing it
sample inputs (weather, traffic, etc.) and known outputs (when
the bus actually arrived) until the algorithm can fit inputs and
ouputs.

14. If your data is inaccurate (for instance the GPS doesn’t work well
in some areas) then your algorithm will learn to make inaccurate
predictions. So you need to be able to judge data quality.

15. If your problem is complex and relies on lots of different data
sets, then you’re going to need more training data. That can be
hard to get hold of. Engineers will get nervous if you keep adding
data sets. So which ones do you really need?

16. High varianceHigh bias
If you don’t have an accurate algorithm, you can at least choose
how to be wrong. Biased consistantly, variable around an
average. In our case it’s better to be biased (towards saying the
bus will be on time) rather than to be right on average.

17. If the data in the layers is unnecessarily complex then the
algorithm may be unreliable, too. So rather than throw raw data
at the algorithm, it’s a good idea to simplify whats in each data
set.

18. Do you need to know precise rainfall times, hour by hour, or just
‘did it rain in the morning’. That affects how much data is in your
data set. Sometimes less data gives better accuracy - like turning
up the contrast on a scanned image of text to make it more legible.

19. At the end of this you’ll have a trained algorithm that delivers the
information you want based on the data you have. But it may still
not be accurate enough. So you’ll need a closed beta or a live
service with a feedback loop to keep up the training.

20. Ariel Luenberger
So we built a prediction
machine. All the way
through there’s a
dialogue between
designer and engineer
about what’s possible
and how to present it.

21. Perhaps as tools and APIs proliferate, designers will take on the
job of training algorithms. But the real place designers add value
is in defining what the outputs should be and how they’re
presented to the user.

22. If you wrap up your
recommendations in
an interface that
promises human-
like interactions
with less than
human manners,
then people will
revolt.

23. Interfaces like this
offer suggestions in
a subtler, less pushy
way. Designing the
etiquette of
suggestions will be
important in next
generation
interaction design.

24. Book me an off peak return from
Bath to London for next Tuesday
with a seat reservation on the way
back at 4:30.
Would you like to add a Zone
1-5 Travelcard for £5.80?
Yes plz
That comes to £78.20 including
booking fee. Want to go ahead?
Trainline BookingMenu My tickets
OK
If you’re dealing with
natural language
interfaces, a lot of the
same rules apply.

25. You need a set of
training data -
transcripts of call
customer service
conversations.
You may need simplify
that data - for instance
by looking for the
successulf
conversations.

26. And you need to think
how to set users’
expectations about
talking to a bot.
The adventure game
Lost Pig has you telling
an Orc what to do. So you
know to keep it simple
and expect errors.
It’s cute, has personality
and humour, and serves
an engineering purpose.

27. You’ll need to map out
conversations as
flowcharts. But there’s a
lot of copywriting you’ll
need to do around those
flows to make it feel
natural. For instance,
you may want to give a
long answer the first
time someone asks a
question and then a
shorter summary the
second time.

28. Book me an off peak return from
Bath to London for next Tuesday
with a seat reservation on the way
back at 4:30.
Would you like to add a Zone
1-5 Travelcard for £5.80?
Yes plz
That comes to £78.20 including
booking fee. Want to go ahead?
Trainline BookingMenu My tickets
OK
I’ve always looked to
human conversation
patterns to figure out
how to solve interaction
design problems.
Now I find that
understanding human to
human conversation is
core design knowledge.

29. And what about Discover
Weekly? Well, a large
part of the design work
there was about
understanding how to
package up the service.
Playlists were familiar.
And limiting the size of
the playlist gave it a feel
of a mix tape from a
friend, rather than a
data dump from an
algorithm.

30. The designers made it
feel elegant and
approachable.
So our core skills are still
important. There’s a rich
future for interaction
design.
But the journey to
evolve our practice and
knowledge is just
begining.

31. Image: Akritasa (cc)
Thank you
@gilescolborne