Martin Charlier @ O'Reilly Design Conference 2018 This is a talk about how video is a powerful tool for rapid low-fi prototyping of connected products & the Internet of Things. This talk argues how this method is useful, shows how it builds on existing prototyping methods and gives a practical example for how you can apply this method. Finally, the talk shows how the same ideas and principles of animated or filmed artefacts can be used at different levels of fidelity and focussed on different purposes.

1. 1
Video as a
prototyping tool for
connected products
@marcharlierO’Reilly Design 2017

2. 2
Hello
Product Manager
Industrial design
User Experience
Service design
Strategy

3. 3
Hello
Tom Metcalfe Martin Charlier
@tommetcalfe @marcharlier

4. 4
Video as a
prototyping tool for
connected products

5. 5
Video as a
prototyping tool for
connected products

6. 6
You might learn more
about user experience
from a wooden block and
a chop stick than from
an high-tech prototype.

7. 7
Through early low-fi
prototyping, you can ask
big and fundamental
questions about your
product before you sink
too much cost.
• Would people use this?
• How would it have to work
to be desirable?
• What would it feel like to
use this?
• Should we build this at all?

8. 8
Video as a
prototyping tool for
connected products

9. 9
Challenges with connected products
• The interface might be controls on a new
kind of device.
• The interface could be non-screen based,
like voice, gesture, proximity, physical
interaction.
• Interaction might happen across different
devices, across space or time.
• The overall experience doesn’t just happen
on a computer or a mobile - it involves
unboxing, manuals, different devices,
installation…
Above: Tado Thermostat
Below: Jordi Parra’s Spotify Box (http://blog.zenona.com/)

10. 10
Challenges with connected products
• The interface might be controls on a new
kind of device.
• The interface could be non-screen based,
like voice, gesture, proximity, physical
interaction.
• Interaction might happen across different
devices, across space or time.
• The overall experience doesn’t just happen
on a computer or a mobile - it involves
unboxing, manuals, different devices,
installation…
Amazon Echo

11. 11
Challenges with connected products
• The interface might be controls on a new
kind of device.
• The interface could be non-screen based,
like voice, gesture, proximity, physical
interaction.
• Interaction might happen across different
devices, across space or time.
• The overall experience doesn’t just happen
on a computer or a mobile - it involves
unboxing, manuals, different devices,
installation…
Fujifilm manual

12. 12
Challenges with connected products
• The interface might be controls on a new
kind of device.
• The interface could be non-screen based,
like voice, gesture, proximity, physical
interaction.
• Interaction might happen across different
devices, across space or time.
• The overall experience doesn’t just happen
on a computer or a mobile - it involves
unboxing, manuals, different devices,
installation…
Photo by Ari Jay Comet (www.arijaycomet.com)

13. 13
Conventional
prototyping methods
aren’t always fit for the
prototyping challenges
of connected products.

14. 14
Bootstrap
HTML/CSS/JS
Objective-C / Swift
Software UX
prototyping tools

15. 15
Bootstrap
HTML/CSS/JS
Objective-C / Swift
• Software UX tools are often too
screen centric too soon.
• They don’t really work for Voice
or Gesture UIs.
• Prototyping user journeys that
happen across multiple screens
or devices isn’t ideal.
• It’s easy to forget the physical
context around the device.

16. 16
Industrial design
& electronics
prototyping

17. 17
• Industrial design and physical
prototyping methods often lack
interactivity at the low-fi level.
• You need electronics - this can
be time-consuming.
• Electronics can become a barrier
quickly.
• Easy to get carried away with
optimisation.

18. 18
Service design
prototyping
Photos from:
D-LABS
http://www.servicedesigntools.org/tools/43
http://uxpamagazine.org/business-origami/

19. Photos from:
D-LABS
http://www.servicedesigntools.org/tools/43
http://uxpamagazine.org/business-origami/
19
• Workshop-based
• Hard to document or share with
people that weren’t there

20. 20
Video as a
prototyping tool for
connected products

21. 21
Video isn’t an obvious
tool for prototyping.

22. 22
The first association for ‘video’
in design is often a polished, CGI
heavy vision film illustrating a
shiny, seamless future.

23. 23
What makes video great
• Effortless to follow
• Easy to share
• Conveys what an experience feels like -
viewers gain empathy with the actors.

24. 24
Dissecting video
• Script/Scenes = Description of action sequence
• Set = The physical context in which action takes place
• Actors = Multiple humans that carry out actions or interact
• Props = Physical things, mock-ups, prototypes
• Special Effects = Things, actions or behaviours that don’t exist

25. 25
Dissecting video
• Script/Scenes = Description of action sequence
• Set = The physical context in which action takes place
• Actors = Multiple humans that carry out actions or interact
• Props = Physical things, mock-ups, prototypes
• Special Effects = Things, actions or behaviours that don’t exist
The user journey you iterate and develop

26. 26
Dissecting video
• Script/Scenes = Description of action sequence
• Set = The physical context in which action takes place
• Actors = Multiple humans that carry out actions or interact
• Props = Physical things, mock-ups, prototypes
• Special Effects = Things, actions or behaviours that don’t exist
How you gain (and convey) empathy
with the user and their context

27. 27
Dissecting video
• Script/Scenes = Description of action sequence
• Set = The physical context in which action takes place
• Actors = Multiple humans that carry out actions or interact
• Props = Physical things, mock-ups, prototypes
• Special Effects = Things, actions or behaviours that don’t exist
Working across different touch
points, interface types, spaces and
taking into account multiple users.

28. 28
Low-fi video prototyping
builds on paper prototyping,
enactment, storyboarding
and wizard-of-oz.

29. 29
30 minute prototype for a connected herb garden sensor.

31. 31
https://instagram.com/explore/tags/solidprototyping/ https://instagram.com/explore/tags/ixdprototyping

32. 32
Gestural interface for a cooking hob
Script / Scenes
Rapid iteration
through alternative
options you’re unsure
about
Non-screen based
interfaces

33. 33
Set
Understanding the
user’s physical
context

34. 34
Connected loyalty coffee cup
Actors & Props
Interactions that
involve multiple
users and cross-
device interactions.

35. 35
Robotic kitchen aid
Special Effects
Interactions with
robots and machines.

36. 36
A connected fishing rod?!
• It’s a way to figure out whether something is a good idea
before investing much time & money.

37. 37
It’s more about the process than the outcome
80% (The process)
• Rapid iteration and decision
making.
• Exploration across physical
context, multiple devices, time and
space.
• Forces you to think in a time-based
sequence of events - a user journey.
• Acting out: Empathy with the user.
How it might ‘feel’ like using this.
20% (The video)
• It’s self-explanatory and shareable.
• Film is a widely understood
medium that is easily digestible.
• Less ‘loss in translation’, relies less
on imagination of others.
• It lets viewers judge plausibility
(they get a degree of empathy).

38. 38
Why Instagram?
How does it work?

39. 39
Conventional filming Instagram
Edit while filming
Film
↓
Edit

40. 40
Forces you to focus on specific interactions & simplify.
1. Select herb type
2. Start paring
3. Confirm on sensor
4. Install sensor
5. See notification

41. Substitution
41
The ‘Stop Trick’ (or ‘Substitution splice’)
Recording
STOP START
• Don’t move the camera.
• Keep the same shot.
• Only move the parts
that change.

42. 42

43. 43

44. 44
See last frame as
transparent overlay.
Move clips around after
filming.

45. Beyond this low-fi
method, there are quick
ways of using video that
have lots of potential.
45

46. 46
1 / 3
More elaborate
low-fi video prototype

47. 47
Sketch-A-Move: Slightly
more elaborate. Both a
demonstration and an
exploration.
Anab Jain & Louise Klinker
http://superflux.in/index.php/work/sketch-a-move/

48. 48
Sketch-A-Move: Slightly more elaborate. Both a demonstration and an exploration.
http://www.superflux.in/work/sketch-move

49. 49
Inspiring a team
through conveying
the vision behind an
idea.
Matthias Kranz, et. al http://www.eislab.fim.uni-passau.de/files/
publications/2006/SketchAMove_preprint.pdf

50. 50
2 / 3
Take storyboards
to the next level

51. 51

52. 52
“What it would feel like being in the theatre
watching the final film”
“Trying to convey that feeling”

53. 53
The storyboard isn’t
just read out, it is
performed.
The emotional feeling
of the scene is
conveyed.

54. 54

55. 55
Stills compositions
with narration
Cooper Economizer
More info: http://www.cooper.com/journal/
2008/12/economizer

56. 56
A storyboard or video prototype can guide the tech requirements
More info: http://www.cooper.com/journal/2008/12/economizer
Design requirements Design requirements Design requirements Design requirements

57. 57
3 / 3
High-fidelity looks, but
low-fidelity prototypes

58. 58

59. 59
Cut & paste prototype
Issue of The
Economist
Economist-styled
mock up paper parts
we glued in
Cut off
headphone
socket, glued in.
+ +

60. 60

61. 61
Still image animated composition - then add wiggle
Still image Animated flat Wiggle path
+ +

62. 62
Keynote prototyping

63. 63
Keynote prototyping
Look & feel: Pinching timeline to group
photos by affinity
Explainer: Group photos by affinity
when pinching timeline & create album
from stack..
Explainer: Tap stack to “uncompress”
timeline and go to selected photos.

64. 64
Summary
Using video for
prototyping

65. 65
Builds on low-fi methods
• Storyboarding
• Enactment
• Paper prototyping / physical prototyping
• Wizard-of-Oz

66. 66
Works at a range of levels of fidelity
Animated storyboard
Instagram sketch
High-fidelity film with low-fi prototypes

67. 67
Has a range of uses
• Iterate and refine your design
• Articulate a joint vision
• Document design requirements
• Test user desirability
• Convince investors or backers

68. 68
Thank you.
designingconnectedproducts.com
@marcharlier