A talk from the Design Track at AWE EU 2018 - the World's #1 XR Conference & Expo in Munich, Germany 18 -19, October, 2018. Filip Healy (Threesixty Reality): Making immersive tech more usable: Involving target users in your design process Understanding user needs, design research and usability testing are common practice today for most digital products. It's rare for a mobile app or website from any serious company to go live without some degree of testing with users. The benefits of involving users in the design process are well known to UX professionals and digital product managers: improved usability, reduced dropouts, optimised conversion rates, higher engagement and better user advocacy. In this talk I'll discuss how this applies to immersive tech and why it's more important than ever to understand actual user behaviour and develop new interaction paradigms that deliver on the potential of VR and AR platforms. I'll talk through the process of running studies with users, when to test the design, how to set things up and what data to focus on in order to get the most benefit from testing with users. What can we learn from this type of research and why is it important for ensuring the success of our product? http://AugmentedWorldExpo.com

1. threesixty
USER EXPERIENCE FOR MIXED REALITIES
MAKING IMMERSIVE TECH MORE USABLE:
INVOLVING TARGET USERS IN YOUR DESIGN
PROCESS

2. Design researcher for VR/AR
Filip Healy
I’ve been delivering design research to inform the
evolution of new technologies for major global
companies since 2002
About me

3. THREESIXTY, is a London-based
consultancy specialising in user
experience for immersive
technologies.
We use research and an understanding of
human interactions with technology to
make VR / AR experiences intuitive and
productive.
My current focus

4. Back
Design research for immersive tech

5. Back
Evaluate the
designs
Specify user
needs
Produce design
solutions
01
.
02
.
03
.
Human centred design
Understand &
specify the user
context
Product that
works for
users
04
.
05.
ISO 9241-210
Iterate as
many times
as necessary

6. Back
Evaluate the
designs
Specify user
needs
Produce design
solutions
01
.
02
.
03
.
Human centred design
Understand &
specify the user
context
Product that
works for
users
04
.
05.
ISO 9241-210
Iterate as
many times
as necessary

7. Back
What interactions do we need to
test?

8. Back
What interactions do we need to
test?

9. How intuitive is immersive tech?

10. Back
Design Challenges for Immersive
3D Virtual environment
Objects and menus
user can interact with
Generally quite
usable
Where we see
most usability
issues
“Just as in the last survey,
respondents said that user
experience was the top
obstacle for mass adoption
of both AR and VR.
2018 Augmented & Virtual Reality Survey,
Perkins Coie

11. Back
UX Challenges for Immersive
Understanding what is
interactive
Navigating
menus
Locating menus
Orientation
(VR)
Learning the
controllers (VR)
Gestures
(AR)
Searching
& typing
Surface
scanning
feedback
(AR)
Gaze
interaction

12. Back
Why is it important
“Every dollar invested in ease of
use returns $10 to $100
Cost-Justifying Usability, Clare-Marie Karat, Ph.D. from
the IBM
“$1 spent during design = $10 to fix
problem during dev or $100 after it’s
released
Software Engineering: A Practitioner’s Guide, Robert
Pressman
Improves productivity,
conversions & other KPIs
Reduces development costs
Evaluate the designsSpecify user needs Produce design
solutions
01
.
Understand & specify the user
context
Product that works for
users
02
.
03
.
04
.
05
.

13. Back
Why is this important
Forrester Research (2008)
Of projects fail due to
lack of user acceptance
70
%
83
%
Jakob Nielsen (2008)
Increase in KPIs
due to UX
90%
Strategic data consulting
special report (2009)
Reduction in support
costs for MacAfee
software after usability
testing

14. Back
Why is this important
“When we started talking to our
customers and seeing how they
used our service, it was the defining
moment of success that turned the
company around”
Joe Gebbia, Co Founder, Air BnB
“In our first year we didn’t spend a
single dollar on advertising… the
best dollars spent are those we use
to improve the customer
experience.”
Jeff Bezos, Founder, Amazon
World’s 1st $1trillion company
Almost bust to $30 billion
company

15. Back
When to test
Evaluate the designsSpecify user needs Produce design
solutions
01
.
Understand & specify the user
context
Product that works for
users
02
.
03
.
04
.
05
.
Early concept Final design
Test Test
Interactive prototype
Increasing
fidelity

16. Back
How to test designs?
A. Validate
design concepts
B. Evaluate
functional design
Do we have the right
features?
Are all the interactions
usable?
● Animations of tasks and
interactive screens
● 3D Paper prototyping
Does an interaction
flow make sense?
● Flat designs to show menu
options
● Storyboards of user journey
● Interactive Unity/Unreal
prototype
● Web VR prototype
● Enterprise workflow
platform
Lab based
usability testing
Rapid design
iteration
Online feedback
tools
Contextual
interviews
Lab based
sessions
Key questions Test stimulus Methods
Are there any UX
barriers undermining
success?

17. Back
How to test designs?
A. Validate
design concepts
B. Evaluate
functional design
Do we have the right
features?
Are all the interactions
usable?
● Animations of tasks and
interactive screens
● 3D Paper prototyping
Does an interaction
flow make sense?
● Flat designs to show menu
options
● Storyboards of user journey
● Interactive Unity/Unreal
prototype
● Web VR prototype
● Enterprise workflow
platform
Lab based
usability testing
Rapid design
iteration
Online feedback
tools
Contextual
interviews
Lab based
sessions
Key questions Test stimulus Methods
Are there any UX
barriers undermining
success?

18. Back
Concept testing
Animated concept mock ups
Paper prototyping

19. Back
How to test designs?
A. Validate
design concepts
B. Evaluate
functional design
Do we have the right
features?
Are all the interactions
usable?
● Animations of tasks and
interactive screens
● 3D Paper prototyping
Does an interaction
flow make sense?
● Flat designs to show menu
options
● Storyboards of user journey
● Interactive Unity/Unreal
prototype
● Web VR prototype
● Enterprise workflow
platform
Lab based
usability testing
Rapid design
iteration
Online feedback
tools
Contextual
interviews
Lab based
sessions
Key questions Test stimulus Methods
Are there any UX
barriers undermining
success?

20. Usability
evaluation
ISO definition of usability:
The extent to which a product
can be used by specified users
to achieve specified goals with
effectiveness, efficiency and
satisfaction in a specified
context of use.

21. Task based
evaluation
Application has specific sequences of steps to
achieve a specific goal:
● Changing settings
● Navigating content
● Communication features
● Creating or editing content
“Please show me how you
would create red sphere and
place it on top of the cube”

22. Play testing
Application leads the user on immersive
entertainment journey
● Video game
● 360 video
● Immersive experience
“Imagine you have downloaded
this new experience and you
have 20 minutes to try it. Please
give it a go as you would at
home.”

23. Back
Example: VR

24. Back
Example: Mobile AR

25. Back
Immersive tech lab

26. Back
Eyetracking

27. Back
Biometrics
GSR & ECG EEG

28. Back
Biometrics
GSR
Positive feelings
Emotional
intensity peaks
High
engagement
GSR & HR
correlation
Heart rate
variations
Cognitive
workload
Distraction
Baseline
normalisatiion
EEG

29. We can design better user experiences for
immersive tech by involving actual users in
the design process

30. Back
Questions