Virtual Reality and Mixed Reality will become the biggest technology disruptions since the smartphone, but most UX/UI design is still focused on flat, 2D screens. What do UX designers and engineers need to do to make the jump to spatial 3D? The futuristic, 3D interfaces of movies and video games are rarely functional in practice. And the tried-and-true rules of flat UX design often fail to translate into 3D. This talk explains the spatial UX/UI toolbox being created by pioneers in the space. We’ll take a look at what works, what doesn’t, why, and how you can apply these UX/UI principles to your own projects. Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com

1. Spatial UX
and UI
John Howard LOOOK, Inc.
john@loook.io
www.loook.io

2. Market Data
“We expect virtual and augmented reality to
become an $80 billion market by 2025, roughly
the size of the desktop PC market today.”
Goldman SachsResearch,
January 2016

3. UX / UI Evolution
1984 2014
Most UX/UI design isstill focused on flat, 2D screens.

4. UX / UI Evolution
Microsoft

5. UX / UI Evolution
2D 3D
GUI Touch CAD Spatial
How do we make the jump
from 2D to 3D?

6. The Dream
It’ll be just like Iron Man !!!

7. This eye thing is annoying.
The Reality
Iron Man 3, Marvel Studios / Paramount Pictures

8. My arms are tired.
Minority Report, DreamWorks
The Reality

9. This menu is actually behind me.
The Division, Ubisoft
The Reality

10. The Reality
“Maybe not like Iron Man…”
Fatigue UX should be tailored to the duration of the experience.
Where to anchor menus? Head-locked UX is annoying.
Occlusion and Clipping UI needs to respect the world.
Z-fighting Near/far focal length switching causes discomfort.
Vestibular discomfort AKA, simulator sickness.
Spatial UX/UI Challenges

11. UX/UI Toolbox

12. UX/UI Toolbox
User Gaze
Gaze Cursor
Voice
SpatialAudio / Haptic Audio
Context SensitiveActions
Extended Desktop
Movement
Hand Tracking
The Spatial UX/UI Toolbox

13. UX/UI Toolbox
User Gaze
Gaze Cursor
Voice
SpatialAudio / Haptic Audio
Context SensitiveActions
Extended Desktop
Movement
Hand Tracking
The Spatial UX/UI Toolbox

14. 01: User Gaze
UX/UI Toolbox
Halo E3 2015, Microsoft

15. Halo E3 2015, Microsoft
01: User Gaze

16. UX / UI Toolbox
Caution! Aggressive orexcessive event triggering with gaze islike pointing a loaded gun;
users get gun-shy and it stresses themout.
01: User Gaze

17. 02: Gaze Cursor
UX/UI Toolbox
Case Western Reserve University School of Medicine

18. CDM Smith / Object Theory
02: Gaze Cursor

19. Case Western Reserve University School of Medicine
02: Gaze Cursor

20. 04: Spatial Sound
UX/UI Toolbox
RoboRaid, Microsoft

21. Case Western Reserve University School of Medicine / Microsoft
04: Spatial Sound

22. 05: Haptic Audio
UX/UI Toolbox

23. Luka Mivsek
05: Haptic Audio

24. Autodesk / Microsoft
05: Haptic Audio

25. Operation, Hasbro
05: Haptic Audio

26. iStock
05: Haptic Audio

27. 06: Context-Sensitive Actions
UX/UI Toolbox
Trimble / Microsoft

28. Case study: Indoor agriculture, LOOOK
06: Context Actions

29. NASA / Jet Propulsion Laboratory / Microsoft
06: Context Actions

30. MediSIM / Carnegie Mellon University

31. MediSIM / Carnegie Mellon University

32. UX / UI Toolbox
Caution! Contextmenusin densely populatedorclose-quartersspace can result in
environment clipping ,occlusionorz-fighting.
Caution! Beware of too many nested fly-outsin pop-up menus.
Considergaze travel distance.
06: Context SensitiveActions & Controls

33. 07: Extended Desktop
UX/UI Toolbox
Autodesk / Microsoft

34. NASA / Jet Propulsion Laboratory / Microsoft
07: Extended Desktop

35. Autodesk / Microsoft
07: Extended Desktop

36. Trimble / Microsoft
07: Extended Desktop

37. UX / UI Toolbox
Caution! Amouse pointershould followsurfaceslike the gaze cursorbut will appearto
"jump" when transitioning between extreme z-depth changes.
“Lerp” to avoid jarring transitionsand display a ray from mouse to destinationso all users
can followthe transition.
07: Extended Desktop

38. In Conclusion…
User Gaze
Gaze Cursor
Voice
SpatialAudio / Haptic Audio
Context SensitiveActions
Extended Desktop
Movement
Hand Tracking
The Spatial UX/UI Toolbox

39. In Conclusion…
“I still want it to be like Iron Man…”

40. Important!
Be like Tony Stark…
“Always Be Prototyping!”

41. Thank You.
John Howard LOOOK, Inc.
john@loook.io
www.loook.io