Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

John Howard (LOOOK Inc) UX/UI Design for VR and Mixed Reality

2,240 views

Published on

Virtual Reality and Mixed Reality will become the biggest technology disruptions since the smartphone, but most UX/UI design is still focused on flat, 2D screens. What do UX designers and engineers need to do to make the jump to spatial 3D?

The futuristic, 3D interfaces of movies and video games are rarely functional in practice. And the tried-and-true rules of flat UX design often fail to translate into 3D.

This talk explains the spatial UX/UI toolbox being created by pioneers in the space. We’ll take a look at what works, what doesn’t, why, and how you can apply these UX/UI principles to your own projects.

Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com

Published in: Technology

John Howard (LOOOK Inc) UX/UI Design for VR and Mixed Reality

  1. 1. Spatial UX and UI John Howard LOOOK, Inc. john@loook.io www.loook.io
  2. 2. Market Data “We expect virtual and augmented reality to become an $80 billion market by 2025, roughly the size of the desktop PC market today.” Goldman SachsResearch, January 2016
  3. 3. UX / UI Evolution 1984 2014 Most UX/UI design isstill focused on flat, 2D screens.
  4. 4. UX / UI Evolution Microsoft
  5. 5. UX / UI Evolution 2D 3D GUI Touch CAD Spatial How do we make the jump from 2D to 3D?
  6. 6. The Dream It’ll be just like Iron Man !!!
  7. 7. This eye thing is annoying. The Reality Iron Man 3, Marvel Studios / Paramount Pictures
  8. 8. My arms are tired. Minority Report, DreamWorks The Reality
  9. 9. This menu is actually behind me. The Division, Ubisoft The Reality
  10. 10. The Reality “Maybe not like Iron Man…” Fatigue UX should be tailored to the duration of the experience. Where to anchor menus? Head-locked UX is annoying. Occlusion and Clipping UI needs to respect the world. Z-fighting Near/far focal length switching causes discomfort. Vestibular discomfort AKA, simulator sickness. Spatial UX/UI Challenges
  11. 11. UX/UI Toolbox
  12. 12. UX/UI Toolbox User Gaze Gaze Cursor Voice SpatialAudio / Haptic Audio Context SensitiveActions Extended Desktop Movement Hand Tracking The Spatial UX/UI Toolbox
  13. 13. UX/UI Toolbox User Gaze Gaze Cursor Voice SpatialAudio / Haptic Audio Context SensitiveActions Extended Desktop Movement Hand Tracking The Spatial UX/UI Toolbox
  14. 14. 01: User Gaze UX/UI Toolbox Halo E3 2015, Microsoft
  15. 15. Halo E3 2015, Microsoft 01: User Gaze
  16. 16. UX / UI Toolbox Caution! Aggressive orexcessive event triggering with gaze islike pointing a loaded gun; users get gun-shy and it stresses themout. 01: User Gaze
  17. 17. 02: Gaze Cursor UX/UI Toolbox Case Western Reserve University School of Medicine
  18. 18. CDM Smith / Object Theory 02: Gaze Cursor
  19. 19. Case Western Reserve University School of Medicine 02: Gaze Cursor
  20. 20. 04: Spatial Sound UX/UI Toolbox RoboRaid, Microsoft
  21. 21. Case Western Reserve University School of Medicine / Microsoft 04: Spatial Sound
  22. 22. 05: Haptic Audio UX/UI Toolbox
  23. 23. Luka Mivsek 05: Haptic Audio
  24. 24. Autodesk / Microsoft 05: Haptic Audio
  25. 25. Operation, Hasbro 05: Haptic Audio
  26. 26. iStock 05: Haptic Audio
  27. 27. 06: Context-Sensitive Actions UX/UI Toolbox Trimble / Microsoft
  28. 28. Case study: Indoor agriculture, LOOOK 06: Context Actions
  29. 29. NASA / Jet Propulsion Laboratory / Microsoft 06: Context Actions
  30. 30. MediSIM / Carnegie Mellon University
  31. 31. MediSIM / Carnegie Mellon University
  32. 32. UX / UI Toolbox Caution! Contextmenusin densely populatedorclose-quartersspace can result in environment clipping ,occlusionorz-fighting. Caution! Beware of too many nested fly-outsin pop-up menus. Considergaze travel distance. 06: Context SensitiveActions & Controls
  33. 33. 07: Extended Desktop UX/UI Toolbox Autodesk / Microsoft
  34. 34. NASA / Jet Propulsion Laboratory / Microsoft 07: Extended Desktop
  35. 35. Autodesk / Microsoft 07: Extended Desktop
  36. 36. Trimble / Microsoft 07: Extended Desktop
  37. 37. UX / UI Toolbox Caution! Amouse pointershould followsurfaceslike the gaze cursorbut will appearto "jump" when transitioning between extreme z-depth changes. “Lerp” to avoid jarring transitionsand display a ray from mouse to destinationso all users can followthe transition. 07: Extended Desktop
  38. 38. In Conclusion… User Gaze Gaze Cursor Voice SpatialAudio / Haptic Audio Context SensitiveActions Extended Desktop Movement Hand Tracking The Spatial UX/UI Toolbox
  39. 39. In Conclusion… “I still want it to be like Iron Man…”
  40. 40. Important! Be like Tony Stark… “Always Be Prototyping!”
  41. 41. Thank You. John Howard LOOOK, Inc. john@loook.io www.loook.io

×