A talk from the Develop Track at AWE USA 2017 - the largest conference for AR+VR in Santa Clara, California May 31- June 2, 2017. Stefano Baldassi (Meta): Designing for the Future of Spatial Computing Join Meta's neuroscience team as we explore together how designing for the future of computing should be: With the human first. For the past two years, Meta's research, neuroscience, and user experience (UX) design teams have been working on design guidelines for augmented reality (AR). These guidelines were created out of our desire to contribute to the larger conversation around how AR experiences can be designed to enhance our abilities to communicate, create, and collaborate. http://AugmentedWorldExpo.com

1. Designing (with Neuroscience)
for the Future of Spatial Computing
Stefano Baldassi, PhD
Sr. Director | Analytics & Neuroscience

2. Sliding doors

3. VISION
 segregation
 integration
 bias correction
 figure-ground
 textures
 contours
MULTISENSORY
 audiovisual
 localization
 timing
 haptic
ATTENTION
 cue modulation
 fixation control
 clutter analysis
 saliency analysis
 distraction analysis
NATURAL
COMPUTER
COGNITION
 task analysis
 A/D task integration
 attention aware design
 targeted +R
 targeted -R
 predictive design

4. META 1
2013
META 2
2016

5. Why neuroscience?

6. The Interaction Machine

7. A UI is a toolset
to interact with digital
content

8. What is the lesson from
neuroscience when it comes
to Spatial UIs?

11. 1. Think Spatial: Place Tools and Content in Space
Replace flat layouts of windows, menus and buttons, which generally favor
only half of the visual system, with a Spatial Interface that arranges tools
and content in 3D space around the user. This leverages the user’s full
perception of form and depth.

12. 2. Minimize Abstractions: Design Tools with Volume and Affordances
Now that we understand how to arrange objects in space, how should we
design the objects themselves? Replace abstract representations (like flat
icons) with volumetric tools featuring physical characteristics that suggest
their use without explicit instruction, leveraging the Neuroscience of
Affordance.

13. 3. You are the OS: Organize holographic files and tools in the user’s environment
Replace the traditional file system—which demands continual decoding of
abstract symbols and hierarchies—with spatial organization of files in the
world that leverages the brain’s spatial memory, naturally tracking the
contents of our personal space.

14. 4. Touch to See: Use the hands to interact directly
Replace remote gestures with direct manipulation, leveraging the
neuroscience of interaction—namely, the brain’s naturally heightened
understanding of objects near the hands

15. 5. Do Not Disturb: Protect the user’s workflow from disruption
Spatial computing demands spatial notifications. Rather than interrupting
the user’s workflow with pop-ups, allow them to designate a separate
container to passively collect incoming notifications. This allows the user to
remain focused on their task—only stopping to check for updates when
desired—and leverages the Neuroscience of Attention.

16. 6. Avoid Surprises and Magic Tricks: Pair actions with intuitive outcomes
Avoid “magical” events that appear unrelated to user behavior or prior
causes, or violate laws of physics to the point of confusion, as they
confound the user’s expectations and trigger Error Neurons

17. 7. The Holographic Campfire: Don’t obscure hands and faces with the UI
Replace UIs that block eye contact with a shared space that promotes it,
and ensure hands are visible while collaborating. Doing so leverages the
power of mirror neurons and face coding.

18. 8. Public by Default: Shared understanding reduces anxiety among users
Replace private UIs—which separate users and cause ambiguity by
obscuring their intentions—with a common digital environment for all Meta
users that leverages the neuroscience of theory of mind and ensures
comfortable awareness for all participants.

19. 9. Augmented, not Mixed Reality: Enhance the user’s perception with relevant information
Rather than block out users’ reality (as in virtual reality) or distort the user’s reality
(as in mixed reality), provide Meta UIs—metadata about the world, near but not
occluding it—that adds something useful to their understanding of their
environment. This is the ultimate aim of this document: to create an informative,
powerful and unobtrusive layer of digital information on top of the real world.

20. This is just the beginning!