Building Holographic
& VR Experiences
Using the Mixed Reality Toolkit for Unity
Nick Landry
Defining Mixed Reality
The state of things as they “actually
exist” through our human senses
without any technology.
Artificially created sensory experiences of
people, environments and objects, which can
include sight, touch, hearing, and smell.
PHYSICAL REALITY VIRTUAL REALITY
MIXED REALITY
VIRTUAL REALITYPHYSICAL REALITY
Physical Reality Digital RealityMixed Reality
Windows Mixed Reality Holographic Device (See-through)
Windows Mixed Reality Immersive Devices (Opaque)
Windows Mixed Reality:
• Includes Perception / Environment Awareness
• Any Mix of People, Places, or Things
Mixed Reality Spectrum
AR Today VR Today
REALITY VIRTUALITY
REALITY VIRTUALITY
REALITY VIRTUALITY
REALITY VIRTUALITY
REALITY VIRTUALITY
Hardware details for development
edition devices from Acer and HP
▪ Two high-resolution LCD at 1440 x 1440
▪ 2.89” diagonal display size (x2)
▪ Front hinged display
▪ 95° horizontal field of view
▪ Display refresh rate up to 90 Hz (native)
▪ Built-in audio out and microphone support through
3.5mm jack
▪ Single cable with HDMI 2.0 (display) and USB 3.0 (data)
for connectivity
▪ 4.00m/0.60m removable cable
▪ Inside-out tracking
Windows Mixed Reality collection
Immersive headsets
Powered by
Windows Mixed Reality
Powered by
Windows Mixed Reality
Hardware details for
Samsung Odyssey HMD
▪ Two high-resolution OLED at 1440 x 1600
▪ 110° horizontal field of view
▪ Display refresh rate up to 90 Hz (native)
▪ built-in premium AKG headphones
and dual array mics
▪ Single cable with HDMI 2.0 (display)
and USB 3.0 (data) for connectivity
▪ 6 DOF inside-out position tracking
What is HoloLens?
Holograms enhance the
real world
Microsoft HoloLens is the first, fully untethered holographic computer
that redefines personal computing and empowers you in new ways.
New ways to connect,
create, and explore
A more natural way to
interact
Advanced technology
from Microsoft
Windows Mixed Reality
Universal Windows Platform
One platform. Multiple Windows devices.
One Universal Windows Platform
Windows Mixed Reality is built into the Windows SDK that
developers already use today.
The Universal Windows Platform makes it easy to build an
application that scales from HoloLens to other mixed reality
headsets.
• Target windows.holographic for holographic headsets
• Target windows.desktop for immersive headsets
Windows is the only platform unifying the mixed reality
ecosystem, providing inside-out tracking for headsets, a
single platform and standardized inputs for developers, and
a consistent interface with a single store for customers.​
Models Materials Textures Shaders Sounds Scripts
Cursor
prefabs
Sharing
prefab
Spatial
prefabs
Utilities Shaders Tests
Gaze
Input
Gesture
Input
Voice
Input
Spatial
Anchors
Spatial
Mapping
Spatial
Sound
https://aka.ms/mrdevtools
https://unity3d.com/unity/qa/patch-releases
https://forum.unity.com/forums/ar-vr-xr-discussion.80/
Try your app on the HoloLens emulator today
You need:
Visual Studio 2015 or 2017
64-bit Windows 10 Pro, Enterprise, or
Education with a 64-bit CPU (4+ cores)
8GB RAM
GPU (DirectX 11, WDDM 1.2 driver)
Windows 10 SDK (10.0.10586 or newer)
HoloLens Emulator (requires, Hardware-
assisted virtualization, SLAT & DEP)
A UWP app that builds for x86
Unity not needed for 2D UWP apps
dev.microsoft.com/windows/mixed-reality
http://aka.ms/mr
“Hello World” Introduction (Origami) Gaze Input
Gesture Input Voice Input Spatial Sound
Spatial Mapping Shared Experiences Sharing Mixed Reality
Galaxy Explorer
Open source application available in the store. The idea was voted by the
community and originally built for HoloLens. It now supports all mixed reality
devices, and is a useful end-to-end demonstration of how to build a Unity app for
Windows Mixed Reality.
Mixed Reality Toolkit
Library with many component that make it easier to build mixed reality apps,
including plug-ins for Unity. Some of the more popular components are the
sharing services for multiplayer mixed reality apps and spectator view.
Academy Tutorial Apps
The tutorials are great starting points for your app, and they improve with
community contributions too.
Work with our teams or our agencies
Join the community
Online tools & resources
dev.windows.com/mixedreality
Download the tools
aka.ms/mrdevtools
Follow the training
aka.ms/holographicacademy
Forums.hololens.com
Questions, discussion, and feedback
from the mixed reality development
community.
Github/microsoft
aka.ms/GalaxyExplorer
aka.ms/holotoolkitunity
aka.ms/holotoolkit
aka.ms/windowsdeviceportalwrapper
aka.ms/holographicacademygithub
aka.ms/MRcommunity
Microsoft Technology Centers
Experience HoloLens & discuss core business uses
aka.ms/iwantmr
Join MR@MS: A program tailored to your needs,
providing guidance and support from ideation to
publish.
aka.ms/MRcommunity
Technical Evangelists - Connect with one of our technical evangelists in your area.
Meetups: join one of the hundreds of meetups around the world
aka.ms/agencies
Agency partners have HoloLens
development expertise and dedicated
teams to take on application
development for clients.
mpcbdesk@microsoft.com
Our teams will work with you from envisioning,
to proof of concepts, all the way to
deployment.
Acer - Windows Mixed Reality
Development Edition
Microsoft HoloLens
Development Edition
Available today on microsoftstore.com
Shipping to : Australia, Canada, France, Germany, Ireland, Japan, New
Zealand, United Kingdom and United States. (pre-order available soon
for China)
English only
HoloLens Development Edition: $3000
HoloLens Commercial Suite: $5000
For more information: www.hololens.com
Sign up today on aka.ms/iwantmr
Shipping to: TBA (please consult with respective OEM announcements)
Languages: English, French, German, Chinese (simplified) and Japanese
Thank You
Nick Landry
Sr. Software Engineer, Microsoft
activenick@microsoft.com
slideshare.net/ActiveNick
github.com/ActiveNick
@ActiveNick
© 2018 Microsoft Corporation. All rights reserved.
Microsoft, Windows, Microsoft Surface and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.
Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft
cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Building Holographic & VR Experiences Using the Mixed Reality Toolkit for Unity

  • 1.
    Building Holographic & VRExperiences Using the Mixed Reality Toolkit for Unity Nick Landry
  • 5.
  • 7.
    The state ofthings as they “actually exist” through our human senses without any technology. Artificially created sensory experiences of people, environments and objects, which can include sight, touch, hearing, and smell. PHYSICAL REALITY VIRTUAL REALITY
  • 8.
  • 9.
    Physical Reality DigitalRealityMixed Reality Windows Mixed Reality Holographic Device (See-through) Windows Mixed Reality Immersive Devices (Opaque) Windows Mixed Reality: • Includes Perception / Environment Awareness • Any Mix of People, Places, or Things Mixed Reality Spectrum AR Today VR Today
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 16.
    Hardware details fordevelopment edition devices from Acer and HP ▪ Two high-resolution LCD at 1440 x 1440 ▪ 2.89” diagonal display size (x2) ▪ Front hinged display ▪ 95° horizontal field of view ▪ Display refresh rate up to 90 Hz (native) ▪ Built-in audio out and microphone support through 3.5mm jack ▪ Single cable with HDMI 2.0 (display) and USB 3.0 (data) for connectivity ▪ 4.00m/0.60m removable cable ▪ Inside-out tracking
  • 17.
    Windows Mixed Realitycollection Immersive headsets Powered by Windows Mixed Reality
  • 18.
    Powered by Windows MixedReality Hardware details for Samsung Odyssey HMD ▪ Two high-resolution OLED at 1440 x 1600 ▪ 110° horizontal field of view ▪ Display refresh rate up to 90 Hz (native) ▪ built-in premium AKG headphones and dual array mics ▪ Single cable with HDMI 2.0 (display) and USB 3.0 (data) for connectivity ▪ 6 DOF inside-out position tracking
  • 23.
    What is HoloLens? Hologramsenhance the real world Microsoft HoloLens is the first, fully untethered holographic computer that redefines personal computing and empowers you in new ways. New ways to connect, create, and explore A more natural way to interact Advanced technology from Microsoft
  • 26.
  • 27.
    One platform. MultipleWindows devices. One Universal Windows Platform Windows Mixed Reality is built into the Windows SDK that developers already use today. The Universal Windows Platform makes it easy to build an application that scales from HoloLens to other mixed reality headsets. • Target windows.holographic for holographic headsets • Target windows.desktop for immersive headsets Windows is the only platform unifying the mixed reality ecosystem, providing inside-out tracking for headsets, a single platform and standardized inputs for developers, and a consistent interface with a single store for customers.​
  • 35.
    Models Materials TexturesShaders Sounds Scripts Cursor prefabs Sharing prefab Spatial prefabs Utilities Shaders Tests Gaze Input Gesture Input Voice Input Spatial Anchors Spatial Mapping Spatial Sound
  • 38.
  • 39.
    Try your appon the HoloLens emulator today You need: Visual Studio 2015 or 2017 64-bit Windows 10 Pro, Enterprise, or Education with a 64-bit CPU (4+ cores) 8GB RAM GPU (DirectX 11, WDDM 1.2 driver) Windows 10 SDK (10.0.10586 or newer) HoloLens Emulator (requires, Hardware- assisted virtualization, SLAT & DEP) A UWP app that builds for x86 Unity not needed for 2D UWP apps dev.microsoft.com/windows/mixed-reality
  • 41.
    http://aka.ms/mr “Hello World” Introduction(Origami) Gaze Input Gesture Input Voice Input Spatial Sound Spatial Mapping Shared Experiences Sharing Mixed Reality
  • 42.
    Galaxy Explorer Open sourceapplication available in the store. The idea was voted by the community and originally built for HoloLens. It now supports all mixed reality devices, and is a useful end-to-end demonstration of how to build a Unity app for Windows Mixed Reality. Mixed Reality Toolkit Library with many component that make it easier to build mixed reality apps, including plug-ins for Unity. Some of the more popular components are the sharing services for multiplayer mixed reality apps and spectator view. Academy Tutorial Apps The tutorials are great starting points for your app, and they improve with community contributions too.
  • 43.
    Work with ourteams or our agencies Join the community Online tools & resources dev.windows.com/mixedreality Download the tools aka.ms/mrdevtools Follow the training aka.ms/holographicacademy Forums.hololens.com Questions, discussion, and feedback from the mixed reality development community. Github/microsoft aka.ms/GalaxyExplorer aka.ms/holotoolkitunity aka.ms/holotoolkit aka.ms/windowsdeviceportalwrapper aka.ms/holographicacademygithub aka.ms/MRcommunity Microsoft Technology Centers Experience HoloLens & discuss core business uses aka.ms/iwantmr Join MR@MS: A program tailored to your needs, providing guidance and support from ideation to publish. aka.ms/MRcommunity Technical Evangelists - Connect with one of our technical evangelists in your area. Meetups: join one of the hundreds of meetups around the world aka.ms/agencies Agency partners have HoloLens development expertise and dedicated teams to take on application development for clients. mpcbdesk@microsoft.com Our teams will work with you from envisioning, to proof of concepts, all the way to deployment.
  • 44.
    Acer - WindowsMixed Reality Development Edition Microsoft HoloLens Development Edition Available today on microsoftstore.com Shipping to : Australia, Canada, France, Germany, Ireland, Japan, New Zealand, United Kingdom and United States. (pre-order available soon for China) English only HoloLens Development Edition: $3000 HoloLens Commercial Suite: $5000 For more information: www.hololens.com Sign up today on aka.ms/iwantmr Shipping to: TBA (please consult with respective OEM announcements) Languages: English, French, German, Chinese (simplified) and Japanese
  • 46.
    Thank You Nick Landry Sr.Software Engineer, Microsoft activenick@microsoft.com slideshare.net/ActiveNick github.com/ActiveNick @ActiveNick
  • 47.
    © 2018 MicrosoftCorporation. All rights reserved. Microsoft, Windows, Microsoft Surface and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.