Brent Insko, Intel Lead Software Architect in Virtual Reality Center of Excellence, presented talk on OpenXR* and the future of AR/VR/MR gaming.

1. OpenXR–AndtheFutureof
AR/VR/MRGaming
Brent Insko, Virtual Reality Center of Excellence
October, 2017

2. Where is VR today?
2

3. 3
Gartner
Hype Cycle
2017
© 2017 Gartner, Inc.

4. What is holding VR back?
4

5. What is holding VR back?
• Content
• Comfort
• Cost
5

6. 6

7. 7
Platforms

8. 8
Platforms VR Platforms

9. 9
3 PCVR
Verticals
Oculus Store
Microsoft
Store
Mixed Reality
App
Mixed Reality
Runtime
Oculus App
Oculus
Runtime
Mixed Reality
HMDs
Oculus
Rift
Steam Store
SteamVR
App
SteamVR
Runtime
HTC Vive
(+ HDK &
others)

10. 10

11. 11

12. 12

13. 13

14. What is a VR Runtime?
14

15. 15
High Level VR Software Stack
VR Runtime
Server Compositor
DirectX (+ other system APIs)
Windows
VR Application Browser
VR
Hardware
OpenXR
APIs
WebVR
Game
Engine
Application
interface
Driver(HW)
interface

16. • Takes Windows surfaces from
multiple applications
• Composites onto a single
desktop in 2.5D
• Handles transparency/
translucency
Windows Compositor… Desktop Window Manager (DWM)

17. VR Compositor
T1 T2 T3
• Composition
• Time Warp
• Barrel Distortion
• Chromatic Aberration
Correction
VR Compositor

18. 18
• Head and
controller positions
• Controller state
• Input events
• Game images
• Haptics events
Information Flow

19. 19

20. Why does Intel care?
20

21. VR Vision – The PC Platform
The PC is doing more than ever to stimulate the user and drive their senses
Physicallydriven audio
Audio effects that are true to life, reflecting off
objects and surface materials, accounting for
distance and orientation.
Intense graphics
Higher resolution, faster frame rates, and depth capable
displays, fully immerse players in the experience.
Hands-free input
Depth cameras incorporated into HMDs and
connected to the PC track arm and hand
movement allow controller-less input
Multiplayer support
Simulation accounts for multiple players in the
same room – including both players VR experience
being driven off of one high-performance PC.
Wireless freedom
High-speed, low-latency wireless
driving of all peripherals.
Haptic feedback
Vest and controllers mimic in-game
interactions to further stimulate the
player and make interactions between
teammates more physical.
Vest reacts with jolt from enemy
weapon fire.
Vest and controllers relay low health
condition.
Physically accept a healing item
handed to you by a teammate. Object
heft is conveyed to you.
1
2
3
We believe VR will evolve to allow multi-modal input and feedback. Here we show a variety of
the technologies that the PC will be providing feedback to the user.
CPU Workloads
Particle effects and physics, Artificial
intelligence, Destruction and Object Persistence,
Initialization & level setup, Spatial binaural 3D
audio, frame rate, mega-tasking workloads

22. VR Vision – The Sensor Platform
Fully immersing users in VR will include full awareness of their state and movement
Facialexpressiontracking
Mask sensors map your reactions to your in
game character, enhancing non-verbal
communication and heightening involvement.
Object scanning
Depth cameras in HMD allow scaling real
world objects to bring in game. Swing a
favorite toy sword or bring a child’s art class
project into social spaces.
Eye tracking
Players eyes are reflected in their in game
character, expressing emotion and intent.
Gaze tracking can unlock UI triggers, and
drive rendering performance focus.
Biometric sensors
Heart rate, pupil dilation and galvanic skin
response can be used to convey
emotional responses such as fear or
excitement.
Responses can be analyzed to tailor game
difficulty, give cues to other players, or to
limit fatigue.
At the same time, the PC will be capturing a wide array of conscious and unconscious input
from the player, and feeding this back to the simulation for processing
Intuitive Controllers
Controllers evolve to intuitively
communicate greater degrees of intent, with
analog triggers, contact sensing, and greater
degrees of affordance
Physics-driven presence
Motion tracking of HMDs and controllers, combined with
3D cameras, and CPU-driven physics are used to track and
confer full body articulation .

23. 23
3rd
Person Camera AI
(Yoann PignoleGamasutra2015)
GFX
Source (PC) Sink (HMD)
Panel
(Output)
Sensors
(Input)
Render
Time Warp
Barrel Distortion
Chroma Correction
Display
(Scalar)
Pose Input
Imaging Data
Tx Rx
USB Display
Displaylink Wireless VR
Intel’s partnership with ISVs
on VR content
Intel Projects

24. 2020 Virtual Reality Vision
Adventure together with friends, compete with rivals,
or bring the world along for the show
Open Innovation & Best of class performance allow VR to thrive and lead on PC
Usage: Truly Immersive multiplayer gaming allows players to travel
to other worlds, whether next to each other or across the globe
Focus areas: Gaming, eSports, Social
Key capability areas (hypotheses)
• Mind-tripping quality: Photo-real, deep behavioral simulation,
responsive enough to exceed sensory reqs. “Believed I was there”
• Wireless HMDs and peripherals and/or wearable form factors for
unhindered and unlimited movement
• Multi sensorystimulation: physiological-transparent quality
display & graphics, physically modeled 3D Audio, (haptic
stimulation), (vestibular dampening),
• Multi-model input, sensing, and responsive simulation: eye-
tracking, expression tracking, tactile feedback, biometric feedback
(galvanic response, heart rate, etc), voice & outside audio cuing,
environment capture
• Social & competitive: Your expressions & movements driving your
presence in virtual interactions with others
New York Los Angeles London
WW Spectators

25. The Future of AR/VR/MR Gaming & OpenXR
• Engaging more senses, better experiences
à more immersion
à more presence
• More innovation
25

26. The Future of AR/VR/MR Gaming & OpenXR
• Engaging more senses, better experiences
à more immersion
à more presence
• More innovation
• Dependent on:
à more content more places
à more hardware more places
à easier cross-platform development & testing
26

27. PlaceholderFooterCopy /BULogo orName Goes Here

28. 28
à more content more places
à more hardware more places
à easier cross-platform development & testing