In this session, Amazon Sumerian leaders provide an overview of the evolution of the augmented reality/virtual reality (AR/VR) industry. They discuss what has changed since Sumerian was announced at re:invent 2017. They also talk about key market trends, and they highlight areas with the highest adoption and potential. This is an immersive session, with examples and demonstrations of immersive experiences created using Amazon Sumerian.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s New in AR/VR: State of the
World Report
Kyle Roche
General Manager
Amazon Sumerian
@kylemroche
A R V 2 0 3
Nell Waliczek
Principal Engineer
Amazon Sumerian
@nellwaliczek
Rose Phillips
SVP Digital Marketing
Sony Pictures
@rosemccotter
Jason Yim
CEO
Trigger
@triggerglobal

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Why did we build Amazon Sumerian?
What makes it unique
WebXR: What is it and why should you care?
Spider-Man: Into the Spider-Verse
What’s new with Amazon Sumerian

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Sumerian

5. AR/VR is coming

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We know it’s coming; that’s why you’re here
• Already working on AR/VR for your business (or personal project)
• You are planning on working on AR/VR projects
• It’s too intimidating to get started
• The Amazon SageMaker session was already full and we’re right across
the hall

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We’ve seen this type of transformation before
Web ImmersiveMobile Cloud

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Invest and Grow – Lessons from shift to cloud
Trade capital
expense for variable
expense
Use the people and
tools you already know
and trust
Test, iterate, scale, and
double down on what
works

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Walled Gardens
Engine Selection
Device Plug-ins
Builds per Platform
Distribution
Getting started in XR

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Sumerian

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud-first approach to assets

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine learning

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sumerian Hosts

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sumerian Hosts

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build once, deploy broadly

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Collaborative Space Planning

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AR/VR for Visualization

20. WebXR is coming

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Upcoming W3C standard
• Provided by browsers
• Consumed in web pages
• API web developers to communicate with AR and VR hardware
• Browser availability expected in 2019
• Replaces previous WebVR experimental API
What is WebXR?

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Relationship to WebGL
• WebGL is used to render pixels
• It is the API web developers use to draw 2D and 3D graphics
• WebGL 1.0 and WebGL 2.0 are both widely available
• WebXR then talks to XR hardware to display the pixels

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WebGL Hello World

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WebGL Hello World

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
More artistic

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3D Engines

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So why learn about WebXR?

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Properties of XR hardware
• VR vs. AR
• See-through vs. pass-through
• Head-worn vs. handheld
• Stationary vs. bounded vs. unbounded

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Work in progress!

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WebXR core concepts
• Session
• Reference Space
• Input

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Session
Active connection used to communicate with XR hardware

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Session
Active connection used to communicate with XR hardware

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Session
Active connection used to communicate with XR hardware

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aside: Security, privacy, and user consent

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Space

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Space
Establishes mobility requirements of an experience

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Space
Establishes mobility requirements of an experience

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Space
Establishes mobility requirements of an experience

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Input

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Input
Common functionality is “aim and select”

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Input
Common functionality is “aim and select”

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Input
Common functionality is “aim and select”

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How to WebXR
1. Check for XR support
2. Request device access
3. Make a reference space
4. Create WebGL resources
5. Register for input events
6. Run the loop

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Checking for XR support
var enterARButton = document.getElementById("enterARButton");
function checkARSupport() {
navigator.xr.supportsSessionMode('immersive-ar’)
.then(() => { enterARButton.disabled = false; })
.catch(() => { enterARButton.disabled = true; });
}
checkARSupport();
navigator.xr.addEventListener('devicechange', checkARSupport);

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Requesting device access
function onClickEnterVR() {
navigator.xr.requestSession({mode: 'immersive-ar’})
.then(onSessionStarted)
.catch(() => { /* Show appropriate error message */ });
}

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Requesting device access
let xrSession = null;
function onSessionStarted(session) {
xrSession = session;
createReferenceSpace()
.then(createWebGLLayer)
.then(() => {
registerForInputEvents();
xrSession.requestAnimationFrame(onFrameCallback);
});
}

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How to WebXR
1. Check for XR support
2. Request device access
3. Make a reference space
4. Create WebGL resources
5. Register for input events
6. Run the loop

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating a reference space
let xrReferenceSpace = null;
function createReferenceSpace() {
let options = { type:'stationary', subtype:'floor-level’ };
let promise = xrSession.requestReferenceSpace(options).then((referenceSpace) => {
xrReferenceSpace = referenceSpace;
});
return promise;
}

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating WebGL resources
let glCanvas = document.getElementById('xrCanvas');
let glContext = glCanvas.getContext('webgl');
function createWebGLLayer() {
let promise = glContext.makeXRCompatible().then(() => {
xrSession.baseLayer = new XRWebGLLayer(xrSession, glContext);
});
}
glCanvas.addEventListener('webglcontextlost', onContextLost);
glCanvas.addEventListener('webglcontextrestored', onContextRestored);

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating WebGL resources
let glCanvas = document.getElementById('xrCanvas');
let glContext = glCanvas.getContext('webgl');
function createWebGLLayer() {
let promise = glContext.makeXRCompatible().then(() => {
xrSession.baseLayer = new XRWebGLLayer(xrSession, glContext);
});
}
glCanvas.addEventListener('webglcontextlost', onContextLost);
glCanvas.addEventListener('webglcontextrestored', onContextRestored);

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Registering for input events
let xrInputSources = null;
function registerForInputEvents() {
xrSession.addEventListener('inputsourceschange', () => {
xrInputSources = xrSession.getInputSources();
});
xrSession.addEventListener('select', onSelect);
xrSession.addEventListener('selectstart', onSelectStart);
xrSession.addEventListener('selectend', onSelectEnd);
}

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How to WebXR
1. Check for XR support
2. Request device access
3. Make a reference space
4. Create WebGL resources
5. Register for input events
6. Run the loop

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frame callback
function onXRFrameCallback(timestamp, frame) {
let pose = frame.getViewerPose(xrReferenceSpace);
if (pose) {
updateInputSources(frame);
runSimulationStep();
drawViews(pose);
}
xrSession.requestAnimationFrame(onXRFrameCallback);
}

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frame callback
function onXRFrameCallback(timestamp, frame) {
let pose = frame.getViewerPose(xrReferenceSpace);
if (pose) {
updateInputSources(frame);
runSimulationStep();
drawViews(pose);
}
xrSession.requestAnimationFrame(onXRFrameCallback);
}

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How to WebXR
1. Check for XR support
2. Request device access
3. Make a reference space
4. Create WebGL resources
5. Register for input events
6. Run the loop

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s next for WebXR?
• Expanded input functionality
• Finalizing AR features
• Enabling HTML/CSS 2D content within XR
• Completing standardization

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spider-Man
Rose Phillips
SVP Digital Marketing
Sony Pictures
@rosemccotter
Jason Yim
CEO
Trigger
@triggerglobal

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s New from Amazon Sumerian
• AWS Amplify Integration
Securely embed scenes in your own web apps
• VR Controller Asset Pack
Easily add interactivity to VR scenes

62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s New from Amazon Sumerian
• New Hosts
Give your on-screen avatar a new look
• Animations as Assets
Share, reuse, and update animation clips
• Japanese Language Support
Use the Sumerian editor in Japanese

63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s New from Amazon Sumerian
• Multi-User Sharing
Let multiple users collaborate on scenes
• Sumerian for ML (labs)
Run robotics simulations in Sumerian

64. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kyle Roche / @kylemroche
Nell Waliczek / @nellwaliczek

65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.