This document provides an overview of VR, AR and MR technologies and the A-Frame framework for building VR experiences. It discusses that VR immerses the user in a simulated environment, AR overlays digital content on the real world, and MR combines both. Key factors that have enabled growth in these areas are improved smartphone cameras and processing power. The document outlines important input sensors, processing techniques like SLAM, graphics APIs, and frameworks like A-Frame that can be used to build VR/AR/MR applications. It encourages experimenting with new opportunities in this emerging field.

1. VR/AR/Mixed Reality
Rabimba Karanjai
@rabimba

2. VR/AR/MR in a
Nutshell
Rabimba Karanjai,
Mozilla Research Mixed Reality
Mozilla Tech Speaker
@rabimba

3. What the Heck Doodle?
ARKit
ARCore
MR
VR
XR
AR
Oculus
Sony
Vive
Meta
HoloLens

4. What You’ll Learn Today
● What is VR / AR / MR?
● Why is this all happening now?
● What do I really need to know?

5. What is VR/AR/MR?
VR: Virtual Reality AR: Augmented Reality
MR: Mixed Reality XR: We don’t know Reality

6. Virtual Reality
● Completely immersive
● Wide field of view
● Fast refresh rate (90+fps)
● Fast sensor refresh

7. Augmented
Reality
● Overlaid on real world
● Anchored to real world
● Non-immersive

9. Partial AR

13. Why Now?

14. The Long History
of VR/AR/MR
Ivan Sutherland, MIT
Sword of Damocles (1968)
3D wireframe rooms

16. 1980s: VPL
● Data Glove
● Eye Phone
● Data Suit

17. Mattel’s Nintendo Power Glove

18. 1990s: Virtuality

20. Virtual Reality
Markup Language
● Scene based
● In the browser
● Slow to download
● Slow to render

22. Smartphone Era

23. Smartphone+ Era
● Dual cameras
● Depth sensing cameras
● Machine Learning in
hardware
● Even more powerful
GPU & CPU

24. What do I
need to
know?

25. Tech Stack for VR & AR
Input: Sensors & Camera Processing: Image & Sensor Fusion
Drawing: 3D Graphics API Application: Framework

26. Input: Sensors & Camera
● Accelerometer: relative to gravity
● Magnetometer: relative to earth’s magnetic field
● Gyroscope: relative angle movement
○ 9DOF sensor breakout board for 15$ @ Adafruit!
● Optical Camera: standard phone camera
● Stereo Camera: dual camera setup
● Depth Sensing Camera: Kinect, Face ID

27. Processing the 3D
Environment
● Markers: Symbols or Beacons
● SLAM: Simultaneous Localization and Mapping
● MonoSLAM: SLAM in real-time with just one camera

28. HTC Vive Sensor
Beacon
two beacons + headset sensors = highly
accurate

29. Processing: VR/AR
APIs
● Vuforia
● ARToolkit
● WebVR
● ARKit
● ARCore
● WebXR

30. Vuforia

31. ARToolkit

32. WebVR
● Mobile: used by web
apps for Google
Cardboard on any
phone.
● Desktop: works with
major HMDs
● Shipping: Firefox for
Windows
● Beta: Chrome, FF for
other platforms

33. ARKit

34. Tango
ARCore

35. WebXR: Work In Progress!
● Proposed Future Web Standard
● Extend WebVR with new features
● Work with ARKit & ARCore
● Polyfill for old browsers using WebRTC + sensors
https://github.com/mozilla/webxr-api
https://github.com/mozilla/webxr-polyfill

36. Drawing: Graphics APIs
● Medium Level
○ OpenGL (ES)
○ DirectX
○ WebGL
● Low Level
○ Metal
○ Vulkan
○ Direct X 12
● High Level
○ Unity
○ Unreal
○ Three.js
○ Babylon.js

37. Application: Frameworks
● Scene Kit
● Cardboard & Daydream
● Hololens Academy
● React VR
● ArgonJS
● A-Frame

38. What do you *really* need to know?
● Start with web stack
● Follow Mozilla and Google and Microsoft
● Play with A-Frame
● Get a Cardboard viewer.

39. What can I do?
● Experiment! We are still in the Model T days!
● No one knows what the killer app will be.
● First new medium in decades
● Go play!

40. State of WebVR and Aframe
The web framework for building VR experiences
@rabimba | Mozilla VR | aframe.io

41. Friction of VR Ecosystems
Gatekeepers Installs Closed

42. WebVR
An open virtual reality platform with the advantages of
the Web
Open Connected Instant

43. Browser APIs that enable WebGL rendering to headsets
and access to VR sensors

44. https://w3c.github.io/webvr/

45. Firefox
Nightly
Chromium
(Experimental)
Samsung
Internet
Mobile
Polyfill

46. A-Frame
A declarative framework for building virtual reality
experiences on the Web

47. Hello World
<a-scene>
<a-box color="#4CC3D9" position="-1 0.5 -3" rotation="0 45 0"></a-box>
<a-cylinder color="#FFC65D" position="1 0.75 -3" radius="0.5" height="1.5
<a-sphere color="#EF2D5E" position="0 1.25 -5" radius="1.25"></a-sphere>
<a-plane color="#7BC8A4" position="0 0 -4" rotation="-90 0 0" width="4" h
<a-sky color="#ECECEC"></a-sky>
</a-scene>

48. Hello World

49. <a-animation> <a-box> <a-camera> <a-circle> <a-collada-model>
<a-cone> <a-cursor> <a-curvedimage> <a-cylinder>
<a-dodecahedron> <a-isocahedron> <a-image> <a-light>
<a-obj-model> <a-plane> <a-ring> <a-tetrahedron> <a-torus>
<a-torus-knot> <a-sky> <a-sound> <a-sphere> <a-videosphere>

50. Registry
Curated collection of A-Frame components

52. Registry
Curated collection of A-Frame components

53. Motion Capture
Record then develop room scale VR on the go
Remix Glitch

54. Community
https://aframe.io/blog/

55. Powerful and Extensible

57. Art - A-Painter
@mozillavr

58. Journalism - Fear of the Sky
Amnesty International UK

59. Journalism - Journey to
Mars
The Washington Post

60. Guri VR
Build your own world in VR

61. Sandbox - Aincraft
@andgokevin

62. Aincraft
Lets play

63. Data Visualization - Adit
@datatitian

64. Gaming - A-Blast
@mozillavr

65. AR - AR.js + A-Frame
@jerome_etienne

66. Come build WebVR with us!
aframe.io/community

67. Thank you
@rabimba | rabimba@mozilla.com| aframe.io