AR/VR in JavaScript Apps discusses emerging technologies for building augmented and virtual reality experiences using web technologies. It introduces key concepts like AR, VR, and XR. It also outlines several JavaScript libraries and frameworks that can be used to create 3D content and immersive experiences, such as Three.js, WebXR, A-Frame, and React 360. The document recommends resources for continued learning and provides examples of how to get started with these technologies.

1. AR/VR in JavaScript Apps
FITC Web Unleashed 2018

2. The Rise of Extended Reality (AR/VR)
• AR (Augmented Reality) – Interaction with physical realty
augmented with computer generated input and output
• VR (Virtual Reality) – Replace physical reality with a computer
generated one
• Hardware costs plummeting (Oculus Go launched F8 2018,
$199)
• Software for building AR and VR experiences getting better
every day

4. Introduction – Hasan Ahmad
• Principal Consultant at DEV6
• Developer Circles from Facebook
(Toronto)
• https://dev6.com
• https://www.facebook.com/groups
/DeveloperCirclesToronto/
• https://twitter.com/has_ibr_ahm

5. Industries that are embracing XR
• Gaming
• Media
• Mobile Apps
• Streaming
• Education
• Industrial
• Retail

6. AR/VR tech-stack for web devs
• Little bit of math & physics background
• Smartphone or VR headset
• Three.js
• WebVR API / Web XR API
• A-Frame
• React 360
• AR.js
• 3D Content

7. Three.js
• https://threejs.org/
• WebGL graphics library
• Scene – What to display
• Camera – What to view
• Renderer – How to display
• Geometry – Objects, textures, etc

8. $ npm install three
...
import { Scene } from 'three’;
const scene = new Scene();
OR
<script
src="https://fastcdn.org/three.js/73/three.min.js">
</script>

9. var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(75,
window.innerWidth / window.innerHeight,
0.1,
1000);
var renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
var geometry = new THREE.BoxGeometry(1, 1, 1);
var material = new THREE.MeshBasicMaterial({color: 0x00ff00});
var cube = new THREE.Mesh(geometry, material);
scene.add(cube);
camera.position.z = 5;

10. WebVR
• Low-level API to gather info about the VR display + capabilities
• Eye parameters, the data to render the scene for each eye
• Field of view data
• Position of the VR display (and velocity and acceleration)

11. WebXR (unstable!)
• Evolution of WebVR spec
• Much faster than WebVR API
• Better architecture to support both VR and AR, multiple view
types
• Desktop
• VR Headset
• Smartphone
• Magic Window
• Touchscreen, Mouse, Gamepad, Controllers

12. The future of the web is immersive (Google I/O ‘18)
https://www.youtube.com/watch?v=1t1gBVykneA

13. WebXR-Polyfill
• Best way to make sure code written to XR spec (proposal) will
actually work
https://github.com/immersive-web/webxr-polyfill
<script src='https://cdn.jsdelivr.net/npm/webxr-
polyfill@latest/build/webxr-polyfill.js’></script>
$ npm install --save webxr-polyfill

14. function initXR() {
xrButton = new XRDeviceButton({
onRequestSession: onRequestSession,
onEndSession: onEndSession
});
document.querySelector('header').appendChild(xrButton.domElement);
if (navigator.xr) {
navigator.xr.requestDevice().then((device) => {
device.supportsSession({immersive: true}).then(() => {
xrButton.setDevice(device);
});
});
}
}
function onRequestSession(device) {
device.requestSession({immersive: true}).then(onSessionStarted);
}

15. function onSessionStarted(session) {
xrButton.setSession(session);
session.addEventListener('end', onSessionEnded);
gl = createWebGLContext({compatibleXRDevice: session.device});
renderer = new Renderer(gl);
scene.setRenderer(renderer);
session.baseLayer = new XRWebGLLayer(session, gl);
session.requestFrameOfReference('eye-level').then((frameOfRef) => {
xrFrameOfRef = frameOfRef;
session.requestAnimationFrame(onXRFrame);
});
}

16. function onXRFrame(t, frame) {
let session = frame.session;
scene.startFrame();
session.requestAnimationFrame(onXRFrame);
let pose = frame.getDevicePose(xrFrameOfRef);
if (pose) {
gl.bindFramebuffer(gl.FRAMEBUFFER, session.baseLayer.framebuffer);
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
for (let view of frame.views) {
let viewport = session.baseLayer.getViewport(view);
gl.viewport(viewport.x, viewport.y,
viewport.width, viewport.height);
scene.draw(view.projectionMatrix, pose.getViewMatrix(view));
}
} else {}
scene.endFrame();
}

17. A-Frame
• Web framework originally from Mozilla for rendering AR and VR
in web pages
• Declarative syntax
• 3D scene graph with markup language

18. <head>
<script src="https://aframe.io/releases/0.8.0/aframe.min.js"></script>
</head>
$ npm install aframe
...
require(‘aframe’);

19. <body>
<a-scene>
<a-box position="-1 0.5 -3"
rotation="0 45 0" color="#4CC3D9"></a-box>
<a-sphere position="0 1.25 -5"
radius="1.25" color="#EF2D5E"></a-sphere>
<a-cylinder position="1 0.75 -3" radius="0.5"
height="1.5" color="#FFC65D"></a-cylinder>
<a-plane position="0 0 -4" rotation="-90 0 0"
width="4" height="4" color="#7BC8A4"></a-plane>
<a-sky color="#ECECEC"></a-sky>
</a-scene>
</body>

20. Loading 3D Models
<a-assets>
<a-asset-item id="cityModel"
src="https://cdn.aframe.io/test-models/models/virtualcity/VC.gltf">
</a-asset-item>
</a-assets>

21. https://aframe.io/
https://aframe.io/aframe-school/#/

22. $ npm install -g react-360-cli
$ react-360 init Hello360
$ cd Hello360
$ npm start
React 360

23. React 360
• You can use React to
build VR web UIs
• Render React Native
components in 3D

24. import { AppRegistry, StyleSheet, Text, View } from'react-360;
export default class Hello360 extends React.Component {
render() {
return (
<View style={styles.panel}>
<View style={styles.greetingBox}>
<Text style={styles.greeting}>
Welcome to React 360
</Text>
</View>
</View>
);
}
};

25. React 360
• Similar in Architecture to React Native
• Uses Web Workers to avoid single-
threaded computation limitation, which
could impact performance, break
immersion

26. React 360
• Can also load 3D models, using Entity (multiple formats)
// to reference a GLTF2 model
<Entity source={{gltf2: asset('myModel.gltf')}} />
// to reference an untextured OBJ model
<Entity source={{obj: asset('myModel.obj')}} />
// to reference an OBJ with matching MTL file
<Entity source={{obj: asset('myModel.obj'), mtl: asset('myModel.mtl')}} />

27. Augmented Reality
• Similar problems solved in VR
• Must be able to identify real-
world geometry
• Capable of marker-based AR at
60fps, even on budget
smartphones
https://aframe.io/blog/arjs/

28. Building AR with A-Frame (AR.js)
<script
src="https://jeromeetienne.github.io/AR.js/aframe/build/aframe-ar.js">
</script>
<a-scene embedded arjs>
<a-marker-camera preset='hiro'></a-marker-camera>

29. Hiro Marker

30. Building AR with A-Frame
<body style='margin : 0px; overflow: hidden;’>
<a-scene embedded arjs>
<!-- create your content here. just a box for now -->
<a-box position='0 0.5 0' material='opacity: 0.5;'></a-box>
<!-- define a camera which will move according to the marker position -->
<a-marker-camera preset='hiro'></a-marker-camera>
</a-scene>
</body>

31. Wikitude SDK
• Paid SDK, for implements sophisticated AR algorithms,
available as a plugin for Native or Cordova projects
• Free trial available (for experiments and education)
• Instant Tracking, SLAM, SMART
• Built on top of ARCore and ARKit
• https://www.wikitude.com/augmented-reality-instant-tracking/

32. 3D Assets
• https://sketchfab.com/
• https://poly.google.com/
• Create or buy 3D content to build amazing AR and VR
experiences.

33. Continued Study
• https://www.khanacademy.org/math/linear-algebra
• https://medium.com/@necsoft/three-js-101-hello-world-part-1-
443207b1ebe1
• https://developer.mozilla.org/en-
US/docs/Web/API/WebGL_API#Guides
• https://github.com/mozilla/aframe-xr
• https://aframe.io/blog/arjs/

34. Summary
• Extended Reality is of increasing interest to many industries
• Web tech can get us quite far, even with today’s experimental
APIs
• There are a number of entry points into this tech stack, pick the
right level of abstraction for you

35. Thank You!