This document discusses building augmented reality (AR) and virtual reality (VR) experiences in the browser. It begins with an overview of AR and VR technologies available today like A-Frame. It then demonstrates creating a VR art gallery and adding AR functionality using AR.js markers. The document emphasizes optimizations needed for media-heavy AR/VR experiences like resizing images, compression formats, and loading only visible assets. It concludes by discussing the future of AR hitting points in browsers using WebXR and encourages building AR/VR applications.

1. @dougsillars
Augmented Reality on the Web:
Building an ARt Gallery
Doug Sillars
April 4, 2019
UXHel

2. @dougsillars
Contact Me:
@DougSillars
Doug.Sillars@gmail.com
www.dougsillars.com
Doug Sillars
Freelance Developer Relations
Performance Audits: Web/Native
Workshops:
Performance/Images/Video/AR
http://bit.ly/HighPerformanceAndroidApps

3. Virtual Reality

4. Virtual Reality
https://www.flickr.com/photos/68158920@N08/17742136272

5. Virtual Reality

6. Virtual Reality

7. Augmented Reality

8. Augmented Reality

9. Augmented Reality
• Wearing fancy occulus with cameras
• Really $$$

10. Augmented Reality –
Smartphone
https://www.flickr.com/photos/bellemarematt/28124392062
https://www.flickr.com/photos/bagogames/28628142444

11. Augmented Reality –
Smartphone
https://www.flickr.com/photos/bellemarematt/28124392062
https://www.flickr.com/photos/bagogames/28628142444

12. AR/VR- in the Browser
• What we can do today
• What is coming tomorrow

13. AR/VR- in the browser
• What we can do today
• A-Frame
• https://aframe.io
• https://github.com/dougsillars/ARtGallery

14. AR/VR- in the browser
• What we can do today
• A-Frame
• https://aframe.io
• https://github.com/dougsillars/ARtGallery

15. Setup
• https://github.com/dougsillars/ARtGallery
git clone https://github.com/<username>/ARtGallery
Seeing your Code
GitHub Account No GitHub Account
Details:
https://gist.github.com/jgravois/5e73b56fa7756fd00b89
Seeing your Code
Local Webserver
python -m SimpleHTTPServer 1337
In Browser: Localhost:1337

16. VR with A-Frame
https://dougsillars.github.io/ARtGallery/aframeHW.html

17. VR with A-Frame
https://dougsillars.github.io/ARtGallery/aframeHW.html

18. VR with A-Frame
https://dougsillars.github.io/ARtGallery/aframeHW.html

19. VR with A-Frame
https://dougsillars.github.io/ARtGallery/aframeHW.html

20. VR with A-Frame
https://dougsillars.github.io/ARtGallery/aframeHW.html

21. Building a Gallery
https://dougsillars.github.io/ARtGallery/aframeroom.html

22. VR ART
https://dougsillars.github.io/ARtGallery/aframeroomart.html

23. VR ART
https://dougsillars.github.io/ARtGallery/justart.html
Front Wall

24. VR ART
https://dougsillars.github.io/ARtGallery/justart.html
Front Wall
2m up

25. VR ART
https://dougsillars.github.io/ARtGallery/justart.html

26. VR Art Animation
https://dougsillars.github.io/ARtGallery/justart1.html

27. VR More Art
https://dougsillars.github.io/ARtGallery/aframeroomart2.html

28. VR More Art
https://dougsillars.github.io/ARtGallery/aframeroomart2.html

29. VR More Art
https://dougsillars.github.io/ARtGallery/aframeroomart2.html

30. GLTF Models

31. GLTF Models

32. VR Models Are Large
We won’t forget optimization!!

33. AR with A-Frame
We’ll use AR.js to add AR functionality to the page
3D objects are placed with Markers:

34. AR Markers
http://bit.ly/ARHiro

35. AR Markers
Create Your Own Markers
https://jeromeetienne.github.io/AR.js/three.js/examples/mark
er-training/examples/generator.html
Creates an image to print and tape to
the wall.
Creates a .patt pattern file for the
browser

36. AR Markers
Create Your Own Markers
https://jeromeetienne.github.io/AR.js/three.js/examples/mark
er-training/examples/generator.html
Creates an image to print and tape to
the wall.
Creates a .patt pattern file for the
browser

37. AR Markers
Create Your Own Markers

38. AR Markers to Create A Scene
https://dougsillars.github.io/ARtGallery/ARt.html

39. Optimisations for AR/VR
• Images are integral to the view – we want them to load quickly

40. Optimisations for AR/VR
• Reduce File Size

41. Optimisations for AR/VR
• Reduce File Size
• Resize images (to power of 2)
https://res.cloudinary.com/dougsillars/image/upload/h_1024/v1552460601/
VanGogh-starry_night_qoohur.jpg

42. Optimisations for AR/VR

43. Optimisations for AR/VR

44. Optimisations for AR/VR
• Optimize Quality
https://res.cloudinary.com/dougsillars/image/upload/h_1024,
q_auto,/v1552460601/VanGogh-starry_night_qoohur.jpg

45. Optimisations for AR/VR
• Optimize Format
https://res.cloudinary.com/dougsillars/image/upload/h_1024,
q_auto,f_auto/v1552460601/VanGogh-
starry_night_qoohur.jpg

46. Optimisations for AR/VR
• Optimize Format
https://res.cloudinary.com/dougsillars/image/upload/h_1024,
q_auto,f_auto/v1552460601/VanGogh-
starry_night_qoohur.jpg

47. Optimisations for AR/VR
• Optimize Format
https://res.cloudinary.com/dougsillars/image/upload/h_1024,
q_auto,f_auto/v1552460601/VanGogh-
starry_night_qoohur.jpg
1.8 MB -> 323 KB

48. Optimisations for AR/VR
• Optimize Format
https://res.cloudinary.com/dougsillars/image/upload/h_512,
q_auto,f_auto/v1552460601/VanGogh-
starry_night_qoohur.jpg
1.8 MB -> 93 KB

49. Optimisations for GLTF
Draco Compression optimizes .bin files

50. AR with WEBXR

51. AR with WEBXR
• All WebXR Demos require:
• Android 8.0+
• Chrome Canary v70-72 (late 2018)
• Flags enabled:
• WebXR Device API
• WebXR Hit Test
• ARCore installed
Protip:
If using WebXR a lot, prevent Canary
from auto-updating in Play Store

52. WEBXR Hit Test
https://codelabs.developers.google.com/codelabs/ar-with-webxr/#4

53. WEBXR Hit Test
https://dougsillars.github.io/ar-with-
webxr/WebXrARt/

54. Optimisations for WebXR
1024X1024 1024X1025

55. https://res.cloudinary.com/dougsillars/image
/upload/x_230,y_128,w_512,h_512,c_crop/
w_256/v1552682248/stabilization_yegbkb.gi
f
850 KB -> 174 KB
Optimisations for aGIF

56. Optimisations for PNG
https://res.cloudinary.com/dougsillars/image/upl
oad/q_auto,w_256/c_crop,w_256,h_256/v15526
82248/Anchor_rzjbco.png
• Optimise quality, shrink to 256
KB wide
• Crop to 256x256
• 36 KB -> 2.2 KB

57. Loading WEbXR Art Gallery (Optimized)

58. Conclusions
• AR on the web is available today with A-Frame
• AR with Hit Points is coming
• AR does not have to be processor intensive, nor utilize huge
amounts of data

59. Let’s build!
• Art Files you can use:
• https://res.cloudinary.com/dougsillars/image/upload/w_1000/v1551634938/scream_qbwybi.jpg
• https://res.cloudinary.com/dougsillars/image/upload/w_1000/v1551634946/monalisa_nlcskz.jpg
• https://res.cloudinary.com/dougsillars/image/upload/w_1000/v1551634942/Mural_del_Gernika_s9ld
ah.jpg
• https://res.cloudinary.com/dougsillars/image/upload/v1552460601/VanGogh-
starry_night_qoohur.jpg
• https://res.cloudinary.com/dougsillars/image/upload/v1553694737/CAPPELLA_SISTINA_Ceiling_oosz
7w.jpg
• https://res.cloudinary.com/dougsillars/image/upload/v1553694758/last_supper_hykuux.jpg
• https://res.cloudinary.com/dougsillars/image/upload/v1553694762/The_Nightwatch_kkzvus.jpg
• Try: Sketchfab.com for 3D Models

60. Links
Code: https://github.com/dougsillars/ARtGallery
https://github.com/dougsillars/ar-with-webxr
WebXR spec: https://www.w3.org/TR/2019/WD-webxr-20190205/
Cloudinary: https://cloudinary.com

61. Love Building with Video and Images?
Media Developer Expert
mde-comm@cloudinary.com
Become a