The document discusses Web augmented reality (AR) and Google's work in this area. It provides an overview of key Web AR libraries and APIs like AR.js, js-aruco, and three.ar.js. It explains how three.ar.js integrates AR capabilities into three.js for WebAR experiences using devices' sensors via ARCore, Tango, or ARKit. It also outlines the architecture for building WebAR apps and how they differ from native AR apps in relying on the browser, WebGL, and device sensors instead of native SDKs.

2. Who are U?
Hirokazu Egashira (eegozilla)
html5j
html5j
html5j Web
Android
LeapMotion Developers JP
GoogleARhirokazu.egashira
eegozilla (@ega1979)
Copyright @Hirokazu Egashira. All right reserved.

3. Google AR
Google ARCore/Tango
EOL
Google AR
Google AR
Copyright @Hirokazu Egashira. All right reserved.

4. /
Copyright @Hirokazu Egashira. All right reserved.

5. Copyright @Hirokazu Egashira. All right reserved.

6. Google AR WG / Google
https://plus.google.com/communities/117782112124490967221
Copyright @Hirokazu Egashira. All right reserved.

7. Web + AR
Copyright @Hirokazu Egashira. All right reserved.

8. Advances in Web Technology for Augument Reality
<video>
<canvas>
WebRTC’s getUserMedia();
WebGL three.jsvideo canvas OpenCV + emscripten
WebVR
Copyright @Hirokazu Egashira. All right reserved.

9. W3C Augument Reality Community Group
Copyright @Hirokazu Egashira. All right reserved.
2011 W3C Community

10. Build Augument Reality Experience with Web
getUserMedia three.js Web AR
JS ARToolkit 5
js-aruco
AR.js
Argon
etc
Copyright @Hirokazu Egashira. All right reserved.

11. AR Library_01 JS ARToolkit 5
JavaScript AR JavaScript ARToolkit Java NyARtoolkit
FLASH FLARToolkit
5 ARToolkit Emscripten Asm.js JavaScript
Copyright @Hirokazu Egashira. All right reserved.

12. cf.(1)The case of loading image
Copyright @Hirokazu Egashira. All right reserved.

13. cf.(2)Using getUserMedia
Copyright @Hirokazu Egashira. All right reserved.

14. Copyright @Hirokazu Egashira. All right reserved.
cf.(3)three.js’s helper API

15. AR Library_02 js-aruco
OpenCV AR ArUco JavaScript
Copyright @Hirokazu Egashira. All right reserved.

16. Install
JavaScript
Copyright @Hirokazu Egashira. All right reserved.

17. AR Library_03 AR.js
ARToolkit Emscripten asm.js JavaScript 3D three.js
Alexandra Etienne Jerome Etienne JavaScript AR
Github A-frame WebVR Polyfill babylon.js
Copyright @Hirokazu Egashira. All right reserved.

18. cf. Using three.js API
Copyright @Hirokazu Egashira. All right reserved.

19. Copyright @Hirokazu Egashira. All right reserved.

20. Copyright @Hirokazu Egashira. All right reserved.

21. Copyright @Hirokazu Egashira. All right reserved.

22. cf. Using WebVR polyfill API
Copyright @Hirokazu Egashira. All right reserved.

23. Copyright @Hirokazu Egashira. All right reserved.

24. Copyright @Hirokazu Egashira. All right reserved.

25. cf. Using A-frame API
Copyright @Hirokazu Egashira. All right reserved.

26. AR Library_04 Argon.js with Argon4
Copyright @Hirokazu Egashira. All right reserved.
Web AR JavaScript
iOS/Android Argon4 WebAR
Vurofia A-frame TypeScript

27. cf. General use
Copyright @Hirokazu Egashira. All right reserved.

28. cf. Argon Aframe
Copyright @Hirokazu Egashira. All right reserved.

29. cf. HTML5 AR
GeoLocation getUserMedia Device orientation
AR Opera12
Copyright @Hirokazu Egashira. All right reserved.

30. cf. mozilla/webxr-polyfill
Copyright @Hirokazu Egashira. All right reserved.

31. Copyright @Hirokazu Egashira. All right reserved.

32. Google
Web AR
Copyright @Hirokazu Egashira. All right reserved.

33. Advances in Web Technology
<video>
<canvas>
WebRTC’s getUserMedia();
WebGL three.jsvideo canvas OpenCV + emscripten
WebVR
Copyright @Hirokazu Egashira. All right reserved.

34. Google & Apple AR Technology
Copyright @Hirokazu Egashira. All right reserved.
ARKit

35. Google AR’s Web AR
three.ar.js JS Web AR
ARCore Tango (Android) ARKit (iOS)
three.ar.js
WebARonARCore
WebARonARkit
https://github.com/google-ar/WebARonARCore
https://github.com/google-ar/three.ar.js
https://github.com/google-ar/WebARonARKit
WebARonTango
https://github.com/google-ar/WebARonTango
Copyright @Hirokazu Egashira. All right reserved.

36. Tools for building Google Web AR
• three.js
• three.ar.js
• WebVR API
•
ARCore Pixel, Pixel XL, Pixel2, Pixel2 XL, Galaxy S8
* Hack Nexus 6P ARCore WebARonARCore
Tango ZenfoneAR, Phab 2 Pro, Yellow Stone(Dev Kit)
ARKit iPad (2017) / iPad Pro (9.7, 10.5 or 12.9 inches) /iPhone 7 and 7 Plus / iPhone8 / iPhone X
• Core Process ARCore or Tango ※iPhone/iPad
•
WebGL & GLSL
Draco
emscripten
AR AR.js
Copyright @Hirokazu Egashira. All right reserved.

37. “three.ar.js” with WebAR on ARCore/Tango/ARKit
Copyright @Hirokazu Egashira. All right reserved.

38. Install the External Browser
Copyright @Hirokazu Egashira. All right reserved.
- Tango / ARCore Devices
1) Core ※Tango Tango Core
Android PC
adb
2)
Android PC
adb
※
Android PC
adb
※
- Tango Devices
- ARCore Devices

39. Install the External Browser
Copyright @Hirokazu Egashira. All right reserved.
- iOS Devices
• iOS11
• Xcode 9
• Apple Developer
• WebARonARKit GitHub clone
• Xcode 9
• clone .xcodeproj
• Project Navigator WebARonARKit ’target’ WebARonARKit
• ”General” signing Team iOS Developer Account / Team
• PC iOS Product >> Destination Run
• Run ”Command+R”

40. How to install and use
path
path
- CDN
-
Copyright @Hirokazu Egashira. All right reserved.

41. How to use
Copyright @Hirokazu Egashira. All right reserved.

42.
How to use init
Copyright @Hirokazu Egashira. All right reserved.

43. How to use update
Copyright @Hirokazu Egashira. All right reserved.

44. How to install and use [module]
How to use [ ES6 ] ※
Copyright @Hirokazu Egashira. All right reserved.

45. ARCore application development stack [Referred from Tango App]
ARCore Service [= Tango]
Unity SDK
Unity App UE App
UE SDK
Java Helper Library
Java App
AIDL
User App
Process
ARCore Process
IBinderInterdace
Native API
Copyright @Hirokazu Egashira. All right reserved.

46. WebAR application development stack
ARCore Service
* arcore-preview.apk
Java Helper
Library
Java App
AIDL
Core
Process
Java App
Tango Service
* Tango Core.apk
Web App [ with three.ar.js + three.js(WebGL) + WebVR ]
Swift / Object-C
App
[WKWebview]User App
Process
WebAR
onARkit
WebAR
onTango
WebAR
onARCore
Copyright @Hirokazu Egashira. All right reserved.
Chromium
[Native]
Java Helper
Library
AIDL
Chromium
[Native]
WebKitARKit
TangoSDK ARCoreSDK
ARKit Process
※
Web

47. WebAR application development stack
ARCore Service
* arcore-preview.apk
Java Helper
Library
Java App
AIDL
Core
Process
Java App
Tango Service
* Tango Core.apk
Web App [ with three.ar.js + three.js(WebGL) + WebVR ]
ARKit Process
※
Swift / Object-C
App
[WKWebview]User App
Process
WebAR
onARkit
WebAR
onTango
WebAR
onARCore
Copyright @Hirokazu Egashira. All right reserved.
Chromium
[Native]
Java Helper
Library
AIDL
Chromium
[Native]
WebKitARKit
TangoSDK ARCoreSDK
Web
Camera Data Camera Data3D Sensor Data
※

48. API overview of three.ar.js
• THREE.ARUtils
• new THREE.ARView(vrDisplay, renderer)
• new THREE.ARReticle(vrDisplay, innerRadius, outerRadius, color, easing)
• new THREE.ARDebug(vrDisplay, scene, config)
• new THREE.ARPerspectiveCamera(vrDisplay, fov, aspect, near, far)
Copyright @Hirokazu Egashira. All right reserved.

49. THREE.ARUtils
※
Promise<VRDisplay> THREE.ARUtils.getARDisplay()
<HTML Element> boolean THREE.ARUtils.isARDisplay(vrDisplay)
boolean THREE.ARUtils.isTango(vrDisplay)
boolean THREE.ARUtils.isARKit(vrDisplay)
AR VRDisplay promise.resolve
null .
VRDisplay AR
※ getARDisplay()
VRDisplay Tango ARCore Android
※ isARDisplay()
VRDisplay ARKit iOS
※ isARDisplay()
Copyright @Hirokazu Egashira. All right reserved.

50. THREE.ARUtils
void THREE.ARUtils.displayUnsupportedMessage()
Element DOM AR
ARDisplay
cf. surfaces.html
Copyright @Hirokazu Egashira. All right reserved.

51. THREE.ARUtils
void THREE.ARUtils.placeObjectAtHit(object, hit, easing=1, applyOrientation=false)
VRDisplay.prototype.hitTest THREE.Object3D VRHit
cf. spawn-at-surface.html
Copyright @Hirokazu Egashira. All right reserved.

52. THREE.ARUtils
Promise<THREE.Group> THREE.ARUtils.loadModel(config)
OBJ MTL
THREE.Group promise resolve
• objPath OBJ
• mtlPath MTL
• OBJLoader THREE OBJLoader window.THREE.OBJLoader ES6
• MTLLoader THREE MTLLoader window.THREE.MTLLoader ES6
Copyright @Hirokazu Egashira. All right reserved.

53. cf. loadmodel.html
Copyright @Hirokazu Egashira. All right reserved.

54. new THREE.ARView(vrDisplay, renderer)
• VRDisplay THREE.WebGLRenderer ARView
• VRDisplay
• ARCore ARKit Scene
render()
Scene
Copyright @Hirokazu Egashira. All right reserved.
onWindowResize()
ARView

55. cf. spawn-at-surface.html
Copyright @Hirokazu Egashira. All right reserved.

56. new THREE.ARPerspectiveCamera(vrDisplay, fov, aspect, near, far)
• THREE.PerspectiveCamera
• AR three.js
• VrDisplay AR
THREE.PerspectiveCamera
• AR ARCore/ARkit/Tango projectionMatrix for
aspect near far
.getProjectionMatrix()
• VRDisplay API
※ getProjectionMatrix() updateProjectionMatrix()
Copyright @Hirokazu Egashira. All right reserved.
.updateProjectionMatrix()
• near/far

57. cf. surfaces.html
Copyright @Hirokazu Egashira. All right reserved.

58. cf. three.ar.js
Copyright @Hirokazu Egashira. All right reserved.

59. new THREE.ARReticle(vrDisplay, innerRadius, outerRadius, color, easing)
VRDisplay THREE.Mesh
.update(x, y)
• x y
•
Copyright @Hirokazu Egashira. All right reserved.

60. cf. reticle.html
Copyright @Hirokazu Egashira. All right reserved.

61. new THREE.ARDebug(vrDisplay, scene, config)
• (hit) (pose) ARDebug
• ARDebug getElement()
• showPlanes: true THREE.Scene scene
•
.open()
.close()
.getElement()
ARDebug l
ARDebug panel
HTML
Copyright @Hirokazu Egashira. All right reserved.

62. cf. surfaces.html
Copyright @Hirokazu Egashira. All right reserved.

63. new THREE.ARAnchorManager(vrDisplay)
• VRDisplay
.add()
.remove()
<event> onAnchorsUpdated_
Anchor
Anchor
Copyright @Hirokazu Egashira. All right reserved.
Anchor
※ARAnchorManager 2017.11.28 API

64. cf. spawn-at-surface-with-anchors.html
Copyright @Hirokazu Egashira. All right reserved.

65. cf. WebAR-Article
Install
Build
Copyright @Hirokazu Egashira. All right reserved.

66. 3D
Copyright @Hirokazu Egashira. All right reserved.

67. Poly
Copyright @Hirokazu Egashira. All right reserved.

68. Copyright @Hirokazu Egashira. All right reserved.

69. Copyright @Hirokazu Egashira. All right reserved.
Google Chrome Media 3D 3D
ZIP
3D Apache License
DRACO

70. Copyright @Hirokazu Egashira. All right reserved.
WebAssembly
WebGL
asm.js asm.js
Mozilla Google Microsoft
Apple 2015

71. Strong Point / Weak Point using Web
※
URL
HTML + CSS + JavaScript
…
JavaScript
Copyright @Hirokazu Egashira. All right reserved.

72. Recap
Web AR
GeoLocation
GoogleAR WebAR ARCore
Web 1
※ Core
Web URL
AR
Copyright @Hirokazu Egashira. All right reserved.

73. Copyright @Hirokazu Egashira. All right reserved.

74. Reference
• https://www.w3.org/community/ar/
• https://www.slideshare.net/robman/computer-vision-now-working-in-over-2-billion-web-browsers
• https://www.slideshare.net/hayatohiratori/firefox-osfpv-52956375
• https://github.com/artoolkit/jsartoolkit5
• https://github.com/jcmellado/js-aruco
• https://github.com/jeromeetienne/AR.js
• https://www.argonjs.io/
• https://github.com/dontcallmedom/html5-augmented-reality
• https://github.com/google-ar/
• https://github.com/google-ar/WebAR-Article
• https://poly.google.com
• http://webassembly.org/
• https://html5experts.jp/shumpei-shiraishi/24409/
Copyright @Hirokazu Egashira. All right reserved.