2017/11/30 GooglePlay App Dojo発表資料

2. Who are U?
Hirokazu Egashira (eegozilla)
html5j
html5j
html5j Web
Android
LeapMotion Developers JP
GoogleARhirokazu.egashira
eegozilla (@ega1979)
Copyright @Hirokazu Egashira. All right reserved.

3. Google AR
Google ARCore/Tango
EOL
Google AR
Google AR
Copyright @Hirokazu Egashira. All right reserved.

4. /
Copyright @Hirokazu Egashira. All right reserved.

5. Copyright @Hirokazu Egashira. All right reserved.

6. Google AR WG / Google
https://plus.google.com/communities/117782112124490967221
Copyright @Hirokazu Egashira. All right reserved.

7. Web + AR
Copyright @Hirokazu Egashira. All right reserved.

8. Advances in Web Technology for Augument Reality
<video>
<canvas>
WebRTC’s getUserMedia();
WebGL three.jsvideo canvas OpenCV + emscripten
WebVR
Copyright @Hirokazu Egashira. All right reserved.

9. W3C Augument Reality Community Group
Copyright @Hirokazu Egashira. All right reserved.
2011 W3C Community

10. Build Augument Reality Experience with Web
getUserMedia three.js Web AR
JS ARToolkit 5
js-aruco
AR.js
Argon
etc
Copyright @Hirokazu Egashira. All right reserved.

11. AR Library_01 JS ARToolkit 5
JavaScript AR JavaScript ARToolkit Java NyARtoolkit
FLASH FLARToolkit
5 ARToolkit Emscripten Asm.js JavaScript
Copyright @Hirokazu Egashira. All right reserved.

12. cf.(1)The case of loading image
Copyright @Hirokazu Egashira. All right reserved.

13. cf.(2)Using getUserMedia
Copyright @Hirokazu Egashira. All right reserved.

14. Copyright @Hirokazu Egashira. All right reserved.
cf.(3)three.js’s helper API

15. AR Library_02 js-aruco
OpenCV AR ArUco JavaScript
Copyright @Hirokazu Egashira. All right reserved.

16. Install
JavaScript
Copyright @Hirokazu Egashira. All right reserved.

17. AR Library_03 AR.js
ARToolkit Emscripten asm.js JavaScript 3D three.js
Alexandra Etienne Jerome Etienne JavaScript AR
Github A-frame WebVR Polyfill babylon.js
Copyright @Hirokazu Egashira. All right reserved.

18. cf. Using three.js API
Copyright @Hirokazu Egashira. All right reserved.

19. Copyright @Hirokazu Egashira. All right reserved.

20. Copyright @Hirokazu Egashira. All right reserved.

21. Copyright @Hirokazu Egashira. All right reserved.

22. cf. Using WebVR polyfill API
Copyright @Hirokazu Egashira. All right reserved.

23. Copyright @Hirokazu Egashira. All right reserved.

24. Copyright @Hirokazu Egashira. All right reserved.

25. cf. Using A-frame API
Copyright @Hirokazu Egashira. All right reserved.

26. AR Library_04 Argon.js with Argon4
Copyright @Hirokazu Egashira. All right reserved.
Web AR JavaScript
iOS/Android Argon4 WebAR
Vurofia A-frame TypeScript

27. cf. General use
Copyright @Hirokazu Egashira. All right reserved.

28. cf. Argon Aframe
Copyright @Hirokazu Egashira. All right reserved.

29. cf. HTML5 AR
GeoLocation getUserMedia Device orientation
AR Opera12
Copyright @Hirokazu Egashira. All right reserved.

30. cf. mozilla/webxr-polyfill
Copyright @Hirokazu Egashira. All right reserved.

31. Copyright @Hirokazu Egashira. All right reserved.

32. Google
Web AR
Copyright @Hirokazu Egashira. All right reserved.

33. Advances in Web Technology
<video>
<canvas>
WebRTC’s getUserMedia();
WebGL three.jsvideo canvas OpenCV + emscripten
WebVR
Copyright @Hirokazu Egashira. All right reserved.

34. Google & Apple AR Technology
Copyright @Hirokazu Egashira. All right reserved.
ARKit

35. Google AR’s Web AR
three.ar.js JS Web AR
ARCore Tango (Android) ARKit (iOS)
three.ar.js
WebARonARCore
WebARonARkit
https://github.com/google-ar/WebARonARCore
https://github.com/google-ar/three.ar.js
https://github.com/google-ar/WebARonARKit
WebARonTango
https://github.com/google-ar/WebARonTango
Copyright @Hirokazu Egashira. All right reserved.

36. Tools for building Google Web AR
• three.js
• three.ar.js
• WebVR API
•
ARCore Pixel, Pixel XL, Pixel2, Pixel2 XL, Galaxy S8
* Hack Nexus 6P ARCore WebARonARCore
Tango ZenfoneAR, Phab 2 Pro, Yellow Stone(Dev Kit)
ARKit iPad (2017) / iPad Pro (9.7, 10.5 or 12.9 inches) /iPhone 7 and 7 Plus / iPhone8 / iPhone X
• Core Process ARCore or Tango ※iPhone/iPad
•
WebGL & GLSL
Draco
emscripten
AR AR.js
Copyright @Hirokazu Egashira. All right reserved.

37. “three.ar.js” with WebAR on ARCore/Tango/ARKit
Copyright @Hirokazu Egashira. All right reserved.

38. Install the External Browser
Copyright @Hirokazu Egashira. All right reserved.
- Tango / ARCore Devices
1) Core ※Tango Tango Core
Android PC
adb
2)
Android PC
adb
※
Android PC
adb
※
- Tango Devices
- ARCore Devices

39. Install the External Browser
Copyright @Hirokazu Egashira. All right reserved.
- iOS Devices
• iOS11
• Xcode 9
• Apple Developer
• WebARonARKit GitHub clone
• Xcode 9
• clone .xcodeproj
• Project Navigator WebARonARKit ’target’ WebARonARKit
• ”General” signing Team iOS Developer Account / Team
• PC iOS Product >> Destination Run
• Run ”Command+R”

40. How to install and use
path
path
- CDN
-
Copyright @Hirokazu Egashira. All right reserved.

41. How to use
Copyright @Hirokazu Egashira. All right reserved.

42.
How to use init
Copyright @Hirokazu Egashira. All right reserved.

43. How to use update
Copyright @Hirokazu Egashira. All right reserved.

44. How to install and use [module]
How to use [ ES6 ] ※
Copyright @Hirokazu Egashira. All right reserved.

45. ARCore application development stack [Referred from Tango App]
ARCore Service [= Tango]
Unity SDK
Unity App UE App
UE SDK
Java Helper Library
Java App
AIDL
User App
Process
ARCore Process
IBinderInterdace
Native API
Copyright @Hirokazu Egashira. All right reserved.

46. WebAR application development stack
ARCore Service
* arcore-preview.apk
Java Helper
Library
Java App
AIDL
Core
Process
Java App
Tango Service
* Tango Core.apk
Web App [ with three.ar.js + three.js(WebGL) + WebVR ]
Swift / Object-C
App
[WKWebview]User App
Process
WebAR
onARkit
WebAR
onTango
WebAR
onARCore
Copyright @Hirokazu Egashira. All right reserved.
Chromium
[Native]
Java Helper
Library
AIDL
Chromium
[Native]
WebKitARKit
TangoSDK ARCoreSDK
ARKit Process
※
Web

47. WebAR application development stack
ARCore Service
* arcore-preview.apk
Java Helper
Library
Java App
AIDL
Core
Process
Java App
Tango Service
* Tango Core.apk
Web App [ with three.ar.js + three.js(WebGL) + WebVR ]
ARKit Process
※
Swift / Object-C
App
[WKWebview]User App
Process
WebAR
onARkit
WebAR
onTango
WebAR
onARCore
Copyright @Hirokazu Egashira. All right reserved.
Chromium
[Native]
Java Helper
Library
AIDL
Chromium
[Native]
WebKitARKit
TangoSDK ARCoreSDK
Web
Camera Data Camera Data3D Sensor Data
※

48. API overview of three.ar.js
• THREE.ARUtils
• new THREE.ARView(vrDisplay, renderer)
• new THREE.ARReticle(vrDisplay, innerRadius, outerRadius, color, easing)
• new THREE.ARDebug(vrDisplay, scene, config)
• new THREE.ARPerspectiveCamera(vrDisplay, fov, aspect, near, far)
Copyright @Hirokazu Egashira. All right reserved.

49. THREE.ARUtils
※
Promise<VRDisplay> THREE.ARUtils.getARDisplay()
<HTML Element> boolean THREE.ARUtils.isARDisplay(vrDisplay)
boolean THREE.ARUtils.isTango(vrDisplay)
boolean THREE.ARUtils.isARKit(vrDisplay)
AR VRDisplay promise.resolve
null .
VRDisplay AR
※ getARDisplay()
VRDisplay Tango ARCore Android
※ isARDisplay()
VRDisplay ARKit iOS
※ isARDisplay()
Copyright @Hirokazu Egashira. All right reserved.

50. THREE.ARUtils
void THREE.ARUtils.displayUnsupportedMessage()
Element DOM AR
ARDisplay
cf. surfaces.html
Copyright @Hirokazu Egashira. All right reserved.

51. THREE.ARUtils
void THREE.ARUtils.placeObjectAtHit(object, hit, easing=1, applyOrientation=false)
VRDisplay.prototype.hitTest THREE.Object3D VRHit
cf. spawn-at-surface.html
Copyright @Hirokazu Egashira. All right reserved.

52. THREE.ARUtils
Promise<THREE.Group> THREE.ARUtils.loadModel(config)
OBJ MTL
THREE.Group promise resolve
• objPath OBJ
• mtlPath MTL
• OBJLoader THREE OBJLoader window.THREE.OBJLoader ES6
• MTLLoader THREE MTLLoader window.THREE.MTLLoader ES6
Copyright @Hirokazu Egashira. All right reserved.

53. cf. loadmodel.html
Copyright @Hirokazu Egashira. All right reserved.

54. new THREE.ARView(vrDisplay, renderer)
• VRDisplay THREE.WebGLRenderer ARView
• VRDisplay
• ARCore ARKit Scene
render()
Scene
Copyright @Hirokazu Egashira. All right reserved.
onWindowResize()
ARView

55. cf. spawn-at-surface.html
Copyright @Hirokazu Egashira. All right reserved.

56. new THREE.ARPerspectiveCamera(vrDisplay, fov, aspect, near, far)
• THREE.PerspectiveCamera
• AR three.js
• VrDisplay AR
THREE.PerspectiveCamera
• AR ARCore/ARkit/Tango projectionMatrix for
aspect near far
.getProjectionMatrix()
• VRDisplay API
※ getProjectionMatrix() updateProjectionMatrix()
Copyright @Hirokazu Egashira. All right reserved.
.updateProjectionMatrix()
• near/far

57. cf. surfaces.html
Copyright @Hirokazu Egashira. All right reserved.

58. cf. three.ar.js
Copyright @Hirokazu Egashira. All right reserved.

59. new THREE.ARReticle(vrDisplay, innerRadius, outerRadius, color, easing)
VRDisplay THREE.Mesh
.update(x, y)
• x y
•
Copyright @Hirokazu Egashira. All right reserved.

60. cf. reticle.html
Copyright @Hirokazu Egashira. All right reserved.

61. new THREE.ARDebug(vrDisplay, scene, config)
• (hit) (pose) ARDebug
• ARDebug getElement()
• showPlanes: true THREE.Scene scene
•
.open()
.close()
.getElement()
ARDebug l
ARDebug panel
HTML
Copyright @Hirokazu Egashira. All right reserved.

62. cf. surfaces.html
Copyright @Hirokazu Egashira. All right reserved.

63. new THREE.ARAnchorManager(vrDisplay)
• VRDisplay
.add()
.remove()
<event> onAnchorsUpdated_
Anchor
Anchor
Copyright @Hirokazu Egashira. All right reserved.
Anchor
※ARAnchorManager 2017.11.28 API

64. cf. spawn-at-surface-with-anchors.html
Copyright @Hirokazu Egashira. All right reserved.

65. cf. WebAR-Article
Install
Build
Copyright @Hirokazu Egashira. All right reserved.

66. 3D
Copyright @Hirokazu Egashira. All right reserved.

67. Poly
Copyright @Hirokazu Egashira. All right reserved.

68. Copyright @Hirokazu Egashira. All right reserved.

69. Copyright @Hirokazu Egashira. All right reserved.
Google Chrome Media 3D 3D
ZIP
3D Apache License
DRACO

70. Copyright @Hirokazu Egashira. All right reserved.
WebAssembly
WebGL
asm.js asm.js
Mozilla Google Microsoft
Apple 2015

71. Strong Point / Weak Point using Web
※
URL
HTML + CSS + JavaScript
…
JavaScript
Copyright @Hirokazu Egashira. All right reserved.

72. Recap
Web AR
GeoLocation
GoogleAR WebAR ARCore
Web 1
※ Core
Web URL
AR
Copyright @Hirokazu Egashira. All right reserved.

73. Copyright @Hirokazu Egashira. All right reserved.

74. Reference
• https://www.w3.org/community/ar/
• https://www.slideshare.net/robman/computer-vision-now-working-in-over-2-billion-web-browsers
• https://www.slideshare.net/hayatohiratori/firefox-osfpv-52956375
• https://github.com/artoolkit/jsartoolkit5
• https://github.com/jcmellado/js-aruco
• https://github.com/jeromeetienne/AR.js
• https://www.argonjs.io/
• https://github.com/dontcallmedom/html5-augmented-reality
• https://github.com/google-ar/
• https://github.com/google-ar/WebAR-Article
• https://poly.google.com
• http://webassembly.org/
• https://html5experts.jp/shumpei-shiraishi/24409/
Copyright @Hirokazu Egashira. All right reserved.