This document discusses augmented reality (AR) and how ARCore can be used for mobile AR applications in Unity. It defines AR, virtual reality and mixed reality. It explains the different types of AR like marker-based, location-based and markerless. It provides an overview of ARCore, describing how it does motion tracking, environmental understanding and light estimation. It outlines the steps to get started with ARCore in Unity and discusses potential next steps like using light estimation, Google Poly and Google Resonance Audio.

1. Beyond reality with ARCore
A Unity’s tale
@doJester13

2. Erik Minarini
@doJester13
Unity and Android Developer @ Touchlabs
Manager @ GDG Bologna
About Me

3. Augmented Reality

4. XR: Cross reality, what is it?
● Augmented reality
● Virtual reality
● Mixed reality

5. AR: Augmented reality
Headset Augmented reality Mobile Augmented reality

6. AR: Augmented reality

7. AR: Augmented reality
Marker based
Location based
Markerless

8. AR: Augmented reality
The models are positioned after the marker is
recognized
Marker based

9. AR: Augmented reality
The models are positioned after recognizing the
geographical position of the device
Location based

10. AR: Augmented reality
The positioning of the models depends on the
algorithms for the surface recognition
Markerless

11. Today

12. Today
● Mobile augmented reality

13. Today
● Mobile augmented reality
● Markerless

14. Today
● Mobile augmented reality
● Markerless
● Using Google ARCore with Unity

15. ARCore

16. ARCore overview
Motion tracking
Environmental understanding
Light estimation

17. ARCore overview
ARCore use concurrent odometry and mapping or COM to understand
where the device is,
detects the features points of the camera image and uses these
points to compute its change in location.
Uses these informations combined with IMU’s measurements to
estimate pose: position and orientation of the camera relative to the
world.
Motion tracking

18. ARCore overview
ARCore looks for clusters of feature points that appear to lie on
common horizontal surfaces, and makes these surfaces available as
planes.
ARCore can also determine each plane's boundary.
Oriented points are used to place virtual objects on non-horizontal
surfaces, ARCore uses clusters of feature points to attempt to
estimate the angle of the surface.
Environmental understanding

19. ARCore overview
Light estimation
ARCore use the average of luminosity of the camera image.
The provided shader multiplies the object color with the global light
estimate.
This adapts the RGB brightness of the final surface color, increasing
the sense of realism.

20. How can we use
ARCore?

21. How can we use
ARCore?

22. Our Application

23. Getting started
● Android SDK version 7.0 (API Level 24) or higher.
● Install Unity 2017.3.0f2 or higher, with the Android Build Support
component.
● Download the ARCore SDK for Unity.
● A supported Android device.

31. What’s next?

32. What’s next?
● Use light estimation.

33. What’s next?
● Google Poly.

34. What’s next?
● Google Resonance Audio.

35. https://github.com/ToughNutToCrack/ToughNutToCrackAR

36. https://toughnuttocrack.github.io/

37. Thank you.
@doJester13