Learn what you can do in Augmented Reality in 2020. Apps and web technologies let you create awesome AR experiences. We give you everything you should know before choosing a technology.

1. .
AR - State of the Art
1

3. .
Technological
differences between
Native and Web
environments
3

4. .4
Native & ARKit/ARCore: World tracking & Scene understanding
With ARKit & ARCore on smartphones, the system is able to track the world around it and to understand the scene it
is in. As such, a virtual object can be positioned with a true sense of reality, without any marker needed.
What’s important to note is that whenever the virtual object is positioned, it stays at its position even if the user
walks in the room and points another zone with its camera.

5. .5
Native & ARKit/ARCore: People occlusion & motion capture
AR content realistically passes behind and in front of people in the real world, making AR experiences more
immersive while also enabling green screen-style effects in almost any environment.
With the new releases of ARKit, developers can now use people occlusion and movement capture. That’s what you
seen in the pictures below.
People occlusion
We continue to see the AR object behind someone
Motion capture
ARKit records people movement and is able to replicate it live

6. .6
Native & ARKit/ARCore: Also with markers
Markers can be used with ARKit/ARCore when one needs to put a virtual object at a very precise location, the
marker can then be:
- A 2D image (e.g. a museum painting — see visual hereunder)
- A square marker / QR Code
- A face
- In some cases, a 3D object

7. .7
Web: Only markers
On a web environment, Augmented Reality can be used only with markers.
Most of the time the marker needs to be a square marker / QR Code marker. 2D images markers and face
recognition markers on the web exist, but their performance can be very deceptive.
The marker must always be visible.

8. .8
Exception for the Web: 3D objects preview with AR
Apple has “AR Quick Look” which gives the ability to preview on a webpage a 3D object within the reality. Under the
hoods it relies on ARKit, but it offers very little interaction (especially, as a developer we cannot bring any
interaction).
Android has an equivalent feature, called “Scene Viewer”: https://developers.google.com/ar/develop/java/scene-viewer
Try it out: https://developer.apple.com/augmented-reality/quick-look/

9. .9
Web: why no World tracking and Scene understanding?
Web technologies by themselves (i.e. Browser / HTML / JavaScript) aren’t powerful enough to support World
tracking and Scene understanding, as it involves advanced software, machine learning and deep access to the
hardware.
A new web standard, WebXR, is being built to bring this support. Basically, WebXR will bridge the web environment
with the underlying ARKit and ARCore on the compatible smartphones and tablets.
However, the WebXR standard is not finalized yet, and only experimental browsers partly support it:
- Firefox WebXR
- Google Chrome Canary editions
Source: https://www.slideshare.net/AugmentedWorldExpo/casper-fabricius-
cimmerse-the-state-of-webxr

10. .
What can be achieved
with ARKit / ARCore
(Native)
10

11. .11
Marker-based concepts: replacing a 2D image with a 3D virtual object
The marker is the image of a painting, and is replaced by a 3D virtual object
Advantages with ARKit / ARCore:
1. Once the virtual object is positioned in the world, the user can navigate around the room or around the object
without “losing” it if the marker is not visible anymore by the camera.
2. The marker can be a 2D image or even a face (no need to be a “square marker” or QR-code-like marker)
/! Need to be careful
about lightning of the
2D image, the angle,
etc. ==> impact on
time of detection

12. .12
Markerless + Manual operation
In a markerless configuration, the most common scenario is the following:
1. The user must first scan the environment (moving around his phone)
2. The user taps on the screen to position the object
3. The object is positioned and locked.
To test this behaviour with an iPhone:
https://developer.apple.com/augmented-reality/quick-look/
This means a manual operation is needed.
Advantage with ARKit / ARCore:
- Markerless AR is only possible with ARKit / ARCore
- The user can walk around in the room, come back and the virtual object
will still be at the same (more or less) position.

13. .13
Markerless + GPS position ==> Precise positioning isn’t an option
For IOC, an interesting case would be to position virtual object at exact GPS coordinates e.g. the Sakura trees in
front of the museum.
While there are numerous projects on the web to achieve such a thing, they all have important limitations:
- It is very hard to display 3D objects when working with GPS coordinates
- The location is not precise, because on one hand the GPS of the phone is not precise (from a couple meters to
tens of meters) and on the other hand it is very hard to have a correct match between GPS coordinates and the
“world detected” by ARKit / ARCore.

14. .14
Markerless + GPS position ==> Approximative positioning
What is feasible is to position a 3D object in the reality using an approximative positioning, it would basically be
geofencing-based:
1. Acquiring GPS position of the user
2. If the user is in a pre-defined radius e.g. a specific lat/long with a tolerance of a few meters => The experience is
activated. We can take into account the compass of the user to be sure of the location he’s looking at.
3. The AR experience is positioned in front of the user.
This is the Pokemon Go principle. Totally OK for characters in the park, not precise enough for Sakuras.

15. .15
Portal doors
Portal doors can be done with Native technologies but they suffer the same problem as GPS positioning: they cannot be
put for sure at one exact location.
However, they can be put at an approximate location, without being a too important problem for the concept.

16. .16
Other advantages of ARKit / ARCore vs Web
Thanks to “World tracking” and “Scene understanding”, ability to:
- Have interactions between virtual objects and the real world: object falling from a table, hitting a wall, etc.
- Have objects scaled to the real world without a marker
- Take into account light in the room
- ARKit 3 / iPhone XS+: People Occlusion
- ARKit 3 / iPhone XS+: Motion capture (Kinect-like)
- ARKit 3 / iPhone XS+: front+back camera at the same time

17. .
What can be achieved
with Web technologies
17

18. .18
With marker
The Web AR can only work with a marker at the moment.
Ideally this must be a “square marker” or QR-code-like marker. With 2D images as a marker and face marker, it is
possible but this can be problematic performance-wise at the moment.
IMPORTANT LIMITATION: the marker must be correctly visible at ALL TIME otherwise the object disappear. If it
disappear and you come back on the image, there is a flickering effect (you see the marker for a fraction of
second, before it is replaced).

19. .19
With marker
The Web AR can only work with a marker at the moment.
Ideally this must be a “square marker” or QR-code-like marker. With 2D images as a marker and face marker, it
seems to be possible but this might be problematic performance-wise at the moment.
IMPORTANT LIMITATION: the marker must be correctly visible at ALL TIME otherwise the object disappear. If it
disappear and you come back on the image, there is a flickering effect (you see the marker for a fraction of
second, before it is replaced).
https://www.youtube.com/watch?v=0MtvjFg7tik

20. .20
Multiple markers
Supposed to work but seems to be complex / not totally reliable:
https://stackoverflow.com/questions/51850930/a-frame-ar-js-multiple-markers-boxes?rq=1

21. .21
Web-based portal door
The web-based portal door is possible, but it requires a marker and this comes with the disadvantages of using a
marker: The marker must be seen at all time (this is a HUGE difficulty UX-wise).
https://www.youtube.com/watch?v=tcudraEerTY