This document discusses the need for common data transmission and exchange protocols between devices like those that exist for the internet (TCP/IP and HTTP). It introduces Resonai's work on building standards for augmented reality experiences through their Vera platform. Vera allows for context-aware, immersive AR experiences that can be consumed across multiple users and devices by understanding the 3D environment through scene scanning and recognition of objects. It provides examples of how AR content could interact with real-world spaces and be synced between multiple agents depending on who owns the rights to project content in public spaces.

1. The Emergence of Digi-Sapience
Creating rich content and
exchange protocols between devices
Emil Alon Founder & CEO, Resonai

2. Within a few years,
we’ll probably all be wearing AR glasses.
How will we sync in?

3. HTTP defines how messages are formatted and transmitted on the WWW, and what actions web
servers and browsers should take in response to various commands.
TCP/IP specifies how data is exchanged over the internet:
- Provides end-to-end communications that identify how data is addressed, transmitted, routed
and received at the destination
- Requires little central management
- Makes networks reliable (recover automatically from failure of any network device)
These protocols are the mechanism of the internet, as we know it.
Standards for TCP/IP & HTTP

4. What about telecommunications & IoT?
No widely accepted standards for data transmission and exchange.
- Little/no communication and synchronization between devices: inefficient, anti-competitive
and restraining to be limited to a single brand ecosystem
- Devices don’t follow the same protocols (e.g. cyber security) → Networks suffer from reliability
issues

5. We need
● Business models that scale with distribution
● Protocols that synchronize between physical and digital ownership and between
multiple devices
AR: The next computational platform

6. OR
Very generic experiences
such as projecting AR stickers
on a surface
Immersive but expensive
experiences, tailored to a single,
specific location
AR today

7. To motivate content creation, we need business
models that scale with content distribution.

8. This requires a smart engine that can map
environments to experiences.

9. When machines understand what they see,
the physical and digital can seamlessly merge.
Resonai translates the visual world to machines.

10. Resonai Visual Search
A visual search engine that understands the
precise geometry and structure of any type of
3D content: scanned, CAD, or sketched.
Visual Perception: Understanding single objects

11. Visual Perception: Understanding full scenes
Resonai Dollhouse
A 3D scene reconstruction engine that transforms scanned
content into matching 3D assets, creating virtual representations
of the real world in 3D.

12. Resonai Vera
The first scalable platform for context-aware AR

13. Scaling AR
Ability to create immersive AR
experiences that adapt to any kind of
environment, anywhere, and can be
consumed by multiple users, devices,
and platforms.

14. Context-Aware
Defined by the
objects in the
environment and
the relationships
between them
Adaptive
Change with
changes in the
semantics of the
scene
Immersive
Virtual objects
interact with real
objects
Natural
Real objects
occlude virtual
objects, and vice
versa
Distributed
Consumed by
multiple users and
devices
Vera brings together developers, content creators, and
end users by enabling a new kind of AR experience:

15. Step #1
Scene Scan
Step #2
Segmentation
& Surface
Extraction
Step #3
Search &
Recognition
Step #4
Scene
Reconstruction
Step #5
Experience
Unlocks
Vera: How does it work?

16. Step #1
Scene Scan
Scan via mobile device.
Data is uploaded to Resonai servers.
Each scanned environment is indexed on
our servers. Next time users want to play
in that location, experiences unlock
instantly.

17. Step #2
Segmentation & Surface
Extraction

18. Step #3
Search & Recognition
Using Resonai Visual Search, the
system identifies the different
objects in the scene, understanding
their precise 3D structure

19. Step #4
Scene Reconstruction
Resonai Dollhouse: Creates a virtual
3D representation of the physical
space.

20. Step #5
Experience Unlocks
Experiences that comply with the
semantics of the environment are
unlocked.
Press to Start!

21. Off Road
The user needs to bring the car from
the starting point to the finish point.
The car’s movement is defined by the
mesh of the room and the objects in it.
This example demonstrates precise
mesh understanding that allows
virtual objects to behave naturally in a
physical setting.
Vera: Example of mesh understanding

22. Off Road game
recording from a
mobile device.
Vera: Example of mesh understanding

23. Steve the Turtle
Steve falls from a portal that opens
near the ceiling. The user needs to
navigate Steve in the room, into a
portal that opens on a computer
screen.
Steve can walk if it has no obstacles,
or, the player can use fans to allow
Steve to fly above obstacles.
Vera: Example of context-aware AR

24. Steve the Turtle
Mobile session
Vera: Example of context-aware AR

25. MatchMaker
Content recommendation
based on semantic
understanding
Vera: Example of context-aware AR

26. How is content going to be synchronized between
multiple agents?

27. ● Who owns the AR projection rights?
● Who owns the R projection rights?
In a building lobby / office / sports venue /
restaurant / any public domain?
Ownership of the default AR projection

28. Sky’s the limit?

29. Thank you
Emil Alon
emil@resonai.com