Augmenta is a creative tracking system that uses 3D cameras and custom software to detect people and objects in an unlimited space. It consists of hardware like multiple 3D cameras and computers, and software for image analysis, data fusion, and providing position data via OSC. Users can get started by listening to the OSC data in their preferred programming language or framework, or using existing libraries for Processing, OpenFrameworks, Unity, and more. Augmenta also provides tools to simulate tracking data and trigger events based on positions.

1. augmenta
CREATIVE TRACKING SYSTEM

2. - intro
- the technology
- getting started

3. the idea
Use an array of 3D cameras and custom
software to allow artists to easily detect
people and objects in an unlimited space
intro

4. workflow
ResultComputerAugmenta
intro

5. examples
Video : https://vimeo.com/126493415
intro

6. the technology

7. definition
Augmenta is a combination of hardware (multiple 3D
cameras, dedicated computers and a server computer)
and software (Image analysis, data fusion, tools, etc.)
which provides the position of every person or object in
scene via OSC, along with other useful data.
the technology

8. installation
Unlimited number of cameras
1 – 6 m
from the
ground
Cameras can
be positioned
at an angle
Overlaps are correctly handled
the technology

9. features
Performance
• Unlimited area size
• Unlimited number of people
• Precision up to a few millimetres
• Fast computation per frame
Simplicity
• Universal data protocol (OSC)
• Plug-and-play
• Web interface for setup
the technology
Augmenta web interface

10. calibration
Augmenta
hardware
Augmenta Merge
calibration software
Augmenta
hardware
Augmenta
hardware
…
Each camera is calibrated relatively to the
other and all the data is merged to track
people seamlessly on an unlimited space
the technology
Augmenta Merge – calibration software
Augmenta
OSC
Augmenta
OSC
Augmenta
OSC
Augmenta
OSC

11. dataflow
Video
Sound
DMX
Other…
the technology
Augmenta
hardware
Augmenta
hardware
Augmenta
hardware
…
Augmenta Merge
Augmenta
OSC
Your app
OSC
OSC
OSC

12. getting started

13. intro
The power of Augmenta is that you don’t have to know how it works to use it.
Just listen to the OSC data in your favourite language/framework or use one
of the existing libs and examples (Processing / OpenFrameworks / Unity / … )
We chose to release everything client-side in open source, so feel free to
contribute, ask for help and share with us and the community !
getting started

14. your app
…
the technology
Augmenta
hardware
Augmenta
hardware
Augmenta
hardware
…
Video
Sound
DMX
Other
Augmenta Merge
Augmenta
OSC
Your app
OSC
OSC
OSC

15. data
/au/personEntered args0 arg1 ...
/au/personWillLeave args0 arg1 …
/au/personUpdated args0 arg1 …
Where args are :
0: pid (int)
1: oid (int)
2: age (int)
3: centroid.x (float 0-1)
4: centroid.y (float 0-1)
5: velocity.x (float 0-1)
6: velocity.y (float 0-1)
7: depth (float)
8: boundingRect.x (float 0-1)
9: boundingRect.y (float 0-1)
10: boundingRect.width (float 0-
1)
11: boundingRect.height (float
0-1)
12: highest.x (float 0-1)
13: highest.y (float 0-1)
14: highest.z (float 0-1)
// Personal ID ex : 42th person to enter has
pid=41
// Ordered ID ex : 3rd person still present has
oid=2
// Time on stage (in frame number)
// Position projected to the ground
// Speed and direction vector
// Distance to sensor (in m) (not implemented)
// Top view bounding box
// Highest point placement (not implemented)
// Height of the person (not implemented)
Messages sent for each detected person :
getting started

16. data
/au/scene args0 arg1 …
Where args are :
0: currentTime (int)
1: percentCovered (float 0-1)
2: numPeople (int)
3: averageMotion.x (float 0-1)
4: averageMotion.y (float 0-1)
5: scene.width (int)
6: scene.height (int)
7: scene.depth (int)
// Time (in frame number)
// Percent covered
// Number of person
// Average motion
// Scene size
// (not implemented)
getting started
Message sent each frame to describe the scene :
Data protocol is up-to-date here :
https://github.com/Theoriz/Augmenta/wiki

17. tools
A tool to simulate Augmenta data for testing
purposes.
- Simulate a moving person with the mouse
- Simulate a grid of people moving
automatically (up to thousands of people)
- Simulate a custom Augmenta scene size
Instructions, download, source :
https://github.com/Theoriz/Augmenta-Simulator
simulator
getting started

18. tools
Optional app to trigger OSC messages
depending of people’s position in the scene
- No coding skills needed
- Trigger custom OSC messages when someone
enters or leaves a custom polygonal area
- Link directly with any audio / video / other OSC
compatible software
…
area2osc
Instructions and download :
https://github.com/Theoriz/Augmenta/releases
getting started

19. libraries
If you want more creative freedom, you can either listen
to the OSC messages in your favourite program or start
using one of the existing libraries and examples
Find the complete list on https://github.com/Theoriz
getting started

20. libraries
- Easy to learn/use
- Multiple examples (2D/3D/assets)
- Complete set of features
- Interactive area cropping, auto adjust scene size,
etc.
- Advanced debug info
Instructions and download :
https://github.com/Theoriz/AugmentaP5
getting started
processing

21. libraries
- Complete set of features
- Advanced debug info
Instructions and download :
https://github.com/Theoriz/ofxAugmenta
openframeworks
getting started

22. libraries
- 3D engine
- Complete set of features
Instructions and download :
https://github.com/Theoriz/AugmentaUnity
getting started
unity

23. libraries
Contribute !
You can help us and the community by sending bug reports,
submitting pull requests and sharing your own examples /
libraries.
Find the guidelines here :
https://github.com/Theoriz/Augmenta/wiki
…
getting started

24. ready ?
start here :
https://github.com/Theoriz/Augmenta/wiki
getting started