Project Tango is a tablet development kit from Google that adds advanced computer vision, depth sensing, and motion tracking capabilities to Android devices. It uses a specialized camera, depth sensor, and Tegra K1 processor to enable applications like area learning, localization, and 3D reconstruction. Developers can access pose, depth, and area description data through APIs to build augmented reality and mobile visual computing experiences.

1. Project Tango
Rouyun Pan

2. What is “Project Tango”
• A new kind of spatial perception to the Android
device platform by adding advanced computer
vision, image processing, and specialized
sensors.

3. HW Spec.
• NVIDIA Tegra K1 processor 4GB of RAM
• 192 CUDA core (~350 GFlop)
• 128GB of storage
• 7.02" 1080p display (323 ppi)
• Stock Android 4.4
• WiFi, Bluetooth LE and 4G LTE
• 120 degree front camera
• 4MP Camera and Depth Sensor
• Motion-tracking Camera

4. HW Diagram

5. HW Architecture
• No external image
processors needed
• Reduces cost, power, latency
• Odometry
• 30 FPS
• ~3-5ms GPU processing/frame
• GPU load: ~15%
• Depth Decoding
• 5 FPS
• ~13-15ms GPU processing/
frame
• GPU load: ~8%
CSI
CSI
CSI
SPI

6. Software Pipeline
• GPU processed sensor data made available to applications Java interfaces or
native access, together with 3D engines such as Unity
• NVIDIA VisionWorks can be used for native vision processing Library of GPU
accelerated vision primitives
• Use OpenVX for optimized execution of graphs of primitives

7. Core functionality
• Motion tracking
• Area learning
• Depth perception

8. Motion Tracking
• Project Tango implements motion tracking using visual-inertial odometry, or VIO, to estimate where a
device is relative to where it started.
• It provides the position and orientation in full six degrees of freedom, referred to as its pose .
• The APIs support two ways to get pose data:
• callbacks
• specific timestamp.

9. Lifecycle of Pose status

10. Area Learning
• Project Tango can remember the key visual features of a
physical space - the edges, corners, other unique features it
has visited. it is known as Simultaneous Localization and
Mapping, or SLAM.
• The Project Tango supports learning, saving, and loading
Area Description Files (ADF).

11. Localization & Drift correction
• Project Tango loads the saved description, it can recognize it is within that
loaded area. This process is called localization.
• Project Tango performs drift correction to fix errors that occur during
prolonged motion tracking by recognizing its position within a known physical
area.

12. Depth Perception
• Project Tango devices are equipped with integrated 3D sensors that measure
the distance (0.5 to 4 meters).
• Project Tango returns depth information in two main formats: XYZ Point Clouds,
and a Project Tango-specific format called XYZij designed to aid meshing.

13. Coordinate system(1)
• Project Tango Coordinate Frames

14. Coordinate system(2)
• OpenGL Coordinate Frames

15. Coordinate system(3)
• Unity Coordinate Frames

16. Coordinate system(4)
• On-Device Coordinate Frames

17. Frames of reference
• The Project Tango APIs provide motion tracking data based on a flexible set of
target and base frame pairs, allowing you to pick the coordinate system that
best fits your use case.

18. Mobile Visual Computing
Enables New Experiences
Need for advanced sensors
and the GPU throughput
Computational
Photography
and Videography
Face, Body
and
Gesture Tracking
3D Scene/Object
Reconstruction
Augmented Reality

19. Demo

20. So where do you get a tango?
• Google store (currently available in the US)
https://store.google.com/product/
project_tango_tablet_development_kit

21. Reference
• https://developers.google.com/project-tango/
developer-overview
• https://plus.google.com/communities/
114537896428695886568
• http://stackoverflow.com/questions/tagged/google-
project-tango
• https://www.passion47.com/project-tango/
• http://www.ustream.tv/recorded/51294136