14. Tracking Requirements
Augmented Reality Information Display
World Stabilized
Body Stabilized
Head Stabilized
Increasing Tracking
Requirements
Head Stabilized Body Stabilized World Stabilized
16. AR Tracking Taxonomy
e.g. AR Toolkit
Low Accuracy
at 15-60 Hz
e.g. IVRD
High Accuracy
& High Speed
Hybrid
Tracking
Limited Range
e.g. HiBall
Many Fiducials
in space/time
but
no GPS
Extended Range
Indoor
Environment
e.g. WLVA
Not Hybridized
GPS or
Camera or
Compass
Low Accuracy &
Not Robust
e.g. BARS
Hybrid Tracking
GPS and
Camera and
Compass
High Accuracy
& Robust
Outdoor
Environment
AR
TRACKING
19. Magnetic Tracker
Idea: difference between a magnetic transmitter
and a receiver
++: 6DOF, robust
-- : wired, sensible to metal, noisy, expensive
Flock of Birds (Ascension)
21. Ultrasonics Tracker
Idea: Time of Flight or Phase-Coherence Sound Waves
++: Small, Cheap
-- : 3DOF, Line of Sight, Low resolution, Affected
Environment Conditon (pressure, temperature)
Ultrasonic
Logitech IS600
22. Inertial Tracker
Idea: measuring linear and angular orientation rates
(accelerometer/gyroscope)
++: no transmitter, cheap, small, high frequency, wireless
-- : drift, hysteris only 3DOF
IS300 (Intersense)
Wii Remote
23. Mobile Sensors
Inertial compass
Earth’s magnetic field
Measures absolute orientation
Accelerometers
Measures acceleration about axis
Used for tilt, relative rotation
Can drift over time
24. Global Positioning System (GPS)
Created by US in 1978
Currently 29 satellites
Satellites send position + time
GPS Receiver positioning
4 satellites need to be visible
Differential time of arrival
Triangulation
Accuracy
5-30m+, blocked by weather, buildings etc
25.
26. Problems with GPS
Takes time to get satellite fix
Satellites moving around
Earths atmosphere affects signal
Assumes consistent speed (the speed of light).
Delay depends where you are on Earth
Weather effects
Signal reflection
Multi-path reflection off buildings
Signal blocking
Trees, buildings, mountains
Satellites send out bad data
Misreport their own position
27. Accurate to < 5cm close to base station (22m/100 km)
Expensive - $20-40,000 USD
28. Assisted-GPS (A-GPS)
Use external location server to send GPS signal
GPS receivers on cell towers, etc
Sends precise satellite position (Ephemeris)
Speeds up GPS Tracking
Makes it faster to search for satellites
Provides navigation data (don’t decode on phone)
Other benefits
Provides support for indoor positioning
Can use cheaper GPS hardware
Uses less battery power on device
30. Cell Tower Triangulation
Calculate phone position
from signal strength
< 50 m in cities
> 1 km in rural
31. WiFi Positioning
Estimate location by using WiFi access points
Can use know locations of WiFi access points
Triangulate through signal strength
Eg. PlaceEngine (www.placeengine.com)
Client software for PC and mobiles
SDK returns position
Accuracy
5 – 100m (depends on WiFi density)
34. Indoor WiFi Location Sensing
Indoor Location
Asset, people tracking
Aeroscout
http://aeroscout.com/
WiFi + RFID
Ekahau
http://www.ekahau.com/
WiFi + LED tracking
35. Integrated Systems
Combine GPS, Cell tower, WiFi signals
Skyhook (www.skyhookwireless.com)
Core Engine
Database of known locations
700 million Wi-Fi access points and cellular towers.
36.
37. Comparative Accuracies
Study testing iPhone 3GS cf. low cost GPS
A-GPS
8 m error
WiFi
74 m error
Cell Tower Positioning
600 m error
Accuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi, and
Cellular Positioning
In GIScience on July 15, 2009 at 8:11 pm By Paul A Zandbergen
Transactions in GIS, Volume 13 Issue s1, Pages 5 - 25
41. Optical Tracking Technologies
Scalable active trackers
InterSense IS-900, 3rd Tech HiBall
Passive optical computer vision
Line of sight, may require landmarks
Can be brittle.
Computer vision is computationally-intensive
3rd Tech, Inc.
42. HiBall Tracking System (3rd Tech)
Inside-Out Tracker
$50K USD
Scalable over large area
Fast update (2000Hz)
Latency Less than 1 ms.
Accurate
Position 0.4mm RMS
Orientation 0.02° RMS
43.
44. Starting simple: Marker tracking
Has been done for more than 10 years
A square marker provides 4 corners
Enough for pose estimation!
Several open source solutions exist
Fairly simple to implement
Standard computer vision methods
49. Tracking challenges in ARToolKit
False positives and inter-marker confusion
(image by M. Fiala)
Image noise
(e.g. poor lens, block coding /
compression, neon tube)
Unfocused camera,
motion blur
Dark/unevenly lit
scene, vignetting
Jittering
(Photoshop illustration)
Occlusion
(image by M. Fiala)
50. Limitations of ARToolKit
Partial occlusions cause tracking failure
Affected by lighting and shadows
Tracking range depends on marker size
Performance depends on number of markers
cf artTag, ARToolKitPlus
Pose accuracy depends on distance to marker
Pose accuracy depends on angle to marker
55. Markerless Tracking
Magnetic Tracker Inertial
Tracker
Ultrasonic
Tracker
Optical
Tracker
Marker-Based
Tracking
Markerless
Tracking
Specialized
Tracking
Edge-Based
Tracking
Template-Based
Tracking
Interest Point
Tracking
No more Markers! Markerless Tracking
56. Natural feature tracking
Tracking from features of the surrounding
environment
Corners, edges, blobs, ...
Generally more difficult than marker tracking
Markers are designed for their purpose
The natural environment is not…
Less well-established methods
Usually much slower than marker tracking
57. Natural Feature Tracking
Use Natural Cues of Real Elements
Edges
Surface Texture
Interest Points
Model or Model-Free
++: no visual pollution
Contours
Features Points
Surfaces
61. Model Based Tracking
Track from 3D model
Eg OpenTL - www.opentl.org
General purpose library for model based visual tracking
62. Marker vs. natural feature tracking
Marker tracking
+ Can require no image database to be stored
+ Markers can be an eye-catcher
+ Tracking is less demanding
- The environment must be instrumented with markers
- Markers usually work only when fully in view
Natural feature tracking
- A database of keypoints must be stored/downloaded
+ Natural feature targets might catch the attention less
+ Natural feature targets are potentially everywhere
+ Natural feature targets work also if partially in view
64. Sensor tracking
Used by many “AR browsers”
GPS, Compass, Accelerometer, (Gyroscope)
Not sufficient alone (drift, interference)
65. Outdoor Hybrid Tracking
Combines
computer vision
- natural feature tracking
inertial gyroscope sensors
Both correct for each other
Inertial gyro - provides frame to frame
prediction of camera orientation
Computer vision - correct for gyro drift
66. Combining Sensors and Vision
Sensors
- Produce noisy output (= jittering augmentations)
- Are not sufficiently accurate (= wrongly placed augmentations)
- Gives us first information on where we are in the world,
and what we are looking at
Vision
- Is more accurate (= stable and correct augmentations)
- Requires choosing the correct keypoint database to track from
- Requires registering our local coordinate frame (online-
generated model) to the global one (world)
72. The Registration Problem
Virtual and Real must stay properly aligned
If not:
Breaks the illusion that the two coexist
Prevents acceptance of many serious applications
73. Sources of registration errors
Static errors
Optical distortions
Mechanical misalignments
Tracker errors
Incorrect viewing parameters
Dynamic errors
System delays (largest source of error)
- 1 ms delay = 1/3 mm registration error
74. Reducing static errors
Distortion compensation
Manual adjustments
View-based or direct measurements
Camera calibration (video)
76. Dynamic errors
Total Delay = 50 + 2 + 33 + 17 = 102 ms
1 ms delay = 1/3 mm = 33mm error
Tracking Calculate
Viewpoint
Simulation
Render
Scene
Draw to
Display
x,y,z
r,p,y
Application Loop
20 Hz = 50ms 500 Hz = 2ms 30 Hz = 33ms 60 Hz = 17ms
77. Reducing dynamic errors (1)
Reduce system lag
Faster components/system modules
Reduce apparent lag
Image deflection
Image warping
78. Reducing System Lag
Tracking Calculate
Viewpoint
Simulation
Render
Scene
Draw to
Display
x,y,z
r,p,y
Application Loop
Faster Tracker Faster CPU Faster GPU Faster Display
79. Reducing Apparent Lag
Tracking
Update
x,y,z
r,p,y
Virtual Display
Physical
Display
(640x480)
1280 x 960
Last known position
Virtual Display
Physical
Display
(640x480)
1280 x 960
Latest position
Tracking Calculate
Viewpoint
Simulation
Render
Scene
Draw to
Display
x,y,z
r,p,y
Application Loop
80. Reducing dynamic errors (2)
Match input streams (video)
Delay video of real world to match system lag
Predictive Tracking
Inertial sensors helpful
Azuma / Bishop 1994
83. Wrap-up
Tracking and Registration are key problems
Registration error
Measures against static error
Measures against dynamic error
AR typically requires multiple tracking technologies
Research Areas: Hybrid Markerless Techniques,
Deformable Surface, Mobile, Outdoors
84. Project List
Mobile
Hybrid Tracking for Outdoor AR
City Scale AR Visualization
Outdoor AR Authoring Tool
Outdoor AR collaborative game
AR interaction for Google Glass
Non-Mobile
AR Face Painting
AR Authoring Tool
Tangible AR puppeteer studio
Gesture based interaction with AR content
85. More Information
• Mark Billinghurst
– mark.billinghurst@hitlabnz.org
• Websites
– www.hitlabnz.org