• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
426 lecture3: AR Tracking

426 lecture3: AR Tracking



COSC 426 Graduate class in Augmented Reality, lecture on AR tracking. Taught by Mark Billinghurst of the HIT Lab NZ at the University of Canterbury, July 25th 2012

COSC 426 Graduate class in Augmented Reality, lecture on AR tracking. Taught by Mark Billinghurst of the HIT Lab NZ at the University of Canterbury, July 25th 2012



Total Views
Views on SlideShare
Embed Views



1 Embed 2

http://pinterest.com 2



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    426 lecture3: AR Tracking 426 lecture3: AR Tracking Presentation Transcript

    • COSC 426: Augmented Reality Mark Billinghurst mark.billinghurst@hitlabnz.org July 25th 2012 Lecture 3: AR Tracking
    • Tracking RequirementsHead Stabilized Body Stabilized World Stabilized  Augmented Reality Information Display   World Stabilized Increasing Tracking   Body Stabilized Requirements   Head Stabilized
    • Tracking Technologies•  Mechanical•  Electromagnetic•  Optical•  Acoustic•  Inertial and dead reckoning•  GPS•  Hybrid
    • AR Tracking Taxonomy AR TRACKING Indoor Outdoor Environment Environment Limited Range Extended Range Low Accuracy & High Accuracy Not Robust & RobustLow Accuracy High Accuracy Many Fiducials Not Hybridized Hybrid Tracking at 15-60 Hz & High Speed in space/time GPS or GPS and Hybrid but Camera or Camera and Tracking no GPS Compass Compasse.g. AR Toolkit e.g. IVRD e.g. HiBall e.g. WLVA e.g. BARS
    • Tracking TypesMagnetic Inertial Ultrasonic Optical MechanicalTracker Tracker Tracker Tracker Tracker Specialized Marker-Based Markerless Tracking Tracking Tracking Edge-Based Template-Based Interest Point Tracking Tracking Tracking
    • Tracking Systems  Mechanical Tracker  Magnetic Tracker  Ultrasonic Tracker  Inertial Tracker  Computer Vision (Optical Tracking)   Specialized (Infrared, Retro-Reflective)   Monocular (DVCam, Webcam)
    • Mechanical Tracker  Idea: mechanical arms with joint sensors Microscribe  ++: high accuracy, haptic feedback  -- : cumbersome, expensive
    • Magnetic Tracker  Idea: difference between a magnetic transmitter and a receiver Flock of Birds (Ascension)  ++: 6DOF, robust  -- : wired, sensible to metal, noisy, expensive
    • Magnetic Tracking Error
    • Ultrasonics Tracker  Idea: Time of Flight or Phase-Coherence Sound Waves Ultrasonic Logitech IS600  ++: Small, Cheap  -- : 3DOF, Line of Sight, Low resolution, Affected Environment Conditon (pressure, temperature)
    • Inertial Tracker  Idea: measuring linear and angular orientation rates (accelerometer/gyroscope)IS300 (Intersense) Wii Remote  ++: no transmitter, cheap, small, high frequency, wireless  -- : drift, hysteris only 3DOF
    • Mobile Sensors  Inertial compass   Earth’s magnetic field   Measures absolute orientation  Accelerometers   Measures acceleration about axis   Used for tilt, relative rotation   Can drift over time
    • Global Positioning System (GPS)  Created by US in 1978   Currently 29 satellites  Satellites send position + time  GPS Receiver positioning   4 satellites need to be visible   Differential time of arrival   Triangulation  Accuracy   5-30m+, blocked by weather, buildings etc
    • Problems with GPS  Takes time to get satellite fix   Satellites moving around  Earths atmosphere affects signal   Assumes consistent speed (the speed of light).   Delay depends where you are on Earth   Weather effects  Signal reflection   Multi-path reflection off buildings  Signal blocking   Trees, buildings, mountains  Satellites send out bad data   Misreport their own position
    • Accurate to < 5cm close to base station (22m/100 km)Expensive - $20-40,000 USD
    • Assisted-GPS (A-GPS)  Use external location server to send GPS signal   GPS receivers on cell towers, etc   Sends precise satellite position (Ephemeris)  Speeds up GPS Tracking   Makes it faster to search for satellites   Provides navigation data (don’t decode on phone)  Other benefits   Provides support for indoor positioning   Can use cheaper GPS hardware   Uses less battery power on device
    • Assisted GPS
    • Cell Tower Triangulation  Calculate phone position from signal strength  < 50 m in cities  > 1 km in rural
    • WiFi Positioning  Estimate location by using WiFi access points   Can use know locations of WiFi access points   Triangulate through signal strength  Eg. PlaceEngine (www.placeengine.com)   Client software for PC and mobiles   SDK returns position  Accuracy   5 – 100m (depends on WiFi density)
    • WiFi Hotspots in New York
    • Indoor WiFi Location Sensing  Indoor Location   Asset, people tracking  Aeroscout   http://aeroscout.com/   WiFi + RFID  Ekahau   http://www.ekahau.com/   WiFi + LED tracking
    • Integrated Systems  Combine GPS, Cell tower, WiFi signals  Skyhook (www.skyhookwireless.com)   Core Engine  Database of known locations   700 million Wi-Fi access points and cellular towers.
    • Comparative Accuracies  Study testing iPhone 3GS cf. low cost GPS  A-GPS   8 m error  WiFi   74 m error  Cell Tower Positioning   600 m errorAccuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi, and Cellular PositioningIn GIScience on July 15, 2009 at 8:11 pm By Paul A Zandbergen Transactions in GIS, Volume 13 Issue s1, Pages 5 - 25
    • Optical Tracking
    • Optical Tracker  Idea: Image Processing and Computer Vision  Specialized   Infrared, Retro-Reflective, Stereoscopic ART Hi-Ball  Monocular Based Vision Tracking
    • Outside-In vs. Inside-Out Tracking
    • Optical Tracking Technologies  Scalable active trackers   InterSense IS-900, 3rd Tech HiBall 3rd Tech, Inc.  Passive optical computer vision   Line of sight, may require landmarks   Can be brittle.   Computer vision is computationally-intensive
    • HiBall Tracking System (3rd Tech)  Inside-Out Tracker   $50K USD  Scalable over large area   Fast update (2000Hz)   Latency Less than 1 ms.  Accurate   Position 0.4mm RMS   Orientation 0.02° RMS
    • Starting simple: Marker tracking  Has been done for more than 10 years  Several open source solutions exist  Fairly simple to implement   Standard computer vision methods  A rectangular marker provides 4 corner points   Enough for pose estimation!
    • Marker Based Tracking: ARToolKithttp://artoolkit.sourceforge.net/
    • Coordinate Systems
    • Tracking Range with Pattern SizeRule of thumb – range = 10 x pattern width
    • Tracking Error with Range
    • Tracking Error with Angle
    • Tracking challenges in ARToolKit Occlusion Unfocused camera, Dark/unevenly lit Jittering(image by M. Fiala) motion blur scene, vignetting (Photoshop illustration) Image noiseFalse positives and inter-marker confusion (e.g. poor lens, block coding / compression, neon tube) (image by M. Fiala)
    • Limitations of ARToolKit  Partial occlusions cause tracking failure  Affected by lighting and shadows  Tracking range depends on marker size  Performance depends on number of markers   cf artTag, ARToolKitPlus  Pose accuracy depends on distance to marker  Pose accuracy depends on angle to marker
    • Tracking, Tracking, Tracking
    • Other Marker Tracking Libraries  arTag   http://www.artag.net/  ARToolKitPlus [Discontinued]   http://studierstube.icg.tu-graz.ac.at/handheld_ar/ artoolkitplus.php  stbTracker   http://studierstube.icg.tu-graz.ac.at/handheld_ar/ stbtracker.php  MXRToolKit   http://sourceforge.net/projects/mxrtoolkit/
    • Markerless Tracking
    • Markerless Tracking  No more Markers! Markerless TrackingMagnetic Tracker Inertial Ultrasonic Optical Tracker Tracker Tracker Specialized Marker-Based Markerless Tracking Tracking Tracking Edge-Based Template-Based Interest Point Tracking Tracking Tracking
    • Natural feature tracking  Tracking from features of the surrounding environment   Corners, edges, blobs, ...  Generally more difficult than marker tracking   Markers are designed for their purpose   The natural environment is not…  Less well-established methods  Usually much slower than marker tracking
    • Natural Feature Tracking Features Points  Use Natural Cues of Real Elements Contours   Edges   Surface Texture   Interest Points  Model or Model-Free  ++: no visual pollution Surfaces
    • Texture Tracking
    • Edge Based Tracking  RAPiD [Drummond et al. 02]   Initialization, Control Points, Pose Prediction (Global Method)
    • Line Based Tracking  Visual Servoing [Comport et al. 2004]
    • Model Based Tracking  Track from 3D model  Eg OpenTL - www.opentl.org   General purpose library for model based visual tracking
    • Marker vs. natural feature tracking  Marker tracking   + Can require no image database to be stored   + Markers can be an eye-catcher   + Tracking is less demanding   - The environment must be instrumented with markers   - Markers usually work only when fully in view  Natural feature tracking   - A database of keypoints must be stored/downloaded   + Natural feature targets might catch the attention less   + Natural feature targets are potentially everywhere   + Natural feature targets work also if partially in view
    • Hybrid Tracking
    • Hybrid TrackingCombining several tracking modalities together
    • Sensor tracking  Used by many “AR browsers”  GPS, Compass, Accelerometer, (Gyroscope)  Not sufficient alone (drift, interference)
    • Outdoor Hybrid Tracking  Combines   computer vision -  natural feature tracking   inertial gyroscope sensors  Both correct for each other   Inertial gyro - provides frame to frame prediction of camera orientation   Computer vision - correct for gyro drift
    • Combining Sensors and Vision  Sensors -  Produce noisy output (= jittering augmentations) -  Are not sufficiently accurate (= wrongly placed augmentations) -  Gives us first information on where we are in the world, and what we are looking at  Vision -  Is more accurate (= stable and correct augmentations) -  Requires choosing the correct keypoint database to track from -  Requires registering our local coordinate frame (online- generated model) to the global one (world)
    • Outdoor AR Tracking SystemYou, Neumann, Azuma outdoor AR system (1999)
    • Robust Outdoor Tracking  Hybrid Tracking   Computer Vision, GPS, inertial  Going Out   Reitmayer & Drummond (Univ. Cambridge)
    • Handheld Display
    • Wrap-up  Tracking and Registration are key problems  Registration error   Measures against static error   Measures against dynamic error  AR typically requires multiple tracking technologies  Research Areas: Hybrid Markerless Techniques, Deformable Surface, Mobile, Outdoors
    • More Information•  Mark Billinghurst –  mark.billinghurst@hitlabnz.org •  Websites –  www.hitlabnz.org