426 lecture3: AR Tracking


Published on

COSC 426 Graduate class in Augmented Reality, lecture on AR tracking. Taught by Mark Billinghurst of the HIT Lab NZ at the University of Canterbury, July 25th 2012

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

426 lecture3: AR Tracking

  1. 1. COSC 426: Augmented Reality Mark Billinghurst mark.billinghurst@hitlabnz.org July 25th 2012 Lecture 3: AR Tracking
  2. 2. Tracking RequirementsHead Stabilized Body Stabilized World Stabilized  Augmented Reality Information Display   World Stabilized Increasing Tracking   Body Stabilized Requirements   Head Stabilized
  3. 3. Tracking Technologies•  Mechanical•  Electromagnetic•  Optical•  Acoustic•  Inertial and dead reckoning•  GPS•  Hybrid
  4. 4. AR Tracking Taxonomy AR TRACKING Indoor Outdoor Environment Environment Limited Range Extended Range Low Accuracy & High Accuracy Not Robust & RobustLow Accuracy High Accuracy Many Fiducials Not Hybridized Hybrid Tracking at 15-60 Hz & High Speed in space/time GPS or GPS and Hybrid but Camera or Camera and Tracking no GPS Compass Compasse.g. AR Toolkit e.g. IVRD e.g. HiBall e.g. WLVA e.g. BARS
  5. 5. Tracking TypesMagnetic Inertial Ultrasonic Optical MechanicalTracker Tracker Tracker Tracker Tracker Specialized Marker-Based Markerless Tracking Tracking Tracking Edge-Based Template-Based Interest Point Tracking Tracking Tracking
  6. 6. Tracking Systems  Mechanical Tracker  Magnetic Tracker  Ultrasonic Tracker  Inertial Tracker  Computer Vision (Optical Tracking)   Specialized (Infrared, Retro-Reflective)   Monocular (DVCam, Webcam)
  7. 7. Mechanical Tracker  Idea: mechanical arms with joint sensors Microscribe  ++: high accuracy, haptic feedback  -- : cumbersome, expensive
  8. 8. Magnetic Tracker  Idea: difference between a magnetic transmitter and a receiver Flock of Birds (Ascension)  ++: 6DOF, robust  -- : wired, sensible to metal, noisy, expensive
  9. 9. Magnetic Tracking Error
  10. 10. Ultrasonics Tracker  Idea: Time of Flight or Phase-Coherence Sound Waves Ultrasonic Logitech IS600  ++: Small, Cheap  -- : 3DOF, Line of Sight, Low resolution, Affected Environment Conditon (pressure, temperature)
  11. 11. Inertial Tracker  Idea: measuring linear and angular orientation rates (accelerometer/gyroscope)IS300 (Intersense) Wii Remote  ++: no transmitter, cheap, small, high frequency, wireless  -- : drift, hysteris only 3DOF
  12. 12. Mobile Sensors  Inertial compass   Earth’s magnetic field   Measures absolute orientation  Accelerometers   Measures acceleration about axis   Used for tilt, relative rotation   Can drift over time
  13. 13. Global Positioning System (GPS)  Created by US in 1978   Currently 29 satellites  Satellites send position + time  GPS Receiver positioning   4 satellites need to be visible   Differential time of arrival   Triangulation  Accuracy   5-30m+, blocked by weather, buildings etc
  14. 14. Problems with GPS  Takes time to get satellite fix   Satellites moving around  Earths atmosphere affects signal   Assumes consistent speed (the speed of light).   Delay depends where you are on Earth   Weather effects  Signal reflection   Multi-path reflection off buildings  Signal blocking   Trees, buildings, mountains  Satellites send out bad data   Misreport their own position
  15. 15. Accurate to < 5cm close to base station (22m/100 km)Expensive - $20-40,000 USD
  16. 16. Assisted-GPS (A-GPS)  Use external location server to send GPS signal   GPS receivers on cell towers, etc   Sends precise satellite position (Ephemeris)  Speeds up GPS Tracking   Makes it faster to search for satellites   Provides navigation data (don’t decode on phone)  Other benefits   Provides support for indoor positioning   Can use cheaper GPS hardware   Uses less battery power on device
  17. 17. Assisted GPS
  18. 18. Cell Tower Triangulation  Calculate phone position from signal strength  < 50 m in cities  > 1 km in rural
  19. 19. WiFi Positioning  Estimate location by using WiFi access points   Can use know locations of WiFi access points   Triangulate through signal strength  Eg. PlaceEngine (www.placeengine.com)   Client software for PC and mobiles   SDK returns position  Accuracy   5 – 100m (depends on WiFi density)
  20. 20. WiFi Hotspots in New York
  21. 21. Indoor WiFi Location Sensing  Indoor Location   Asset, people tracking  Aeroscout   http://aeroscout.com/   WiFi + RFID  Ekahau   http://www.ekahau.com/   WiFi + LED tracking
  22. 22. Integrated Systems  Combine GPS, Cell tower, WiFi signals  Skyhook (www.skyhookwireless.com)   Core Engine  Database of known locations   700 million Wi-Fi access points and cellular towers.
  23. 23. Comparative Accuracies  Study testing iPhone 3GS cf. low cost GPS  A-GPS   8 m error  WiFi   74 m error  Cell Tower Positioning   600 m errorAccuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi, and Cellular PositioningIn GIScience on July 15, 2009 at 8:11 pm By Paul A Zandbergen Transactions in GIS, Volume 13 Issue s1, Pages 5 - 25
  24. 24. Optical Tracking
  25. 25. Optical Tracker  Idea: Image Processing and Computer Vision  Specialized   Infrared, Retro-Reflective, Stereoscopic ART Hi-Ball  Monocular Based Vision Tracking
  26. 26. Outside-In vs. Inside-Out Tracking
  27. 27. Optical Tracking Technologies  Scalable active trackers   InterSense IS-900, 3rd Tech HiBall 3rd Tech, Inc.  Passive optical computer vision   Line of sight, may require landmarks   Can be brittle.   Computer vision is computationally-intensive
  28. 28. HiBall Tracking System (3rd Tech)  Inside-Out Tracker   $50K USD  Scalable over large area   Fast update (2000Hz)   Latency Less than 1 ms.  Accurate   Position 0.4mm RMS   Orientation 0.02° RMS
  29. 29. Starting simple: Marker tracking  Has been done for more than 10 years  Several open source solutions exist  Fairly simple to implement   Standard computer vision methods  A rectangular marker provides 4 corner points   Enough for pose estimation!
  30. 30. Marker Based Tracking: ARToolKithttp://artoolkit.sourceforge.net/
  31. 31. Coordinate Systems
  32. 32. Tracking Range with Pattern SizeRule of thumb – range = 10 x pattern width
  33. 33. Tracking Error with Range
  34. 34. Tracking Error with Angle
  35. 35. Tracking challenges in ARToolKit Occlusion Unfocused camera, Dark/unevenly lit Jittering(image by M. Fiala) motion blur scene, vignetting (Photoshop illustration) Image noiseFalse positives and inter-marker confusion (e.g. poor lens, block coding / compression, neon tube) (image by M. Fiala)
  36. 36. Limitations of ARToolKit  Partial occlusions cause tracking failure  Affected by lighting and shadows  Tracking range depends on marker size  Performance depends on number of markers   cf artTag, ARToolKitPlus  Pose accuracy depends on distance to marker  Pose accuracy depends on angle to marker
  37. 37. Tracking, Tracking, Tracking
  38. 38. Other Marker Tracking Libraries  arTag   http://www.artag.net/  ARToolKitPlus [Discontinued]   http://studierstube.icg.tu-graz.ac.at/handheld_ar/ artoolkitplus.php  stbTracker   http://studierstube.icg.tu-graz.ac.at/handheld_ar/ stbtracker.php  MXRToolKit   http://sourceforge.net/projects/mxrtoolkit/
  39. 39. Markerless Tracking
  40. 40. Markerless Tracking  No more Markers! Markerless TrackingMagnetic Tracker Inertial Ultrasonic Optical Tracker Tracker Tracker Specialized Marker-Based Markerless Tracking Tracking Tracking Edge-Based Template-Based Interest Point Tracking Tracking Tracking
  41. 41. Natural feature tracking  Tracking from features of the surrounding environment   Corners, edges, blobs, ...  Generally more difficult than marker tracking   Markers are designed for their purpose   The natural environment is not…  Less well-established methods  Usually much slower than marker tracking
  42. 42. Natural Feature Tracking Features Points  Use Natural Cues of Real Elements Contours   Edges   Surface Texture   Interest Points  Model or Model-Free  ++: no visual pollution Surfaces
  43. 43. Texture Tracking
  44. 44. Edge Based Tracking  RAPiD [Drummond et al. 02]   Initialization, Control Points, Pose Prediction (Global Method)
  45. 45. Line Based Tracking  Visual Servoing [Comport et al. 2004]
  46. 46. Model Based Tracking  Track from 3D model  Eg OpenTL - www.opentl.org   General purpose library for model based visual tracking
  47. 47. Marker vs. natural feature tracking  Marker tracking   + Can require no image database to be stored   + Markers can be an eye-catcher   + Tracking is less demanding   - The environment must be instrumented with markers   - Markers usually work only when fully in view  Natural feature tracking   - A database of keypoints must be stored/downloaded   + Natural feature targets might catch the attention less   + Natural feature targets are potentially everywhere   + Natural feature targets work also if partially in view
  48. 48. Hybrid Tracking
  49. 49. Hybrid TrackingCombining several tracking modalities together
  50. 50. Sensor tracking  Used by many “AR browsers”  GPS, Compass, Accelerometer, (Gyroscope)  Not sufficient alone (drift, interference)
  51. 51. Outdoor Hybrid Tracking  Combines   computer vision -  natural feature tracking   inertial gyroscope sensors  Both correct for each other   Inertial gyro - provides frame to frame prediction of camera orientation   Computer vision - correct for gyro drift
  52. 52. Combining Sensors and Vision  Sensors -  Produce noisy output (= jittering augmentations) -  Are not sufficiently accurate (= wrongly placed augmentations) -  Gives us first information on where we are in the world, and what we are looking at  Vision -  Is more accurate (= stable and correct augmentations) -  Requires choosing the correct keypoint database to track from -  Requires registering our local coordinate frame (online- generated model) to the global one (world)
  53. 53. Outdoor AR Tracking SystemYou, Neumann, Azuma outdoor AR system (1999)
  54. 54. Robust Outdoor Tracking  Hybrid Tracking   Computer Vision, GPS, inertial  Going Out   Reitmayer & Drummond (Univ. Cambridge)
  55. 55. Handheld Display
  56. 56. Wrap-up  Tracking and Registration are key problems  Registration error   Measures against static error   Measures against dynamic error  AR typically requires multiple tracking technologies  Research Areas: Hybrid Markerless Techniques, Deformable Surface, Mobile, Outdoors
  57. 57. More Information•  Mark Billinghurst –  mark.billinghurst@hitlabnz.org •  Websites –  www.hitlabnz.org