Mobile AR Lecture 2 - Technology
•
6 likes•1,828 views
Lecture 2 from a course on Mobile Based Augmented Reality Development taught by Mark Billinghurst and Zi Siang See on November 29th and 30th 2015 at Johor Bahru in Malaysia. This lecture provides an introduction to Mobile AR Technology. Look for the other 9 lectures in the course.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Mobile AR Lecture 2 - Technology
Similar to Mobile AR Lecture 2 - Technology (20)
More from Mark Billinghurst
More from Mark Billinghurst (20)
Recently uploaded
Recently uploaded (20)
Mobile AR Lecture 2 - Technology
- 1. LECTURE 2: AR TECHNOLOGY Mark Billinghurst mark.billinghurst@unisa.edu.au Zi Siang See zisiang@reina.com.my November 29th-30th 2015 Mobile-Based Augmented Reality Development
- 2. TECHNOLOGY
- 3. Augmented Reality Definition • Defining Characteristics • Combines Real andVirtual Images • Display Technology • Interactive in real-time • Interaction Technology • Registered in 3D • Tracking Technology
- 4. DISPLAY
- 5. Display Technologies ! Types (Bimber/Raskar 2003) ! Head attached • Head mounted display/projector ! Body attached • Handheld display/projector ! Spatial • Spatially aligned projector/monitor
- 7. Types of Head Mounted Displays Occluded See-thru Multiplexed
- 8. Optical see-through HMD Virtual images from monitors Real World Optical Combiners
- 9. Epson Moverio BT-200 ▪ Stereo see-through display ($700) ▪ 960 x 540 pixels, 23 degree FOV, 60Hz, 88g ▪ Android Powered, separate controller ▪ VGA camera, GPS, gyro, accelerometer
- 10. Strengths of optical see-through • Simpler (cheaper) • Direct view of real world • Full resolution, no time delay (for real world) • Safety • Lower distortion • No eye displacement • see directly through display
- 12. Vuzix Wrap 1200DXAR ▪ Stereo video see-through display ($1500) ■ Twin 852 x 480 LCD displays, 35 deg. FOV ■ StereoVGA cameras ■ 3 DOF head tracking
- 13. Strengths of Video See-Through • True occlusion • Block image of real world • Digitized image of real world • Flexibility in composition • Matchable time delays • More registration, calibration strategies • Wide FOV is easier to support • wide FOV camera
- 14. Multiplexed Displays • Above or below line of sight • Strengths • User has unobstructed view of real world • Simple optics/cheap • Weaknesses • Direct information overlay difficult • Display/camera offset from eyeline • Wide FOV difficult
- 15. Google Glass ▪ Monocular see-through multiplexed display ▪ 640 x 360 microprojector, 15 degree FOV ▪ 5 MP camera, gyro, accelerometer
- 16. DisplayTechnology • Curved Mirror • off-axis projection • curved mirrors in front of eye • high distortion, small eye-box • Waveguide • use internal reflection • unobstructed view of world • large eye-box
- 17. See-through thin displays • Waveguide techniques for thin see-through displays • Wider FOV, enable AR applications • Social acceptability Opinvent Ora Lumus DK40
- 18. SpatialAugmented Reality • Project onto irregular surfaces • Geometric Registration • Projector blending, High dynamic range • Book: Bimber “Spatial Augmented Reality”
- 19. Projector-basedAR Examples: Raskar, MIT Media Lab Inami, Tachi Lab, U. Tokyo Projector Real objects with retroreflective covering User (possibly head-tracked)
- 20. HMD vs.HMPD Head Mounted Display Head Mounted Projected Display
- 21. CastAR - http://technicalillusions.com/ • Stereo head worn projectors • Interactive wand • Rollable retro-reflective sheet
- 22. • Designed for shared interaction
- 23. Video MonitorAR Video cameras Monitor Graphics Combiner Video Stereo glasses
- 24. Examples
- 26. TRACKING
- 27. Objects Registered in 3D • Registration • Positioning virtual object wrt real world • Tracking • Continually locating the users viewpoint • Position (x,y,z), Orientation (r,p,y)
- 28. Tracking Technologies " Active • Mechanical, Magnetic, Ultrasonic • GPS, Wifi, cell location " Passive • Inertial sensors (compass, accelerometer, gyro) • Computer Vision • Marker based, Natural feature tracking " Hybrid Tracking • Combined sensors (eg Vision + Inertial)
- 30. Example:Marker tracking • Available for more than 10 years • Several open source solutions exist • ARToolKit, ARTag, ATK+, etc • Fairly simple to implement • Standard computer vision methods • A rectangle provides 4 corner points • Enough for pose estimation!
- 31. Marker Based Tracking: ARToolKit http://artoolkit.sourceforge.net/
- 33. Tracking challenges inARToolKit False positives and inter-marker confusion (image by M. Fiala) Image noise (e.g. poor lens, block coding / compression, neon tube) Unfocused camera, motion blur Dark/unevenly lit scene, vignetting Jittering (Photoshop illustration) Occlusion (image by M. Fiala)
- 34. Markerless Tracking Magnetic Tracker Inertial Tracker Ultrasonic Tracker Optical Tracker Marker-Based Tracking Markerless Tracking Specialized Tracking Edge-Based Tracking Template-Based Tracking Interest Point Tracking • No more Markers! #Markerless Tracking Mechanical Tracker
- 35. Natural Feature Tracking • Use Natural Cues of Real Elements • Edges • Surface Texture • Interest Points • Model or Model-Free • No visual pollution Contours Features Points Surfaces
- 36. TextureTracking
- 37. Edge Based Tracking • RAPiD [Drummond et al. 02] • Initialization, Control Points, Pose Prediction (Global Method)
- 38. Line Based Tracking • Visual Servoing [Comport et al. 2004]
- 39. Model BasedTracking • OpenTL - www.opentl.org • General purpose library for model based visual tracking
- 40. Marker vs.natural feature tracking • Marker tracking • ++ Markers can be an eye-catcher • ++ Tracking is less demanding • -- The environment must be instrumented with markers • -- Markers usually work only when fully in view • Natural feature tracking • -- A database of keypoints must be stored/downloaded • ++ Natural feature targets might catch the attention less • ++ Natural feature targets are potentially everywhere • ++ Natural feature targets work also if partially in view
- 41. Example: Outdoor Hybrid Tracking • Combines • computer vision • natural feature tracking • inertial gyroscope sensors • Both correct for each other • Inertial gyro - provides frame to frame prediction of camera orientation • Computer vision - correct for gyro drift
- 42. Robust OutdoorTracking • Hybrid Tracking • ComputerVision, GPS, inertial • Going Out • Reitmayr & Drummond (Univ. Cambridge) Reitmayr, G., & Drummond, T. W. (2006). Going out: robust model-based tracking for outdoor augmented reality. In Mixed and Augmented Reality, 2006. ISMAR 2006. IEEE/ACM International Symposium on (pp. 109-118). IEEE.
- 43. Handheld Display
- 44. INTERACTION
- 45. • Interface Components • Physical components • Display elements • Visual/audio • Interaction metaphors Physical Elements Display ElementsInteraction MetaphorInput Output AR Interface Elements
- 46. AR Design Space Reality Virtual Reality Augmented Reality Physical Design Virtual Design
- 47. Interface Design Path 1/ Prototype Demonstration 2/ Adoption of Interaction Techniques from other interface metaphors 3/ Development of new interface metaphors appropriate to the medium 4/ Development of formal theoretical models for predicting and modeling user actions Desktop WIMP Virtual Reality Augmented Reality
- 48. Interaction Development • Information Browsing • Camera movement • Limited interaction • 3D AR Interaction • HMD, hand tracking • 3D UI/VR techniques • Specialized input devices
- 49. Tangible User Interfaces (Ishii 97) • Augmented Surfaces • Rekimoto 1998 • Multiple projection surfaces • Tangible prop interaction • i/O Brush (2004) • Ryokai, Marti, Ishii • Sensor enhanced real brush
- 50. Other Examples • Triangles (Gorbert 1998) • Triangular based story telling • ActiveCube (Kitamura 2000-) • Cubes with sensors
- 51. Lessons fromTangible Interfaces • Benefits • Physical objects make us smart (affordances) • Objects aid collaboration • Objects increase understanding (cognitive artifacts) • Limitations • Difficult to change object properties • Limited display capabilities (2D surface) • Separation between object and display
- 52. Orthogonal Nature ofAR Interfaces
- 53. TangibleAR Interaction • AR overcomes limitation of TUIs • enhance display possibilities • merge task/display space • provide public and private views • TUI + AR = Tangible AR • Apply TUI methods to AR interface design
- 54. Tangible AR Design Principles • Tangible AR Interfaces use TUI principles • Physical controllers for moving virtual content • Support for spatial 3D interaction techniques • Support for multi-handed interaction • Match object affordances to task requirements • Support parallel activity with multiple objects • Allow collaboration between multiple users
- 55. VOMAR -TangibleAR Interface • Use of natural physical object to control virtual objects • Physical objects • Catalog book: • Turn over the page • Paddle operation: • Push, shake, incline, hit, scoop Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., & Tachibana, K. (2000). Virtual object manipulation on a table-top AR environment. In Augmented Reality, 2000.(ISAR 2000). Proceedings. IEEE and ACM International Symposium on (pp. 111-119). Ieee.
- 56. Interaction Evolution • Browsing Interfaces • simple (conceptually!), unobtrusive • 3D AR Interfaces • expressive, creative, require attention • Tangible Interfaces • Embedded into conventional environments • Tangible AR • Combines TUI input + AR display
- 57. AR APPLICATIONS
- 58. • Web based AR • Flash, HTML 5 based AR • Marketing, education • Outdoor Mobile AR • GPS, compass tracking • Viewing Points of Interest in real world • Eg: Junaio, Layar, Wikitude • Handheld AR • Vision based tracking • Marketing, gaming • Location Based Experiences • HMD, fixed screens • Museums, point of sale, advertising Typical AR Experiences
- 59. CityViewARApplication • Visualize Christchurch before the earthquakes
- 60. User Experience • MultipleViews • MapView,ARView, ListView • Multiple Data Types • 2D images, 3D content, text, panoramas
- 61. Warp Runner • Puzzle solving game • Deform real world terrain
- 62. Demo:colAR • Turn colouring books pages into AR scenes • Markerless tracking, use your own colours.. • Try it yourself: http://www.colARapp.com/
- 63. What Makes a GoodAR Experience? • Compelling • Engaging,‘Magic’ moment • Intuitive, ease of use • Uses existing skills • Anchored in physical world • Seamless combination of real and digital
- 64. Conclusion • AR seamlessly blends real and virtual imagery • Interactive in real time, fixed in space • AR has developed into a mass market technology • Education, engineering, entertainment • The technologies to create AR are available • Display, tracking, interaction