2014 COSC 426 Lecture 2: Augmented Reality Technology

1,816 views
1,658 views

Published on

This is the second lecture in the COSC 426

Published in: Technology

2014 COSC 426 Lecture 2: Augmented Reality Technology

  1. 1. COSC 426: Augmented Reality Mark Billinghurst mark.billinghurst@hitlabnz.org July 23rd 2014 Lecture 2: AR Technology mark.billinghurst@hitlabnz.org
  2. 2. Recap
  3. 3. Augmented Reality Definition   Defining Characteristics [Azuma 97]   Combines Real and Virtual Images -  Both can be seen at the same time   Interactive in real-time -  Virtual content can be interacted with   Registered in 3D -  Virtual objects appear fixed in space
  4. 4. What is not Augmented Reality?   Location-based services   Barcode detection (QR-codes)   Augmenting still images   Special effects in movies   …   … but they can be combined with AR!
  5. 5. AR vs VR   Virtual Reality: Replaces Reality   Scene Generation: requires realistic images   Display Device: fully immersive, wide FOV   Tracking and Sensing: low accuracy is okay   Augmented Reality: Enhances Reality   Scene Generation: minimal rendering okay   Display Device: non-immersive, small FOV   Tracking and Sensing: high accuracy needed
  6. 6. Milgram’s Reality-Virtuality continuum Mixed Reality Reality - Virtuality (RV) Continuum Real Environment Augmented Reality (AR) Augmented Virtuality (AV) Virtual Environment "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
  7. 7. A Brief History of AR (1)   1960’s: Sutherland / Sproull’s first HMD system was see- through
  8. 8. History Summary   1960’s – 80’s: Early Experimentation   1980’s – 90’s: Basic Research   Tracking, displays   1995 – 2005: Tools/Applications   Interaction, usability, theory   2005 - : Commercial Applications   Games, Medical, Industry
  9. 9. Google Searches for AR
  10. 10. 2008 - Browser Based AR   Flash + camera + 3D graphics   High impact   High marketing value   Large potential install base   1.6 Billion web users   Ease of development   Lots of developers, mature tools   Low cost of entry   Browser, web camera
  11. 11. 2005 - Mobile Phone AR   Mobile Phones   camera   processor   display   AR on Mobile Phones   Simple graphics   Optimized computer vision   Collaborative Interaction
  12. 12. 2009 - Outdoor Information Overlay   Mobile phone based   Tag real world locations   GPS + Compass input   Overlay graphics data on live video   Applications   Travel guide, Advertising, etc   Wikitude, Layar, Junaio, etc..   Android based, Public API released
  13. 13. AR Today  Key Technologies Available -  Robust tracking (Computer Vision, GPS/sensors) -  Display (Handheld, HMDs) -  Input Devices (Kinect, etc) -  Developer tools (Qualcomm, Metaio, ARTW)  Commercial Business Growing -  Gaming, GPS/Mobile, Online Advertisement •  >$5 Billion USD by 2016 (Markets andMarkets) •  >$1.5 Billion USD in Mobile AR by 2014 (Juniper Research)
  14. 14. Sample AR Applications
  15. 15. Applications   Medicine   Manufacturing   Information overlay   Architecture   Museum   Marketing   Gaming
  16. 16. Applications: medical   “X-ray vision” for surgeons   Aid visualization, minimally-invasive operations. Training. MRI, CT data.   Ultrasound project, UNC Chapel Hill. Courtesy UNC Chapel Hill
  17. 17. Medical AR Trials   Sauer et al. 2000 at Siemens Corporate Research, NJ   Stereo video see through F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial, MICCAI 2002
  18. 18. Assembly and maintenance © 1993 S. Feiner, B. MacIntyre, & D. Seligmann, Columbia University © 1996 S. Feiner, B. MacIntyre, & A. Webster, Columbia University
  19. 19. PS3 - Eye of Judgment (2007)   Computer Vision Tracking   Card based battle game   Collaborative AR   October 24th 2007
  20. 20. AR Books – Markerless Tracking
  21. 21. AR Annotations Columbia University HRL © 1993 S. Feiner, B. MacIntyre, M. Haupt, & E. Solomon, Columbia University © 1997 S. Feiner, B. MacIntyre, T. Höllerer, & A. Webster, Columbia University
  22. 22. Broadcast TV
  23. 23. Interactive Museum Experiences   BlackMagic   Virtual America’s Cup   410,000 people in six months   MagicPlanet   TeManawa science museum   Virtual Astronomy   Collaborative AR experience   ARVolcano   Interactive AR kiosk   Scienceworks museum, Melbourne
  24. 24. Digital Binocular Station http://www.DigitalBinocularStation.com/
  25. 25. Museum Archeology   LifePlus (2002-2004)   Natural feature tracking   Virtual characters   Mobile AR system   Archeoguide (2000-2002)   Cultural heritage on-site guide   Hybrid tracking   Virtual overlay
  26. 26. Sales and Marketing   Connect with brands and branded objects   Location Based Experiences   Lynx Angels   Web based   Rayban glasses   Mobile   Ford Ka campaign   Print based   Red Bull Magazine
  27. 27. Summary   AR technology can be used to develop a wide range of applications   Promising application areas include   Games   Education   Engineering   Medicine   Museums   Etc..
  28. 28. AR Experience Design
  29. 29. “The product is no longer the basis of value.The experience is.” Venkat Ramaswamy The Future of Competition.
  30. 30. Experience Economy
  31. 31. experiences services products components Value Sony CSL © 2004 Gilmore + Pine: Experience Economy Function Emotion
  32. 32. Good Experience Design   Reactrix   Top down projection   Camera based input   Reactive Graphics   No instructions   No training
  33. 33. Improve the experience of picking up rubbish?
  34. 34. World’s Deepest Rubbish Bin   The Fun Theory – http://www.funtheory.com
  35. 35. Improve the experience of walking up stairs?
  36. 36. Musical Stairs   The Fun Theory – http://www.funtheory.com
  37. 37. Apple: The Value of Good Design   Good Experience Design Dominates Markets iPod Sales 2002-2007
  38. 38. Using the N-gage
  39. 39. SideTalking   http://www.sidetalkin.com
  40. 40. Interaction Design “Designing interactive products to support people in their everyday and working lives” Preece, J., (2002). Interaction Design   Design of User Experience with Technology   Higher in the value chain than product design
  41. 41.   Interaction Design involves answering three questions:   What do you do? - How do you affect the world?   What do you feel? – What do you sense of the world?   What do you know? – What do you learn?
  42. 42. Interaction Design is All About You   Users should be involved throughout the Design Process   Consider all the needs of the user
  43. 43. Interaction Design Process
  44. 44. experiences applications tools components Building Compelling AR Experiences Tracking, Display Authoring Interaction Usability
  45. 45. Summary   In order to build AR applications you need to focus on the user experience   Great user experience is based on   Low level AR component technology   Authoring tools   Application/Interaction design   User experience texting
  46. 46. AR Technology
  47. 47. experiences applications tools components Sony CSL © 2004 Building Compelling AR Experiences Display, Tracking
  48. 48. Core Technologies  Combining Real and Virtual Images •  Display technologies  Interactive in Real-Time •  Input and interactive technologies  Registered in 3D •  Viewpoint tracking technologies Display Processing Input Tracking
  49. 49. AR Displays
  50. 50. AR Displays e.g. window reflections Virtual Images seen off windows e.g. Reach-In Projection CRT Display using beamsplitter Not Head-Mounted e.g. Shared Space Magic Book Liquid Crystal Displays LCDs Head-Mounted Display (HMD) Primarily Indoor Environments e.g. WLVA and IVRD Cathode Ray Tube (CRT) or Virtual Retinal Display (VRD) Many Military Applications & Assistive Technologies Head-Mounted Display (HMD) e.g. Head-Up Display (HUD) Projection Display Navigational Aids in Cars Military Airborne Applications Not Head Mounted (e.g. vehicle mounted) Primarily Outdoor (Daylight) Environments AR Visual Displays
  51. 51. Display Technologies  Types (Bimber/Raskar 2003)  Head attached •  Head mounted display/projector  Body attached •  Handheld display/projector  Spatial •  Spatially aligned projector/monitor
  52. 52. Display Taxonomy
  53. 53. Head Mounted Displays
  54. 54. Head Mounted Displays (HMD) -  Display and Optics mounted on Head -  May or may not fully occlude real world -  Provide full-color images -  Considerations •  Cumbersome to wear •  Brightness •  Low power consumption •  Resolution limited •  Cost is high?
  55. 55. Key Properties of HMD   Field of View   Human eye 95 degrees horizontal, 60/70 degrees vertical   Resolution   > 320x240 pixel   Refresh Rate   Focus   Fixed/manual   Power   Size
  56. 56. Types of Head Mounted Displays The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The ima ge can not be dis play ed. You r co mp uter Occluded See-thru Multiplexed
  57. 57. Immersive VR Architecture Head! Tracker Host! Processor Data Base! Model Rendering! Engine Frame! Buffer head position/orientation to network Display! Driver Non see-thru! Image source & optics virtual object Virtual World
  58. 58. See-thru AR Architecture Head! Tracker Host! Processor Data Base! Model Rendering! Engine Frame! Buffer head position/orientation to network Display! Driver see-thru! combiner Virtual Image superimposed! over real world object real world Image source
  59. 59. Optical see-through head-mounted display Virtual images from monitors Real World Optical Combiners
  60. 60. Optical See-Through HMD
  61. 61. Epson Moverio BT-200 ▪  Stereo see-through display ($700) ▪  960 x 540 pixels, 23 degree FOV, 60Hz, 88g ▪  Android Powered, separate controller ▪  VGA camera, GPS, gyro, accelerometer
  62. 62. View Through Optical See-Through HMD
  63. 63. The Virtual Retinal Display   Image scanned onto retina   Commercialized through Microvision   Nomad System - www.mvis.com
  64. 64. Strengths of optical see-through AR   Simpler (cheaper)   Direct view of real world   Full resolution, no time delay (for real world)   Safety   Lower distortion   No eye displacement   but COASTAR video see-through avoids this
  65. 65. Video AR Architecture Head! Tracker Host! Processor Graphics! renderer Digital! Mixer Frame! Buffer head position/orientation to network Display! Driver Non see-thru! Image source & optics Head-mounted camera aligned to display optics Video! Processor Video image of real world Virtual image inset into video of real world
  66. 66. Video see-through HMD Video cameras Monitors Graphics Combiner Video
  67. 67. Video See-Through HMD
  68. 68. Vuzix Wrap 1200DXAR ▪  Stereo video see-through display ($1500) ■ Twin 852 x 480 LCD displays, 35 deg. FOV ■ Stereo VGA cameras ■ 3 DOF head tracking
  69. 69. View Through a Video See-Through HMD
  70. 70. Strengths of Video See-Through AR   True occlusion   Virtual images can block view of real world   Digitized image of real world   Flexibility in composition   Matchable time delays   More registration, calibration strategies   Wide FOV is easier to support
  71. 71. Optical vs. Video AR Summary   Both have proponents   Video is more popular today?   Likely because lack of available optical products   Depends on application?   Manufacturing: optical is cheaper   Medical: video for calibration strategies
  72. 72. Eye multiplexed AR Architecture Head! Tracker Host! Processor Data Base! Model Rendering! Engine Frame! Buffer head position/orientation to network Display! Driver Virtual Image inset into! real world scene real world Opaque! Image source
  73. 73. Virtual Image ‘inset’ into real
  74. 74. Google Glass
  75. 75. View Through Google Glass
  76. 76. Vuzix M-100 ▪  Monocular multiplexed display ($1000) ■ 852 x 480 LCD display, 15 deg. FOV ■ 5 MP camera, HD video ■ GPS, gyro, accelerometer
  77. 77. Display Types ▪  Curved Mirror ▪  off-axis projection ▪  curved mirrors in front of eye ▪  high distortion, small eye-box ▪  Waveguide ▪  use internal reflection ▪  unobstructed view of world ▪  large eye-box
  78. 78. See-through thin displays ▪  Waveguide techniques for thin see-through displays ▪  Wider FOV, enable AR applications ▪  Social acceptability Opinvent Ora
  79. 79. Waveguide Methods See: http://optinvent.com/HUD-HMD-benchmark#benchmarkTable Holographic Hologram diffracts light Limited FOV Colour bleeding Diffractive Slanted gratings Total internal reflection Costly, small FOV
  80. 80. Waveguide Methods See: http://optinvent.com/HUD-HMD-benchmark#benchmarkTable Clear-Vu Reflective Several reflective elements Thinner light guide Large FOV, eye-box Reflective Simple reflective elements Lower cost Size is function of FOV
  81. 81. Comparison Chart

×