2013 426 Lecture 2:  Augmented Reality Technology
Upcoming SlideShare
Loading in...5

2013 426 Lecture 2: Augmented Reality Technology



The second lecture in the 426 graduate class on Augmented Reality taught thy Mark Billinghurst at the HIT Lab NZ, University of Canterbury. The class was taught on July 19th 2013

The second lecture in the 426 graduate class on Augmented Reality taught thy Mark Billinghurst at the HIT Lab NZ, University of Canterbury. The class was taught on July 19th 2013



Total Views
Views on SlideShare
Embed Views



1 Embed 1

https://twitter.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

2013 426 Lecture 2:  Augmented Reality Technology 2013 426 Lecture 2: Augmented Reality Technology Presentation Transcript

  • COSC 426: Augmented Reality Mark Billinghurst mark.billinghurst@hitlabnz.org July 19th 2013 Lecture 2: AR Technology
  • Key Points from Lecture 1
  • Augmented Reality Definition   Defining Characteristics [Azuma 97]   Combines Real and Virtual Images -  Both can be seen at the same time   Interactive in real-time -  Virtual content can be interacted with   Registered in 3D -  Virtual objects appear fixed in space
  • What is not Augmented Reality?   Location-based services   Barcode detection (QR-codes)   Augmenting still images   Special effects in movies   …   … but they can be combined with AR!
  • Milgram’s Reality-Virtuality Continuum Mixed Reality Reality - Virtuality (RV) Continuum Real Environment Augmented Reality (AR) Augmented Virtuality (AV) Virtual Environment
  • Metaverse
  • AR History Summary   1960’s – 80’s: Early Experimentation   1980’s – 90’s: Basic Research   Tracking, displays   1995 – 2005: Tools/Applications   Interaction, usability, theory   2005 - : Commercial Applications   Games, Medical, Industry
  • Applications   Medicine   Manufacturing   Information overlay   Architecture   Museum   Marketing   Gaming
  • AR Experience Design
  • “The product is no longer the basis of value.The experience is.” Venkat Ramaswamy The Future of Competition.
  • experiences services products components Value Sony CSL © 2004 Gilmore + Pine: Experience Economy Function Emotion
  • The Value of Good User Experience 20c 50c $3.50
  • Good Experience Design   Reactrix   Top down projection   Camera based input   Reactive Graphics   No instructions   No training
  • Apple: The Value of Good Design   Good Experience Design Dominates Markets iPod Sales 2002-2007
  • Using the N-gage
  • SideTalking   http://www.sidetalkin.com
  • Interaction Design “Designing interactive products to support people in their everyday and working lives” Preece, J., (2002). Interaction Design   Design of User Experience with Technology   Higher in the value chain than product design
  •   Interaction Design involves answering three questions:   What do you do? - How do you affect the world?   What do you feel? – What do you sense of the world?   What do you know? – What do you learn?
  • Interaction Design is All About You   Users should be involved throughout the Design Process   Consider all the needs of the user
  • Interaction Design Process
  • Gabbard Model for AR Design 1. user task analysis 2. expert guidelines-based evaluation 3. formative user-centered evaluation 4. summative comparative evaluations Gabbard, J.L.; Swan, J.E.; , "Usability Engineering for Augmented Reality: Employing User-Based Studies to Inform Design,” Visualization and Computer Graphics, IEEE Transactions on, vol.14, no.3, pp.513-525, May- June 2008
  • Gabbard Model in Context
  • experiences applications tools components Building Compelling AR Experiences Tracking, Display Authoring Interaction Usability
  • Summary   In order to build AR applications you need to focus on the user experience   Great user experience is based on   Low level AR component technology   Authoring tools   Application/Interaction design   User experience texting
  • AR Technology
  • experiences applications tools components Sony CSL © 2004 Building Compelling AR Experiences Display, Tracking
  • Core Technologies  Combining Real and Virtual Images •  Display technologies  Interactive in Real-Time •  Input and interactive technologies  Registered in 3D •  Viewpoint tracking technologies Display Processing Input Tracking
  • AR Displays
  • AR Displays e.g. window reflections Virtual Images seen off windows e.g. Reach-In Projection CRT Display using beamsplitter Not Head-Mounted e.g. Shared Space Magic Book Liquid Crystal Displays LCDs Head-Mounted Display (HMD) Primarily Indoor Environments e.g. WLVA and IVRD Cathode Ray Tube (CRT) or Virtual Retinal Display (VRD) Many Military Applications & Assistive Technologies Head-Mounted Display (HMD) e.g. Head-Up Display (HUD) Projection Display Navigational Aids in Cars Military Airborne Applications Not Head Mounted (e.g. vehicle mounted) Primarily Outdoor (Daylight) Environments AR Visual Displays
  • Display Technologies  Types (Bimber/Raskar 2003)  Head attached •  Head mounted display/projector  Body attached •  Handheld display/projector  Spatial •  Spatially aligned projector/monitor
  • Display Taxonomy
  • Head Mounted Displays
  • Head Mounted Displays (HMD) -  Display and Optics mounted on Head -  May or may not fully occlude real world -  Provide full-color images -  Considerations •  Cumbersome to wear •  Brightness •  Low power consumption •  Resolution limited •  Cost is high?
  • Key Properties of HMD   Field of View   Human eye 95 degrees horizontal, 60/70 degrees vertical   Resolution   > 320x240 pixel   Refresh Rate   Focus   Fixed/manual   Power   Size
  • Types of Head Mounted Displays The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The ima ge can not be dis play ed. You r co mp uter Occluded See-thru Multiplexed
  • Immersive VR Architecture Head! Tracker Host! Processor Data Base! Model Rendering! Engine Frame! Buffer head position/orientation to network Display! Driver Non see-thru! Image source & optics virtual object Virtual World
  • See-thru AR Architecture Head! Tracker Host! Processor Data Base! Model Rendering! Engine Frame! Buffer head position/orientation to network Display! Driver see-thru! combiner Virtual Image superimposed! over real world object real world Image source
  • Optical see-through head-mounted display Virtual images from monitors Real World Optical Combiners
  • Optical See-Through HMD
  • Epson Moverio Display   Stereo see-through display   960 x 540 pixels, 23 degree Field of View, 60 Hz   $700 USD   Android Powered
  • View Through Optical See-Through HMD
  • DigiLens   www.digilens.com   Compact HOE   Solid state optics   Switchable Bragg Grating   Stacked SBG   Fast switching   Ultra compact
  • The Virtual Retinal Display   Image scanned onto retina   Commercialized through Microvision   Nomad System - www.mvis.com
  • Strengths of optical see-through AR   Simpler (cheaper)   Direct view of real world   Full resolution, no time delay (for real world)   Safety   Lower distortion   No eye displacement   but COASTAR video see-through avoids this
  • Video AR Architecture Head! Tracker Host! Processor Graphics! renderer Digital! Mixer Frame! Buffer head position/orientation to network Display! Driver Non see-thru! Image source & optics Head-mounted camera aligned to display optics Video! Processor Video image of real world Virtual image inset into video of real world
  • Video see-through HMD Video cameras Monitors Graphics Combiner Video
  • Video See-Through HMD
  • Video see-through HMD MR Laboratory’s COASTAR HMD (Co-Optical Axis See-Through Augmented Reality) Parallax-free video see-through HMD
  • TriVisio   www.trivisio.com   Stereo video input   PAL resolution cameras   2 x SVGA displays   30 degree FOV   User adjustable convergence   $6,000 USD
  • View Through a Video See-Through HMD
  • Vuzix Display   www.vuzix.com   Wrap 920   $350 USD   Twin 640 x 480 LCD displays   31 degree diagonal field of view   Weighs less than three ounces
  • Strengths of Video See-Through AR   True occlusion   Kiyokawa optical display that supports occlusion   Digitized image of real world   Flexibility in composition   Matchable time delays   More registration, calibration strategies   Wide FOV is easier to support
  • Optical vs. Video AR Summary   Both have proponents   Video is more popular today?   Likely because lack of available optical products   Depends on application?   Manufacturing: optical is cheaper   Medical: video for calibration strategies
  • Eye multiplexed AR Architecture Head! Tracker Host! Processor Data Base! Model Rendering! Engine Frame! Buffer head position/orientation to network Display! Driver Virtual Image inset into! real world scene real world Opaque! Image source
  • Virtual Image ‘inset’ into real
  • Virtual Vision Personal Eyewear
  • Virtual image inset into real world
  • Google Glass
  • View Through Google Glass
  • What's Inside Google Glass?
  • Spatial/Projected AR
  • Spatial Augmented Reality   Project onto irregular surfaces   Geometric Registration   Projector blending, High dynamic range   Book: Bimber, Rasker “Spatial Augmented Reality”
  • Projector-based AR Examples: Raskar, MIT Media Lab Inami, Tachi Lab, U. Tokyo Projector Real objects with retroreflective covering User (possibly head-tracked)
  • Example of projector-based AR Ramesh Raskar, UNC, MERL
  • Example of projector-based AR Ramesh Raskar, UNC Chapel Hill
  • The I/O Bulb   Projector + Camera   John Underkoffler, Hiroshi Ishii   MIT Media Lab
  • Head Mounted Projector   Head Mounted Projector   Jannick Rolland (UCF)   Retro-reflective Material   Potentially portable
  • Head Mounted Projector   NVIS P-50 HMPD   1280x1024/eye   Stereoscopic   50 degree FOV   www.nvis.com
  • HMD vs. HMPD Head Mounted Display Head Mounted Projected Display
  • Pico Projectors   Microvision - www.mvis.com   3M, Samsung, Philips, etc
  • MIT Sixth Sense   Body worn camera and projector   http://www.pranavmistry.com/projects/sixthsense/
  • Other AR Displays
  • Video Monitor AR Video cameras Monitor Graphics Combiner Video Stereo glasses
  • Examples
  • Virtual Showcase   Mirrors on a projection table   Head tracked stereo   Up to 4 users   Merges graphic and real objects   Exhibit/museum applications   Fraunhofer Institute (2001)   Bimber, Frohlich
  • Augmented Paleontology Bimber et. al. IEEE Computer Sept. 2002
  • Alternate Displays LCD Panel Laptop PDA
  • Handheld Displays   Mobile Phones   Camera   Display   Input
  • Other Types of AR Display   Audio   spatial sound   ambient audio   Tactile   physical sensation   Haptic   virtual touch
  • Haptic Input   AR Haptic Workbench   CSIRO 2003 – Adcock et. al.
  • Phantom   Sensable Technologies (www.sensable.com)   6 DOF Force Feedback Device
  • AR Haptic Interface   Phantom, ARToolKit, Magellan