Augmented Reality: The Next 20 Years

1. AUGMENTED REALITY: THE NEXT 20 YEARS Mark Billinghurst mark.billinghurst@unisa.edu.au September 16th 2015

2. 1977 – StarWars

3. Augmented Reality 1.  Combines Real andVirtual Images • Both can be seen at the same time 2.  Interactive in real-time • The virtual content can be interacted with 3.  Registered in 3D • Virtual objects appear fixed in space Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.

4. 50 Years of Progress (1965-2015) • Moving from lab to living room • AR devices available in every pocket 1968: First AR HMD 1980’s: SuperCockpit 1997: Outdoor AR 2005: Mobile AR

5. Example: 1998 vs. 2008 CPU: 300 Mhz HDD; 9GB RAM: 512 mb Camera: VGA 30fps Graphics: 500K poly/sec 1998: SGI O2 2008: Nokia N95 CPU: 332 Mhz HDD; 8GB RAM: 128 mb Camera: VGA 30 fps Graphics: 2m poly/sec

6. AR in 2015 • Large growing market •  $600 Million USD in 2014 •  Many companies • Many available devices •  HMD, phones, tablets, HUDs • Robust developer tools •  Vuforia, ARToolKit, Unity, Wikitude, etc • Large number of applications •  > 200K developers, > 20K mobile apps • Strong research/business communities •  ISMAR, AWE conferences, AugmentedReality.org, etc

7. Looking to the Future What’s Next?

8. Key Enabling Technologies 1.  Combines Real andVirtual Images Display Technology 2.  Interactive in real-time Interaction Technologies 3.  Registered in 3D Tracking Technologies

9. DISPLAY

10. • Past •  Bulky Head mounted displays • Current •  Handheld, lightweight head mounted • Future •  Projected AR •  Wide FOV see through •  Retinal displays •  Contact lens Evolution in Displays

11. See-through thin displays •  Waveguide techniques for thin see-through displays •  Wider FOV, enable AR applications •  Social acceptability Opinvent Ora Lumus DK40

12. Projected AR (1-3 years) • Use stereo head mounted projectors • Rollable retro-reflective sheet •  Wide FOV, shared interaction • Eg CastAR (http://castar.com) •  $400 USD, available Q4 2015

13. Wide FOV See-Through (3+ years) • Waveguide techniques •  Wider FOV •  Thin see through •  Socially acceptable • Pinlight Displays •  LCD panel + point light sources •  110 degree FOV •  UNC/Nvidia Lumus DK40 Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., & Fuchs, H. (2014). Pinlight displays: wide field of view augmented reality eyeglasses using defocused point light sources. In ACM SIGGRAPH 2014 Emerging Technologies (p. 20). ACM.

14. Retinal Displays (5+ years) • Photons scanned into eye •  Infinite depth of field •  Bright outdoor performance •  Overcome visual defects •  True 3D stereo with depth modulation • Microvision (1993-) •  Head mounted monochrome • MagicLeap (2013-) •  Projecting light field into eye

15. Contact Lens (10 – 15 + years) • Contact Lens only •  Unobtrusive •  Significant technical challenges •  Power, data, resolution •  Babak Parviz (2008) • Contact Lens + Micro-display •  Wide FOV •  socially acceptable •  Innovega (innovega-inc.com) http://spectrum.ieee.org/biomedical/bionics/augmented-reality-in-a-contact-lens/

16. INTERACTION

17. Evolution of Interaction • Past •  Limited interaction •  Viewpoint manipulation • Present •  Screen based, simple gesture •  tangible interaction • Future •  Natural gesture, Multimodal •  Intelligent Interfaces •  Physiological/Sensor based

18. Natural Gesture (2-5 years) • Freehand gesture input •  Depth sensors for gesture capture •  Move beyond simple pointing •  Rich two handed gestures • Eg Microsoft Research Hand Tracker •  3D hand tracking, 30 fps, single sensor • Commercial Systems •  Meta, MS Hololens, Occulus, Intel, etc Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Leichter, D. K. C. R. I., ... & Izadi, S. (2015, April). Accurate, Robust, and Flexible Real-time Hand Tracking. In Proc. CHI (Vol. 8).

19. Multimodal Input (5+ years) • Combine gesture and speech input •  Gesture good for qualitative input •  Speech good for quantitative input •  Support combined commands •  “Put that there” + pointing • Eg HIT Lab NZ multimodal input •  3D hand tracking, speech •  Multimodal fusion module •  Complete tasks faster with MMI, less errors Billinghurst, M., Piumsomboon, T., & Bai, H. (2014). Hands in Space: Gesture Interaction with Augmented-Reality Interfaces. IEEE computer graphics and applications, (1), 77-80.

20. Intelligent Interfaces (10+ years) • Move to Implicit Input vs. Explicit •  Recognize user behaviour •  Provide adaptive feedback •  Support scaffolded learning •  Move beyond check-lists of actions • Eg AR + Intelligent Tutoring •  Constraint based ITS + AR •  PC Assembly (Westerfield (2015) •  30% faster, 25% better retention Westerfield, G., Mitrovic, A., & Billinghurst, M. (2015). Intelligent Augmented Reality Training for Motherboard Assembly. International Journal of Artificial Intelligence in Education, 25(1), 157-172.

21. TRACKING

22. Evolution of Tracking • Past •  Location based, marker based, •  magnetic/mechanical • Present •  Image based, hybrid tracking • Future •  Ubiquitous •  Model based •  Environmental

23. Model Based Tracking (1-3 yrs) • Track from known 3D model •  Use depth + colour information •  Match input to model template •  Use CAD model of targets • Recent innovations •  Learn models online •  Tracking from cluttered scene •  Track from deformable objects Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013). Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In Computer Vision–ACCV 2012 (pp. 548-562). Springer Berlin Heidelberg.

24. Environmental Tracking (3+ yrs) • Environment capture •  Use depth sensors to capture scene & track from model • InifinitAM (www.robots.ox.ac.uk/~victor/infinitam/) •  Real time scene capture on mobiles, dense or sparse capture •  Dynamic memory swapping allows large environment capture •  Cross platform, open source library available

25. Wide Area Outdoor Tracking (5+ yrs) • Process •  Combine panorama’s into point cloud model (offline) •  Initialize camera tracking from point cloud •  Update pose by aligning camera image to point cloud •  Accurate to 25 cm, 0.5 degree over very wide area Ventura, J., & Hollerer, T. (2012). Wide-area scene mapping for mobile visual tracking. In Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on (pp. 3-12). IEEE.

26. ENHANCED EXPERIENCE

27. AR Business Today • Around $600 Million USD in 2014 (>$2B 2015) • > 80% Games and Marketing applications

28. Market Projections cf. 2014 computer game market = $84 Billion USD

29. Crossing the Chasm - 5-10 years http://www.gartner.com/newsroom/id/3114217

30. Getting from Here to There • New markets •  Medical •  Education •  Industry •  Etc • New applications enabled •  Training •  Collaboration •  Information Presentation •  Etc

31. Example: Commercial Systems • Ngrain • http://www.ngrain.com/ • Training authoring tool • Model based AR tracking • ScopeAR • http://www.scopear.com/ • Remote assistance • Image based tracking

32. Example: Social Panoramas • Google Glass • Capture live image panorama (compass + camera) • Remote device (tablet) • Immersive viewing, live annotation Reichherzer, C., Nassani, A., & Billinghurst, M. (2014). Social panoramas using wearable computers. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on (pp. 303-304). IEEE.

33. Example: AR remote collaboration • Local user uses AR display •  Move real objects using AR cues • Remote expert on desktop interface •  Place 3D objects with independent view Tait, M., & Billinghurst, M. The Effect of View Independence in a Collaborative AR System. Computer Supported Cooperative Work (CSCW), 1-27.

34. Research Needed in Many Areas • Social Acceptance •  Overcome social problems with AR • Cloud Services •  Cloud based storage/processing • Ubiquitous AR •  Using AR with Ubicomp/IoT technologies • Collaborative Experiences •  AR teleconferencing • Etc..

35. SocialAcceptance • People don’t want to look silly •  Only 12% of 4,600 adults would be willing to wear AR glasses •  20% of mobile AR browser users experience social issues • Acceptance more due to Social than Technical issues •  Needs further study (ethnographic, field tests, longitudinal)

36. CONCLUSIONS

37. Conclusions • AR is becoming commonly available • In order to achieve significant growth AR needs to •  Expand into new markets •  Move onto new platforms •  Create new types of applications • New AR technologies will enable this to happen •  Display, Interaction, Tracking technologies • However there are still significant areas for research •  Social Acceptance, Cloud Services, Ubiquitous AR, Etc

38. www.empathiccomputing.org @marknb00 mark.billinghurst@unisa.edu.au

Augmented Reality: The Next 20 Years

You are reading a preview.

Check these out next

Augmented Reality: The Next 20 Years

More Related Content

Slideshows for you (20)

Similar to Augmented Reality: The Next 20 Years (20)

More from Mark Billinghurst (20)

Recently uploaded (20)

Augmented Reality: The Next 20 Years