0
COSC 426: Augmented Reality            Mark Billinghurst      mark.billinghurst@hitlabnz.org             Sept 19th 2012   ...
Looking to the Future
The Future is with usIt takes at least 20 years for new      technologies to go from the lab to the      lounge..“The tech...
Research Directions  experiences                 Usability  applications   Interaction      tools      Authoring  componen...
Research Directions  Components    Markerless tracking, hybrid tracking    Displays, input devices  Tools    Authorin...
HMD Design
Occlusion with See-through HMD  The Problem      Occluding real objects with virtual      Occluding virtual objects wit...
ELMO (Kiyokawa 2001)  Occlusive see-through HMD     Masking LCD     Real time range finding
ELMO Demo
ELMO Design                             Virtual images                             from LCD        Depth        Sensing   ...
ELMO Results
Future Displays  Always on, unobtrusive
Google Glasses
Contact Lens Display  Babak Parviz    University Washington  MEMS components    Transparent elements    Micro-sensors...
Contact Lens Prototype
Applications
Interaction Techniques  Input techniques      3D vs. 2D input      Pen/buttons/gestures  Natural Interaction     Spee...
Flexible Displays  Flexible Lens Surface     Bimanual interaction     Digital paper analogy                    Red Plan...
Sony CSL © 2004
Sony CSL © 2004
Tangible User Interfaces (TUIs)  GUMMI bendable display prototype    Reproduced by permission of Sony CSL
Sony CSL © 2004
Sony CSL © 2004
Lucid Touch  Microsoft Research & Mitsubishi Electric Research Labs  Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J...
Auditory Modalities  Auditory     auditory icons     earcons     speech synthesis/recognition     Nomadic Radio (Sawh...
Gestural interfaces  1. Micro-gestures     (unistroke, smartPad)  2. Device-based gestures     (tilt based examples) ...
Natural Gesture Interaction on Mobile  Use mobile camera for hand tracking    Fingertip detection
Evaluation  Gesture input more than twice as slow as touch  No difference in naturalness
Haptic Modalities      Haptic interfaces            Simple uses in mobiles? (vibration instead of ringtone)            ...
Haptic Input  AR Haptic Workbench    CSIRO 2003 – Adcock et. al.
AR Haptic Interface  Phantom, ARToolKit, Magellan
Natural Interaction
The Vision of AR
To Make the Vision Real..  Hardware/software requirements    Contact lens displays    Free space hand/body tracking   ...
Natural Interaction  Automatically detecting real environment    Environmental awareness    Physically based interactio...
Environmental Awareness
AR MicroMachines  AR experience with environment awareness   and physically-based interaction    Based on MS Kinect RGB-...
Operating Environment
Architecture  Our framework uses five libraries:    OpenNI    OpenCV    OPIRA    Bullet Physics    OpenSceneGraph
System Flow  The system flow consists of three sections:     Image Processing and Marker Tracking     Physics Simulatio...
Physics Simulation  Create virtual mesh over real world  Update at 10 fps – can move real objects  Use by physics engin...
RenderingOcclusion               Shadows
Natural Gesture Interaction
Mo#va#on	                                    AR	  MicroMachines	  and	  PhobiAR	   • 	  	  Treated	  the	  environment	  a...
Mo#va#on	                                                   Occlusion	  Issues	  AR	  MicroMachines	  only	  achieved	  re...
HITLabNZ’s Gesture Library   Architecture                  5. Gesture•  Static Gestures•  Dynamic Gestures•  Context based...
HITLabNZ’s Gesture Library                               Architecture         5. Gesture                     o  Supports P...
HITLabNZ’s Gesture Library                               Architecture         5. Gesture                 o  Segment images...
HITLabNZ’s Gesture Library                               Architecture         5. Gesture                     o  Identify a...
HITLabNZ’s Gesture Library                               Architecture         5. Gesture                     o  Hand Recog...
Method	  Represent	  models	  as	  collec#ons	  of	  spheres	  moving	  with	  the	     models	  in	  the	  Bullet	  physi...
Method	  Render	  AR	  scene	  with	  OpenSceneGraph,	  using	  depth	  map	     for	  occlusion	                       Sh...
Results
HITLabNZ’s Gesture Library                               Architecture         5. Gesture                    o  Static (han...
Multimodal Interaction
Multimodal Interaction  Combined speech input  Gesture and Speech complimentary    Speech     -  modal commands, quanti...
1. Marker Based Multimodal Interface  Add speech recognition to VOMAR  Paddle + speech commands
Commands Recognized  Create Command "Make a blue chair": to create a virtual   object and place it on the paddle.  Dupli...
System Architecture
Object Relationships"Put chair behind the table”Where is behind?                               View specific regions
User Evaluation  Performance time     Speech + static paddle significantly faster  Gesture-only condition less accurate...
Subjective Surveys
2. Free Hand Multimodal Input  Use free hand to interact with AR content  Recognize simple gestures  No marker tracking...
Multimodal Architecture
Multimodal Fusion
Hand Occlusion
User Evaluation  Change object shape, colour and position  Conditions    Speech only, gesture only, multimodal  Measur...
Experimental SetupChange object shape  and colour
Results  Average performance time (MMI, speech fastest)     Gesture: 15.44s     Speech: 12.38s     Multimodal: 11.78s...
Intelligent Interfaces
Intelligent Interfaces  Most AR systems stupid    Don’t recognize user behaviour    Don’t provide feedback    Don’t ad...
Intelligent Interfaces  AR interface + intelligent tutoring system    ASPIRE constraint based system (from UC)    Const...
Domain Ontology
Intelligent Feedback  Actively monitors user behaviour     Implicit vs. explicit interaction  Provides corrective feedb...
Evaluation Results  16 subjects, with and without ITS  Improved task completion  Improved learning
Intelligent Agents  AR characters    Virtual embodiment of system    Multimodal input/output  Examples    AR Lego, We...
Context Sensing
Context Sensing  TKK Project  Using context to   manage information  Context from    Speech    Gaze    Real world  ...
Gaze Interaction
AR View
More Information Over Time
Experiences
Novel Experiences  Crossing Boundaries     Ubiquitous VR/AR  Collaborative Experiences  Massive AR     AR + Social Ne...
Crossing Boundaries           Jun Rekimoto, Sony CSL
Invisible Interfaces            Jun Rekimoto, Sony CSL
Milgram’s Reality-Virtuality continuum                       Mixed Reality   Real        Augmented           Augmented    ...
The MagicBookReality   Augmented      Augmented         Virtuality          Reality (AR)   Virtuality (AV)
Invisible Interfaces            Jun Rekimoto, Sony CSL
Example: Visualizing Sensor Networks  Rauhala et. al. 2007 (Linkoping)  Network of Humidity Sensors    ZigBee wireless ...
Invisible Interfaces            Jun Rekimoto, Sony CSL
UbiVR – CAMAR                           CAMAR Controller          CAMAR Viewer                         CAMAR CompanionGIST...
ubiHome @ GIST        Media services       Light service        MR window    ubiTrack Where/When     Tag-it               ...
CAMAR - GIST (CAMAR: Context-Aware Mobile  Augmented Reality)
  UCAM: Architecture                           wear-UCAM                          Content    Sensor                      ...
Hybrid User InerfacesGoal: To incorporate AR into normal meeting environment  Physical Components      Real props  Disp...
Hybrid User Interfaces    1                   2                 3                   4   PERSONAL         TABLETOP        W...
Ubiquitous                          UbiComp                                      Ubi AR                                   ...
MassiveMulti User                                     Ubiquitous                                 r                        ...
Remote Collaboration
AR Client  HMD and HHD    Showing virtual images over real world    Images drawn by remote expert    Local interaction
Shared Visual Context (Fussell ,1999)  Remote video collaboration    Shared manual, video viewing    Compared Video, Au...
WACL(Kurata,2004)  Wearable Camera/Laser Pointer    Independent pointer control    Remote panorama view
WACL(Kurata,2004)  Remote Expert View    Panorama viewing, annotation, image capture
As If Being There (Poelman, 2012)  AR + Scene Capture    HMD viewing, remote expert    Gesture input    Scene capture ...
As If Being There (Poelman, 2012)  Gesture Interaction    Hand postures recognized    Menu superimposed on hands
Real World Capture  Using Kinect for 3D Scene Capture    Camera tracking    AR overlay    Remote situational awareness
Remote scene capture with AR annotations added
Future Directions             SLIDE 116                   Massive Multiuser  Handheld AR for the first time allows extrem...
Massive MultiUser  2D Applications     MSN – 29 million     Skype – 10 million     Facebook – 100m+  3D/VR Applicatio...
BASIC VIEW
PERSONAL VIEW
Augmented Reality 2.0 Infrastructure
Leveraging Web 2.0  Content retrieval using HTTP  XML encoded meta information     KML placemarks + extensions  Querie...
Content  Content creation and delivery    Content creation pipeline    Delivering previously unknown content  Streamin...
ARML (AR Markup Language)
Scaling Up  AR on a City Scale  Using mobile phone as ubiquitous sensor  MIT Senseable City Lab    http://senseable.mi...
WikiCity Rome (Senseable City Lab MIT)
Conclusions
AR Research in the HIT Lab NZ  Gesture interaction     Gesture library  Multimodal interaction     Collaborative speec...
More Information•  Mark Billinghurst  –  mark.billinghurst@hitlabnz.org•  Websites  –  http://www.hitlabnz.org/  –  http:/...
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
Upcoming SlideShare
Loading in...5
×

426 Lecture 9: Research Directions in AR

3,997

Published on

The final lecture in the COSC 426 graduate course in Augmented Reality. Taught by Mark Billinghurst from the HIT Lab NZ at the University of Canterbury on Sept. 19th 2012

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,997
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
281
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "426 Lecture 9: Research Directions in AR"

  1. 1. COSC 426: Augmented Reality Mark Billinghurst mark.billinghurst@hitlabnz.org Sept 19th 2012 Lecture 9: AR Research Directions
  2. 2. Looking to the Future
  3. 3. The Future is with usIt takes at least 20 years for new technologies to go from the lab to the lounge..“The technologies that will significantly affect our lives over the next 10 years have been around for a decade.The future is with us.The trick is learning how to spot it.The commercialization of research, in other words, is far more about Oct 11th 2004 prospecting than alchemy.” Bill Buxton
  4. 4. Research Directions experiences Usability applications Interaction tools Authoring components Tracking, Display Sony CSL © 2004
  5. 5. Research Directions  Components   Markerless tracking, hybrid tracking   Displays, input devices  Tools   Authoring tools, user generated content  Applications   Interaction techniques/metaphors  Experiences   User evaluation, novel AR/MR experiences
  6. 6. HMD Design
  7. 7. Occlusion with See-through HMD  The Problem   Occluding real objects with virtual   Occluding virtual objects with real Real Scene Current See-through HMD
  8. 8. ELMO (Kiyokawa 2001)  Occlusive see-through HMD   Masking LCD   Real time range finding
  9. 9. ELMO Demo
  10. 10. ELMO Design Virtual images from LCD Depth Sensing LCD MaskRealWorld Optical Combiner   Use LCD mask to block real world   Depth sensing for occluding virtual images
  11. 11. ELMO Results
  12. 12. Future Displays  Always on, unobtrusive
  13. 13. Google Glasses
  14. 14. Contact Lens Display  Babak Parviz   University Washington  MEMS components   Transparent elements   Micro-sensors  Challenges   Miniaturization   Assembly   Eye-safe
  15. 15. Contact Lens Prototype
  16. 16. Applications
  17. 17. Interaction Techniques  Input techniques   3D vs. 2D input   Pen/buttons/gestures  Natural Interaction   Speech + gesture input  Intelligent Interfaces   Artificial agents   Context sensing
  18. 18. Flexible Displays  Flexible Lens Surface   Bimanual interaction   Digital paper analogy Red Planet, 2000
  19. 19. Sony CSL © 2004
  20. 20. Sony CSL © 2004
  21. 21. Tangible User Interfaces (TUIs)  GUMMI bendable display prototype  Reproduced by permission of Sony CSL
  22. 22. Sony CSL © 2004
  23. 23. Sony CSL © 2004
  24. 24. Lucid Touch  Microsoft Research & Mitsubishi Electric Research Labs  Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., Shen, C. LucidTouch: A See-Through Mobile Device In Proceedings of UIST 2007, Newport, Rhode Island, October 7-10, 2007, pp. 269–278.
  25. 25. Auditory Modalities  Auditory   auditory icons   earcons   speech synthesis/recognition   Nomadic Radio (Sawhney) -  combines spatialized audio -  auditory cues -  speech synthesis/recognition
  26. 26. Gestural interfaces  1. Micro-gestures   (unistroke, smartPad)  2. Device-based gestures   (tilt based examples)  3. Embodied interaction   (eye toy)
  27. 27. Natural Gesture Interaction on Mobile  Use mobile camera for hand tracking   Fingertip detection
  28. 28. Evaluation  Gesture input more than twice as slow as touch  No difference in naturalness
  29. 29. Haptic Modalities   Haptic interfaces   Simple uses in mobiles? (vibration instead of ringtone)   Sony’s Touchengine -  physiological experiments show you can perceive two stimulus 5ms apart, and spaced as low as 0.2 microns 4 µm n層 28 µm n層 V
  30. 30. Haptic Input  AR Haptic Workbench   CSIRO 2003 – Adcock et. al.
  31. 31. AR Haptic Interface  Phantom, ARToolKit, Magellan
  32. 32. Natural Interaction
  33. 33. The Vision of AR
  34. 34. To Make the Vision Real..  Hardware/software requirements   Contact lens displays   Free space hand/body tracking   Environment recognition   Speech/gesture recognition   Etc..
  35. 35. Natural Interaction  Automatically detecting real environment   Environmental awareness   Physically based interaction  Gesture Input   Free-hand interaction  Multimodal Input   Speech and gesture interaction   Implicit rather than Explicit interaction
  36. 36. Environmental Awareness
  37. 37. AR MicroMachines  AR experience with environment awareness and physically-based interaction   Based on MS Kinect RGB-D sensor  Augmented environment supports   occlusion, shadows   physically-based interaction between real and virtual objects
  38. 38. Operating Environment
  39. 39. Architecture  Our framework uses five libraries:   OpenNI   OpenCV   OPIRA   Bullet Physics   OpenSceneGraph
  40. 40. System Flow  The system flow consists of three sections:   Image Processing and Marker Tracking   Physics Simulation   Rendering
  41. 41. Physics Simulation  Create virtual mesh over real world  Update at 10 fps – can move real objects  Use by physics engine for collision detection (virtual/real)  Use by OpenScenegraph for occlusion and shadows
  42. 42. RenderingOcclusion Shadows
  43. 43. Natural Gesture Interaction
  44. 44. Mo#va#on   AR  MicroMachines  and  PhobiAR   •     Treated  the  environment  as            sta/c  –  no  tracking   •     Tracked  objects  in  2D  More  realis#c  interac#on  requires  3D  gesture  tracking      
  45. 45. Mo#va#on   Occlusion  Issues  AR  MicroMachines  only  achieved  realis/c  occlusion  because  the  user’s  viewpoint  matched  the  Kinect’s  Proper  occlusion  requires  a  more  complete  model  of  scene  objects  
  46. 46. HITLabNZ’s Gesture Library Architecture 5. Gesture•  Static Gestures•  Dynamic Gestures•  Context based Gestures 4. Modeling•  Hand recognition/modeling•  Rigid-body modeling 3. Classification/Tracking 2. Segmentation 1. Hardware Interface
  47. 47. HITLabNZ’s Gesture Library Architecture 5. Gesture o  Supports PCL, OpenNI, OpenCV, and Kinect SDK. o  Provides access to depth, RGB, XYZRGB.•  Static Gestures o  Usage: Capturing color image, depth image and concatenated•  Dynamic Gestures point clouds from a single or multiple cameras o  For example:•  Context based Gestures 4. Modeling•  Hand recognition/ modeling Kinect for Xbox 360•  Rigid-body modeling 3. Classification/Tracking Kinect for Windows 2. Segmentation Asus Xtion Pro Live 1. Hardware Interface
  48. 48. HITLabNZ’s Gesture Library Architecture 5. Gesture o  Segment images and point clouds based on color, depth and space.•  Static Gestures o  Usage: Segmenting images or point clouds using color•  Dynamic Gestures models, depth, or spatial properties such as location, shape and size.•  Context based Gestures o  For example: 4. Modeling•  Hand recognition/ modeling•  Rigid-body modeling Skin color segmentation 3. Classification/Tracking 2. Segmentation Depth threshold 1. Hardware Interface
  49. 49. HITLabNZ’s Gesture Library Architecture 5. Gesture o  Identify and track objects between frames based on XYZRGB.•  Static Gestures o  Usage: Identifying current position/orientation of the tracked•  Dynamic Gestures object in space.•  Context based Gestures o  For example: 4. Modeling•  Hand recognition/ Training set of hand modeling poses, colors•  Rigid-body modeling represent unique regions of the hand. 3. Classification/Tracking 2. Segmentation Raw output (without- cleaning) classified on real hand input 1. Hardware Interface (depth image).
  50. 50. HITLabNZ’s Gesture Library Architecture 5. Gesture o  Hand Recognition/Modeling   Skeleton based (for low resolution•  Static Gestures approximation)•  Dynamic Gestures   Model based (for more accurate•  Context based Gestures representation) o  Object Modeling (identification and tracking rigid- 4. Modeling body objects)•  Hand recognition/ o  Physical Modeling (physical interaction) modeling   Sphere Proxy•  Rigid-body modeling   Model based   Mesh based 3. Classification/Tracking o  Usage: For general spatial interaction in AR/VR environment 2. Segmentation 1. Hardware Interface
  51. 51. Method  Represent  models  as  collec#ons  of  spheres  moving  with  the   models  in  the  Bullet  physics  engine  
  52. 52. Method  Render  AR  scene  with  OpenSceneGraph,  using  depth  map   for  occlusion   Shadows  yet  to  be  implemented  
  53. 53. Results
  54. 54. HITLabNZ’s Gesture Library Architecture 5. Gesture o  Static (hand pose recognition) o  Dynamic (meaningful movement recognition)•  Static Gestures o  Context-based gesture recognition (gestures with context,•  Dynamic Gestures e.g. pointing) o  Usage: Issuing commands/anticipating user intention and high•  Context based Gestures level interaction. 4. Modeling•  Hand recognition/ modeling•  Rigid-body modeling 3. Classification/Tracking 2. Segmentation 1. Hardware Interface
  55. 55. Multimodal Interaction
  56. 56. Multimodal Interaction  Combined speech input  Gesture and Speech complimentary   Speech -  modal commands, quantities   Gesture -  selection, motion, qualities  Previous work found multimodal interfaces intuitive for 2D/3D graphics interaction
  57. 57. 1. Marker Based Multimodal Interface  Add speech recognition to VOMAR  Paddle + speech commands
  58. 58. Commands Recognized  Create Command "Make a blue chair": to create a virtual object and place it on the paddle.  Duplicate Command "Copy this": to duplicate a virtual object and place it on the paddle.  Grab Command "Grab table": to select a virtual object and place it on the paddle.  Place Command "Place here": to place the attached object in the workspace.  Move Command "Move the couch": to attach a virtual object in the workspace to the paddle so that it follows the paddle movement.
  59. 59. System Architecture
  60. 60. Object Relationships"Put chair behind the table”Where is behind? View specific regions
  61. 61. User Evaluation  Performance time   Speech + static paddle significantly faster  Gesture-only condition less accurate for position/orientation  Users preferred speech + paddle input
  62. 62. Subjective Surveys
  63. 63. 2. Free Hand Multimodal Input  Use free hand to interact with AR content  Recognize simple gestures  No marker tracking Point Move Pick/Drop
  64. 64. Multimodal Architecture
  65. 65. Multimodal Fusion
  66. 66. Hand Occlusion
  67. 67. User Evaluation  Change object shape, colour and position  Conditions   Speech only, gesture only, multimodal  Measure   performance time, error, subjective survey
  68. 68. Experimental SetupChange object shape and colour
  69. 69. Results  Average performance time (MMI, speech fastest)   Gesture: 15.44s   Speech: 12.38s   Multimodal: 11.78s  No difference in user errors  User subjective survey   Q1: How natural was it to manipulate the object? -  MMI, speech significantly better   70% preferred MMI, 25% speech only, 5% gesture only
  70. 70. Intelligent Interfaces
  71. 71. Intelligent Interfaces  Most AR systems stupid   Don’t recognize user behaviour   Don’t provide feedback   Don’t adapt to user  Especially important for training   Scaffolded learning   Moving beyond check-lists of actions
  72. 72. Intelligent Interfaces  AR interface + intelligent tutoring system   ASPIRE constraint based system (from UC)   Constraints -  relevance cond., satisfaction cond., feedback
  73. 73. Domain Ontology
  74. 74. Intelligent Feedback  Actively monitors user behaviour   Implicit vs. explicit interaction  Provides corrective feedback
  75. 75. Evaluation Results  16 subjects, with and without ITS  Improved task completion  Improved learning
  76. 76. Intelligent Agents  AR characters   Virtual embodiment of system   Multimodal input/output  Examples   AR Lego, Welbo, etc   Mr Virtuoso -  AR character more real, more fun -  On-screen 3D and AR similar in usefulness
  77. 77. Context Sensing
  78. 78. Context Sensing  TKK Project  Using context to manage information  Context from   Speech   Gaze   Real world  AR Display
  79. 79. Gaze Interaction
  80. 80. AR View
  81. 81. More Information Over Time
  82. 82. Experiences
  83. 83. Novel Experiences  Crossing Boundaries   Ubiquitous VR/AR  Collaborative Experiences  Massive AR   AR + Social Networking  Usability
  84. 84. Crossing Boundaries Jun Rekimoto, Sony CSL
  85. 85. Invisible Interfaces Jun Rekimoto, Sony CSL
  86. 86. Milgram’s Reality-Virtuality continuum Mixed Reality Real Augmented Augmented VirtualEnvironment Reality (AR) Virtuality (AV) Environment Reality - Virtuality (RV) Continuum
  87. 87. The MagicBookReality Augmented Augmented Virtuality Reality (AR) Virtuality (AV)
  88. 88. Invisible Interfaces Jun Rekimoto, Sony CSL
  89. 89. Example: Visualizing Sensor Networks  Rauhala et. al. 2007 (Linkoping)  Network of Humidity Sensors   ZigBee wireless communication  Use Mobile AR to Visualize Humidity
  90. 90. Invisible Interfaces Jun Rekimoto, Sony CSL
  91. 91. UbiVR – CAMAR CAMAR Controller CAMAR Viewer CAMAR CompanionGIST - Korea
  92. 92. ubiHome @ GIST Media services Light service MR window ubiTrack Where/When Tag-it ubiKey ©ubiHome Who/What/What/When/How When/How PDA Couch Sensor Door Sensor Who/What/When/How When/How When/How
  93. 93. CAMAR - GIST (CAMAR: Context-Aware Mobile Augmented Reality)
  94. 94.   UCAM: Architecture wear-UCAM Content Sensor Service (Integrator,Manager, Interpreter,ServiceProvider) Context Interface Network Interface ubi-UCAM BAN/PAN TCP/IP (BT) (Discovery,Control,Event) Operating System vr-UCAM
  95. 95. Hybrid User InerfacesGoal: To incorporate AR into normal meeting environment  Physical Components   Real props  Display Elements   2D and 3D (AR) displays  Interaction Metaphor   Use multiple tools – each relevant for the task
  96. 96. Hybrid User Interfaces 1 2 3 4 PERSONAL TABLETOP WHITEBOARD MULTIGROUPPrivate Display Private Display Private Display Private Display Group Display Public Display Group Display Public Display
  97. 97. Ubiquitous UbiComp Ubi AR Ubi VRWeiser Mobile AR Desktop AR VR Terminal Reality Virtual Reality Milgram From: Joe Newman
  98. 98. MassiveMulti User Ubiquitous r Weise TerminalSingle User Reality Milg ram VR
  99. 99. Remote Collaboration
  100. 100. AR Client  HMD and HHD   Showing virtual images over real world   Images drawn by remote expert   Local interaction
  101. 101. Shared Visual Context (Fussell ,1999)  Remote video collaboration   Shared manual, video viewing   Compared Video, Audio, Side-by-side collaboration   Communication analysis
  102. 102. WACL(Kurata,2004)  Wearable Camera/Laser Pointer   Independent pointer control   Remote panorama view
  103. 103. WACL(Kurata,2004)  Remote Expert View   Panorama viewing, annotation, image capture
  104. 104. As If Being There (Poelman, 2012)  AR + Scene Capture   HMD viewing, remote expert   Gesture input   Scene capture (PTAM), stereo camera
  105. 105. As If Being There (Poelman, 2012)  Gesture Interaction   Hand postures recognized   Menu superimposed on hands
  106. 106. Real World Capture  Using Kinect for 3D Scene Capture   Camera tracking   AR overlay   Remote situational awareness
  107. 107. Remote scene capture with AR annotations added
  108. 108. Future Directions SLIDE 116 Massive Multiuser  Handheld AR for the first time allows extremely high numbers of AR users  Requires   New types of applications/games   New infrastructure (server/client/peer-to-peer)   Content distribution…
  109. 109. Massive MultiUser  2D Applications   MSN – 29 million   Skype – 10 million   Facebook – 100m+  3D/VR Applications   SecondLife > 50K   Stereo projection - <500  Augmented Reality   Shared Space (1999) - 4   Invisible Train (2004) - 8
  110. 110. BASIC VIEW
  111. 111. PERSONAL VIEW
  112. 112. Augmented Reality 2.0 Infrastructure
  113. 113. Leveraging Web 2.0  Content retrieval using HTTP  XML encoded meta information   KML placemarks + extensions  Queries   Based on location (from GPS, image recognition)   Based on situation (barcode markers)  Queries also deliver tracking feature databases  Everybody can set up an AR 2.0 server  Syndication:   Community servers for end-user content   Tagging  AR client subscribes to arbitrary number of feeds
  114. 114. Content  Content creation and delivery   Content creation pipeline   Delivering previously unknown content  Streaming of   Data (objects, multi-media)   Applications  Distribution   How do users learn about all that content?   How do they access it?
  115. 115. ARML (AR Markup Language)
  116. 116. Scaling Up  AR on a City Scale  Using mobile phone as ubiquitous sensor  MIT Senseable City Lab   http://senseable.mit.edu/
  117. 117. WikiCity Rome (Senseable City Lab MIT)
  118. 118. Conclusions
  119. 119. AR Research in the HIT Lab NZ  Gesture interaction   Gesture library  Multimodal interaction   Collaborative speech/gesture interfaces  Mobile AR interfaces   Outdoor AR, interaction methods, navigation tools  AR authoring tools   Visual programming for AR  Remote Collaboration   Mobile AR for remote interaction
  120. 120. More Information•  Mark Billinghurst –  mark.billinghurst@hitlabnz.org•  Websites –  http://www.hitlabnz.org/ –  http://artoolkit.sourceforge.net/ –  http://www.osgart.org/ –  http://www.hitlabnz.org/wiki/buildAR/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×