Tangible AR Interface


Published on

HIT Lab.. NZ

1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Tangible AR Interface

  1. 1. Augmented Reality Interfaces for Visual Analytics Mark Billinghurst HIT Lab NZ University of Canterbury
  2. 2. We are here
  3. 3. What Year is This From ? 1982 2.77 Mhz
  4. 4. Back to Reality
  5. 5. Invisible Interfaces Jun Rekimoto, Sony CSL
  6. 6. Augmented Reality • Augmented Reality: Enhances Reality • Key Characteristics • Combines real and virtual images • Interactive in Real-Time • Registered in 3D
  7. 7. Medical Imaging Devices X-ray, US, CT, PET, MR, PET/CT, (Capsule) Endoscopy, Optical Imaging, Open MR, 4D US, …
  8. 8. Medical Imaging Data X-ray, Contract Agents, Image intensifier, US, CT, PET, MR, …
  9. 9. 3D imaging data is often displayed in 2D!
  10. 10. There are as many displays as imaging modalities!
  11. 11. Current CAS interfaces MediVisi on
  12. 12. AR Medical Trials Sauer et al. 2000 at Siemens Corporate Research, NJ Stereo video see through F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial, MICCAI 2002
  13. 13. AR History Summary 1960’s – 80’s: Early Experimentation 1980’s – 90’s: Basic Research Tracking, displays 1995 – 2005: Tools/Applications Interaction, usability, theory 2005 - : Commercial Applications Games, Medical, Industry
  14. 14. MIT Technology Review March 2007 list of the 10 most exciting technologies
  15. 15. Visual Analytics Prototype using Tangible AR Prototype developed by Nick Hedley, at SFU. Design based on empirical findings of several tangible AR studies compared with other conventional 3D visualizations Image courtesy of N. Hedley / SIRL (2008)
  16. 16. AR Interface Design
  17. 17. Interface Design Path 1/ Prototype Demonstration 2/ Adoption of Interaction Techniques from other interface metaphors Augmented Reality 3/ Development of new interface metaphors appropriate to the medium Virtual Reality 4/ Development of formal theoretical models for predicting and modeling user actions Desktop WIMP
  18. 18. AR Design Principles Interface Components Physical components Display elements - Visual/audio Interaction metaphors Physical Display Elements Interaction Elements Metaphor Input Output
  19. 19. AR User Interface Design Space Reality Virtual Reality Augmented Reality Physical Design Virtual Design NO STANDARD PRINCIPLES !! STANDARD PRINCIPLES STANDARD PRINCIPLES SOME CLUES
  20. 20. Tangible Interfaces (Ishii 97) Create digital shadows for physical objects Foreground graspable UI Background ambient interfaces
  21. 21. Tangible Interfaces Dangling String Jeremijenko 1995 Ambient ethernet monitor Relies on peripheral cues Ambient Fixtures Dahley, Wisneski, Ishii 1998 Use natural material qualities for information display
  22. 22. TerraVision (ART+COM 1994) Goal use virtual earth to visualize information Features seamless zooming tangible interface separation of physical + virtual
  23. 23. Lessons from Tangible Interfaces Physical objects make us smart Norman’s “Things that Make Us Smart” encode affordances, constraints Objects aid collaboration establish shared meaning Objects increase understanding serve as cognitive artifacts
  24. 24. Limitations Difficult to change object properties can’t tell state of digital data Limited display capabilities pinwheels = 1D, projection screen = 2D dependent on physical display surface Separation between object and display TerraVision
  25. 25. Back to the Real World AR overcomes limitation of TUIs enhance display possibilities merge task/display space provide public and private views TUI + AR = Tangible AR Apply TUI methods to AR interface design
  26. 26. Tangible AR Design Principles Tangible AR Interfaces use TUI principles Physical controllers for moving virtual content Support for spatial 3D interaction techniques Time and space multiplexed interaction Support for multi-handed interaction Match object affordances to task requirements Support parallel activity with multiple objects Allow collaboration between multiple users
  27. 27. Design of Objects Objects Purposely built – affordances “Found” – repurposed Existing – already at use in marketplace Make affordances obvious (Norman) Object affordances visible Give feedback Provide constraints Use natural mapping Use good cognitive model
  28. 28. Tangible AR: VOMAR Interface (Kato 2000) Use of natural physical object manipulations to control virtual objects VOMAR Demo Catalog book: - Turn over the page Paddle operation: - Push, shake, incline, hit, scoop
  29. 29. VOMAR Interface
  30. 30. Case Study: 3D AR Lens Goal: Develop a lens based AR interface MagicLenses Developed at Xerox PARC in 1993 View a region of the workspace differently to the rest Overlap MagicLenses to create composite effects
  31. 31. 3D MagicLenses MagicLenses extended to 3D (Veiga et. al. 96) Volumetric and flat lenses
  32. 32. AR Lens Design Principles Physical Components Lens handle - Virtual lens attached to real object Display Elements Lens view - Reveal layers in dataset Interaction Metaphor Physically holding lens
  33. 33. 3D AR Lenses: Model Viewer Displays models made up of multiple parts Each part can be shown or hidden through the lens Allows the user to peer inside the model Maintains focus + context
  34. 34. Examples Simple Wireframe Terrain layers Non-photorealistic rendering
  35. 35. Flexible Lens Red Planet, 2000 (5.2/10 IMDB) Flexible Lens Surface Bimanual interaction Digital paper analogy Looks expensive
  36. 36. Flexible Lens
  37. 37. Flexible Lens – Gesture Input Default Stretch Twist Bend Rotate
  38. 38. Milgram’s Continuum (1994) Mixed Reality (MR) Reality Augmented Augmented Virtuality (Tangible Reality (AR) Virtuality (AV) (Virtual Interfaces) Reality)
  39. 39. MagicBook Metaphor
  40. 40. Collaboration
  41. 41. Communication Cues A wide variety of communication cues used. Visual Audio Gaze Speech Gesture Paralinguistic Face Expression Paraverbals Body Position Prosodics Intonation Object Manipulation Environmental Writing/Drawing Spatial Relationship Object Presence
  42. 42. Collaboration in the Future ? Remote Conferencing Face to face Conferencing
  43. 43. Remote AR Conferencing Moves conferencing from the desktop to the workspace
  44. 44. Face-to-face collaboration People surround a table It is easy to see each other Communication Space Task Space Computer supported collaboration People sit side by side It is hard to see each other Communication Space Task Space
  45. 45. Collaborative Augmented Reality Seamless Interaction Natural Communication Attributes Virtuality Augmentation Cooperation Independence Individuality
  46. 46. Hybrid User Interfaces 1 2 3 4 PERSONAL TABLETOP WHITEBOARD MULTIGROUP Private Display Private Display Private Display Private Display Group Display Public Display Group Display Public Display
  47. 47. UbiVR (GIST, Korea)
  48. 48. Shared Design Space (M. Haller, Hagenberg) Enhanced face to face collaboration Tangible tools Digital pens Active desktop Multiple users
  49. 49. Digital Pen, ANOTO Captures position (x, y) in absolute coordinates, time (t), pressure (p), and status (up, down) www.anoto.com
  50. 50. Presentation Area Design Area Shared Workspace Private Workspace
  51. 51. Evaluation
  52. 52. Interaction Design is All About You Users should be involved throughout the Design Process Consider all the needs of the user
  53. 53. Interaction Design Process
  54. 54. Survey of AR Papers Edward Swan (2005) Surveyed major conference/journals (1992-2004) - Presence, ISMAR, ISWC, IEEE VR Summary 1104 total papers 266 AR papers 38 AR HCI papers (Interaction) 21 AR user studies Only 21 from 266 AR papers have formal user study (<8% of all AR papers)
  55. 55. AR Papers
  56. 56. Types of Experiments Perception How is virtual content perceived ? What perceptual cues are most important ? Interaction How can users interact with virtual content ? Which interaction techniques are most efficient ? Collaboration How is collaboration in AR interface different ? Which collaborative cues can be conveyed best ?
  57. 57. Looking to the Future
  58. 58. The Future is with us It takes at least 20 years for new technologies to go from the lab to the lounge.. “The technologies that will significantly affect our lives over the next 10 years have been around for a decade. The future is with us. The trick is learning how to spot it. The commercialization of research, in Oct 11th 2004 other words, is far more about prospecting than alchemy.” Bill Buxton
  59. 59. VR vs. AR Don’t oversell it.. Be Honest
  60. 60. Mobile Phone AR Mobile Phones camera processor display AR on Mobile Phones Simple graphics Optimized computer vision Collaborative Interaction
  61. 61. Collaborative AR AR Tennis Virtual tennis court Two user game Audio + haptic feedback Bluetooth messaging
  62. 62. Collaborative AR
  63. 63. UbiVR Cellphone for service discovery Visualization of real world sensor data
  64. 64. Real Time Rome (Ratti, MIT)
  65. 65. System Design
  66. 66. Results
  67. 67. NZ Cell Tower Locations
  68. 68. Sameer’s Facebook Friends
  69. 69. TouchGraph http://www.touchgraph.com/
  70. 70. Physical + Social Networking Location based social networking Where are my friends now? Where are people with common interests? Trend analysis How do these people behave? Where is my friend likely to be?
  71. 71. Conclusions
  72. 72. Conclusions AR allows us to return to the real world Invisible Interfaces Enhanced Interaction Enhanced Collaboration Future Research Directions Evaluation Massive AR
  73. 73. More Information • Mark Billinghurst – mark.billinghurst@hitlabnz.org • Websites – www.hitlabnz.org