Download as PDF, PPTX
![Augmented Reality Definition
• Defining Characteristics [Azuma 97]
• Combines Real andVirtual Images
• Both can be seen at the same time
• Registered in 3D
• Virtual objects appear fixed in space
• Interactive in real-time
• The virtual content can be interacted with
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.](https://image.slidesharecdn.com/lecture3-160229094553/75/2016-AR-Summer-School-Lecture3-2-2048.jpg)
!["...the term affordance refers to the perceived
and actual properties of the thing, primarily
those fundamental properties that determine
just how the thing could possibly be used. [...]
Affordances provide strong clues to the
operations of things.”
(Norman, The Psychology of EverydayThings 1988, p.9)](https://image.slidesharecdn.com/lecture3-160229094553/75/2016-AR-Summer-School-Lecture3-14-2048.jpg)
Uploaded byMark Billinghurst
2016 AR Summer School Lecture3
This document summarizes a lecture on interaction design for augmented reality. It discusses several types of AR interfaces including: (1) AR information browsers that allow viewing and manipulating virtual content registered in the real world, (2) 3D AR interfaces that allow interacting with and manipulating 3D virtual objects, and (3) tangible interfaces that use physical objects to interact with and control virtual objects. It also presents case studies of specific AR applications and discusses design principles for AR interaction including using physical affordances, feedback, and natural mappings.
More Related Content
Similar to 2016 AR Summer School Lecture3
2016 AR Summer School Lecture3
- 1. LECTURE 3: INTERACTION FORAUGMENTED REALITY Mark Billinghurst AR Summer School February 15th – 19th 2016 University of South Australia
- 2. Augmented Reality Definition • DefiningCharacteristics [Azuma 97] • Combines Real andVirtual Images • Both can be seen at the same time • Registered in 3D • Virtual objects appear fixed in space • Interactive in real-time • The virtual content can be interacted with Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
- 3. Interacting inAugmented Reality • You can see spatially registered AR.. how can you interact with it?
- 4. AR Interaction • Designing ARSystem = Interface Design • Using different input and output technologies • Objective is a high quality of user experience • Ease of use and learning • Performance and satisfaction
- 5. User Interface andTool • Human ! User Interface/Tool " Machine/Object • Human Machine Interface
- 6. User Interface: Characteristics • Input:mono or multimodal • Output: mono or multi-sensorial • Technique/Metaphor/Paradigm © Andreas Dünser Output Sensation of movement Input Metaphor “Push” to accelerate “Turn” to rotate
- 7. Human Computer Interface • Human! User Interface "Computer System • Human Computer Interface= Hardware + Software • Computer is everywhere now# HCI electronic devices, Home Automation, Transport vehicles, etc
- 8. • Interface Components • Physical components • Displayelements • Visual/audio • Interaction metaphors Physical Elements Display ElementsInteraction MetaphorInput Output AR Interface Elements
- 9. AR Design Space RealityVirtual Reality Augmented Reality Physical Design Virtual Design
- 10. Types of Objects • Objects • Purposely built – affordances • “Found” – repurposed • Existing – already at use in marketplace
- 14. "...the term affordance refers tothe perceived and actual properties of the thing, primarily those fundamental properties that determine just how the thing could possibly be used. [...] Affordances provide strong clues to the operations of things.” (Norman, The Psychology of EverydayThings 1988, p.9)
- 15. Physical Affordances • Physical affordances: Howdo the following physical objects afford? Are they obvious?
- 16. VirtualAffordances • Virtual affordances How dothe following screen objects afford? What if you were a novice user? Would you know what to do with them?
- 17. • AR Interface =physical affordance + virtual affordance • Physical • Tangible controllers and objects • Virtual • Virtual graphics and audio
- 18. Design of Objects • Makeaffordances obvious • Object affordances visible • Give feedback • Provide constraints • Use natural mapping • Use good cognitive model
- 19. Interface Design Path 1/Prototype Demonstration 2/ Adoption of Interaction Techniques from other interface metaphors 3/ Development of new interface metaphors appropriate to the medium 4/ Development of formal theoretical models for predicting and modeling user actions Desktop WIMP Virtual Reality Augmented Reality
- 20. AR Interaction Metaphors • BrowsingInterfaces • simple (conceptually!), unobtrusive • 3D AR Interfaces • expressive, creative, require attention • Tangible Interfaces • Embedded into conventional environments • Tangible AR • Combines TUI input + AR display
- 21. AR Interfaces asData Browsers • 2D/3D virtual objects are registered in 3D • “VR in Real World” • Interaction • 2D/3D virtual viewpoint control • Applications • Visualization, training
- 22. AR Information Browsers • Information is registered to real-world context • Hand held AR displays • Interaction • Manipulation of a window into information space • Applications • Context-aware information displays Rekimoto, et al. 1997
- 23.
- 24. Current AR InformationBrowsers • Mobile AR • GPS + compass • Many Applications • Layar • Wikitude • Acrossair • PressLite • Yelp • AR Car Finder • …
- 25. Layar (www.layar.com) • Location baseddata • GPS + compass location • Map + camera view • AR Layers on real world • Customized data • Audio, 3D, 2D content • Easy authoring • Android, iPhone
- 26. Advantages and Disadvantages • Importantclass of AR interfaces • Wearable computers • AR simulation, training • Limited interactivity • Modification of virtual content is difficult Rekimoto, et al. 1997
- 27. 3D AR Interfaces • Virtual objects displayed in 3D physical space and manipulated • HMDs and 6DOF head-tracking • 6DOF hand trackers for input • Interaction • Viewpoint control • Traditional 3D user interface interaction: manipulation, selection, etc. Kiyokawa, et al. 2000
- 28.
- 29.
- 30. Advantages and Disadvantages • Important class of AR interfaces • Entertainment, design, training • Advantages • User can interact with 3D virtual object everywhere in space • Natural, familiar interaction • Disadvantages • Usually no tactile feedback • User has to use different devices for virtual and physical objects Oshima, et al. 2000
- 31. Augmented Surfaces andTangible Interfaces • Basic principles • Virtual objects are projected on a surface • Physical objects are used as controls for virtual objects • Support for collaboration
- 32. Augmented Surfaces • Rekimoto, etal. 1998 • Front projection • Marker-based tracking • Multiple projection surfaces Rekimoto, J., & Saitoh, M. (1999, May). Augmented surfaces: a spatially continuous work space for hybrid computing environments. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 378-385). ACM.
- 33. Tangible User Interfaces(Ishii 97) • Create digital shadows for physical objects • Foreground • graspable UI • Background • ambient interfaces Ishii, H., & Ullmer, B. (1997, March). Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (pp. 234-241). ACM.
- 34. Tangible Interfaces -Ambient • Dangling String • Jeremijenko 1995 • Ambient ethernet monitor • Relies on peripheral cues • Ambient Fixtures • Dahley, Wisneski, Ishii 1998 • Use natural material qualities for information display
- 35. Tangible Interface: ARgroove • Collaborative Instrument • Exploring Physically Based Interaction • Map physical actions to Midi output • Translation, rotation • Tilt, shake
- 36.
- 37. Visual Feedback • Continuous VisualFeedback is Key • Single Virtual Image Provides: • Rotation • Tilt • Height
- 38. i/O Brush (Ryokai,Marti, Ishii)
- 39. Other Examples • Triangles (Gorbert1998) • Triangular based story telling • ActiveCube (Kitamura 2000-) • Cubes with sensors
- 40. Lessons from TangibleInterfaces • Physical objects make us smart • Norman’s “Things that Make Us Smart” • encode affordances, constraints • Objects aid collaboration • establish shared meaning • Objects increase understanding • serve as cognitive artifacts
- 41. TUI Limitations • Difficult tochange object properties • can’t tell state of digital data • Limited display capabilities • projection screen = 2D • dependent on physical display surface • Separation between object and display • ARgroove
- 42. Advantages and Disadvantages • Advantages • Natural - users hands are used for interacting with both virtual and real objects. • No need for special purpose input devices • Disadvantages • Interaction is limited only to 2D surface • Full 3D interaction and manipulation is difficult
- 43. Orthogonal Nature ofARInterfaces
- 44. Orthogonal Nature ofARInterfaces
- 45. Tangible AR Interaction • ARovercomes limitation of TUIs • enhance display possibilities • merge task/display space • provide public and private views • TUI + AR = Tangible AR • Apply TUI methods to AR interface design
- 46. TangibleAR Design Principles • TangibleAR Interfaces use TUI principles • Physical controllers for moving virtual content • Support for spatial 3D interaction techniques • Support for multi-handed interaction • Match object affordances to task requirements • Support parallel activity with multiple objects • Allow collaboration between multiple users
- 47. Space vs.Time -Multiplexed • Space-multiplexed • Many devices each with one function • Quicker to use, more intuitive, clutter • Real Toolbox • Time-multiplexed • One device with many functions • Space efficient • mouse
- 48. TangibleAR:Tiles (Space Multiplexed) • Tilessemantics • data tiles • operation tiles • Operation on tiles • proximity • spatial arrangements • space-multiplexed Poupyrev, I., Tan, D., Billinghurst, M., Kato, H., Regenbrecht, H., & Tetsutani, N. (2001). Tiles: A mixed reality authoring interface. In INTERACT 2001 Conference on Human Computer Interaction (pp. 334-341).
- 49.
- 50.
- 51. Object Based Interaction:MagicCup • IntuitiveVirtual Object Manipulation on a Table-Top Workspace • Time multiplexed • Multiple Markers • Robust Tracking • Tangible User Interface • Intuitive Manipulation • Stereo Display • Good Presence
- 52. MagicCup system $ Main table,Menu table, Cup interface Billinghurst, M., Kato, H., & Myojin, S. (2009). Advanced interaction techniques for augmented reality applications. In Virtual and Mixed Reality (pp. 13-22). Springer Berlin Heidelberg.
- 54. TangibleAR:Time-multiplexed Interaction • Use ofnatural physical object manipulations to control virtual objects • VOMAR Demo • Catalog book: • Turn over the page • Paddle operation: • Push, shake, incline, hit, scoop Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., & Tachibana, K. (2000). Virtual object manipulation on a table-top AR environment. In Augmented Reality, 2000.(ISAR 2000). Proceedings. IEEE and ACM International Symposium on (pp. 111-119). Ieee.
- 55.
- 56. Advantages and Disadvantages • Advantages • Natural interaction with virtual and physical tools • No need for special purpose input devices • Spatial interaction with virtual objects • 3D manipulation with virtual objects anywhere in physical space • Disadvantages • Requires Head Mounted Display
- 57. Wrap-up • Browsing Interfaces • simple(conceptually!), unobtrusive • 3D AR Interfaces • expressive, creative, require attention • Tangible Interfaces • Embedded into conventional environments • Tangible AR • Combines TUI input + AR display
- 58. • Interface Components • Physical components • Displayelements • Visual/audio • Interaction metaphors Physical Elements Display ElementsInteraction MetaphorInput Output AR Interface Elements
- 59. Case Study 1:3DARLens Goal: Develop a lens based AR interface • MagicLenses • Developed at Xerox PARC in 1993 • View a region of the workspace differently to the rest • Overlap MagicLenses to create composite effects
- 60. 3D MagicLenses MagicLenses extendedto 3D (Veiga et. al. 96) % Volumetric and flat lenses
- 61. AR Lens DesignPrinciples • Physical Components • Lens handle • Virtual lens attached to real object • Display Elements • Lens view • Reveal layers in dataset • Interaction Metaphor • Physically holding lens
- 62. 3DAR Lenses:ModelViewer % Displaysmodels made up of multiple parts % Each part can be shown or hidden through the lens % Allows the user to peer inside the model % Maintains focus + context
- 63.
- 64. AR Lens Implementation StencilBuffer Outside Lens Inside Lens Virtual Magnifying Glass
- 65. AR FlexiLens Real handles/controllerswith flexible AR lens
- 66. Case Study 2:LevelHead (Julian Oliver) • Block based game
- 67. Case Study 2:LevelHead • PhysicalComponents • Real blocks • Display Elements • Virtual person and rooms • Interaction Metaphor • Blocks are rooms
- 69. Case Study 3:ARChemistry (Fjeld 2002) • Tangible AR chemistry education Fjeld, M., Fredriksson, J., Ejdestig, M., Duca, F., Bötschi, K., Voegtli, B., & Juchli, P. (2007, April). Tangible user interface for chemistry education: comparative evaluation and re-design. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 805-808). ACM.
- 70. Goal: An ARapplication to test molecular structure in chemistry • Physical Components • Real book, rotation cube, scoop, tracking markers • Display Elements • AR atoms and molecules • Interaction Metaphor • Build your own molecule
- 71. AR Chemistry InputDevices
- 73. Case Study 4:TransitionalInterfaces Goal: An AR interface supporting transitions from reality to virtual reality • Physical Components • Real book • Display Elements • AR andVR content • Interaction Metaphor • Book pages hold virtual scenes
- 74. Milgram’s Continuum (1994) Reality (Tangible Interfaces) Virtuality (Virtual Reality) Augmented Reality(AR) Augmented Virtuality (AV) Mixed Reality (MR) Central Hypothesis • The next generation of interfaces will support transitions along the Reality-Virtuality continuum
- 75. Transitions • Interfaces of thefuture will need to support transitions along the RV continuum • Augmented Reality is preferred for: • co-located collaboration • ImmersiveVirtual Reality is preferred for: • experiencing world immersively (egocentric) • sharing views • remote collaboration
- 76.
- 77. Features • Seamless transition betweenReality andVirtuality • Reliance on real decreases as virtual increases • Supports egocentric and exocentric views • User can pick appropriate view • Computer becomes invisible • Consistent interface metaphors • Virtual content seems real • Supports collaboration
- 78. Collaboration • Collaboration on multiplelevels: • Physical Object • AR Object • ImmersiveVirtual Space • Egocentric + exocentric collaboration • multiple multi-scale users • IndependentViews • Privacy, role division, scalability
- 79. Technology • Reality • No technology • Augmented Reality • Camera– tracking • Switch – fly in • Virtual Reality • Compass – tracking • Press pad – move • Switch – fly out
- 80. Summary • When designing ARinterfaces, think of: • Physical Components • Physical affordances • Virtual Components • Virtual affordances • Interface Metaphors • Connecting physical and virtual
- 81.