Download as PDF, PPTX
![Affordances
”… the perceived and actual properties of the thing, primarily
those fundamental properties that determine just how the
thing could possibly be used. [...]
Affordances provide strong clues to the operations of things.”
(Norman, The Psychology of Everyday Things 1988, p.9)](https://image.slidesharecdn.com/comp4010-lecture7-final-novideo-2021-210908172228/75/Comp4010-Lecture7-Designing-AR-Systems-97-2048.jpg)
Uploaded byMark Billinghurst
Comp4010 Lecture7 Designing AR Systems
The document provides an overview of AR prototyping tools and methodologies, distinguishing between low and high fidelity approaches. It discusses various digital authoring tools, immersive authoring options, and design considerations, emphasizing human factors, user needs, and effective UI principles for AR experiences. Key tools mentioned include Unity, Vuforia, and Adobe Aero, along with design guidelines aimed at improving user interactions and cognitive processing in AR environments.
More Related Content
Similar to Comp4010 Lecture7 Designing AR Systems
![[BDD 2025 - Mobile Development] Exploring Apple’s On-Device FoundationModels](https://cdn.slidesharecdn.com/ss_thumbnails/md-exploringappleson-devicefoundationmodels-251124030840-d690542c-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Full-Stack Development] PHP in AI Age: The Laravel Way. (Rizqy Hi...](https://cdn.slidesharecdn.com/ss_thumbnails/fs-phpinaiagethelaravelway-251125012602-ef9d330e-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DevFest Strasbourg 2025] - NodeJs Can do that !!](https://cdn.slidesharecdn.com/ss_thumbnails/devfeststrasbourg2025-nodejscandothat-251127142731-da65b6fd-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Full-Stack Development] Digital Accessibility: Why Developers nee...](https://cdn.slidesharecdn.com/ss_thumbnails/fs-digitalaccessibilitywhydevelopersneedtoknowandcarein2025-251127011019-0674441d-thumbnail.jpg?width=640&height=640&fit=bounds)
Comp4010 Lecture7 Designing AR Systems
- 1. DESIGNING AR SYSTEMS COMP4010 Lecture Seven Mark Billinghurst September 7th 2021 mark.billinghurst@unisa.edu.au
- 2.
- 3. XR Prototyping Tools LowFidelity (Concept, visual design) • Sketching • Photoshop • PowerPoint • Video High Fidelity (Interaction, experience design) • Interactive sketching • Desktop & on-device authoring • Immersive authoring & visual scripting • XR development toolkits
- 4. XR Prototyping Techniques Lo- Fi Hi- Fi Easy Hard Digital Authoring Immersive Authoring Web-Based Development* Cross-Platform Development* Native Development* *requires scripting and 3D programming skills Sketching Paper Prototyping Video Prototyping Wireframing Bodystorming Wizard of Oz
- 5. Interactive Sketching •Pop App ●Pop - https://marvelapp.com/pop ● Combining sketching and interactivity on mobiles ● Take pictures of sketches, link pictures together
- 6. Proto.io • Web basedprototyping tool • Visual drag and drop interface • Rich transitions • Scroll, swipe, buttons, etc • Deploy on device • mobile, PC, browser • Ideal for mobile interfaces • iOS, Android template • For low and high fidelity prototypes
- 7. Digital Authoring Toolsfor AR Vuforia Studio Lens Studio • Support visual authoring of marker- based and/or marker-less AR apps • Provide default markers and support for custom markers • Typically enable AR previews through emulator but need to deploy to AR device for testing
- 8. Zappar • Zapworks Studio •Code-free interactivity • Desktop authoring for mobile AR • Integrated computer vision (ARkit, ARCore) • Scripting, visual programming • Multiple publishing options • Zappar App, WebAR, App enabled • Zapbox • Inexpensive mobile AR HMD solution • Two handed input ZapBox
- 9. Snap LensStudio -https://lensstudio.snapchat.com/ Author and preview AR prototypes ● Tool behind Snapchat Lenses, but also a powerful AR prototyping tool ● Can do face (using front camera) and world lenses (rear camera) ● Simulated previews using webcam Deploy and use advanced AR features ● Can deploy to phone running Snapchat app via Snapcode ● Has advanced AR tracking and segmentation capabilities
- 10. Immersive Authoring Toolsfor AR • Enable visual authoring of 3D content in AR • Make it possible to edit while previewing AR experience in the environment • Provide basic support for interactive behaviors • Sometimes support export to WebXR Apple Reality Composer Adobe Aero
- 11. Creating On Device •AdobeAero •Create AR on mobile devices •Touch based interaction and authoring •Only iOS support for now •https://www.adobe.com/nz/products/aero.html
- 12. Apple Reality Composer •Rapidly create 3D scenes and AR experiences • Creation on device (iPhone, iPad) • Drag and drop interface • Loading 2D/3D content • Simple interactivity – trigger/action • Anchor content in real world (AR view) • Planes (vertical, horizontal), faces, images
- 13.
- 14. XR Tools Landscape Digital& Immersive Authoring Proto.io, Tour Creator, ... Tilt Brush, Blocks, Quill, … Web-Based Development THREE.js, Babylon.js, … A-Frame, AR.js, … Cross-Platform Development Unity + SDKs Unreal + SDKs Native Development Cardboard/Oculus/Vive/... SDK Vuforia/ARCore/ARKit/… SDK
- 15. XR Tools Landscape Digital& Immersive Authoring Good for storyboarding but limited support for interactions Web-Based Development Good for basic XR apps but often interactions feel unfinished Native Development Good for full-fledged XR apps but limited to a particular platform Cross-Platform Development Good for full-fledged XR apps but usually high learning curve
- 16. XR Toolkits Card- board AR Kit AR Core Oculus VIVE Holo Lens WMR Web Cam A-Frame AR.js SteamVR MRTK Vuforia ARFoundation XR Interaction WebXR
- 17. WebXR: A-Frame • Basedon Three.js and WebGL • New HTML tags for 3D scenes • A-Frame Inspector (not editor) • Asset management (img, video, audio, & 3D models) • ECS architecture with many open- source components • Cross-platform XR
- 18. AR.js – WebXRTracking • Web based AR tracking library • Marker tracking: ARToolkit markers • Image tracking: Natural feature tracking • Location tracking: GPS and compass • Key Features • Very Fast : It runs efficiently even on phones • Web-based : It is a pure web solution, so no installation required. • Full javascript based on three.js + A-Frame + jsartoolkit5 • Open Source : It is completely open source and free of charge! • Standards : It works on any phone with webgl and webrtc • See https://ar-js-org.github.io/AR.js-Docs/
- 19. Unity – unity.com •Started out as game engine • Has integrated support for many types of XR apps • Powerful scene editor • Asset management & store • Basically all XR device vendors provide Unity SDKs
- 20. Vuforia • Highly optimizedcomputer vision tracking • Multiple types of tracking • Image tracking, object tracking, model tracking, area tracking, etc. • Interaction features • Virtual buttons, occlusion, visual effects, • Multi-platform • Mobile AR, AR headsets See https://www.vuforia.com/
- 21. AR Foundation • Aunified Framework for AR • Multi-platform API • Includes core features from ARKit, ARCore, Magic Leap, and HoloLens • Set of behaviours and API with following features • Tracking, light estimation, occlusion, meshing , video pass-through, etc. • Integrates with Unity MARS • See https://unity.com/unity/features/arfoundation
- 22. Unity XR InterationToolKit (preview package) • Easy way to add interactivity to AR/VR experience • Object interactions • UI interactions • Locomotion • Enabling common interactions without writing code • AR gesture, object placement, annotations • https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@1.0/
- 23. Unity MARS • Features •Visually author AR apps (WYSIWYG) • Test apps in Unity editor • Develop apps that can interact with real world • Intelligent real-world recognition • Multi-platform development • Based on ARFoundation • ARKit, ARCore, Magic Leap and Hololens • See unity.com/mars
- 24. Mixed Reality ToolKit(MRTK) • Open-Source Mixed Reality ToolKit • Set of Unity modules/Unreal plugin • Interaction Models • Controllers, gesture, gaze, voice, etc. • UX elements • Foundational elements • Material, text, light, etc. • Controls and behaviours • button, menu, slider, etc. • Tutorials, documentation, guidelines • See https://github.com/microsoft/MixedRealityToolkit-Unity
- 25.
- 26. Design in InteractionDesign Key Prototyping Steps
- 27. Good vs. BadAR Design
- 28.
- 29. AR. Design Considerations •1. Design for Humans • Use Human Information Processing model • 2. Design for Different User Groups • Different users may have unique needs • 3. Design for the Whole User • Social, cultural, emotional, physical cognitive • 4. Use UI Best Practices • Adapt known UI guidelines to AR/VR • 5. Use of Interface Metaphors/Affordances • Decide best metaphor for AR/VR application
- 30. 1. Design forHuman Information Processing • High level staged model from Wickens and Carswell (1997) • Relates perception, cognition, and physical ergonomics Perception Cognition Ergonomics
- 31. Design for Perception •Need to understand perception to design AR • Visual perception • Many types of visual cues (stereo, oculomotor, etc.) • Auditory system • Binaural cues, vestibular cues • Somatosensory • Haptic, tactile, kinesthetic, proprioceptive cues • Chemical Sensing System • Taste and smell
- 32. Depth Perception Problems •Without proper depth cues AR interfaces look unreal
- 33. Which of thesePOI are near or far?
- 34. Types of DepthCues
- 35.
- 36. Cutaway Example • Providingdepth perception cues for AR https://www.youtube.com/watch?v=2mXRO48w_E4
- 37. Design for Cognition •Design for Working and Long-term memory • Working memory • Short term storage, Limited storage (~5-9 items) • Long term memory • Memory recall trigger by associative cues • Situational Awareness • Model of current state of user’s environment • Used for wayfinding, object interaction, spatial awareness, etc.. • Provide cognitive cues to help with situational awareness • Landmarks, procedural cues, map knowledge • Support both ego-centric and exo-centric views
- 38. Micro-Interactions ▪ Using mobilephones people split their attention between the display and the real world
- 39. Time Looking atScreen Oulasvirta, A. (2005). The fragmentation of attention in mobile interaction, and what to do with it. interactions, 12(6), 16-18.
- 40. Dividing Attention toWorld • Number of times looking away from mobile screen
- 41. Design for MicroInteractions ▪ Design interaction for less than a few seconds • Tiny bursts of interaction • One task per interaction • One input per interaction ▪ Benefits • Use limited input • Minimize interruptions • Reduce attention fragmentation
- 42. NHTSA Guidelines -www.nhtsa.gov For technology in cars: • Any task by a driver should be interruptible at any time. • The driver should control the pace of task interactions. • Tasks should be completed with glances away from the roadway of 2 seconds or less • Cumulative time glancing away from the road <=12 secs.
- 43. Make it Glanceable •Seek to rigorously reduce information density. Successful designs afford for recognition, not reading. Bad Good
- 44. Reduce Information Chunks Youare designing for recognition, not reading. Reducing the total # of information chunks will greatly increase the glanceability of your design. 1 2 3 1 2 3 4 5 (6) Eye movements For 1: 1-2 460ms For 2: 1 230ms For 3: 1 230ms ~920ms Eye movements For 1: 1 230ms For 2: 1 230ms For 3: 1 230ms For 4: 3 690ms For 5: 2 460ms ~1,840ms
- 45. Ego-centric and Exo-centricviews • Combining ego-centric and exo-centric cue for better situational awareness
- 46. Cognitive Issues inMobile AR • Information Presentation • Amount, Representation, Placement, View combination • Physical Interaction • Navigation, Direct manipulation, Content creation • Shared Experience • Social context, Bodily Configuration, Artifact manipulation, Display space Li, N., & Duh, H. B. L. (2013). Cognitive issues in mobile augmented reality: an embodied perspective. In Human factors in augmented reality environments (pp. 109-135). Springer, New York, NY.
- 47. Information Presentation • Consider •The amount of information • Clutter, complexity • The representation of information • Navigation cues, POI representation • The placement of information • Head, body, world stabilized • Using view combinations • Multiple views
- 48. Example: Twitter 360 •www.twitter-360.com • iPhone application • See geo-located tweets in real world • Twitter.com supports geo tagging
- 49. But: Information Clutterfrom Many Tweets Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah
- 50.
- 51.
- 52.
- 53. • Show POIoutside FOV • Zooms between map and panorama views Zooming Views
- 54.
- 55. Design for PhysicalErgonomics • Design for the human motion range • Consider human comfort and natural posture • Design for hand input • Coarse and fine scale motions, gripping and grasping • Avoid “Gorilla arm syndrome” from holding arm pose
- 56. Gorilla Arm inAR • Design interface to reduce mid-air gestures
- 57. XRgonomics • Uses physiologicalmodel to calculate ergonomic interaction cost • Difficulty of reaching points around the user • Customizable for different users • Programmable API, Hololens demonstrator • GitHub Repository • https://github.com/joaobelo92/xrgonomics Evangelista Belo, J. M., Feit, A. M., Feuchtner, T., & Grønbæk, K. (2021, May). XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-11).
- 58.
- 59. 2. Designing forDifferent User Groups • Design for Difference Ages • Children require different interface design than adults • Older uses have different needs than younger • Prior Experience with AR systems • Familiar with HMDs, AR input devices • People with Different Physical Characteristics • Height and arm reach, handedness • Perceptual, Cognitive and Motor Abilities • Colour perception varies between people • Spatial ability, cognitive or motor disabilities
- 60. Designing for Children •HMDS • inter pupillary distance, head fit, size and weight • Tablets • Poor dexterity, need to hold large tablet • Content • Reading ability, spatial perception
- 61. 3. Design forthe Whole User
- 62. Consider Your User •Consider context of user • Physical, social, emotional, cognitive, etc. • Mobile Phone AR User • Probably Mobile • One hand interaction • Short application use • Need to be able to multitask • Use in outdoor or indoor environment • Want to enhance interaction with real world
- 63. Would you wearthis HMD?
- 64. Whole User Needs •Social • Don’t make your user look stupid • Cultural • Follow local cultural norms • Physical • Can the user physically use the interface? • Cognitive • Can the user understand how the interface works? • Emotional • Make the user feel good and in control
- 65. Example: Social Acceptance •People don’t want to look silly • Only 12% of 4,600 adults would be willing to wear AR glasses • 20% of mobile AR browser users experience social issues • Acceptance more due to Social than Technical issues • Needs further study (ethnographic, field tests, longitudinal)
- 66.
- 69. 4. Use UIBest Practices • General UI design principles can be applied to AR • E.g. Shneiderman’s UI guidelines from 1998 • Providing interface feedback • Mixture of reactive, instrumental and operational feedback • Maintain spatial and temporal correspondence • Use constraints • Specify relations between variables that must be satisfied • E.g. physical constraints reduce freedom of movement • Support Two-Handed control • Use Guiard’s framework of bimanual manipulation • Dominant vs. non-dominant hands
- 70. Follow Good HCIPrinciples • Provide good conceptual model/Metaphor • customers want to understand how UI works • Make things visible • if object has function, interface should show it • Map interface controls to customer s model • infix -vs- postfix calculator -- whose model? • Provide feedback • what you see is what you get!
- 71. Example: Guiard’s modelof bimanual manipulation Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19, 486-517. Dominant hand Non-dominant hand Dominant hand Non-dominant hand Non-Dominant: Leads, set spatial reference frame, performs coarse motions Dominant: Follows, works in reference frame, performs fine motions
- 72. Adapting Existing Guidelines •Mobile Phone AR • Phone HCI Guidelines • Mobile HCI Guidelines • HMD Based AR • 3D User Interface Guidelines • VR Interface Guidelines • Desktop AR • Desktop UI Guidelines
- 73. Example: Apple iOSInterface Guidelines • Make it obvious how to use your content. • Avoid clutter, unused blank space, and busy backgrounds. • Minimize required user input. • Express essential information succinctly. • Provide a fingertip-sized target for all controls. • Avoid unnecessary interactivity. • Provide feedback when necessary From: https://developer.apple.com/ios/human-interface-guidelines/
- 74. Applying Principles toMobile AR • Clean • Large Video View • Large Icons • Text Overlay • Feedback
- 75. •Interface Components • Physicalcomponents • Display elements • Visual/audio • Interaction metaphors Physical Elements Display Elements Interaction Metaphor Input Output 5. Use Interface Metaphors
- 76. AR Interfaces Tangible AR Tangibleinput AR overlay Direct interaction Natural AR Freehand gesture Speech, gaze Tangible UI Augmented surfaces Object interaction Familiar controllers Indirect interaction 3D AR 3D UI Dedicated controllers Custom devices Browsing Simple input Viewpoint control Expressiveness, Intuitiveness
- 77. AR Interfaces Tangible AR Tangibleinput AR overlay Direct interaction Natural AR Freehand gesture Speech, gaze Tangible UI Augmented surfaces Object interaction Familiar controllers Indirect interaction 3D AR 3D UI Dedicated controllers Custom devices Browsing Simple input Viewpoint control Design for Layers
- 78. Information Layers • Head-stabilized •Heads-up display • Body-stabilized • E.g., virtual tool-belt • World-stabilized • E.g., billboard or signpost
- 79. Head stabilized • Informationattached to view – always visible
- 80. Body Stabilized • Informationmoves with person
- 81. Body Stabilized Interface •Elements you want always available, but not always visible
- 82. World Stabilized • Informationfixed in world
- 83. • Elements youwant fixed relative to real world objects
- 84. “Diegetic UI” • Integratedwith world
- 85. Example: Fragments • UIElements embedded in real world • Real world occlusion
- 86.
- 87. Design to DeviceConstraints • Understand the platform and design for limitations • Hardware, software platforms • E.g. Handheld AR game with visual tracking • Use large screen icons • Consider screen reflectivity • Support one-hand interaction • Consider the natural viewing angle • Do not tire users out physically • Do not encourage fast actions • Keep at least one tracking surface in view Art of Defense Game
- 88. Handheld AR Constraints/Affordances •Camera and screen are linked • Fast motions a problem when looking at screen • Intuitive “navigation” • Phone in hand • Two handed activities: awkward or intuitive • Extended periods of holding phone tiring • Awareness of surrounding environment • Small screen • Extended periods of looking at screen tiring • In general, small awkward platform • Vibration, sound • Can provide feedback when looking elsewhere
- 89. Common Mobile ARMetaphors • Tangible AR Lens Viewing • Look through screen into AR scene • Interact with screen to interact with AR content • Touch screen input • E.g. Invisible Train • Metaphor – holding a window into the AR world
- 90.
- 91. Common Mobile ARMetaphors • Tangible AR Lens Manipulation • Select AR object and attach to device • Physically move phone to move AR object • Use motion of device as input • E.g. AR Lego • Metaphor – phone as physical handle for device
- 92.
- 93. AR Interfaces Tangible AR Tangibleinput AR overlay Direct interaction Natural AR Freehand gesture Speech, gaze Tangible UI Augmented surfaces Object interaction Familiar controllers Indirect interaction 3D AR 3D UI Dedicated controllers Custom devices Browsing Simple input Viewpoint control Design for Affordances
- 94. Tangible AR Metaphor •AR overcomes limitation of TUIs • enhance display possibilities • merge task/display space • provide public and private views • TUI + AR = Tangible AR • Apply TUI methods to AR interface design
- 95. Tangible AR DesignPrinciples • Tangible AR Interfaces use TUI principles • Physical controllers for moving virtual content • Support for spatial 3D interaction techniques • Time and space multiplexed interaction • Support for multi-handed interaction • Match object affordances to task requirements • Support parallel activity with multiple objects • Allow collaboration between multiple users
- 96. AR Design Space RealityVirtual Reality Augmented Reality Physical Design Virtual Design
- 97. Affordances ”… the perceivedand actual properties of the thing, primarily those fundamental properties that determine just how the thing could possibly be used. [...] Affordances provide strong clues to the operations of things.” (Norman, The Psychology of Everyday Things 1988, p.9)
- 98.
- 99.
- 100. Affordance Matrix Fake door Hiddendoor Real door No door
- 101. Physical vs. VirtualAffordances • Physical Affordance • Look and feel of real objects • Shape, texture, colour, weight, etc. • Industrial Design • Virtual Affordance • Look of virtual objects • Copy real objects • Interface Design
- 102. •AR design ismixture of physical affordance and virtual affordance •Physical •Tangible controllers and objects •Virtual •Virtual graphics and audio
- 103. Affordances in AR •Design AR interface objects to show how they are used • Use visual and physical cues to show possible affordances • Perceived affordances should match actual affordances • Physical and virtual affordances should match Merge Cube Tangible Molecules
- 104. Case Study 1:3D AR Lens Goal: Develop a lens based AR interface • MagicLenses • Developed at Xerox PARC in 1993 • View a region of the workspace differently to the rest • Overlap MagicLenses to create composite effects
- 105. 3D MagicLenses MagicLenses extendedto 3D (Veiga et. al. 96) § Volumetric and flat lenses
- 106. AR Lens DesignPrinciples • Physical Components • Lens handle • Virtual lens attached to real object • Display Elements • Lens view • Reveal layers in dataset • Interaction Metaphor • Physically holding lens
- 107. 3D AR Lenses:Model Viewer § Displays models made up of multiple parts § Each part can be shown or hidden through the lens § Allows the user to peer inside the model § Maintains focus + context
- 108.
- 109. AR Lens Implementation StencilBuffer Outside Lens Inside Lens Virtual Magnifying Glass
- 110. Case Study 2: LevelHead • Block based game
- 111. Case Study 2:LevelHead • Physical Components • Real blocks • Display Elements • Virtual person and rooms • Interaction Metaphor • Blocks are rooms
- 113.
- 114. Case Study 3:AR Chemistry (Fjeld 2002) • Tangible AR chemistry education
- 115. Goal: An ARapplication to teach molecular structure in chemistry •Physical Components • Real book, rotation cube, scoop, tracking markers •Display Elements • AR atoms and molecules • Interaction Metaphor • Build your own molecule
- 116. AR Chemistry InputDevices
- 118.
- 119. Case Study 4:Transitional Interfaces Goal: An AR interface supporting transitions from reality to virtual reality •Physical Components • Real book •Display Elements • AR and VR content • Interaction Metaphor • Book pages hold virtual scenes
- 120. Milgram’s Continuum (1994) Reality (Tangible Interfaces) Virtualit y (Virtual Reality) Augmented Reality(AR) Augmented Virtuality (AV) Mixed Reality (MR) Central Hypothesis • The next generation of interfaces will support transitions along the Reality-Virtuality continuum
- 121. Transitions • Interfaces ofthe future will need to support transitions along the RV continuum • Augmented Reality is preferred for: • co-located collaboration • Immersive Virtual Reality is preferred for: • experiencing world immersively (egocentric) • sharing views • remote collaboration
- 122. The MagicBook •Design Goals: •Allowsuser to move smoothly between reality and virtual reality •Support collaboration
- 123.
- 124.
- 125. Features • Seamless transitionfrom Reality to Virtuality • Reliance on real decreases as virtual increases • Supports egocentric and exocentric views • User can pick appropriate view • Computer becomes invisible • Consistent interface metaphors • Virtual content seems real • Supports collaboration
- 126. Collaboration in MagicBook •Collaboration on multiple levels: • Physical Object • AR Object • Immersive Virtual Space • Egocentric + exocentric collaboration • multiple multi-scale users • Independent Views • Privacy, role division, scalability
- 127. Technology • Reality • Notechnology • Augmented Reality • Camera – tracking • Switch – fly in • Virtual Reality • Compass – tracking • Pressure pad – move • Switch – fly out
- 128. Summary •When designing ARinterfaces, think of: • Physical Components • Physical affordances • Virtual Components • Virtual affordances • Interface Metaphors • Tangible AR or similar
- 129.
- 130. Design Guidelines By Vendors Platformdriven By Designers User oriented By Practitioners Experience based By Researchers Empirically derived
- 131. Design Patterns “Each patterndescribes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem in such a way that you can use this solution a million times over, without ever doing it the same way twice.” – Christopher Alexander et al. Use Design Patterns to Address Reoccurring Problems C.A. Alexander, A Pattern Language, Oxford Univ. Press, New York, 1977.
- 132. Example UI DesignPatterns • http://ui-patterns.com/patterns
- 134. Design Patterns forHandheld AR • Set of design patterns for Handheld AR • Title: a short phase that is memorable. • Definition: what experiences the prepattern supports • Description: how and why the prepattern works, what aspects of game design it is based on. • Examples: Illustrate the meaning of the pre-pattern. • Using the pre-patterns: reveal the challenges and context of applying the pre-patterns. Xu, Y., Barba, E., Radu, I., Gandy, M., Shemaka, R., Schrank, B., ... & Tseng, T. (2011, October). Pre-patterns for designing embodied interactions in handheld augmented reality games. In 2011 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (pp. 19-28). IEEE.
- 135. Handheld AR DesignPatterns Title Meaning Embodied Skills Device Metaphors Using metaphor to suggest available player actions Body A&S Naïve physics Control Mapping Intuitive mapping between physical and digital objects Body A&S Naïve physics Seamful Design Making sense of and integrating the technological seams through game design Body A&S World Consistency Whether the laws and rules in physical world hold in digital world Naïve physics Environmental A&S Landmarks Reinforcing the connection between digital- physical space through landmarks Environmental A&S Personal Presence The way that a player is represented in the game decides how much they feel like living in the digital game world Environmental A&S Naïve physics Living Creatures Game characters that are responsive to physical, social events that mimic behaviours of living beings Social A&S Body A&S Body constraints Movement of one’s body position constrains another player’s action Body A&S Social A&S Hidden information The information that can be hidden and revealed can foster emergent social play Social A&S Body A&S *A&S = awareness and skills
- 137. Example: Seamless Design •Design to reduce seams in the user experience • Eg: AR tracking failure, change in interaction mode • Paparazzi Game • Change between AR tracking to accelerometer input Yan Xu , et.al. , Pre-patterns for designing embodied interactions in handheld augmented reality games, Proceedings of the 2011 IEEE International Symposium on Mixed and Augmented Reality--Arts, Media, and Humanities, p.19-28, October 26-29, 2011
- 138. Demo: Paparazzi Game •https://www.youtube.com/watch?v=MIGH5WGMnbs
- 139. Example: Living Creatures •Virtual creatures should respond to real world events • eg. Player motion, wind, light, etc • Creates illusion creatures are alive in the real world • Sony EyePet • Responds to player blowing on creature
- 149. Google ARCore InterfaceGuidelines https://developers.google.com/ar/design
- 150. ARCore Elements App •Mobile AR app demonstrating interface guidelines • Multiple Interface Guidelines • User interface • User environment • Object manipulation • Off-screen markers • Etc.. • Test on Device • https://play.google.com/store/apps/details?id=com.google.ar.unity.ddelements
- 151.
- 152. ARKit Interface Guidelines •developer.apple.com/design/human-interface-guidelines/ios/system-capabilities/augmented-reality/
- 153. Microsoft Mixed RealityDesign Guidelines • https://docs.microsoft.com/en-us/windows/mixed-reality/design/design
- 154. MRTK Interface Examples •Examples of UX Building Blocks • http://aka.ms/MRTK
- 155. The Trouble withAR Design Guidelines 1) Rapidly evolving best practices Still a moving target, lots to learn about AR design Slowly emerging design patterns, but often change with OS updates Already major differences between device platforms 2) Challenges with scoping guidelines Often too high level, like “keep the user safe and comfortable” Or, too application/device/vendor-specific 3) Best guidelines come from learning by doing Test your designs early and often, learn from your own “mistakes” Mind differences between VR and AR, but less so between devices
- 156.