Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rapid Prototyping For Augmented Reality

15,426 views

Published on

Talk on Rapid Prototyping for Augmented Reality, given by Mark Billinghurst on April 5th 2016. Given to students at Stanford University's Augmented Reality class

Published in: Technology
  • Login to see the comments

Rapid Prototyping For Augmented Reality

  1. 1. RAPID PROTOTYPING FOR AUGMENTED REALITY Mark Billinghurst mark.billinghurst@unisa.edu.au April 5th 2016
  2. 2. How do you Design and Test this?
  3. 3. Or this..
  4. 4. Google Glass
  5. 5. View Through Glass
  6. 6. Google Glass Prototyping http://ed.ted.com/lessons/rapid-prototyping-google-glass-tom-chi
  7. 7. Rapid Prototyping Glass • Q: How long would it take you to make the first working version of the Glass headset display? • A: One day..
  8. 8. Interacting with Glass
  9. 9. Prototyping Interaction • Q: How long would it take you to prototype a Minority Report interaction technique? • A: 45 minutes..
  10. 10. Tom Chi’s Prototyping Rules 1.  Find the quickest path to experience 2.  Doing is the best kind of thinking 3.  Use materials that move at the speed of thought to maximize your rate of learning
  11. 11. The Road to Glass
  12. 12. Earlyprototyping
  13. 13. DESIGN PROCESSES
  14. 14. Interaction Design Process
  15. 15. DesignThinking -Three Phase Model http://dschool.stanford.edu/redesigningtheater/the-design-thinking-process/
  16. 16. Elaboration and Reduction •  Elaborate - generate solutions.These are the opportunities •  Reduce - decide on the ones worth pursuing •  Repeat - elaborate and reduce again on those solutions Source: Laseau,P. (1980) Graphic Thinking for Architects & Designers. John Wiley and Sons
  17. 17. Elaboration vs.Reduction • Elaboration: Sketching/Storytelling • Creation, exploration • Quick, conceptual • Reduction: Prototyping/Testing • Functional • Idea presentation • Decision making
  18. 18. RAPID PROTOTYPING
  19. 19. How can we quickly prototype Augmented Reality experiences with little or no coding?
  20. 20. Why Prototype? ▪  Quick visual design ▪  Capture key interactions ▪  Focus on user experience ▪  Communicate design ideas ▪  “Learn by doing/experiencing”
  21. 21. Design/PrototypingTools
  22. 22. Typical Development Steps ▪  Sketching ▪  Storyboards ▪  UI Mockups ▪  Interaction Flows ▪  Video Prototypes ▪  Interactive Prototypes ▪  Final Native Application Increased Fidelity & Interactivity
  23. 23. Sketching Sketching is not about drawing It is about design. Sketching is a tool to help you: •  express •  develop, and •  communicate design ideas Sketching is part of a process: •  idea generation, •  design elaboration •  design choices, •  engineering
  24. 24. Figure 41: A Sketch of a Dialogue with a Sketch The “conversation” between the sketch (right bubble) and the mind (left bubble). A sketch is created from current knowledge (top arrow). Reading, or interpreting the resulting representation (bottom arrow), creates new knowledge. The creation results from what Goldschmidt calls “seeing that” reasoning, and the extraction of new knowledge results from what she calls “seeing as.” sketch representation mind (new) knowledge Create (seeing that) Read (seeing as) Sketching is about the activity not the result – Bill Buxton 25
  25. 25. AR Hockey Concept https://stereoscopica.wordpress.com/2013/05/29/augmented-reality-hockey-concept/
  26. 26. Sketched Interfaces ▪  Sketch + Powerpoint/Photoshop/Illustrator
  27. 27. Sketched Concepts http://www.objecttheory.com/
  28. 28. Example Storyboard
  29. 29. Example Storyboard p.2
  30. 30. Persona + Storyboard
  31. 31. Paper Prototyping (Low Fidelity) Quick and simple means of sketching interfaces Use office materials Easier to criticize, quick to change Creative process (develop in team) Can also use for usability test (focus on interaction flow) Used a lot to test out concepts before real design begins.
  32. 32. Paper Prototyping
  33. 33. The Basic Materials • Post-its • 5x8 in. index cards • Scissors, X-acto knives • Overhead transparencies • Large, heavy, white paper (11 x 17) • Tape, stick glue, correction tape • Pens & markers (many colors & sizes)
  34. 34. Physical Prototype
  35. 35. • Early ideation • Think through ideas • Force you to visualize how things come together • Communicate ideas to others to inspire new designs • Active brainstorming Why is Sketching/Paper Prototyping Useful?
  36. 36. Paper Prototypes Pros and Cons • Good •  Quick and dirty •  No software needed •  Very inclusive • Bad •  Not quick enough •  Too dirty (lack context and scale) •  Hard to share among team •  Still needs a separate documentation step
  37. 37. From Sketches toWorking Systems Early design Late design Brainstorm different ideas and representations Choose a representation Rough out interface style Multitude of sketches Sketch variations and details Sketch or low fidelity prototypesTask centered walkthrough and redesign Fine tune interface, screen design Heuristic evaluation and redesign Usability testing and redesign Low to medium fidelity prototypes Limited field testing Alpha/Beta tests High fidelity prototypes Working systems
  38. 38. From Sketches to Prototypes •  Sketches: early ideation stages of design •  Prototypes: capturing /detailing the actual design
  39. 39. Prototyping • Create physical form of ideas • Allow people to experience and interact with them • Why Prototyping? • Empathy gaining- deepen understanding of design space • Exploration – build to think • Testing – test solutions with end users • Inspiration – help others catch your vision
  40. 40. Sketch vs.Prototype Sketch Prototype Invite A)end Suggest Describe Explore Refine Ques;on Answer Propose Test Provoke Resolve Tenta;ve, non commi)al Specific Depic;on The primary differences are in the intent
  41. 41. Sketching to Prototype
  42. 42. Wireframe • It’s about • Functional specs • Navigational systems • Functionality and layout • Notes about the intended functionality • How interface elements work together • Lack of typographic style, colour or graphics • Leaving room for the design to be created
  43. 43. Wireframes
  44. 44. Mockup • It’s about •  Look and feel •  Build on the wireframe with graphics and polish •  May adjust layout slightly but stays within the general guide of the wireframe
  45. 45. FunMe Mobile AR Wireframe/Mockup http://www.yunnuocheng.com/funme/
  46. 46. GlassSim – http://glasssim.com/ ▪  Simulate the view through Google Glass ▪  Multiple card templates
  47. 47. GlassSim Card Builder ▪  Use HTML for card details ▪  Multiple templates ▪  Change background ▪  Own image ▪  Camera view
  48. 48. GlassSim Samples
  49. 49. Application Storyboard ▪  http://dsky9.com/glassfaq/google-glass- storyboard-template-download/
  50. 50. Application Flow
  51. 51. Acting/Role Playing
  52. 52. Wireframe/Mockup Pros and Cons ▪  Positives ▪  Good for documenting screens ▪  Can show application flow ▪  Negatives ▪  No interactivity/transitions ▪  Can’t be used for testing ▪  Can’t deploy on device ▪  Can be time consuming to create
  53. 53. Transitions
  54. 54. ▪ Series of still photos in a movie format. ▪ Demonstrates the experience of the product ▪ Discover where concept needs fleshing out. ▪ Communicate experience and interface ▪ You can use whatever tools, from Flash to iMovie. Video Sketching
  55. 55. UI Concept Movies
  56. 56. Video Sketch Pros and Cons • Good •  Excellent communication tool •  Shows interactions •  High fidelity concept • Bad •  No interaction •  Difficult to use for user testing •  May not be able to implement vision •  Too good to be true
  57. 57. EXAMPLE: SKETCH TO VIDEO
  58. 58. AR Hockey •  Concept – Air Hockey in the real world •  Mix materiality and real surfaces with digital forms •  React to real body movement and player interaction https://stereoscopica.wordpress.com/2013/05/29/augmented-reality-hockey-concept/
  59. 59. AR Hockey Concept https://stereoscopica.wordpress.com/2013/05/29/augmented-reality-hockey-concept/
  60. 60. Sketched Projected Content
  61. 61. Game Setup
  62. 62. Paper/Experience Prototypes •  Having people act out the game
  63. 63. Physical Mockup
  64. 64. Concept Video https://vimeo.com/79285725
  65. 65. WORKING PROTOTYPES
  66. 66. Interactive Sketching • Pop App • Pop - https://popapp.in/ • Combining sketching and interactivity on mobiles • Take pictures of sketches, link pictures together
  67. 67. Using Pop
  68. 68. Interactive Wireframing ▪  Developing interactive interfaces/wireframes ▪  Transitions, user feedback, interface design ▪  Web based tools ▪  UXpin - http://www.uxpin.com/ ▪  proto.io - http://www.proto.io/ ▪  Native tools ▪  Justinmind - http://www.justinmind.com/ ▪  Axure - http://www.axure.com/
  69. 69. Proto.io - http://www.proto.io/ ▪  Web based mobile prototyping tool ▪  Features ▪  Prototype for multiple devices ▪  Gesture input, touch events, animations ▪  Share with collaborators ▪  Test on device
  70. 70. Proto.io - Interface
  71. 71. Limitations ▪  No access to sensor data ▪  Camera, orientation sensor ▪  No multimedia playback ▪  Audio, video ▪  Simple transitions ▪  No conditional logic ▪  No networking
  72. 72. Wireframe vs.Prototype vs.Mockup • Wireframe •  Low fidelity representation of design •  What UI elements, where UI are placed • Mockup •  High fidelity static design •  Visual design draft • Prototype •  Medium to high fidelity •  Supports user interaction
  73. 73. Interactive Coding - Processing ▪ Programming tool for Artists/Designers ▪ http://processing.org ▪ Easy to code, Free, Open source, Java based ▪ 2D, 3D, audio/video support ▪ Processing For Android ▪ http://wiki.processing.org/w/Android ▪ Strong Android support, builds .apk file
  74. 74. Processing + Augmented Reality •  Support for Augmented Reality •  Camera input •  GPS/Sensor input •  3D Graphics •  Books •  Prototyping Augmented Reality (Mullins) •  Rapid Android Development (Sauter) •  Software •  Ketai Library - http://ketai.org/ •  Android library support – all sensors on Android devices
  75. 75. http://toxiclibs.org/
  76. 76. Other Tools ▪  Wireframing ▪  pidoco ▪  FluidUI ▪  Rapid Development ▪  Phone Gap ▪  AppMachine ▪  Interactive ▪  App Inventor ▪  WearScript
  77. 77. EXAMPLE: CONCEPT TO DEMO
  78. 78. NASA Hololens Concept Demo • Vision: Work on Mars from your office • Story and sketches based on vision • Led to working Demo
  79. 79. Chesley Bonestell (1940s)
  80. 80. Hololens Story
  81. 81. HoloLens Concept Sketches
  82. 82. Final NASA HoloLens OnSight Demo https://www.youtube.com/watch?v=o-GP3Kx6-CE
  83. 83. HARDWARE PROTOTYPING
  84. 84. Build Your Own Wearable ▪  MyVu display + phone + sensors
  85. 85. Beady-i ▪  http://www.instructables.com/id/DIY- Google-Glasses-AKA-the-Beady-i/
  86. 86. Physical Input Devices ▪  Can we develop unobtrusive input devices ? ▪  Reduce need for speech, touch pad input ▪  Socially more acceptable ▪  Examples ▪  Ring, ▪  pendant, ▪  bracelet, ▪  gloves, etc
  87. 87. Prototyping Platform Arduino Kit Bluetooth Shield Google Glass
  88. 88. Example: Glove Input ▪  Buttons on fingertips ▪  Map touches to commands
  89. 89. Example: Ring Input ▪  Touch strip, button, accelerometer ▪  Tap, swipe, flick actions
  90. 90. How it works Bracelet Armband Gloves 1,2, 3,4 Values/ output
  91. 91. CASE STUDY
  92. 92. ITERATIVE DESIGN PROCESS Design (redesign) Prototype Test Design with Users (Note problems) Evaluate (Fix Issues)
  93. 93. MOBILE AUGMENTED REALITY FOR SPATIAL NAVIGATION Sharon Brosnan Bachelor of Science in Digital Media Design http://www.sharonbrosnan.com/portfolio/mobile-augmented-reality.php
  94. 94. BUNRATTY FOLK PARK • Irish visitor attraction run by Shannon Heritage • 19th century life is recreated • Buildings from the mid-west have been relocated to the 26-land surrounding Bunratty Castle • 30 buildings are set in a rural or village setting there.
  95. 95. AUGMENTED REALITY In Bunratty Folk Park: •  Allows the visitor to point a camera at an exhibit, the device recognises its by it’s location and layers digital information on to the display •  3-dimensional virtual objects can be positioned with real ones on display •  Leads to dynamic combination of a live camera view and information
  96. 96. NAVIGATIONALAID Smartphone Platform Most people carry mobile phones Ideal Augmented Reality Technology •  Global tracking tools •  Wireless communication capabilities •  Location based computing •  Large display for interaction
  97. 97. DESIGNING FOR THEVISITOR
  98. 98. HUMAN CENTRED DESIGN Goal of the Navigational Aid •  Easy to use, clear and understandable •  Useful to visitors •  Creating interaction between the visitor and the aid through the user interface Engage the visitor •  It is necessary to understand the visitor to Bunratty •  Identify visitor motives and goals while going through the Folk Park.
  99. 99. HUMAN CENTRED DESIGN Understanding the technology and Related Works ! Literature ! Similar Technologies ! Electronic Tours in Museum Settings ! Interactions design in Outdoor Museums Understanding the User over time !  Observations !  Interviews
  100. 100. HUMAN CENTRED DESIGN Findings •  Most visitors do not use the map •  Most visitors have mobile phones •  Visitors want more information •  View the Folk Park at their own pace •  Information should be straight to the point •  Large social interaction within groups
  101. 101. FUNCTIONALITY • View Options •  CameraView •  MapView •  ListView • Sub-Options •  Places •  Events •  Restaurants ! AR Features ! Text Information ! 3D Objects ! 3D Tour Guide ! 3D Placement of Buildings
  102. 102. STORYBOARD
  103. 103. ITERATIVE DESIGN PROCESS
  104. 104. ITERATIVE DESIGN PROCESS Prototyping and User Testing •  Low Fidelity Prototyping • Sketches • Paper Prototyping • Post-It Prototyping • PowerPoint Prototyping •  High Fidelity Prototyping • Wikitude
  105. 105. INITIAL SKETCHES - Photoshop Pros: • Good for idea generation • Cheap • Concepts seem feasible Cons: • Not great feedback gained • Photoshop not fast enough for making changes
  106. 106. POST IT PROTOTYPING First DraJ Camera View with 3D Second DraJ Third DraJ •  Selec;on highlighted in blue •  Home bu)on added for easy naviga;on to main menu
  107. 107. POWERPOINT PROTOTYPING Benefits • Used for User Testing • Interactive • Functionalities work • Quick • Easy arrangement of slides User Testing • Participants found • 15 minute sessions screen captured • ‘Talk Allowed’ technique used • Notes taken • Post-Interview
  108. 108. WIKITUDE •  Popular AR browser •  Mapping •  Point of Interest abilities •  Multiplatform •  Shows points of interest •  Markers can be selected in and information pop-up appears
  109. 109. FINAL CONCEPT DESIGN
  110. 110. FINAL DESIGN CONCEPT Key Issues •  Fix issues found in previous sessions •  Design with guidelines in mind •  Appealing to the Mental Model •  Icon Design •  Aesthetic Design •  Colour/Font •  Buttons •  Look
  111. 111. VIDEO PROTOTYPE " Flexible tool for capturing the use of an interface " Elaborate simula;on of how the naviga;onal aid will work " Does not need to be realis;c in every detail " Gives a good idea of how the finished system will work
  112. 112. SUMMARY
  113. 113. Tom Chi’s Prototyping Rules 1.  Find the quickest path to experience 2.  Doing is the best kind of thinking 3.  Use materials that move at the speed of thought to maximize your rate of learning
  114. 114. Conclusions • Prototyping is one of the most important skills you can have as an ARinterface designer • There are many tools available: •  Sketching, Paper Prototyping, Video, etc • Always keep the user at the heart of the process •  User centered design •  User testing/feedback
  115. 115. Consider theWhole User
  116. 116. www.empathiccomputing.org @marknb00 mark.billinghurst@unisa.edu.au

×