The Glass Class: Rapid Prototyping for Wearable Computers

3,256 views

Published on

Talk that was to be given by Mark Billinghurst at the AWE New York conference on March 25th 2014. Unfortunately, due to technical problems this wasn't able to go ahead.

Published in: Technology

The Glass Class: Rapid Prototyping for Wearable Computers

  1. 1. The Glass Class: Rapid Prototyping for Wearable Computing Mark Billinghurst HIT Lab NZ University of Canterbury March 25th 2014
  2. 2. How can I quickly build my own wearable experiences?
  3. 3. Wearables  Vuzix M100  $999, profession  Recon Jet  $600, more sensors, sports  Opinvent  500 Euro, multi-view mode  Motorola Golden-i  Rugged, remote assistance
  4. 4. Common Elements  Different form factor but common elements  Android OS  Monocular Display  Camera  On-board sensors - Compass, Accelerometer, etc  Connectivity  Designed for wearability
  5. 5. WTF is This?
  6. 6. Rapid Prototyping for Non-Programmers
  7. 7. Interaction Design Process
  8. 8. Paper Prototyping: Widgets (Low Fidelity)
  9. 9. Tethered Prototypes (High Fidelity) Goal: Communicate Vision
  10. 10. Rapid Prototyping Approach  Quick design  Capture key interactions  Focus on user interface and experience  No programming required
  11. 11. BUNRATTY FOLK PARK Sharon Brosnan 0651869 Bachelor of Science in Digital Media Design
  12. 12. EXAMPLE: BUNRATTY PARK Irish visitor attraction run by Shannon Heritage 19th century life is recreated Buildings from the mid-west have been relocated to Bunratty Castle 30 buildings are set in a rural or village setting there.
  13. 13. AUGMENTED REALITY Want to develop AR tour guide Runs on smart phones Overlay virtual information on real objects Provides navigation aid Adds story/game elements
  14. 14. STORYBOARD
  15. 15. SKETCHES – PHOTOSHOP Pros: • Good for idea generation • Cheap • Concepts seem feasible Cons: • Not great feedback gained • Photoshop not fast enough for making changes
  16. 16. POST IT PROTOTYPING First Draft Camera View with 3D Second Draft Third Draft • Selection highlighted in blue • Home button added for easy navigation to main menu
  17. 17. POWERPOINT PROTOTYPING Benefits • Used for User Testing • Quick, Interactive • Functionalities work with storyboard • Easy arrangement of slides User Testing • Participants found • 15 minute sessions screen captured • ‘Talk Allowed’ technique used • Notes taken, Post-Interview
  18. 18. WIKITUDE Popular AR browser Mapping Point of Interest abilities Multiplatform Shows the points of interest of Bunratty Folk Park
  19. 19. Benefits of RAPID Prototyping  Fast and inexpensive  Identifies problems before they’re coded  Elicits more and better feedback from users  Helps developers think creatively  Gets users involved early in the process  Helps decide design directions
  20. 20. Rapid Prototyping Tools for Wearables
  21. 21. Rapid Prototyping Tools  Low-fi/Static Tools  Sketching  User interface templates  Storyboards/Application flows  High-fi/Interactive Tools  Wireframing tools  Mobile prototyping  Coding
  22. 22. GlassSim – http://glasssim.com/  Simulate the view through Google Glass  Multiple card templates
  23. 23. GlassSim Card Builder  Use HTML for card details  Multiple templates  Change background  Own image  Camera view
  24. 24. GlassSim Samples
  25. 25. Glass UI Templates  Google Glass Photoshop Templates  http://glass-ui.com/  http://dsky9.com/glassfaq/the-google-glass-psd- template/
  26. 26. Sample Slides From Templates
  27. 27. Glass Application Storyboard  http://dsky9.com/glassfaq/google-glass- storyboard-template-download/
  28. 28. Application Flow
  29. 29. Wireframing Tool  Add transitions, limited interactions  UXPin Wireframing Tool  http://uxpin.com/  Web based  Glass templates
  30. 30. UXpin
  31. 31. Proto.io - http://www.proto.io/  Web based mobile prototyping tool  Features  Prototype for multiple devices  Gesture input, touch events, animations  Share with collaborators  Test on device
  32. 32. Proto.io - Interface
  33. 33. Demo Building a Simple Flow
  34. 34. Gesture Flow Scr1 Scr2 Scr3 Scr4 Scr5 Scr6 Tap Swipe
  35. 35. Start Transitions
  36. 36. Demo
  37. 37. Justinmind – justinmind.com  Wireframing tool – interactive prototypes  Google Glass widget library
  38. 38. Wireframe Limitations  Can’t deploy on Wearable  No access to sensor data  Camera, orientation sensor  No multimedia playback  Audio, video  Simple transitions  No conditional logic
  39. 39. App Inventor  http://appinventor.mit.edu/  Visual Programming for Android Apps  Features  Access to Android Sensors  Multimedia output  Drag and drop web based interface  Designer view – app layout  Blocks view – program logic/control
  40. 40. Device Setup  Emulator  Use aiStarter on Windows/Linux  Live view (run on connected device)  Make sure device is in Debug mode  Install companion app (sideload on Glass) - MITAI2Companion.apk
  41. 41. App Inventor Designer View
  42. 42. App Inventor Blocks View
  43. 43. Orientation Demo  Use orientation sensor
  44. 44. Rapid Prototyping using Processing
  45. 45. Processing  Easy to use language for Interaction  Programming tool for Artists/Designers  http://processing.org  Easy to code, Free, Open source, Java based  2D, 3D, audio/video support  Processing For Android  http://wiki.processing.org/w/Android  Generates Android Ready .apk file
  46. 46. Development Environment
  47. 47. Hello World //called initially at the start of the Processing sketch void setup() { size(640, 360); background(0); } //called every frame to draw output void draw() { background(0); //draw a white text string showing Hello World fill(255); text("Hello World", 50, 50); }
  48. 48. Demo
  49. 49. Hello World Image PImage img; // Create an image variable void setup() { size(640, 360); //load the ok glass home screen image img = loadImage("okGlass.jpg"); // Load the image into the program } void draw() { // Displays the image at its actual size at point (0,0) image(img, 0, 0); }
  50. 50. Demo
  51. 51. Touch Pad Input  Tap recognized as DPAD input void keyPressed() { if (key == CODED){ if (keyCode == DPAD) { // Do something ..  Java code to capture rich motion events  import android.view.MotionEvent;
  52. 52. Motion Event //Glass Touch Events - reads from touch pad public boolean dispatchGenericMotionEvent(MotionEvent event) { float x = event.getX(); // get x/y coords float y = event.getY(); int action = event.getActionMasked(); // get code for action switch (action) { // let us know which action code shows up case MotionEvent.ACTION_DOWN: touchEvent = "DOWN"; fingerTouch = 1; break; case MotionEvent.ACTION_MOVE: touchEvent = "MOVE"; xpos = myScreenWidth-x*touchPadScaleX; ypos = y*touchPadScaleY; break;
  53. 53. Demo
  54. 54. Sensors  Ketai Library for Processing  https://code.google.com/p/ketai/  Support all phone sensors  GPS, Compass, Light, Camera, etc  Include Ketai Library  import ketai.sensors.*;  KetaiSensor sensor;
  55. 55. Using Sensors  Setup in Setup( ) function  sensor = new KetaiSensor(this);  sensor.start();  sensor.list();  Event based sensor reading void onAccelerometerEvent(…) { accelerometer.set(x, y, z); }
  56. 56. Sensor Demo
  57. 57. PhoneGap - http://phonegap.com/  Developing mobile apps using web tools  HTML, CSS, JavaScript  Access to sensor data  Camera, compass, accelerometer  Cross platform  iOS, Android, Windows Phone, etc
  58. 58. WearScript – wearscript.com  JavaScript for Glass  Access to sensors, touch input, etc  Power of Android, Ease of website development
  59. 59. Design Guidelines
  60. 60. "Computing should just be more comfortable" "Google should do the hard work, and you should have a chance to live, have a good life, and get on with it."
  61. 61. As technology becomes more personal and immediate, it can start to disappear. Distant Intimate
  62. 62. Last year Last week NowForever The Now machine Focus on location, contextual and timely information, and communication.
  63. 63. 1. Design For the Device  Simple, relevant information  Complement existing devices
  64. 64. 2. Don’t Get in the Way  Enhance, not replace, real world interaction
  65. 65. 3. Keep it Relevant  Information at the right time and place
  66. 66. 4. Avoid the Unexpected  Don’t send unexpected content at wrong times  Make it clear to users what your glassware
  67. 67. 5. Build for People  Use imagery, voice interaction, natural gestures  Focus on fire and forget interaction model
  68. 68. Micro Interactions The position of the display and limited input ability makes longer interactions less comfortable. Using it shouldn’t take longer than taking out your phone.
  69. 69. It's like a rear view mirror Don't overload the user. Stick to the absolutely essential, avoid long interactions. Be explicit.
  70. 70. Examples
  71. 71. Virtual Exercise Companion  GlassFitGames  http://www.glassfitgames.com
  72. 72. GlassFitGames Video
  73. 73. CityViewAR  Using AR to visualize Christchurch city buildings  3D models of buildings, 2D images, text, panoramas  AR View, Map view, List view
  74. 74. CityViewAR on Glass  AR overlay of virtual buildings in Christchurch
  75. 75. CItyViewAR on Glass Demo
  76. 76. Conclusions
  77. 77. Conclusions  Rapid prototyping vital for wearables  Display concepts, user feedback  Range of different tools available  Sketching, storyboarding, wireframe, interactive  Need to follow design guidelines  Don’t get in the way, build for people
  78. 78. Books  Programming Google Glass  Eric Redmond  Rapid Android Development: Build Rich, Sensor-Based Applications with Processing  Daniel Sauter  Building Android Apps in Easy Steps: Using App Inventor  Mike McGrath
  79. 79. Web Resources  Main Developer Website  https://developers.google.com/glass/  Glass Apps Developer Site  http://glass-apps.org/glass-developer  Google Glass Emulator  http://glass-apps.org/google-glass-emulator  AR for Glass Website  http://www.arforglass.org/
  80. 80. More Information  Mark Billinghurst  Email: mark.billinghurst@hitlabnz.org  Twitter: @marknb00  HIT Lab NZ  http://www.hitlabnz.org/

×