Talk that was to be given by Mark Billinghurst at the AWE New York conference on March 25th 2014. Unfortunately, due to technical problems this wasn't able to go ahead.

1. The Glass Class: Rapid
Prototyping for Wearable
Computing
Mark Billinghurst
HIT Lab NZ
University of Canterbury
March 25th 2014

6. How can I quickly build my own
wearable experiences?

7. Wearables
 Vuzix M100
 $999, profession
 Recon Jet
 $600, more sensors, sports
 Opinvent
 500 Euro, multi-view mode
 Motorola Golden-i
 Rugged, remote assistance

8. Common Elements
 Different form factor but common
elements
 Android OS
 Monocular Display
 Camera
 On-board sensors
- Compass, Accelerometer, etc
 Connectivity
 Designed for wearability

9. WTF is This?

10. Rapid Prototyping for
Non-Programmers

11. Interaction Design Process

12. Paper Prototyping: Widgets (Low Fidelity)

13. Tethered Prototypes (High Fidelity)
Goal:
Communicate Vision

14. Rapid Prototyping Approach
 Quick design
 Capture key interactions
 Focus on user interface and experience
 No programming required

15. BUNRATTY FOLK PARK
Sharon Brosnan
0651869
Bachelor of Science in Digital Media Design

16. EXAMPLE: BUNRATTY PARK
Irish visitor attraction run by Shannon
Heritage
19th century life is recreated
Buildings from the mid-west have been
relocated to Bunratty Castle
30 buildings are set in a rural or village
setting there.

17. AUGMENTED REALITY
Want to develop AR tour guide
Runs on smart phones
Overlay virtual information on real objects
Provides navigation aid
Adds story/game elements

18. STORYBOARD

19. SKETCHES – PHOTOSHOP
Pros:
• Good for idea generation
• Cheap
• Concepts seem feasible
Cons:
• Not great feedback gained
• Photoshop not fast enough
for making changes

20. POST IT PROTOTYPING
First Draft
Camera View with 3D
Second Draft Third Draft
• Selection highlighted in blue • Home button added for easy
navigation to main menu

21. POWERPOINT
PROTOTYPING
Benefits
• Used for User Testing
• Quick, Interactive
• Functionalities work with storyboard
• Easy arrangement of slides
User Testing
• Participants found
• 15 minute sessions screen captured
• ‘Talk Allowed’ technique used
• Notes taken, Post-Interview

22. WIKITUDE
Popular AR browser
Mapping
Point of Interest abilities
Multiplatform
Shows the points of
interest of Bunratty Folk
Park

23. Benefits of RAPID
Prototyping
 Fast and inexpensive
 Identifies problems before they’re
coded
 Elicits more and better feedback from
users
 Helps developers think creatively
 Gets users involved early in the process
 Helps decide design directions

24. Rapid Prototyping Tools for
Wearables

25. Rapid Prototyping Tools
 Low-fi/Static Tools
 Sketching
 User interface templates
 Storyboards/Application flows
 High-fi/Interactive Tools
 Wireframing tools
 Mobile prototyping
 Coding

26. GlassSim – http://glasssim.com/
 Simulate the view through Google Glass
 Multiple card templates

27. GlassSim Card Builder
 Use HTML for card details
 Multiple templates
 Change background
 Own image
 Camera view

28. GlassSim Samples

29. Glass UI Templates
 Google Glass Photoshop Templates
 http://glass-ui.com/
 http://dsky9.com/glassfaq/the-google-glass-psd-
template/

30. Sample Slides From Templates

31. Glass Application Storyboard
 http://dsky9.com/glassfaq/google-glass-
storyboard-template-download/

32. Application Flow

33. Wireframing Tool
 Add transitions, limited interactions
 UXPin Wireframing Tool
 http://uxpin.com/
 Web based
 Glass templates

34. UXpin

35. Proto.io - http://www.proto.io/
 Web based mobile prototyping tool
 Features
 Prototype for multiple devices
 Gesture input, touch events, animations
 Share with collaborators
 Test on device

36. Proto.io - Interface

37. Demo Building a Simple Flow

38. Gesture Flow
Scr1
Scr2 Scr3
Scr4 Scr5 Scr6
Tap
Swipe

39. Start Transitions

40. Demo

41. Justinmind – justinmind.com
 Wireframing tool – interactive prototypes
 Google Glass widget library

42. Wireframe Limitations
 Can’t deploy on Wearable
 No access to sensor data
 Camera, orientation sensor
 No multimedia playback
 Audio, video
 Simple transitions
 No conditional logic

43. App Inventor
 http://appinventor.mit.edu/
 Visual Programming for Android Apps
 Features
 Access to Android Sensors
 Multimedia output
 Drag and drop web based interface
 Designer view – app layout
 Blocks view – program logic/control

44. Device Setup
 Emulator
 Use aiStarter on Windows/Linux
 Live view (run on connected device)
 Make sure device is in Debug mode
 Install companion app (sideload on Glass)
- MITAI2Companion.apk

45. App Inventor Designer View

46. App Inventor Blocks View

47. Orientation Demo
 Use orientation sensor

48. Rapid Prototyping using
Processing

49. Processing
 Easy to use language for Interaction
 Programming tool for Artists/Designers
 http://processing.org
 Easy to code, Free, Open source, Java based
 2D, 3D, audio/video support
 Processing For Android
 http://wiki.processing.org/w/Android
 Generates Android Ready .apk file

52. Development Environment

53. Hello World
//called initially at the start of the Processing sketch
void setup() {
size(640, 360);
background(0);
}
//called every frame to draw output
void draw() {
background(0);
//draw a white text string showing Hello World
fill(255);
text("Hello World", 50, 50);
}

54. Demo

55. Hello World Image
PImage img; // Create an image variable
void setup() {
size(640, 360);
//load the ok glass home screen image
img = loadImage("okGlass.jpg"); // Load the image into the program
}
void draw() {
// Displays the image at its actual size at point (0,0)
image(img, 0, 0);
}

56. Demo

57. Touch Pad Input
 Tap recognized as DPAD input
void keyPressed() {
if (key == CODED){
if (keyCode == DPAD) {
// Do something ..
 Java code to capture rich motion events
 import android.view.MotionEvent;

58. Motion Event
//Glass Touch Events - reads from touch pad
public boolean dispatchGenericMotionEvent(MotionEvent event) {
float x = event.getX(); // get x/y coords
float y = event.getY();
int action = event.getActionMasked(); // get code for action
switch (action) { // let us know which action code shows up
case MotionEvent.ACTION_DOWN:
touchEvent = "DOWN";
fingerTouch = 1;
break;
case MotionEvent.ACTION_MOVE:
touchEvent = "MOVE";
xpos = myScreenWidth-x*touchPadScaleX;
ypos = y*touchPadScaleY;
break;

59. Demo

60. Sensors
 Ketai Library for Processing
 https://code.google.com/p/ketai/
 Support all phone sensors
 GPS, Compass, Light, Camera, etc
 Include Ketai Library
 import ketai.sensors.*;
 KetaiSensor sensor;

61. Using Sensors
 Setup in Setup( ) function
 sensor = new KetaiSensor(this);
 sensor.start();
 sensor.list();
 Event based sensor reading
void onAccelerometerEvent(…)
{
accelerometer.set(x, y, z);
}

62. Sensor Demo

63. PhoneGap -
http://phonegap.com/
 Developing mobile apps using web tools
 HTML, CSS, JavaScript
 Access to sensor data
 Camera, compass, accelerometer
 Cross platform
 iOS, Android, Windows Phone, etc

64. WearScript – wearscript.com
 JavaScript for Glass
 Access to sensors, touch input, etc
 Power of Android, Ease of website
development

65. Design Guidelines

67. "Computing
should just be
more comfortable"
"Google should do the hard
work, and you should have a
chance to live, have a good
life, and get on with it."

68. As technology becomes more
personal and immediate, it can
start to disappear.
Distant Intimate

69. Last year Last week NowForever
The Now machine
Focus on location, contextual
and timely information, and
communication.

70. 1. Design For the Device
 Simple, relevant information
 Complement existing devices

71. 2. Don’t Get in the Way
 Enhance, not replace, real world
interaction

72. 3. Keep it Relevant
 Information at the right time and place

73. 4. Avoid the Unexpected
 Don’t send unexpected content at wrong
times
 Make it clear to users what your glassware

74. 5. Build for People
 Use imagery, voice interaction, natural
gestures
 Focus on fire and forget interaction model

75. Micro
Interactions
The position of the display and
limited input ability makes
longer interactions less
comfortable.
Using it shouldn’t take longer
than taking out your phone.

76. It's like a rear view mirror
Don't overload the user. Stick to the
absolutely essential, avoid long
interactions. Be explicit.

77. Examples

78. Virtual Exercise Companion
 GlassFitGames
 http://www.glassfitgames.com

79. GlassFitGames Video

80. CityViewAR
 Using AR to visualize Christchurch city
buildings
 3D models of buildings, 2D images, text,
panoramas
 AR View, Map view, List view

81. CityViewAR on Glass
 AR overlay of virtual buildings in
Christchurch

82. CItyViewAR on Glass Demo

83. Conclusions

84. Conclusions
 Rapid prototyping vital for wearables
 Display concepts, user feedback
 Range of different tools available
 Sketching, storyboarding, wireframe, interactive
 Need to follow design guidelines
 Don’t get in the way, build for people

85. Books
 Programming Google Glass
 Eric Redmond
 Rapid Android Development:
Build Rich, Sensor-Based
Applications with Processing
 Daniel Sauter
 Building Android Apps in Easy
Steps: Using App Inventor
 Mike McGrath

86. Web Resources
 Main Developer Website
 https://developers.google.com/glass/
 Glass Apps Developer Site
 http://glass-apps.org/glass-developer
 Google Glass Emulator
 http://glass-apps.org/google-glass-emulator
 AR for Glass Website
 http://www.arforglass.org/

87. More Information
 Mark Billinghurst
 Email: mark.billinghurst@hitlabnz.org
 Twitter: @marknb00
 HIT Lab NZ
 http://www.hitlabnz.org/