Exemplar: Designing Sensor-based interactions by demonstration... (a CHI2007 Best Paper)

6,317 views
6,166 views

Published on

Authoring Sensor-based Interactions by Demonstration with Direct Manipulation and Pattern Recognition

Björn Hartmann, Leith Abdulla, Manas Mittal, Scott Klemmer

Contributes method and tool for rapidly designing sensor-based interactions by demonstration; emphasizes control of generalization criteria through integrating direct manipulation and pattern recognition; offers theoretical and first-use lab evaluations.

Published in: Economy & Finance, Technology
1 Comment
6 Likes
Statistics
Notes
  • Excellent presentation. I have taken a number of the structure graphics and adapted to my startup
    Teisha
    http://dashinghealth.com http://healthimplants.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
6,317
On SlideShare
0
From Embeds
0
Number of Embeds
44
Actions
Shares
0
Downloads
1
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

Exemplar: Designing Sensor-based interactions by demonstration... (a CHI2007 Best Paper)

  1. 1. Authoring Sensor-Based Interactions by Demonstration with Direct Manipulation and Pattern Recognition Björn Hartmann, Leith Abdulla Manas Mittal, Scott R. Klemmer 1
  2. 2. How would you prototype… …a workout monitoring system? [Apple, Nike] 2
  3. 3. How would you explore… …motion-based game controllers? [Nintendo] 3
  4. 4. How would you build… …a proximity-aware whiteboard? [Ju, Lee, Akers] 4
  5. 5. Sensor-based Interaction Design 5
  6. 6. d.tools [Hartmann et al., UIST ’06] 6
  7. 7. d.Tools d.tools Student Project 7
  8. 8. Designing Sensor-based Interactions PROTOTYPE APPLICATION LOGIC SPECIFY RELATIONSHIP BETWEEN SENSOR DATA AND APPLICATION LOGIC PROVIDE SOFTWARE ABSTRACTION FOR HARDWARE 8
  9. 9. Types of Sensors Force Sensitive Resistors Accelerometers RFID IR/Ultrasonic Rangers Phototransistors Discrete Continuous Categorical 9
  10. 10. Current Tools: Oscilloscopes [CC image by Timo Arnall] 10
  11. 11. Current Tools: LabView LabView 11
  12. 12. Representation Matters Accelerometer X axis Accelerometer Y axis //detect accelerometer peaks //read data sample xVal[t++]=readA2DValue(xPin); //look for changes in derivative if(((xVal[t]-xVal[t-1]) >= 0 && (xVal[t-1]-xVal[t-2]) < 0) (((xVal[t]-xVal[t-1]) < 0 && (xVal[t-1]-xVal[t-2]) >= 0 //peak detected //send message oscSendMessageInt("/x/peak",1); } else { //no peak } 12
  13. 13. Idea: Programming by Demonstration 13
  14. 14. Idea: Programming by Demonstration 14
  15. 15. Tacit Knowledge 15
  16. 16. Crux: Generalization 16
  17. 17. Crux: Generalization 17
  18. 18. Crux: Generalization 18
  19. 19. Exemplar 19
  20. 20. Exemplar 20
  21. 21. Exemplar 21
  22. 22. Exemplar 22
  23. 23. 23
  24. 24. Exemplar UI 24
  25. 25. Exemplar UI 25
  26. 26. Exemplar UI 26
  27. 27. Exemplar UI 27
  28. 28. Exemplar UI 28
  29. 29. Exemplar UI 29
  30. 30. Exemplar UI 30
  31. 31. [Flash animations from the talk are not included in this presentation] 31
  32. 32. Dynamic Time Warping Demonstration Signal Matching Input Signal [Sakoe, H. Chiba, S. ‘78] 32
  33. 33. Filters Offset Y-Axis Scaling Smoothing Rate of Change 33
  34. 34. Extensible Filter Set 34
  35. 35. Understanding the Exemplar Approach What? Why? Cognitive Dimensions of Analysis of Exemplar as a Notation (CDN) Inspection visual authoring environment First-Use Laboratory Study Threshold and usability Class Deployment Real-world stress test Used as Design Tools for Complexity ceiling for “Building Upon Everyday Play” knowledgeable users CHI Interactivity Exhibit 35
  36. 36. Lab Study [sketches by Wendy Ju] 36
  37. 37. Lab Study 26 min 18 min 27 min 47 min 22 min 31 min [sketches by Wendy Ju] 37
  38. 38. 38
  39. 39. Mini Joystick Bend Sensor taped to table Navigation Navigation Test Play with Accelerometer with XY Joystick 39
  40. 40. Mini Joystick Bend Sensor taped to table Trigger with Test Play Refine trigger Test Play Bend sensor detection 40
  41. 41. Post-Test Questionnaire 41
  42. 42. Post-Test Questionnaire 42
  43. 43. CHI Interactivity [Control Freaks by Haiyan Zhang] 43
  44. 44. CHI Interactivity Compulsory latency of pattern matching precludes use for fast-paced actions Threshold Pattern crossed here matched here time [Control Freaks by Haiyan Zhang] 44
  45. 45. Related Work PBD Tools Musical controller Commercial for Ubicomp Design tools Dsp software a Capella HID Toolkit LabView/ [Dey et al., CHI 04] [Steiner, NIME 05] Lego Mindstorms NXT Crayons MnM [Fails & Olsen, CHI 03] [Bevilacqua et al., NIME 05] Monet FlexiGesture [Li, Landay, UIST 05] [Merrill & Paradiso, CHI 05] Papier-Mâché [Klemmer et al., CHI 04] 45
  46. 46. Related Work A CAPella (Dey et al., CHI 04) 46
  47. 47. Related Work FlexiGesture (Merrill CHI05) 47
  48. 48. Related Work LabView 48
  49. 49. Related Work Lego Mindstorms NXT 49
  50. 50. Future Directions Export: Generate code + specifications Programming sensor networks by demonstration d.mix: Programming by A Sample for Web Service APIs Integration of direct manipulation environments with textual programming 50
  51. 51. Acknowledgments We thank MediaX/DNP for funding, Intel for equipment donation, Wendy Ju for illustrations, David Merrill & Timo Arnall for photos. 51
  52. 52. Download Exemplar: http://hci.stanford.edu/exemplar CHI Interactivity “Building Upon Everyday Play” Talk tomorrow, 11:30am, Room C2 52

×