Mobile Interaction Based on Human Gesture Analysis Ricardo Gamboa Tiago Guerreiro, Joaquim Jorge {rjssg,tjvg,jaj}@immi.ine...
Actual Interaction Successful & Appropriate?
Screen and Keypad size are  Limited
Multi-Task Graphical-based devices
Slow and Visually Demanding Interaction Mobility Issues!
Possible solution Key Shortcuts
Memory Issues – Where is each shortcut?
Other Solution Voice shortcuts
Voice Recognition  Issues
Low  Social Acceptance
Proposed Solution Gestural Interface!
Gestures ease  Communication
Task Analysis  -   User observation   Actual panorama on mobile usage and shortcutting habits. Results proved: Mobile Inte...
Propposed Approach –   Mnemonical Body Shortcuts “ Phone in Silence Mode”
Propposed Approach –   Mnemonical Body Shortcuts “ Phone your Girlfriend”
Related Work RFID  Accelerometers  EMG  Cameras   Touch Screens
RFID Prototype Pocket LOOX 720 ACG RF  RFID reader RFID Tags
Evaluation
Mnemonical Body Shortcuts  Evaluation Mouth Hand Chest Head Wrist Eye Finger Ear SMS 10 1 6 Call 3 1 12 Contacts 3 5 2 1 C...
20   Users 5   chosen Applications 20  Shortcuts Key Shortcuts Vs Mnemonics 94%  Recognition Rate
Accelerometer ADXL 330 MEMS Bioplux4 Device
Why Accelerometers?
Accelerometer Applicable in  Contextual  &  Explicit   Human Motion
Contextual Interaction
Stopped Holding Picked Walking Running Stopped Movement Analysis - Amplitude Holding Time (s) Acc (g) Amp = √( x^2+y^2+z^2 )
Fall Detection
Explicit Interaction Explicit Interaction
Mnemonical Body Shortcuts X Y
Mnemonical Body Shortcuts Position – Y axis Acceleration Calibrated Acceleration Thresholded
Mnemonical Body Shortcuts Position – Y axis ∫ dt ∫ dt Velocity Position
Mnemonical Body Shortcuts Position + Final Rotation
Mnemonical Body Shortcuts Evaluation – Default Gestures TOTAL RECOGNITION  82% Mouth  -  85% Chest –  85% Navel –  90% Sho...
Mnemonical Body Shortcuts Evaluation – Trainable Gestures TOTAL RECOGNITION  71% 10  users –  5  Gestures – 5x Train for e...
Mnemonical Body Shortcuts Discussion <ul><li>Default Gestures  </li></ul><ul><ul><li>Good Recognition Rate. </li></ul></ul...
Tilt – Angle Calculation Time (s/1000) Angle (degrees) X Y Z
Tilt  - Centralization and Joining Y & Z X Y + Z Angle (degrees)  - start position variation Time (s/1000)
Tilt  - Thresholding X Y + Z Angle (degrees)  - start position variation Time (s/1000)
Tilt  - Decision LEFT RIGHT UP DOWN X Y + Z Time (s/1000) Angle (degrees)
Tilt Evaluation
Tilt Results - Recognition LEFT TILT 85% RIGHT TILT 94% UP TILT 75% DOWN TILT 93% TOTAL RECOGNITION  86%
Future Work <ul><li>Mobility tests on Mnemonical Body Shortcuts </li></ul><ul><li>Try a Feature Based Algorithm  </li></ul...
Upcoming SlideShare
Loading in...5
×

Mobile Interaction Based on Human Gesture Analysis

3,590

Published on

Details in:
http://m-accessibility.blogspot.com

This presentation is focused on mnemonical body shortcuts (check http://immi.inesc-id.pt/~tjvg/ for Publications on the subject). In the presentation we detail information we can obtain from an accelerometer and how it can be used to improve mobile device interaction.

The presentation was performed at ISHF 2007.

Published in: Business, Technology
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,590
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
14
Embeds 0
No embeds

No notes for slide

Transcript of "Mobile Interaction Based on Human Gesture Analysis"

  1. 1. Mobile Interaction Based on Human Gesture Analysis Ricardo Gamboa Tiago Guerreiro, Joaquim Jorge {rjssg,tjvg,jaj}@immi.inesc-id.pt Hugo Gamboa [email_address]
  2. 2. Actual Interaction Successful & Appropriate?
  3. 3. Screen and Keypad size are Limited
  4. 4. Multi-Task Graphical-based devices
  5. 5. Slow and Visually Demanding Interaction Mobility Issues!
  6. 6. Possible solution Key Shortcuts
  7. 7. Memory Issues – Where is each shortcut?
  8. 8. Other Solution Voice shortcuts
  9. 9. Voice Recognition Issues
  10. 10. Low Social Acceptance
  11. 11. Proposed Solution Gestural Interface!
  12. 12. Gestures ease Communication
  13. 13. Task Analysis - User observation Actual panorama on mobile usage and shortcutting habits. Results proved: Mobile Interaction is “keystroke consuming” Shortcuts are ineffective.
  14. 14. Propposed Approach – Mnemonical Body Shortcuts “ Phone in Silence Mode”
  15. 15. Propposed Approach – Mnemonical Body Shortcuts “ Phone your Girlfriend”
  16. 16. Related Work RFID Accelerometers EMG Cameras Touch Screens
  17. 17. RFID Prototype Pocket LOOX 720 ACG RF RFID reader RFID Tags
  18. 18. Evaluation
  19. 19. Mnemonical Body Shortcuts Evaluation Mouth Hand Chest Head Wrist Eye Finger Ear SMS 10 1 6 Call 3 1 12 Contacts 3 5 2 1 Clock 10 1 Photos 2 8 Calculator 3 Mp3 2 Agenda 1 3 1 Alarm-clock 2 2 2 3
  20. 20. 20 Users 5 chosen Applications 20 Shortcuts Key Shortcuts Vs Mnemonics 94% Recognition Rate
  21. 21. Accelerometer ADXL 330 MEMS Bioplux4 Device
  22. 22. Why Accelerometers?
  23. 23. Accelerometer Applicable in Contextual & Explicit Human Motion
  24. 24. Contextual Interaction
  25. 25. Stopped Holding Picked Walking Running Stopped Movement Analysis - Amplitude Holding Time (s) Acc (g) Amp = √( x^2+y^2+z^2 )
  26. 26. Fall Detection
  27. 27. Explicit Interaction Explicit Interaction
  28. 28. Mnemonical Body Shortcuts X Y
  29. 29. Mnemonical Body Shortcuts Position – Y axis Acceleration Calibrated Acceleration Thresholded
  30. 30. Mnemonical Body Shortcuts Position – Y axis ∫ dt ∫ dt Velocity Position
  31. 31. Mnemonical Body Shortcuts Position + Final Rotation
  32. 32. Mnemonical Body Shortcuts Evaluation – Default Gestures TOTAL RECOGNITION 82% Mouth - 85% Chest – 85% Navel – 90% Shoulder - 75% Neck – 100% Ear - 60% Head – 85% Leg – 80% Wrist – 85% Eye – 75% 10 users – 5 Gestures – No training – 20 Recognitions each
  33. 33. Mnemonical Body Shortcuts Evaluation – Trainable Gestures TOTAL RECOGNITION 71% 10 users – 5 Gestures – 5x Train for each gesture – 20 Recognitions Leg - 85% Mouth – 94% Navel – 64% Neck – 95% Ear - 55% 5 most common gestures
  34. 34. Mnemonical Body Shortcuts Discussion <ul><li>Default Gestures </li></ul><ul><ul><li>Good Recognition Rate. </li></ul></ul><ul><ul><li>Limited to 10 pre-defined gestures. </li></ul></ul><ul><ul><li>Users have to learn the gestures. </li></ul></ul><ul><li>Treinable gestures </li></ul><ul><ul><li>Lower Recognition Rate – Similar gestures are choosen. </li></ul></ul><ul><ul><li>Position isn’t very effective to desambiguate gestures outside x,y plan. </li></ul></ul><ul><ul><li>One training error spoils the recognition – outlier detection is needed. </li></ul></ul>
  35. 35. Tilt – Angle Calculation Time (s/1000) Angle (degrees) X Y Z
  36. 36. Tilt - Centralization and Joining Y & Z X Y + Z Angle (degrees) - start position variation Time (s/1000)
  37. 37. Tilt - Thresholding X Y + Z Angle (degrees) - start position variation Time (s/1000)
  38. 38. Tilt - Decision LEFT RIGHT UP DOWN X Y + Z Time (s/1000) Angle (degrees)
  39. 39. Tilt Evaluation
  40. 40. Tilt Results - Recognition LEFT TILT 85% RIGHT TILT 94% UP TILT 75% DOWN TILT 93% TOTAL RECOGNITION 86%
  41. 41. Future Work <ul><li>Mobility tests on Mnemonical Body Shortcuts </li></ul><ul><li>Try a Feature Based Algorithm </li></ul><ul><li>Design a full prototype </li></ul><ul><li>( + feedback and shortcuts) </li></ul>

×