Gestures For Mobile

12,117 views
11,913 views

Published on

Slides from workshop at Design For Mobile. We look at gestures at, for, on, and with mobiles.

Published in: Design, Technology, Business

Gestures For Mobile

  1. 1. Gesture Design for (and with) Mobiles Design For Mobile April 20, 2009 Lawrence, KS
  2. 2. Who am I? Barbara Ballard Principal at Little Springs Design Curator of Design for Mobile MBA from the University of Kansas Author of Designing the Mobile User Experience Mobilist 2
  3. 3. Setting the scene Culture Technology 3
  4. 4. Minority Report 4
  5. 5. Inspired 5
  6. 6. Jeff Han multi-touch 6
  7. 7. What are gestures? Kinesthetic Communicative Inherent meaning to others Intentional or not 7
  8. 8. Gestures & language Key part of language Part of speech production Cognitive representation Unique to speaker Viewers absorb meaning 8
  9. 9. Why gestures now? Everybody is blind sometimes 9
  10. 10. Multi-touch 10
  11. 11. Efficient interaction Mouse click: x, y, t Key click: i, t Mouse move: x, x’, y, y’, t, t’ Touch screen swipe: x, x’, y, y’, t, t’ and path Human hand: indefinite, 3D Gesture: meaning Richer interaction. 11
  12. 12. Path? So? Swype Shapewriter 12
  13. 13. Gestures & brand Iconographic gestures Branding without product WWE fans? West Ham? Protest? 13
  14. 14. Mobile gesture taxonomy (Human to environment) Device to environment Between devices At device With device On device 15
  15. 15. ASL sign taxonomy Transparent: guessable Translucent: explainable Opaque: inexplicable 16
  16. 16. TAT gesture taxonomy Iconic Metaphoric Symbolic Transparent Translucent Opaque Instantly Advanced users Easily understood remembered Separate from UI Slow Depends on UI Difficult to Context elements remember dependent Can be used to Feedback useful Relies on discover new fn Magic quality affordances Gesture is the Gesture is an Gesture is its own meaning enabler layer 17
  17. 17. Mobile gesture taxonomy (Human to environment) Device to environment Between devices At device With device On device 18
  18. 18. Human to environment 19
  19. 19. Human to environment 20
  20. 20. Human to environment Canesta Kiosk systems Museum displays No mobile phone! 21
  21. 21. Device to environment Text 22
  22. 22. Device to environment 23
  23. 23. Available technologies Challenge: device must communicate with environment NFC, RFID: rare, forthcoming Bluetooth: common, somewhat scary to many Wi-Fi: small number of devices ... iPhone, Android What can we do with this, now? 24
  24. 24. Between devices Information transfer Bluetooth, network, NFC Security issues Future interaction 25
  25. 25. Between devices Phone locked, loud when in purse? 26
  26. 26. At device Camera input 27
  27. 27. At device Sonic input 28
  28. 28. With device Accelerometers 29
  29. 29. With device Apple patent on changing UI when vibrating 30
  30. 30. With device Pressure sensors recognize hand grip 31
  31. 31. With device Shake Tilt Rotate Marble game Koi pond Driving game 32
  32. 32. On device Behind screen 33
  33. 33. On device Eyes-free touch dialing 5 is where finger hits 34
  34. 34. On device 35
  35. 35. On device Flick Swipe Pinch Consider OS norms Patents 36
  36. 36. Prototyping & Research 37
  37. 37. Design issues Human factors Industrial design Visual design 38
  38. 38. Technologies Camera Accelerometers Touch screen Multi-touch NFC, RFID, Bluetooth Pressure sensor Light sensor Platform!! 39
  39. 39. Rules for design (Saffer) Discoverable Trustworthy Responsive Appropriate Meaningful Smart Clever Playful Pleasurable Good 40
  40. 40. Goals for design Sharable Replaceable Memorable Symmetrical Responsive HF principles 41
  41. 41. Responsive Interaction must feel smooth to achieve trust Focus on apps Visual, auditory, tactile Immediate 42
  42. 42. Anatomy of gesture (Saffer) Presence Duration Position Motion Pressure Size Orientation Other objects Combination of touch points Sequence (# participants) 43
  43. 43. Detecting context Sensors detect Movement Light etc. Change UI 44
  44. 44. How many gestures? Motivation Predictability Affordances Practice 3 (pinch, swipe, flick) 1,000 (ASL) 45
  45. 45. Motivation Ekapad 12-key, 3-finger chording 46
  46. 46. or Alt P & RELEASE to R hand L hand Motivation AltR or activate commands CapsShift qwerty right side !R or CtrlR OptionR ctrlR or wwR ShiftR EkaTetra Cheat Sheet v 5.7 Display.pdf ekatetra.com Copyright © EkaTetra 2008 All rights reserved 2 x-ray views Start here P&R = Press & Release P&R boxed chord 1st - then P&R Caps if shown For a chorded P&R Caps P&R chord character 1 = + P&R with symbol % 2 Every command, prefix, & character (except one finger 1 alphas and numerals) is available whatever the state. x-ray views are thumb side views of keys & chords. Right hand sot aei fk b m zy qu hrn gjw pdl vcx . period Space Return Tab Left hand Pressing any key completes and/or ends a command action uq P&R Alpha NonBreak , comma Space Enter esc to return to sot aei hrn lower case ( ) [ ] $;: +=- @* / { } ‹ › GridPad Numerical keypad on full size qwertyboards « » %!_ &• quot; l' ? Backspace Forward Delete Nlock Delete Caps CapsLock SOT 321 654 AEI 987 HRN Windows: In Mac use Navig 12 21 0 P&R Nlock to exit P&R CapsLock to exit Nlock default layout is N pad p (phone) HelpInsert Navigation includes Deletes Accent ¸¨ ! pU P&R : quot; End # Accent ´ ˆ `~ to move cursor. H $ pD .+mark P&R Navig to exit + letter or Space Macintosh: Post PostCaps 3postCaps 3post same as Caps Use 3post for 1 1 symbols with Caps in Nlock or anywhere. 2 2 Extended character sets § ø¬ å € # …ƒ ß µ æ ¥ œ ¢ | <° ÷ ^` ~ ª ç° £†¶ blue symbols vary by font ±> – | ‚ „ ‰¡— ” ’ ¿ - “‘ + = * / GridPad Enter white symbols unique to each OS Macintosh additional character set ¯ !ı √ Ø™ %& ' Æ$Œ #flfi ˙˝ ˚ Åquot; ! ˛ *+, ˘© (Çquot; )‡ ⁄® ˇ Windows additional character set ! Ø™ ³²¹ ¾ ½¼ ¤ quot; þ ð ¯ Å Æ Œ # × ÞÇ& Ї - ©¦ $®% soft Print Scroll keyboard Windows only Relic Screen Lock Pause Power P P ShortCut & Keep Store 99 ShortCuts & 99 Keeps: for 1 digit: Enter # entry Enter; for 2 digit: Enter Enter # # entry Enter. To retrieve: Access # or A A ##. To clear: Enter # Enter or EE## E. ShortCut Keep Enter Access Enter Access Code */ + 321 321 321 321 - 654 654 654 654 987 987 987 987 0 all empty 0 0 all empty 0 Config + c Code+password+KeepAccess unlocks Keeps if passsword protected. Create 1 to 10 ShortCut sets, each with ten 1-finger chords. To make it easy to access sets of ShortCuts for an application, organize them into ranges like 10,11,…19 or 50,51,…59 entering up to ten ShortCuts in a range. Accessed by 10-ones chord + first digit, each set gives repeated one finger access to ShortCuts in the range. Volume F delete/ To Access a set of one finger ShortCuts: P&R 10-ones, then P&R first digit of two digit location. 10-ones first digit repeated 1-finger chords 3 2 1 1-finger chord 3 2 1 More actions 6 5 4 does ShortCut 6 5 4 until exit with + 9 8 7 stored in 2nd 9 8 7 Alpha. digit location 0 0 Mute Up Down Eject Clear Commands Primary: qwerty keyboard left side ! or Ctrl Option ctrl or ww ShiftL or Alt P & RELEASE to R hand L hand AltR or activate commands CapsShift qwerty right side !R or CtrlR OptionR ctrlR or wwR ShiftR L R 2 1 Pressing any key completes and/or ends a command action GridPad Numerical keypad on full size qwertyboards Windows: In Mac use Navig to move cursor. Macintosh: LR - + = * / GridPad Enter */ + - Volume F delete/ Mute Up Down Eject Clear L R L R LR L R . period Alpha exits Gnum Numlock 987 Windows 654 only 321 0 Numlock switches between & Page Insert Home End Up Down . period Arrow Forward Delete Left Right Up Down Alpha exits Gnum Numlock Configs change settings; Config chord + 1st letter chord Config OS select GridPad Reset FingerUp WM 987 M 654 W 321 0 Resets in blue 1st~Last OS=Config+o+m or w Code Q Set 987 Create If Alph On, Alph OnOff (empty) password with password EkaPad Q set. Unlock 3 2 1 returns to Keeps with 6 5 4 AlphaState Code. after 5 987 Ekapad Code+password Q Set+pass- minutes of Off is default Windows no chords word+KAccess + KeepAccess Npad (Config+n+letter) Sleep phone ten right left 654 Mac dialog box w 321 987 741 369 e 654 654 852 258 w 987 321 963 147 only Wake Npd+p 0 Npd+t 0 Npd+r 0 Npd+l 0 Mac: UnPlug EkaPad to wake Trigger Repeat & Hold-Down Version# v 321 To hold down Shift, %, Option, ctrl, Alt, press for Trigger delay 654 e 321 time (0.5 sec-2 secs) and then 987 vr release. To Repeat, hold down ver together Config+t+ chord and release. P&R any key to Displays the current 1 to 8 sets exit. 9 = off version of EkaPad delay time. software. Functions 12-key, 3-finger P&R F once 3 2 1 P&R F twice Function 321 numbers 1 0 Numlock switches between for one digit 6 5 4 for two digit 654 through 24 & function function 987 987 work. 0 then P&R the number key(s) Restart = FF99 Restart stops & restarts computer Page Names of State chords are colored: Alpha Navig Nlock CapsLock ShortCut&KeepEnter GridPad 10-ones chording EkaTetra CheatSheet for the USB EkaPad Find additional Tools at www.ekatetra.com/support EkaTetra 16580 Maple Circle, Lake Oswego OR 97034 USA Insert Home End Up Down 503-697-0604 www. ekatetra.com EkaTetra<Eka_T_USB_CheatSheet One hand it – with EkaPad! ® 47
  47. 47. Motivation 48
  48. 48. Affordances Design physical & visual Avoid instructions 49
  49. 49. Metaphors Direct manipulation Symbolic Gestures as part of language 50
  50. 50. Thank you! P.O. Box 44-2302 Lawrence, KS 66044 785 838 3003 littlespringsdesign.com 51

×