Your SlideShare is downloading. ×
0
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Eyes Free Android Julian Harty
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Eyes Free Android Julian Harty

1,953

Published on

Published in: Technology, News & Politics
1 Comment
0 Likes
Statistics
Notes
  • watch this presentation

    http://skillsmatter.com/podcast/os-mobile-server/julian-harty-eyes-free-ui-and-accessibility/zx-489
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
1,953
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Eyes-Free User Interaction Julian Harty Google http://www.google.co.uk September 18, 2009
  • 2. Overview Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Eyes-Free Interaction Techshare September 2009 – 2 / 39
  • 3. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Android Programming
  • 4. Android: Engineer’s Dream! Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Smart phones — computer in your pocket. s Open platform — liberates innovation. s New I/O — minimize user interaction. Bend technology to your will! Eyes-Free Interaction Techshare September 2009 – 4 / 39
  • 5. Innovative User Interfaces Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Create innovative end-user solutions that: s Fit into the user’s mode of working, s Avoid one-size fits all solutions, s Design context-aware solutions, s Tailor solutions to user’s needs. Eyes-Free Interaction Techshare September 2009 – 5 / 39
  • 6. Eyes-Free Interaction Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Not just for blind users! s Eyes-busy environments. s One-handed use. s Minimize explicit user interaction. Rethink all assumptions about the user. Eyes-Free Interaction Techshare September 2009 – 6 / 39
  • 7. Devices That Sense Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Mobile phones make UI design exciting! s Can capture user intent — keyboard, . . .. s Can see — camera. s Can hear and speak — speech I/O. s Can sense and stimulate touch — touch screen and vibration. s Can sense motion — accelerometer, compass. s Can sense position — GPS, cell towers. . . s .. Our Android has many eyes and ears! Eyes-Free Interaction Techshare September 2009 – 7 / 39
  • 8. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Touch Screen Interaction, Eyes-Free!
  • 9. Using A Touch Screen Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion These are atomic tasks: s Identifying the item to touch. s Touching item to activate it. s Eyes-free use requires help with the former. s Activation should produce appropriate feedback. Eyes-Free Interaction Techshare September 2009 – 9 / 39
  • 10. On-Screen Touchpads Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Eyes-Free use of an on-screen touchpad: s Need to see the buttons to know where to touch. s Because: buttons use absolute positioning. s Conclusion: (wrong) — you need to see to use a touch screen! s Solution: Relax constraint of absolute positioning. What if we used relative positioning instead? Eyes-Free Interaction Techshare September 2009 – 10 / 39
  • 11. Phone Keypad Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Eyes-free phone dialer: s Keypad centered where you touch — relative positioning. s Users already know layout of a phone keypad. s Can stroke from the center for a desired digit. Enables eyes-free one-handed dialing! Eyes-Free Interaction Techshare September 2009 – 11 / 39
  • 12. Dialer Feedback Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Enables one-handed, eyes-free use: s Buttons vibrate when you move over them. s Augmented by auditory feedback. s Pressing buttons produces spoken and tactile feedback. s Shake to erase input. Synchronized auditory and haptic feedback is key. Eyes-Free Interaction Techshare September 2009 – 12 / 39
  • 13. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Text Input Via Touch Screen
  • 14. Contact Manager Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion No one dials phone numbers any more! s Browsing contacts using touch screen. s Finding contacts. s Managing contacts. Eyes-Free Interaction Techshare September 2009 – 14 / 39
  • 15. Entering Text Input Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Extending stroke dialer for textual input: s Arrange letters in 4 concentric circles. s Strokes access distinct circles. s Choose desired letter by tracing chosen circle. Any letter is at most 3 steps away! Eyes-Free Interaction Techshare September 2009 – 15 / 39
  • 16. Keypad Layout Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Gesture: Pick circle, and trace to desired character. A I Q Letter Gesture A Stroke up to the left ⊙ Y B Stroke A, trace right U M E E Stroke down to the right Eyes-Free Interaction Techshare September 2009 – 16 / 39
  • 17. Managing Contacts Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Contacts live in the cloud. s Contacts managed over the network. s Updated automatically on the device. s No more editing contacts with two fingers! Eyes-Free Interaction Techshare September 2009 – 17 / 39
  • 18. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Text To Speech On Android
  • 19. TTS Library Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Create speech-enabled apps. s Developer compiles against the TTS library stub. s Text To Speech is an Android service. s Class TTS manages communication with the service. s Enables multiple apps to use TTS without code duplication. s TTS library can be updated without a need to recompile clients. Eyes-Free Interaction Techshare September 2009 – 19 / 39
  • 20. TTS Method Summary Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s speak(String text, int queueMode, String[] params) s stop() s isSpeaking() s Also has methods for synthesizing to a file, specifying the language, etc. Eyes-Free Interaction Techshare September 2009 – 20 / 39
  • 21. TTS Object Properties Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Can automatically prompt the user to install the TTS Service. s When absent, fail silently (methods become no-ops) See: http://eyes-free.googlecode.com/ Eyes-Free Interaction Techshare September 2009 – 21 / 39
  • 22. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Gesture Library For Stroke Input
  • 23. Gesture Overlay Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Overlay watches for touch events s Returns identified gesture s Exposes same UI as the stroke dialer s Implemented as a custom transparent View with callbacks Eyes-Free Interaction Techshare September 2009 – 23 / 39
  • 24. Gesture Overlay Implementation Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Implement Interface GestureListener , s Create a new TouchGestureControlOverlay with it, s Set a FrameLayout as the base View, s Add the View with the content as a child to the FrameLayout, s Add the TouchGestureControlOverlay as a child Can easily enable/disable the TouchGestureControlOverlay Eyes-Free Interaction Techshare September 2009 – 24 / 39
  • 25. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion A Simple Music File Browser
  • 26. Music File Browser Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Barebones version: s Uses a simple ListView s Based on a FileBrowser tutorial on Anddev.org s Browse directories and mp3 files on SDCard. s Play music by clicking on the file. s Directory path is the first entry. s Click this to cycle through files in current level. s Tactile feedback when scrolling through list items. Eyes-Free Interaction Techshare September 2009 – 26 / 39
  • 27. Adding Spoken Output Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Create object TTS in method onCreate. s Result: Runs method TTS.onInit as part of setup. s Can add application-specific code to method TTS.onInit. s Result: Application comes up speaking. Eyes-Free Interaction Techshare September 2009 – 27 / 39
  • 28. Music Player Talks Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion private InitListener i = new InitListener() { public void onInit(int version) { tts.speak("Music File Browser started.", 0, null); } }; Eyes-Free Interaction Techshare September 2009 – 28 / 39
  • 29. Speaking List Items Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Add TTS call to speak list items. s Can use event handler that produces tactile feedback. Eyes-Free Interaction Techshare September 2009 – 29 / 39
  • 30. Speaking List Items Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Augment event handler with a TTS call. public boolean onTrackballEvent(MotionEvent event) { String filename = directoryEntries.get(selectedId); tts.speak(filename.substring(1), 0, null); } Self-voicing functionality added with just 13 additional lines of code! Eyes-Free Interaction Techshare September 2009 – 30 / 39
  • 31. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Adding Gesture Input To Music Browser
  • 32. Advantages Of An Overlay Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s GestureOverlay is a transparent overlay. s Can be layered on an existing View. s Does not interfere with visual appearance. Eyes-Free Interaction Techshare September 2009 – 32 / 39
  • 33. Gestures For Music Browser Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Add GestureListener to handle user input. s Gesture Tap: play/pause. s Gesture Right: Browse to next track. Add application-specific code in method onGestureFinish. Eyes-Free Interaction Techshare September 2009 – 33 / 39
  • 34. Steps In Adding Gesture Input Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion private GestureListener g = new GestureListener() { public void onGestureChange(Gesture arg0) {} public void onGestureFinish(Gesture arg0) { if (arg0 == Gesture.CENTER) { // play/pause music } else if (arg0 == Gesture.RIGHT) { playNext(); } } public void onGestureStart(Gesture arg0) {} }; Eyes-Free Interaction Techshare September 2009 – 34 / 39
  • 35. Adding Overlay To Music Player Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion myFrame = new FrameLayout(this); myFrame.addView(myList); myG = new TouchGestureControlOverlay(this, gestureListener); overlayActive = false; setContentView(myFrame); Eyes-Free Interaction Techshare September 2009 – 35 / 39
  • 36. Toggling Gesture Support Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion if (overlayActive) { myFrame.removeView(myG); overlayActive = false; tts.speak("Gestures disabled.", 0, null); } else { myFrame.addView(myG); overlayActive = true; tts.speak("Gestures activated.", 0, null); } Gesture support enabled by adding about 40 lines of code. Eyes-Free Interaction Techshare September 2009 – 36 / 39
  • 37. Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Conclusion
  • 38. Summary Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion s Programmable smart phones are an engineer’s dream! s I/O peripherals open up new UI avenues. s Provide desired information with minimal user interaction. Profound impact on how we work and play! Eyes-Free Interaction Techshare September 2009 – 38 / 39
  • 39. Get in touch and get involved Android Dialer Contacts TTS Gestures MusicBrowser Overlays Conclusion Julian Harty jharty@google.com http://eyes-free.googlecode.com/ Eyes-Free Interaction Techshare September 2009 – 39 / 39

×