• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Trailblazer: Enabling Blind Web Users to Blaze Trails Through the Web

on

  • 3,948 views

IUI talk on TrailBlazer, work by Jeffrey P. Bigham, Tessa Lau, and Jeffrey Nichols.

IUI talk on TrailBlazer, work by Jeffrey P. Bigham, Tessa Lau, and Jeffrey Nichols.

Statistics

Views

Total Views
3,948
Views on SlideShare
3,947
Embed Views
1

Actions

Likes
0
Downloads
3
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Trailblazer:  Enabling Blind Web Users to Blaze Trails Through the Web Trailblazer: Enabling Blind Web Users to Blaze Trails Through the Web Presentation Transcript

  • Jeffrey P. Bigham University of Washington CSE [email_address] Enabling Blind Users to Blaze Trails Through the Web Tessa Lau and Jeffrey Nichols IBM Almaden Research {tessalau, jwnichols}@us.ibm.com start
  • Scale of Access
    • 37 Million Blind People in the World
      • (125 million with low vision)
    • Increasing
      • Aging Population
      • Situational Impairments
    • Access technology benefits 25% of web users 1
    [1] “The Market for Accessible Technology. “ Forrester Research, 2003.
  • Web Potential for Blind People
    • Web Advantages
      • Convertible to Accessible Form
      • Shop From Home
        • Blind People Read 7x as Many Books
    • The Law
      • U.S. Government Requires Accessibility 1,2
      • NFB vs. Target Corporation 2
    • Making Access Possible is Not Enough
      • Should be Intuitive and Fun
    [1] Section 508 of the U.S. Rehabilitation Act. [2] Americans with Disabilities Act.
  •  
  •  
  • Keyboard Shortcuts X X X X h h h e b b START
  • Keyboard Shortcuts X X X X <span class=“heading”> e b b START <span class=“heading”> <span class=“heading”> .heading { font-weight: bold; }
  • Task: Placing Book on Hold 1. Enter “Book You Want” 2. Click “Find” Button 3. Find Correct Result 4. Click “Place Hold” Button 5. Click “Request” Button 6. Verify Queue Position
  • Task: Placing Book on Hold 1. Enter “Book You Want” 2. Click “Find” Button 3. Find Correct Result 4. Click “Place Hold” Button 5. Click “Request” Button 6. Verify Queue Position
  • Task: Placing Book on Hold 1. Enter “Book You Want” 2. Click “Find” Button 3. Find Correct Result 4. Click “Place Hold” Button 5. Click “Request” Button 6. Verify Queue Position
  • Task: Placing Book on Hold 1. Enter “Book You Want” 2. Click “Find” Button 3. Find Correct Result 4. Click “Place Hold” Button 5. Click “Request” Button 6. Verify Queue Position
  • How can blind people more effectively complete web tasks?
    • Avoid Linear Searches
      • Direct Users to Important Content
    • Users Maintain Control
      • Interface should be understandable
      • Guide users not direct them
    • Flexible and Adaptable
      • Work in full web domain
    • Compatibility with Tools Users Know
      • Supplement to existing tools (screen readers)
  • CoScripter 1 : Recording and Replaying Tasks
    • Pseudo-Natural Language Commands
    [1] Little et al. “Koala: capture, share, automate, personalize business processes on the web.” CHI 2007.
    • go to “http://www.united.com”
    • enter your “flight number” (e.g., 0168) into the “Flight number” textbox
    • enter your “airport” (e.g., SJC) into the “From:” textbox
    • click the “Check” button
    CoScript Repository
  • Example TrailBlazer Placing a book on hold at the Seattle Public Library.
  • Set Focus & Provide Contextual Help Step description. Inline Playback Controls
  • Formative Evaluation
  • Generalizing with Suggestions Choose from one of the following suggestions: Click the “Check” button. Enter into the “From:” textbox. Enter into the “To:” textbox. Select from the “Date:” listbox. Select from the “Time:” listbox.
  • Providing Suggestions
    • User enters a short task description (query)
    • “ flight status on united”
    • “ place hold on book at Seattle library”
    • (etc.)
    • Machine learning model ranks all actions.
    • Features
      • Task Description
      • Related Scripts from the Script Repository
      • User’s History of Actions
    • Apply existing scripts to new tasks or new sites
    • No need to explicitly find script first
    Click the “profile” link. Click the “Home” link. Click the “Books” link. … Enter into the “Name” textbox. Enter into the “Zip” textbox. … Click the “Submit” button. Click the “Check Now” button. … Turn on the “Yes” Radio Button. … . Example Action List x 1 x 2 x 6 … y
  • Generalizing Scripts to New Sites 0.08 0.27 0.10 0.76 0.20 “ flight status on united” Task Description 1. go to “www.aa.com” 2. enter your “Flight Number” (e.g. 144) into the “Flight Number:” textbox 3. click the fifth “GO” button Flight Status on American Airlines 1. go to “http://www.southwest.com/” 2. select “SJC” from the “Select the Departure City:” listbox 3. enter your “flight number” into the first “Enter the Flight Number” textbox 4. click the first “View Flight Status Information” button Flight Status on Southwest Airlines Click “my profile” link Click “Flight Information” link Enter into “Mileage Plus” textbox Enter into “flight number” textbox Click “Check” button Possible Actions similarity ( , ) Repository
  • Using History to Match Sub-Tasks Click “my profile” link Enter into ‘password’ password box Click “Login” button Enter into “flight number” textbox Click “Check” button 0.08 0.88 0.10 0.36 0.20 Last Action Update Facebook Status Reserve UW Meeting Room Possible Actions 1. go to “www.facebook.com” 2. enter your “facebook username” into the “username” textbox 3. enter your “facebook password” into the password box 4. Click “Login” button 5. … 7. … 8. enter your “CSE login” into the “username” textbox 9. enter your “CSE password” into the password box 10. … 0.00 0.87 0.46 0.14 0.04 0.00 0.75 0.42 0.13 “ enter into ‘username’ textbox” similarity ( , ) Repository
  • Evaluation
    • Tested on 102 Actions
      • 15 Most Popular CoScripts
      • User-provided title used as task description
      • 6.8 steps per script
      • 42 possible actions
    • Leave-One-Out Cross-Validation
      • Removed from repository too
    • Top 5 in 76% of cases
    (5.1 words on avg.) Suggestions Provided CoScript Repository
  • Ongoing and Future Work
    • User Studies
      • What happens when TrailBlazer is wrong?
      • Incentives vs. Cost
    • Better Model
      • Continuous Learning
      • Popularity, Past Behavior, etc.
    • Improved Web Access
      • Voice Over Mobile Phones
      • Small Screens
  • Conclusion
    • TrailBlazer makes web access more efficient
      • Record, replay and share trails
      • Targets non-visual use
      • Uses existing repository of trails
    • Suggestion Component
      • Finds appropriate trails
      • Generalizes existing scripts to new tasks and sites
  • The End Thanks to : Allen Cypher, Clemens Drews, Jimmy Lin, Yevgen Borodin, Charles Chen and T.V. Raman. Want to hire me? [email_address] www.jeffreybigham.com
  • Directing Users to Content
    • Direct to Content
      • Avoid inefficient linear searches
    • 5 Participants/20 Regions
      • Mechanical Turk
    • 3 Commonalities
      • Semantic Desciption (78%)
      • Heading of Region (53%)
      • Words Used in Region (37%)
    Clip the “search results” region
  • Directing Users to Content
    • Direct to Content
      • Avoid linear searches
    • 5 Descriptions of 20 Regions
      • Mechanical Turk
      • 27 Users
    Clip the “search results” region Clip the region containing “batting average”
  • Generalizing Scripts to New Sites 0.08 0.27 0.10 0.76 0.20 “ flight status on united” Task Description 1. go to “www.aa.com” 2. enter your “Flight Number” (e.g. 144) into the “Flight Number:” textbox 3. click the fifth “GO” button Flight Status on American Airlines 1. go to “http://www.southwest.com/” 2. select “SJC” from the “Select the Departure City:” listbox 3. enter your “flight number” into the first “Enter the Flight Number” textbox 4. click the first “View Flight Status Information” button Flight Status on Southwest Airlines Click “my profile” link Click “Flight Information” link Enter into “Mileage Plus” textbox Enter into “flight number” textbox Click “Check” button Possible Actions Actions a Scripts s 1: enter 1: flight 1: number 1: textbox 2: button 1: city 2: click 1: departure 3: enter 4: flight 3: go 1: information 1: listbox 4: number 1: select 1: SJC 1: status 2: textbox 1: view 2: your Similarity = cos( a , s ) Repository
  • Matching Partial Scripts Click “my profile” link Enter into ‘password’ password box Click “Login” button Enter into “flight number” textbox Click “Check” button 0.08 0.88 0.10 0.76 0.20 Last Action Update Facebook Status Reserve UW Meeting Room Possible Actions Action a Scripts s 1: enter 2: password 1: box box: 0.9 button: 0.1 click: 0.0 CSE: 1.2 enter: 2.5 facebook: 1.2 go: 0.0 login: 0.1 password: 1.8 textbox: 1.6 W(s i ) = d * W(s i-1 ) + cos( t , s i ) 1. go to “www.facebook.com” 2. enter your “facebook username” into the “username” textbox 3. enter your “facebook password” into the password box 4. Click “Login” button 5. … 7. … 8. enter your “CSE login” into the “username” textbox 9. enter your “CSE password” into the password box 10. … 0.30 * 0.00 0.30 * 0.00 + 0.87 0.30 * 0.87 + 0.20 0.30 * 0.46 + 0.00 0.30 * 0.14 + 0.00 0.30 * 0.00 + 0.00 0.30 * 0.00 + 0.75 0.30 * 0.75 + 0.20 0.30 * 0.43 + … 0.00 0.87 0.46 0.14 0.04 0.00 0.75 0.42 0.13 d = 0.30 “ enter into ‘username’ textbox” Repository