Typing Performance of Blind Users: An Analysis of Touch Behaviors, Learning, and In-Situ Usage
•Download as PPTX, PDF•
1 like•419 views
The document summarizes a study on the typing performance of blind users on mobile devices. Over 8 weeks, 5 novice blind users participated in weekly 20-minute typing tasks on an Android device. Results showed users slowly improved their words per minute over time due to factors like land-on accuracy and movement efficiency increasing. Substitution errors were most common, though users corrected most errors. However, corrections took significant time. The study provides insights into challenges blind users face with typing and suggests areas for future work like more efficient correction methods and leveraging touch movement models.
More Related Content
Similar to Typing Performance of Blind Users: An Analysis of Touch Behaviors, Learning, and In-Situ Usage
Similar to Typing Performance of Blind Users: An Analysis of Touch Behaviors, Learning, and In-Situ Usage (20)
More from Hugo Nicolau
More from Hugo Nicolau (19)
Recently uploaded
Recently uploaded (20)
Typing Performance of Blind Users: An Analysis of Touch Behaviors, Learning, and In-Situ Usage
- 1. TYPING PERFORMANCE OF BLIND USERS HUGO NICOLAU KYLE MONTAGUE TIAGO GUERREIRO ANDRÉ RODRIGUES VICKI L. HANSON
- 2. accessibility :: explore by touch “Messages. Double-tap to open”
- 3. text-entry :: explore by touch
- 4. previous :: alternative techniques Bonner et al., 2010 Azenkot et al., 2012 Oliveira et al., 2011 Southern et al., 2012 Yfantidis et al., 2006 Guerreiro et al., 2015
- 5. problem:: lack of understanding Words per Minute Error Rate
- 6. problem:: how to improve? ? ? ? ?
- 7. contribution :: new knowledge
- 8. contribution :: how? why? 1.6 4.0 1 2 3 4 5 6 7 8 WordsperMinute
- 9. contribution :: longitudinal study weeks
- 11. apparatus :: S3 mini
- 12. procedure :: weekly assessment week week week week week week week week 20-minute typing tasks
- 13. apparatus :: in-situ usage TinyBlackBox Supporting Mobile In-The-Wild Studies
- 14. results
- 16. performance :: words per minute 1 2 3 4 5 1 2 3 4 5 6 7 8 WordsperMinute P2 P4 P1 P3 P5
- 17. performance :: words per minute 1 2 3 4 5 1 2 3 4 5 6 7 8 WordsperMinute P2 P4 P1 P3 P5
- 18. performance :: words per minute 1 2 3 4 5 1 2 3 4 5 6 7 8 WordsperMinute P2 P4 P1 P3 P5
- 19. performance :: words per minute 1 2 3 4 5 1 2 3 4 5 6 7 8 WordsperMinute P2 P4 P1 P3 P5
- 21. wpm :: land-on accuracy 27% (SD=12) 48% (SD=15) week 1 week 8
- 22. wpm :: movement efficiency P1 Week 1 P1 Week 8
- 23. wpm :: less re-entries 6.6 0.8 1 2 3 4 5 6 7 8 week
- 24. error rates how?
- 25. performance :: total error rate 0% 10% 20% 30% 40% 50% 1 2 3 4 5 6 7 8 P1 P2 P3 P4 P5
- 26. performance :: uncorrected error rate 0% 5% 10% 15% 20% 25% 1 2 3 4 5 6 7 8 P1 P2 P3 P4 P5
- 27. performance :: corrected error rate 3out of 10deleted characters were correct
- 28. performance :: correction time 32 13 1 2 3 4 5 6 7 8 CorrectionTime(%)
- 29. types of errors :: performance 0% 5% 10% 15% 20% 25% 1 2 3 4 5 6 7 8 Substitutions Omissions Insertions
- 31. types of errors :: slips 38%
- 32. types of errors :: feedback delay 64%
- 33. substitutions :: key not visited 36%
- 34. major results
- 35. summary :: major results Improve performance at slow rate Due to several factors Substitutions are predominant Correct most errors Corrections are time consuming
- 36. summary :: major results Improve performance at slow rate Due to several factors Substitutions are predominant Correct most errors Corrections are time consuming
- 37. summary :: major results Improve performance at slow rate Due to several factors Substitutions are predominant Correct most errors Corrections are time consuming
- 38. summary :: major results Improve performance at slow rate Due to several factors Substitutions are predominant Correct most errors Corrections are time consuming
- 39. summary :: major results Improve performance at slow rate Due to several factors Substitutions are predominant Correct most errors Corrections are time consuming
- 40. implications
- 41. correction :: easier and efficient
- 42. synchronization :: output and input “H” “B” “N”“M”
- 43. touch models :: leverage movement
- 44. HUGO NICOLAU http://web.ist.utl.pt/hugo.nicolau TYPING PERFORMANCE OF BLIND USERS SOS
Editor's Notes
- Hi everyone, my name is Hugo Nicolau and the work I’ll be presenting today entitled “typing performance of blind users” was done during my stay at the Rochester Institute of Technology, in collaboration with Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki Hanson.
- As most of you know, since 2008 Apple and latter Google included accessibility features in their smartphones allowing blind users to explore an interface by dragging their fingers on the screen while earing what they are touching, and then by performing a double tap or a split tap they can select the intended target.
- And this simple exploration technique allows blind people to use a multitude of applications and perform multiple tasks with their device, including text-entry, which is the focus of this work.
- Since 2006, many text-entry techniques have been proposed from gestural interfaces to Braille-based techniques, all of them trying to improve the non-visual typing process on touchscreen devices
- However, looking back although there is a considerable body of work, we realized that there isn’t much knowledge about the actual typing process. Most research is limited to performance comparisons between input techniques.
- And comparisons are great to establish that differences occur in terms of speed and errors, but they fail to provide insights into why they occurred. Essentially, there is a lack of knowledge on users’ actual typing behaviors. And without this knowledge, it is unclear how to improve current input techniques.
- With this in mind, we aim to bridge this gap and contribute with new insights about blind users typing process on touchscreen mobile devices
- We were particularly interested in observing users’ learning experience, how performance evolved, and why did participants achieved that performance level. Our ultimate goal is to identify new opportunities to reduce the learning overhead and support better non-visual input on mobile devices.
- To do that, we conducted a longitudinal study of 8 weeks
- With 5 novice blind users. Participants age ranged from 23 to 55 years and non had prior experience with touchscreen screen readers.
- We basically gave them a new Samsung S3 mini and some basic training on how to use the device and a virtual keyboard. And asked them to use it as their main mobile phone.
- We conducted controlled weekly typing sessions of 20 minutes for 8 weeks
- We also collected real-world usage measures. I hope you had the chance to talk with Kyle during yesterday’s poster session, about our data collection framework, if not he’s still around In this paper, we used that data to control for device usage of individual participants and relate to lab performance. Although if you are interested in in-the-wild performance measures come talk with me afterwards.
- Participants entered a total of 32,764 characters over eight weeks. They spent a total of 51 hours actively entering text. However, there is a high variance in usage results both between participants and weeks. For instance, while P2 and P3 were particularly active in the fourth week, P4 were more active in the last two weeks. P5 was the least active with an average usage of 12.5 minutes per week, while P2 spent on average more than 2 hours typing per week.
- Starting with speed, how did typing speed evolved over time?
- Participants improved on average 2.4 wpm in 8 weeks; from 1.6 wpm to 4 wpm. Although all participants improved over time, learning rate was slow with an improvement of 0.3 wpm per week,.
- We also noticed that P2 and P4 had atypical changes in performance in week four and seven, respectively. Without having collect real-world device usage it would have been very hard to understand these changes, since they were clearly influenced by usage
- For instance, for P2, we notice that performance dropped after she installed a 3rd party app, WhatsApp. When debriefing P2 she mentioned perceiving the speech feedback being slower while typing. In fact, this is a known issue with this particular application. Although we are not able to confirm that speech feedback changed, we can show that both number of pauses and duration of pauses during typing increased
- Regarding P4, the abrupt increase is most likely related with the increase of usage in week seven. he mentioned that he was finally using his phone to the fullest, particularly sending and receiving text messages, skype messages and so forth.
- So, overall why did participants improve typing speed over time? Previous work has not looked into this question, and in fact it is impossible to answer it if we only look at performance measures. There’s a need to understand how users actually type and explore the keyboard. They did improve because they move their fingers faster while exploring the keyboard? How fast? Do they start to land their fingers closer to targets? Do they move their fingers less? How exactly does that change over time? These are the kinds of questions we aimed to answer
- First of all, participants significantly improve their land-on accuracy in the 8-week period, in week one only 27% of landing positions were inside the target. In week eight this value increases to 48. Also, by week eight, 91% of the times, participants land either inside the intended key or an adjacent key. This shows that users start to gain a better spatial understanding of the keyboard and the device.
- Regarding movement, and exploration paths, how did they improve? First they start to make more efficient explorations; meaning that they get closer and closer to the optimal path from their landing position to the intended target. As a results the number of visited keys also decreases over time from an average of 5 to 2 visited keys by week 8.
- We also looked at target re-entries, which consist of entering the same target for the second time. These are particularly important because users receive audio feedback every time they enter a target. Overall, target re-entries decreased from 6.6 (SD=3.2) to 0.8 (SD=0.3), which suggests that over time it is easier to find keys and with less errors (PAUSE)In summary speed improvements are due to a combination of factors, from landing accuracy to more efficient movements
- Regarding error rates, how does that change over time?
- We looked at total error rate, which represents the percentage of incorrect characters that were entered, even those that were corrected. Overall participants started with an average total error rate of 26% (SD=11.7%) and finished with 7.4% (SD=1.7%) . The most significant drop occurred in week 2
- Looking at the errors in the final sentence, we found that when given the chance, users tend to correct most errors, resulting in high quality transcribed sentences. This goes in line with previous findings for sighted users. There is a decrease over the 8 weeks from 9% to 1.6%
- In terms of correction performance it was fairly constant throughout the 8 weeks. And we found that on average 3 out of 10 characters were unnecessarily deleted. The reason for this to happen is that sometimes participants only notice errors when receiving word feedback, and then they need to delete several characters to correct the error and some of those were correct
- Examining the time spent on those corrections, they spent on average 13% of their time correcting text in the last week. This corresponds to nearly 14 seconds correcting on a 5 word sentence
- Going into further detail, and analyzing types of errors. Substitution errors were consistently higher than insertions and omissions. Substitutions errors are just incorrect characters. Although there was a significant decrease over time, from 24% (SD=12%) to 6% (SD=1%), they still remain significantly higher than the remaining types of errors. We want to understand why this happen
- Overall, participants had similar error rates across all intended keys. No row, column, or side patterns emerged from weekly data. Unlike sighted users that experience substitution patterns towards a predominant direction [5, 17], blind users’ patterns are less clear. This is most likely related with the differences between visual and auditory feedback when acquiring keys. Some of the keys have fewer or no data because they are less common in the Portuguese letter
- Again, traditional performance measures do not provide any insights into why an outcome occurred or how to improve it In the particular case of substitution, for sighted users these usually occur when their finger is inside the intended target and when lifting the finger, it slips to an adjacent key
- However, we didn’t find this to be true with our user group. Overall, only 38% of substitution errors were slips (constant over time). Taking into account that users should receive speech feedback before selecting the intended key, this can seem a counter-intuitive result. So, we started analyzing whether participants’ finger paths crossed the intended target at some point during movement.
- And that happen for 64% (SD=9.8%) of substitution; however, although they were inside the target they failed to select it. We were puzzled by this result so we started doing a manual examination of the recorded videos. We noticed that most of these cases were related to a significant delay between speech feedback and input, which resulted in a mismatch between the key being heard and the one being touched that moment. This still leaves 36% of substitution errors unaccounted for.
- In these cases participants did not even hear the key they were aiming for. In these ones it is harder for us to understand why. We believe it can be due to several reasons, including accidental touches, phonetically similar letters, users just gave up during exploration, or overconfidence of aiming to the key without hearing it
- So, major results from this study. I didn’t have time to go through all of them, please refer to the paper for a more complete analysis. But what I showed here today,
- Is that users improve both entry speed and accuracy, although at slow rate. This has been previously reported
- However, with this work we uncover why those improvements happen. They are mostly due to a combination of factors, such as landing closer to intended keys, performing more efficient keyboard explorations, lower number of target re-entries, and lower movement times.
- Also, the most common type of errors are substitutions.
- Regarding correction strategies, users correct most typing errors, which …
- consumes on average 13% of input time.
- Regarding implications or lessons learned
- Because corrections are still time consuming and inefficient we need better correction techniques. For instance, none of our participants used cursor-positioning operations throughout the study. Also, participants did not use auto-correct or auto-complete suggestions. We are not sure why, if it was because it was too hard, or maybe it wasn’t useful, or simply because they unaware of these features. Future work should investigate auditory interfaces for these features. As far as we know, this is an unexplored topic.
- Also, we observed that 64% of substitution errors could be due to a mismatch between speech output and touch information. Also, as we have seen in the case of one participants it can affect the way to type as well. So, future non-visual keyboards should prioritize synchronization between input and output modalities.
- The majority of omission errors (68%) go by undetected and therefore uncorrected. Language- based solutions such as spellcheckers seem to be the only plausible solution. Again, we still don’t know whether these features would be useful or desirable.
- For those of you interested in building touch models to compensate errors, although blind users do not show a predominant touch offset direction, most substitution errors were adjacent keys. Also, finger movement data can provide evidence of what particular key users are trying to select; however, this is more relevant for novice users. Expert users already land on the intended target most of the times (or at least very close). So, touch models that adapt to skill level and learning rate are probably a good idea.