Background, why do we need this comparison? We work at the Icelandic National Institute for the Blind (INIB). INIB is a government organization that serves approximately 1500 individuals in Iceland (a small island with a population of 320.000). INIB‘s role is to provide services to all individuals in Iceland who have a diagnosed visual impairment, regardless of age or location within the country. Our services range from providing appropriate glasses and other vision aids to preparing alternative formats for text books and other instructional material for visually impaired students. A major component of our services is providing appropriate Assistive Technology (hardware and software) as well as necessary A.T. training to our users. We have between 50 - 100 active A.T. users ranging from age 5 to 90 with varying degrees of vision impairment and technology skills. Our job involves carrying out an assessment of our user’s technology needs and competencies. Also developing an individualized appropriate Assistive Technology plan and submitting the information as input to our manager, education consultants and funding committees. This process is somewhat similar to the U.S. IEP (Individual Education Plan), but in our case the user‘s age and situation is much more varied than that of U.S. students in grades 1 through 12.Some of our users are university students, or professionals, who need advanced functionality and braille support for school work or to facilitate complex jobs, others are either very young or elderly individuals getting their first taste of reading email, Facebook, blogs and newspapers on the internet. Once our recommendations have been reviewed and approved and necessary equipment purchased, it is our job is to carry out suitable follow up training. Often this requires face to face training with the user, but other times writing instruction manuals or recording podcasts will suffice. As our users skill set evolves we may reevaluate and update their Assistive Technology, as the situation may call for.
Until recently we enjoyed a relatively simple kind of life. We simply provided the user a copy of a certain Screen Reader we, as an organization, had contracted for, regardless of the technology activities that user would engage in. This was an acceptable procedure at the time, since we had sufficient funding and the availability of Assistive Technology software for the Windows platform was rather limited. All Screen Readers fell, broadly speaking, into the same price range and the functional differences between them were never studied in detail, and considered to be more of a personal preference than anything else. But then two important things happened: 1. New Screen Readers with varying functionalities and price tags started to appear on the market, both for the Windows O.S. as well as other operating systems and2. We experienced budget restrictions and our management encouraged us to find more cost effective solutions without negatively affecting our user’s experience. We concluded that individuals who simply want to use the computer for email, web browsing etc. simply didn´t need advanced Screen Reading software. Add to that, said users usually don´t read braille. Therefore we started developing an Assistive Technology evaluation process that we could combine with the individual Assistive Technology needs and skills assessment. By effectively combining the two we hope to give each user the most appropriate, useful and cost effective Assistive Technology solution available.
We took over an office, supplied ourselves with an almost limitless amount of coffee and serious amounts of patience and started our testing....
Email: Apple Mail, Thunderbird, MS OutlookWord processing: MS Word, OO Writer, iWork PagesSpreadsheet: MS Excel, OO Calc, iWork NumbersPowerpoint presentations: MS Powerpoint, OO Impress, iWork Keynote
Safari, Firefox 3.5, IE 8
Adium, Miranda, Pidgin, Windows Live Messenger
Pigin, Skype 4.xx
Adobe Reader, Preview
This should give us some indication of how the tested screen readers stack up against each other.
Developing a framework for comparing different screen readers and various operating systems
5 Screen Readers 3 Operating Systems 1 WeekPractical screen reader comparison: a user oriented approachIcelandic National Center for the Blind, Deaf Blind and Visually Impaired
Who are we?• We work for the Icelandic National Institute for the Blind, Deaf Blind and Visually Impaired (INIB)• We provide Assistive Technology (A.T.) and training for clients nationwide of all ages• We are responsible for recommending the most suitable A.T. setup for each client of the INIB• Our clients require various types of A.T. solutions
The purpose of this study• To understand the abilities of each of the screenreaders and the operating systems they work on• This way we can match the most suitable screen reader to each user• To create mini tutorials for the five screen readers that were tested• To examine if the A.T. budget is being spent effectively
Screen readers tested• HAL 11 – Dolphin Computer Access - Win7 (32bit)• JAWS 11 – Freedom Scientific - Win7 (32bit)• NVDA 2010.1 – NV Access – Win7 (32bit)• VoiceOver SnowLeopard – Apple• Orca – Ubuntu 10.04• The study was carried out in July 2010
The screen readers were tested in 7 catagories• Screen reader functionality• Screen reader and O.S. operability• Office applications• Internet• Instant Messaging• Skype• PDF
Screen reader functionality• Braille support• Virtual focus• User customization• To name but a few...• Scoring: 15 points available• HAL 15 points• JAWS 14.5 points• VoiceOver 12 points• NVDA 8 points• Orca 8 points
Screen reader and O.S. operability• Navigate all files and folders• Add/Remove programs• Read tooltips, error messages and other O.S. information• Scoring: 13 points available• HAL 13 points• JAWS 13 points• NVDA 12 points• VoiceOver 10.5 points• Orca 10.5 points
Internet• To navigate by HTML elements (edit fields, radio buttons, check boxes, headings, frames)• Work with JAVA script (Jump Menu s)• Interact with an HTML form• Work with Accessible Flash• Work with Landmarks• Scoring: 17 points available• JAWS 17 points• HAL 16 points• NVDA 15.5 points• VoiceOver 12.5 points• Orca 9 points
Instant Messaging• Announcing user login• Automatically read incoming message, even when another window has focus• Ability to review the entire conversation• Scoring: 6 points available• JAWS 6 points• HAL 6 points• Orca 6 points• NVDA 4 points• VoiceOver 3.5 points
Skype• Read contact list with online status• Incoming caller info announced• Skype Instant Messaging• Scoring: 6 points available• Orca 6 points• HAL 6 points• JAWS 5 points• NVDA 4 points• VoiceOver 3 points
PDF• Reads a tagged PDF document• Navigates a tagged PDF document (by links and headings)• Can interact with an accessible PDF form• Scoring: 5 points available• NVDA 5 points• JAWS 5 points• VoiceOver 2 points• HAL 2 points• Orca 0 points
Total Score for the study• The study produced these scores for the screen readers• JAWS 93%• HAL 93%• NVDA 75%• VoiceOver 63%• Orca 59%
Results and observations• HAL and JAWS still lead the way with the best customization and braille support• HAL and JAWS are still the only options for effective use of Office applications• We are still most comfortable with using JAWS for webpage browsing• In our view NVDA is comparable to JAWS and HAL if you do need not require Braille support + it s a truly portable screen reader
So what now?• We will publish our study at the INIB s homepage: www.midstod.is along with all the data we gathered• We hope to do this kind of study annually as we feel that it s an important resource• We hope to get input from users of all screen readers to improve our study
Contact information• Birkir R. Gunnarsson: firstname.lastname@example.org• Hlynur M. Hreinsson: email@example.com• Consultants for the Icelandic National Institute for the Blind, Deaf Blind and Visually Impaired