The team from the University of Central Florida and University of Michigan evaluated the accessibility of the Canvas iOS app using screen readers. They found several issues including inconsistent labeling of controls, a lack of color contrast meeting standards, videos failing to display captions, and alerts not announcing using screen readers or LEDs. Recommendations were provided to Instructure to address these problems and improve the equitable experience for assistive technology users.

1. Canvas iOS App
Accessibility Evaluation
University of Central Florida
University of Michigan
CSUN 2017

2. ATHEN collaboration
Access Technology Higher Education Network (ATHEN), athenpro.org
ATHEN has been investigating the accessibility of Canvas since 2014 under the
leadership of Terrill Thompson
Hadi Rangin et al. published Canvas a11y study and presented results at CSUN
2016
Spin-off group, consisting of UCF and U-M staff, decided to investigate Canvas
iOS app and present at CSUN 2017
Will share results and file tickets with Instructure

3. The team
University of Central Florida:
John Raible - John.Raible@ucf.edu
Munawar Bijani - munawarb@gmail.com
Ashley Salter - ashley.salter@ucf.edu
University of Michigan:
Scott Williams - swims@umich.edu
Walter Stover - wstover@umich.edu
Brandon Werner - brwern@umich.edu

4. Purpose
Determine the status of the accessibility and usability of the Instructure Canvas
iOS app
Recommend, where possible, ways to improve the accessibility—and usability
of the application for the benefit of everyone
Increase awareness of the importance of the accessibility/usability of mobile
apps in higher education

5. Methodology
Small-scale usability test with the testers employing iOS accessibility features
while using Canvas
Our VoiceOver testers were not sighted
Results verified by sighted testers
Functional accessibility—validation, not determination of technical accessibility
Was it an equitable experience for assistive technology users?

6. Methodology (cont.’d)
Sub-divided each feature into functional tasks
Rated accessibility of tasks using AT:
1. Completely inaccessible
2. Technically accessible but not equitable, being time-consuming,
obscure, difficult to manipulate, etc.
3. Equitably accessible, requiring roughly the same amount of effort for
those using, and those not using, assistive technology

7. Disclaimer
Obvious that Instructure is making good-faith efforts to improve accessibility
We noticed improvements as we were testing
Shared responsibility between Instructure and Apple
Not an exhaustive study. Limited resources. Found representative errors.
VO testing with keyboard occurred in 2016, non-keyboard 2017
Student-facing features only

8. iOS VoiceOver Demo

9. Results

10. VoiceOver with and without keyboard
Inconsistent labeling of controls
Calendar, Discussions, Quizzes, and Chat were problematic
Color used to denote meaning
Testers found HTML rendering better than native app controls
Chat messages not announced in real time
Confirmation alerts not announced to VO

11. VoiceOver with and without keyboard (Cont.’d)
Gesture actions need control analogues, e.g., when using swipe-to-refresh or
swipe-to-expose controls. Affects keyboard use as well
Three-finger scroll gesture ineffective, navigation through a long Messages
thread almost impossible
Item types in Notifications were not announced by VO

12. Color contrast and text size
Color settings of the app failed to meet the WCAG 2.0 AA contrast standard in
many instances
Login screen was particularly bad
Calendar, Messages had their contrast issues
Color alone used to denote meaning as well
Larger and Largest Text worked inconsistently throughout the application
Inverting colors also yielded poor contrast in certain instances
Using the windowed zoom, the tester encountered difficulty moving the zoom
window down to the menu bar along the bottom of the screen

13. Captioning and alerts
Canvas uses a modified version of Kaltura’s Open Source Video Player for the
built-in record/upload media functionality
During our evaluation, multiple videos with .srt files were tested
All videos failed to display closed captions videos via the mobile app
During our testing, LED Alerts were enabled in Apple iOS settings
LED Alert was not activated upon receiving push notifications

14. Recommendations
Review all developer accessibility resources in Apple’s iOS in order to ensure a
holistic approach to accessibility and avoid a screen-reader-centric focus
Debug accessibility in the iOS Simulator from the earliest development
iterations with the Accessibility Inspector. Work with Apple
Add in-app Help or at least have in-app link to web help page
Ensure that an adequate color contrast of 4.50:1 is used throughout the
application
Enable dynamic text in all areas of the application

15. Recommendations (Cont.’d)
Label all controls consistently throughout the application
Where possible add confirmation for actions performed by the user
Add an “agenda” (list) view for Calendar. Allow it to be set as default
Add LED compatibility for alerts
Add accessibility help that outlines known issues and target dates for future bug
fixes

16. Resources
Presentation:
Slideshare: https://www.slideshare.net/swimsy/csun2017
Report: https://goo.gl/o96zwt
swims@umich.edu, John.Raible@ucf.edu
Developer:
https://developer.apple.com/accessibility/ios/
Accessibility Inspector: https://goo.gl/GWSVAY
https://athenpro.org/