Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ai and inclusion - Challenges and Benefits for those with disabilities.


Published on

Description of a series of challenges and benefits related to AI and Inclusion for those with disabilities. Discussion around automating web accessibility checks and supporting augmentative and alternative communication symbol searches with better classification using linked data, image recognition and machine learning.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Ai and inclusion - Challenges and Benefits for those with disabilities.

  1. 1. AI and Inclusion Seeking to understand the design and deployment of AI and inclusion to benefit those with disabilities, which will also provide digital accessibility for all members of society Professor Mike Wald E.A. Draffan Dr Chaohai Ding
  2. 2. Challenges… AI and Inclusion
  3. 3. Make algorithmic systems fair, transparent, and ethical Of the nine protected characteristics identified by the Equality Act 2010, disability is the least homogeneous and so techniques need to be developed to ensure algorithms work fairly for these ‘edge cases/outliers’
  4. 4. Health and Inclusion • Social vs Medical model of disability • Large data sets for diagnosis • Lack of data to overcome barriers
  5. 5. Some Principles • Assistive/Augmentative AI • Include disabled people in design • Design for ‘edge cases/outliers’
  6. 6. Some Examples • Speech recognition • Captioning speech and sounds • Sign Language Synthesis & Recognition • Intelligent personalised interfaces • Artificial Assistance • Symbol communication • Simplification of text • Describing images • Recommendations of AT solutions • Autonomous mobility guides
  7. 7. Speech Recognition more accurate than Human Transcribers • Professionals had 5.9% and 11.3% error rates • Speech recognition had 5.9% and 11.1% error rates Achieving Human Parity in Conversational Speech Recognition : W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig: Microsoft Research Technical Report MSR-TR-2016-71 February 2017
  8. 8. A proof-of-concept demonstration of SpeechBubbles. The user (right) equipped with a Microsoft HoloLens views speech bubbles adjacent to two speakers (left and middle). Peng et al 2018 CHI Conference.
  9. 9. AI can identify speakers with 92% accuracy
  10. 10. AI can caption some sounds (Applause, music, laughter)
  11. 11. AI can be used to improve lecture capture
  12. 12. https://s3-us-west- Speech emotion detection
  13. 13. Paralletdots
  14. 14. Using % of colours to represent emotions in a caption
  15. 15. Lip reading
  16. 16. Average experienced lipreader performance is only 52% LipNet goes up to 93%
  17. 17. Lip Reading Sentences in the Wild
  18. 18. 50% Word accuracy on the held-out test set. Many of the errors are tenses, plurals, etc.
  19. 19. Also improves Speech Recognition Performance
  20. 20. 3D Cameras provide even better Lipreading and Speech Recognition accuracy
  21. 21. Could also replace face with a more lipreadable one
  22. 22. Reenactment Pipeline
  23. 23. Captioning
  24. 24. Tags
  25. 25. Analyse video in near real-time • Use computer vision APIs with your video files by extracting frames of the video from your device • Send those frames to the API calls of your choice. • Get results from your videos more quickly
  26. 26. Challenge… • Automatic audio description of videos requires reasoning and understanding subtle meanings and context to identify what visual information is important. e.g. If a person leaves a room is it important to know they did not hear what was said after they left?
  27. 27. Real time sign language translation
  28. 28. Avatar
  29. 29. Challenge… • AI can provide automatic sign language translation of captions using human video clips or avatars • But the quality of translation for a visual language is not as good as translations between written languages which have vast amounts of data available for training the AI systems.
  30. 30. Mobility guides CMU and Pittsburgh Airport launch smart suitcase for blind passengers
  31. 31. Recommendations of AT solutions • Inclusive Employment • Workplace Reasonable Adjustment • What we have? • 20,000 + assessments reports over 5 years • Word document/PDF • Multiple-version of documents layouts • Tasks • Extract useful content to free-text • Data anonymisation • NLP + Deep Learning
  32. 32. Challenges… • AI can help with workplace assessment • Speed up the process • Reduce the cost • Remote or self assessment • AT recommendation • But there is still a need for • More data (new ATs) • Explainable algorithms • End-to-End solutions
  33. 33. Web Accessibility checking • Image recognition providing alternative text for images • Automatic context sensitive captions for images • Detecting contextual hyperlink text and flagging for correction • Form completion support
  34. 34. Machine learning – Intelligent personalised interfaces, text simplification, summarisation and writing support • Chatbots and Conversational AI as digital assistants • Complex text clarified for those with cognitive impairments • Summarisation for confusing convoluted sentences • Blog writing made easier with targeted support
  35. 35. Challenges… • Need to augment human abilities allowing for empathy and other human creativity • Increase data representing diversity • Develop understanding of the human condition at the time of use to be truly assistive • Solve misrecognition - simplification, summarisation and support can be too general and fail to respond to context or miss main facts.
  36. 36. Complex Communication Needs • Machine learning to adapt to Augmentative and Alternative Communication user’s situation and skills Livox (2mins)
  37. 37. Global Symbols • Using machine learning to classify the symbol's category based on ConceptNet. • Image recognition for concept detection • Deep learning for symbol style transfer and generation
  38. 38. Final Thoughts … “Some global institutions are beginning to examine how AI can impact and contribute to the social good, but there is much work to be done. This emerging “AI Divide” - if allowed to continue - could jeopardize equal treatment of people within and among nations. This asymmetry is a critical issue that must be addressed locally and globally.” Berkman Klein Center for Internet & Society "The top priority is creating a diverse, inclusive, substantial pipeline of ethical, competent and talented MSc graduates highly skilled in Machine Learning and Artificial Intelligence.." (British Computer Society)
  39. 39. Thank You Discussion AI and Inclusion Team