Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Technologies for Dyslexia

105 views

Published on

Ted Drake from Intuit and this year's general chair of Web4All Conference 2020 is organising a series of lunch and learn session to take advantage of people working from home! The W4A keynotes of Vivienne Conway, Director (Web Key IT Pty Ltd) and Ricardo Baeza-Yates (NTENT & Khoury College of Computer Sciences Northeastern University at Silicon Valley) have been already shared. Now, they ask me to give an overview of research and technologies around dyslexia which is next Thursday on the 14th of May !

Published in: Education
  • Be the first to comment

  • Be the first to like this

Technologies for Dyslexia

  1. 1. Technologies for Dyslexia Maria Rauschenberger Max Planck Institute for Software Systems, Saarland, Germany. 14th May 2020, at Intuit Rauschenberger (MPI) Twitter: Rauschii 1 / 1 14th May 2020, at Intuit 1 / 28
  2. 2. Outline 1 Common Characteristics of Dyslexia 2 Intervention and Assisted Technology 3 Screening of Dyslexia Early and playful Screening of Dyslexia Methodology Game and Content Design Results Conclusion 4 Take Away Rauschenberger (MPI) Twitter: Rauschii 2 / 2 14th May 2020, at Intuit 2 / 28
  3. 3. Common Characteristics of Dyslexia Common Characteristics of Dyslexia Rauschenberger (MPI) Twitter: Rauschii 3 / 3 14th May 2020, at Intuit 3 / 28
  4. 4. Common Characteristics of Dyslexia Definition of Dyslexia What is dyslexia? (American Psychiatric Association, 2013) Rauschenberger (MPI) Twitter: Rauschii 4 / 4 14th May 2020, at Intuit 4 / 28
  5. 5. Common Characteristics of Dyslexia Dyslexia is ... Definition of Dyslexia A specific learning disorder/difficulty (American Psychiatric Association, 2013) Reading and writing impairments Wrongly associated to reduced intelligent Important cause of school failure Rauschenberger (MPI) Twitter: Rauschii 5 / 5 14th May 2020, at Intuit 5 / 28
  6. 6. Common Characteristics of Dyslexia Dyslexia is ... Definition of Dyslexia A specific learning disorder/difficulty (American Psychiatric Association, 2013) Reading and writing impairments Wrongly associated to reduced intelligent Important cause of school failure Affects 5–15% of the population Multiple factors investigated to discover its cause (Catts et al., 2017; De Zubicaray and Schiller, 2018) Dyslexia is a combination of characteristics (De Zubicaray and Schiller, 2018) Rauschenberger (MPI) Twitter: Rauschii 5 / 5 14th May 2020, at Intuit 5 / 28
  7. 7. Common Characteristics of Dyslexia What kind of Errors do People with Dyslexia make? Rauschenberger (MPI) Twitter: Rauschii 6 / 6 14th May 2020, at Intuit 6 / 28
  8. 8. Common Characteristics of Dyslexia What kind of Errors do People with Dyslexia make? Error Classification: Substitution, Insertion, Omission, ... Depending on the Orthography of the language (Rauschenberger et al., 2016) Rauschenberger (MPI) Twitter: Rauschii 6 / 6 14th May 2020, at Intuit 6 / 28
  9. 9. Common Characteristics of Dyslexia Language-Independent Indicators Evidence from literature & strongly related to dyslexia Similarities between languages, German, English and Spanish (Rello et al., 2016; Rauschenberger et al., 2016, 2017) — Error category Substitution: exchanging a letter for another one (Rello et al., 2016; Rauschenberger et al., 2017) Examples: — b, d, p, q — versbrochen (versprochen, ‘promise’) — Walt (Wald, ‘forest’) (Rello et al., 2016; Rauschenberger et al., 2016) What happens if the error produces a new word? Rauschenberger (MPI) Twitter: Rauschii 7 / 7 14th May 2020, at Intuit 7 / 28
  10. 10. Intervention and Assisted Technology Real World Errors (Rauschenberger et al., 2015) Rauschenberger (MPI) Twitter: Rauschii 8 / 8 14th May 2020, at Intuit 8 / 28
  11. 11. Intervention and Assisted Technology Intervention with Errors (Rauschenberger et al., 2015) https://itunes.apple.com/de/app/dyseggxia/id534986729?mt=8 Rauschenberger (MPI) Twitter: Rauschii 9 / 9 14th May 2020, at Intuit 9 / 28
  12. 12. Intervention and Assisted Technology Assisted Reading for Dyslexia Text Customization e.g., Font size (>16pt) Font Type such as Arial (italics and serif should be avoided (Rello and Baeza-Yates, 2016, 2017; British Dyslexia Association, 2018).) Text Simplification “Since dyslexia is a learning disorder and not a cognitive disability, the simplification depends more on the typographical errors and not on the complexity of the content.” (Rauschenberger et al., 2019b) Text to Speech Rauschenberger (MPI) Twitter: Rauschii 10 / 10 14th May 2020, at Intuit 10 / 28
  13. 13. Intervention and Assisted Technology Assisted Writing for Dyslexia “[...] people with dyslexia experience more often negative feedback on the writing which can trigger or increase their stress and anxiety.” (Reynolds and Wu, 2018) Spelling Correction Text Suggestions Dictation Rauschenberger (MPI) Twitter: Rauschii 11 / 11 14th May 2020, at Intuit 11 / 28
  14. 14. Screening of Dyslexia Screening How to detect if you have dyslexia? Rauschenberger (MPI) Twitter: Rauschii 12 / 12 14th May 2020, at Intuit 12 / 28
  15. 15. Screening of Dyslexia Examples of Dyslexia Screening Digital Readers Dytective GraphoGame Lexercise Screener Nessy Pre-Readers AGTB 5–12 DYSL–X Game–Collection Lexa Others Diagnostischer Rechtschreibtest Hamburger Leseprobe Special hardware, i.e., — MRI or fMRI Scans (Paulesu et al., 2014; Tamboer et al., 2016; Paz-Alonso et al., 2018) — Eye-tracking (Asvestopoulou et al., 2019) Specialist staff, i.e., — learning therapists (Rauschenberger et al., 2019b) Rauschenberger (MPI) Twitter: Rauschii 13 / 13 14th May 2020, at Intuit 13 / 28
  16. 16. Screening of Dyslexia To sum up... All reading and spelling tests need a ... — minimum knowledge of phonological awareness, — grammar, — and vocabulary ... to predict risk of dyslexia is a specific language task. Rauschenberger (MPI) Twitter: Rauschii 14 / 14 14th May 2020, at Intuit 14 / 28
  17. 17. Screening of Dyslexia Early and playful Screening of Dyslexia What we have done! Screening a person without using linguistic knowledge while having fun! Rauschenberger (MPI) Twitter: Rauschii 15 / 15 14th May 2020, at Intuit 15 / 28
  18. 18. Screening of Dyslexia Methodology Screening Risk of Dyslexia through a Web-Game using Language-Independent Content and Machine Learning. Challenge → Approach Hidden disorder → Indicators Different languages → Language-independent Content → Iterative design Design to measure → Human-centered design No global standards → Data science External factors → Transparency Collecting health data → Ethical standards Small data → Avoid over-fitting Rauschenberger et al. (2019a) Rauschenberger (MPI) Twitter: Rauschii 16 / 16 14th May 2020, at Intuit 16 / 28
  19. 19. Screening of Dyslexia Methodology Steps 1 Designing a language-independent content game — Visual and auditory cues 2 Measuring the interaction (e.g., score, duration) — Between children with and without dyslexia — Age 7-12 year’s old 3 Analysing the interaction measures — Traditional statistical methods — Existing machine learning classification Child playing. Rauschenberger et al. (2019a) Rauschenberger (MPI) Twitter: Rauschii 17 / 17 14th May 2020, at Intuit 17 / 28
  20. 20. Screening of Dyslexia Game and Content Design Game and Content Design Visual part with the priming of the target cue symbol (left) and then the (right) nine-squared design including the distractors for each symbol. Auditory part for the first two clicks on two sound cards (left) and then when a pair of equal sounds is found (right). Rauschenberger et al. (2019a) Rauschenberger (MPI) Twitter: Rauschii 18 / 18 14th May 2020, at Intuit 18 / 28
  21. 21. Screening of Dyslexia Game and Content Design Prototype MusVis — Visual Part Rauschenberger (MPI) Twitter: Rauschii 19 / 19 14th May 2020, at Intuit 19 / 28
  22. 22. Screening of Dyslexia Game and Content Design Prototype MusVis — Auditory Part Rauschenberger (MPI) Twitter: Rauschii 20 / 20 14th May 2020, at Intuit 20 / 28
  23. 23. Screening of Dyslexia Game and Content Design Related Language-Independent Content Related with auditory and visual content that refers mainly to one single acoustic or visual indicator e.g., frequency or horizontal similarity Rauschenberger (MPI) Twitter: Rauschii 21 / 21 14th May 2020, at Intuit 21 / 28
  24. 24. Screening of Dyslexia Results There are significant statistical differences between children with and without dyslexia when playing a game with auditory and visual content! Data set Visual Auditory ES (n =153) total clicks, first click, hits, and efficiency (4) 4th click, duration, and average (3) DE (n =149) / / ALL (n = 313) total clicks (1) / Overview of the significant dependent variables of MusVis. Since all variables were not normally distributed (Shapiro-Wilk test), we applied the Mann-Whitney U Test. ⇒ Results let us assume, we can measure over different languages with the same content. Rauschenberger et al. (2018) Rauschenberger (MPI) Twitter: Rauschii 22 / 22 14th May 2020, at Intuit 22 / 28
  25. 25. Screening of Dyslexia Results There are significant statistical differences between children with and without dyslexia when playing a game with auditory and visual content! Data set Visual Auditory ES (n =153) total clicks, first click, hits, and efficiency (4) 4th click, duration, and average (3) DE (n =149) / / ALL (n = 313) total clicks (1) / Overview of the significant dependent variables of MusVis. Since all variables were not normally distributed (Shapiro-Wilk test), we applied the Mann-Whitney U Test. ⇒ Results let us assume, we can measure over different languages with the same content. Rauschenberger et al. (2018) Rauschenberger (MPI) Twitter: Rauschii 22 / 22 14th May 2020, at Intuit 22 / 28
  26. 26. Screening of Dyslexia Results Is it possible to predict risk of dyslexia based on language-independent auditory and visual content using a game and machine learning for different languages? Rauschenberger (MPI) Twitter: Rauschii 23 / 23 14th May 2020, at Intuit 23 / 28
  27. 27. Screening of Dyslexia Results Prediction Results Clas. Data set Feat. Recall Precision F1 Accuracy RF DE 5 0.77 0.78 0.75 0.74 RFW DE 5 0.75 0.75 0.74 0.73 Baseline DE 0.60 0.37 0.46 0.50 ETC ES 20 0.76 0.76 0.75 0.69 RF ES 5 0.74 0.73 0.72 0.65 Baseline ES 0.68 0.46 0.55 0.50 GB ALL 20 0.66 0.65 0.65 0.61 GB ALL 5 0.64 0.64 0.63 0.59 Baseline ALL 0.63 0.40 0.49 0.50 Best results of the different classifiers, features and data sets. Results are ordered by the best F1-score and accuracy. Rauschenberger (MPI) Twitter: Rauschii 24 / 24 14th May 2020, at Intuit 24 / 28
  28. 28. Screening of Dyslexia Results Prediction Discussion Why is ALL NOT performing better than DE? Caused by difference in the informative features Features canceling each other out ⇒ Expected due to very different languages. Is the F1-Score and Accuracy high enough? Indicators are probably not as strong as reading and writing mistakes Early indication can equal earlier intervention. ⇒ Improve with content related to more characteristics ⇒ Therefore, we aim to optimize the recall and F1-score by finding as many participants with dyslexia as possible. Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
  29. 29. Screening of Dyslexia Results Prediction Discussion Why is ALL NOT performing better than DE? Caused by difference in the informative features Features canceling each other out ⇒ Expected due to very different languages. Is the F1-Score and Accuracy high enough? Indicators are probably not as strong as reading and writing mistakes Early indication can equal earlier intervention. ⇒ Improve with content related to more characteristics ⇒ Therefore, we aim to optimize the recall and F1-score by finding as many participants with dyslexia as possible. Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
  30. 30. Screening of Dyslexia Results Prediction Discussion Why is ALL NOT performing better than DE? Caused by difference in the informative features Features canceling each other out ⇒ Expected due to very different languages. Is the F1-Score and Accuracy high enough? Indicators are probably not as strong as reading and writing mistakes Early indication can equal earlier intervention. ⇒ Improve with content related to more characteristics ⇒ Therefore, we aim to optimize the recall and F1-score by finding as many participants with dyslexia as possible. Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
  31. 31. Screening of Dyslexia Results Prediction Discussion Why is ALL NOT performing better than DE? Caused by difference in the informative features Features canceling each other out ⇒ Expected due to very different languages. Is the F1-Score and Accuracy high enough? Indicators are probably not as strong as reading and writing mistakes Early indication can equal earlier intervention. ⇒ Improve with content related to more characteristics ⇒ Therefore, we aim to optimize the recall and F1-score by finding as many participants with dyslexia as possible. Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
  32. 32. Screening of Dyslexia Conclusion Conclusions Screening Risk of Dyslexia through a Web-Game using Language-Independent Content and Machine Learning. 1 To the best of our knowledge this is the first time that risk of dyslexia is screened using a language-independent content web-based game and machine-learning. 2 Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. 3 However, different models are needed for the prediction in different languages, something that in retrospect made sense. Rauschenberger (MPI) Twitter: Rauschii 26 / 26 14th May 2020, at Intuit 26 / 28
  33. 33. Screening of Dyslexia Conclusion Conclusions Screening Risk of Dyslexia through a Web-Game using Language-Independent Content and Machine Learning. 1 To the best of our knowledge this is the first time that risk of dyslexia is screened using a language-independent content web-based game and machine-learning. 2 Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. 3 However, different models are needed for the prediction in different languages, something that in retrospect made sense. Rauschenberger (MPI) Twitter: Rauschii 26 / 26 14th May 2020, at Intuit 26 / 28
  34. 34. Screening of Dyslexia Conclusion Conclusions Screening Risk of Dyslexia through a Web-Game using Language-Independent Content and Machine Learning. 1 To the best of our knowledge this is the first time that risk of dyslexia is screened using a language-independent content web-based game and machine-learning. 2 Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. 3 However, different models are needed for the prediction in different languages, something that in retrospect made sense. Rauschenberger (MPI) Twitter: Rauschii 26 / 26 14th May 2020, at Intuit 26 / 28
  35. 35. Take Away Web Accessibility - Technologies for Dyslexia There is a lot of Technology ... Assisted Reading for Dyslexia Assisted Writing for Dyslexia Dyslexia Screening Dyslexia Intervention Rauschenberger et al. (2019b) https://link.springer.com/chapter/10.1007/978-1-4471-7440-0_31 Rauschenberger (MPI) Twitter: Rauschii 27 / 27 14th May 2020, at Intuit 27 / 28
  36. 36. Take Away Questions? Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28
  37. 37. References American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders. American Psychiatric Association, London, England. Asvestopoulou, T., Manousaki, V., Psistakis, A., Smyrnakis, I., Andreadakis, V., Aslanides, I. M., and Papadopouli, M. (2019). Dyslexml: Screening tool for dyslexia using machine learning. British Dyslexia Association (2018). Dyslexia Style Guide 2018. Catts, H. W., McIlraith, A., Bridges, M. S., and Nielsen, D. C. (2017). Viewing a phonological deficit within a multifactorial model of dyslexia. Reading and Writing, 30(3):613–629. De Zubicaray, G. and Schiller, N. O. (2018). The Oxford handbook of neurolinguistics. Oxford University Press, New York, NY. Paulesu, E., Danelli, L., and Berlingeri, M. (2014). Reading the dyslexic brain: multiple dysfunctional routes revealed by a new meta-analysis of PET and fMRI activation studies. Frontiers in human neuroscience, 8:830. Paz-Alonso, P. M., Oliver, M., Lerma-Usabiaga, G., Caballero-Gaudes, C., Quiñones, I., Suárez-Coalla, P., Duñabeitia, J. A., Cuetos, F., and Carreiras, M. (2018). Neural correlates of phonological, orthographic and semantic reading processing in dyslexia. NeuroImage. Clinical, 20:433–447. Rauschenberger, M., Füchsel, S., Rello, L., Bayarri, C., and Thomaschewski, J. (2015). Exercises for German-speaking children with dyslexia. In Human-Computer Interaction–INTERACT 2015, pages 445–452, Bamberg, Germany. Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28
  38. 38. References Rauschenberger, M., Füchsel, S., Rello, L., and Thomaschewski, J. (2017). DysList German resource: A language resource of German errors written by children with dyslexia. https://zenodo.org/record/809801#.XOVWRFMzYWo. [Online, accessed 06-June-2019]. Rauschenberger, M., Rello, L., and Baeza-Yates, R. (2019a). Predicting Dyslexia through a Web-Game using Language-Independent Content (in progress. In in progress, N.N. in progress. Rauschenberger, M., Rello, L., and Baeza-Yates, R. (2019b). Technologies for Dyslexia. In Yesilada, Y. and Harper, S., editors, Web Accessibility Book, volume 1, pages 603–627. Springer-Verlag London, London, 2 edition. Rauschenberger, M., Rello, L., Baeza-Yates, R., and Bigham, J. P. (2018). Towards language independent detection of dyslexia with a web-based game. In W4A ’18: The Internet of Accessible Things, pages 4–6, Lyon, France. ACM. Rauschenberger, M., Rello, L., Füchsel, S., and Thomaschewski, J. (2016). A language resource of German errors written by children with dyslexia. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France. European Language Resources Association (ELRA). Rello, L. and Baeza-Yates, R. (2016). The Effect of Font Type on Screen Readability by People with Dyslexia. ACM Transactions on Accessible Computing, 8(4):1–33. Rello, L. and Baeza-Yates, R. (2017). How to present more readable text for people with dyslexia. Universal Access in the Information Society, 16(1):29–49. Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28
  39. 39. References Rello, L., Baeza-Yates, R., and Llisterri, J. (2016). A resource of errors written in Spanish by people with dyslexia and its linguistic, phonetic and visual analysis. Language Resources and Evaluation, 51(2):1–30. Reynolds, L. and Wu, S. (2018). "I’m Never Happy with What I Write": Challenges and Strategies of People with Dyslexia on Social Media. In Proceedings of the 12th International Conferenceon Web and Social Media, page 280. The AAAI Press, Palo Alto, California USA. Tamboer, P., Vorst, H. C. M., Ghebreab, S., and Scholte, H. S. (2016). Machine learning and dyslexia: Classification of individual structural neuro-imaging scans of students with and without dyslexia. NeuroImage. Clinical, 11:508–514. Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28

×