Technologies for Dyslexia

Technologies for Dyslexia
Maria Rauschenberger
Max Planck Institute for Software Systems, Saarland, Germany.
14th May 2020, at Intuit
Rauschenberger (MPI) Twitter: Rauschii 1 / 1 14th May 2020, at Intuit 1 / 28

Outline
1 Common Characteristics of Dyslexia
2 Intervention and Assisted Technology
3 Screening of Dyslexia
Early and playful Screening of Dyslexia
Methodology
Game and Content Design
Results
Conclusion
4 Take Away

Common Characteristics of Dyslexia

Deﬁnition of Dyslexia
What is dyslexia?
(American Psychiatric Association, 2013)

Dyslexia is ...
A speciﬁc learning disorder/diﬃculty
Reading and writing impairments
Wrongly associated to reduced intelligent
Important cause of school failure

Dyslexia is ...
A specific learning disorder/difficulty
Reading and writing impairments
Wrongly associated to reduced intelligent
Important cause of school failure
Affects 5–15% of the population
Multiple factors investigated to discover its cause
(Catts et al., 2017; De Zubicaray and Schiller, 2018)
Dyslexia is a combination of characteristics
(De Zubicaray and Schiller, 2018)

What kind of Errors do People with Dyslexia make?

What kind of Errors do People with Dyslexia make?
Error Classiﬁcation: Substitution, Insertion, Omission, ...
Depending on the Orthography of the language
(Rauschenberger et al., 2016)

Language-Independent Indicators
Evidence from literature & strongly related to dyslexia
Similarities between languages, German, English and Spanish
(Rello et al., 2016; Rauschenberger et al., 2016, 2017)
— Error category Substitution: exchanging a letter for another one
(Rello et al., 2016; Rauschenberger et al., 2017)
Examples:
— b, d, p, q
— versbrochen (versprochen, ‘promise’)
— Walt (Wald, ‘forest’)
(Rello et al., 2016; Rauschenberger et al., 2016)
What happens if the error produces a new word?

Intervention and Assisted Technology
Real World Errors

Intervention with Errors
https://itunes.apple.com/de/app/dyseggxia/id534986729?mt=8

Assisted Reading for Dyslexia
Text Customization
e.g., Font size (>16pt)
Font Type such as Arial
(italics and serif should be avoided (Rello and Baeza-Yates, 2016,
2017; British Dyslexia Association, 2018).)
Text Simpliﬁcation
“Since dyslexia is a learning disorder and not a cognitive disability, the
simpliﬁcation depends more on the typographical errors and not on
the complexity of the content.” (Rauschenberger et al., 2019b)
Text to Speech

Assisted Writing for Dyslexia
“[...] people with dyslexia experience more often negative feedback on the
writing which can trigger or increase their stress and anxiety.”
(Reynolds and Wu, 2018)
Spelling Correction
Text Suggestions
Dictation

Screening of Dyslexia
Screening
How to detect if you have dyslexia?

Examples of Dyslexia Screening
Digital
Readers
Dytective
GraphoGame
Lexercise Screener
Nessy
Pre-Readers
AGTB 5–12
DYSL–X
Game–Collection
Lexa
Others
Diagnostischer Rechtschreibtest
Hamburger Leseprobe
Special hardware, i.e.,
— MRI or fMRI Scans
(Paulesu et al., 2014; Tamboer et al., 2016;
Paz-Alonso et al., 2018)
— Eye-tracking
(Asvestopoulou et al., 2019)
Specialist staﬀ, i.e.,
— learning therapists
(Rauschenberger et al., 2019b)

To sum up...
All reading and spelling tests need a ...
— minimum knowledge of phonological awareness,
— grammar,
— and vocabulary
... to predict risk of dyslexia is a speciﬁc language task.

Screening of Dyslexia Early and playful Screening of Dyslexia
What we have done!
Screening a person without using linguistic knowledge
while having fun!

Screening of Dyslexia Methodology
Screening Risk of Dyslexia through a Web-Game
using Language-Independent Content and Machine Learning.
Challenge → Approach
Hidden disorder → Indicators
Diﬀerent languages → Language-independent
Content → Iterative design
Design to measure → Human-centered design
No global standards → Data science
External factors → Transparency
Collecting health data → Ethical standards
Small data → Avoid over-ﬁtting
Rauschenberger et al. (2019a)

Screening of Dyslexia Methodology
Steps
1 Designing a language-independent content
game
— Visual and auditory cues
2 Measuring the interaction (e.g., score, duration)
— Between children with and without dyslexia
— Age 7-12 year’s old
3 Analysing the interaction measures
— Traditional statistical methods
— Existing machine learning classiﬁcation
Child playing.

Screening of Dyslexia Game and Content Design
Game and Content Design
Visual part with the priming of the
target cue symbol (left) and then the
(right) nine-squared design including the
distractors for each symbol.
Auditory part for the ﬁrst two clicks on
two sound cards (left) and then when a
pair of equal sounds is found (right).

Prototype MusVis — Visual Part

Prototype MusVis — Auditory Part

Related Language-Independent Content
Related with auditory and visual content that refers mainly to one
single acoustic or visual indicator
e.g., frequency or horizontal similarity

Screening of Dyslexia Results
There are significant statistical differences
between children with and without dyslexia
when playing a game with auditory and visual content!
Data set Visual Auditory
ES (n =153) total clicks, first click, hits,
and efficiency (4)
4th click, duration, and
average (3)
DE (n =149) / /
ALL (n = 313) total clicks (1) /
Overview of the significant dependent variables of MusVis. Since all variables
were not normally distributed (Shapiro-Wilk test), we applied the
Mann-Whitney U Test.
⇒ Results let us assume, we can measure over different languages
with the same content.
Rauschenberger et al. (2018)

Is it possible to predict risk of dyslexia
based on language-independent
auditory and visual content
using a game and machine learning
for diﬀerent languages?

Prediction Results
Clas. Data set Feat. Recall Precision F1 Accuracy
RF DE 5 0.77 0.78 0.75 0.74
RFW DE 5 0.75 0.75 0.74 0.73
Baseline DE 0.60 0.37 0.46 0.50
ETC ES 20 0.76 0.76 0.75 0.69
RF ES 5 0.74 0.73 0.72 0.65
Baseline ES 0.68 0.46 0.55 0.50
GB ALL 20 0.66 0.65 0.65 0.61
GB ALL 5 0.64 0.64 0.63 0.59
Baseline ALL 0.63 0.40 0.49 0.50
Best results of the diﬀerent classiﬁers, features and data sets. Results are ordered
by the best F1-score and accuracy.

Prediction Discussion
Why is ALL NOT performing better than DE?
Caused by difference in the informative features
Features canceling each other out
⇒ Expected due to very different languages.
Is the F1-Score and Accuracy high enough?
Indicators are probably not as strong as reading and writing
mistakes
Early indication can equal earlier intervention.
⇒ Improve with content related to more characteristics
⇒ Therefore, we aim to optimize the recall and F1-score by
finding as many participants with dyslexia as possible.

Screening of Dyslexia Conclusion
Conclusions
Screening Risk of Dyslexia through a Web-Game using Language-Independent Content
and Machine Learning.
1 To the best of our knowledge this is the first time that risk of
dyslexia is screened using a language-independent content
web-based game and machine-learning.
2 Our method yields an accuracy of 0.74 for German and 0.69 for
Spanish as well as a F1-score of 0.75 for German and 0.75 for
Spanish, using Random Forests and Extra Trees, respectively.
3 However, different models are needed for the prediction in
different languages, something that in retrospect made sense.

Take Away
Web Accessibility - Technologies for Dyslexia
There is a lot of Technology ...
Assisted Reading for Dyslexia
Assisted Writing for Dyslexia
Dyslexia Screening
Dyslexia Intervention
Rauschenberger et al. (2019b)
https://link.springer.com/chapter/10.1007/978-1-4471-7440-0_31

Take Away
Questions?

References
American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental
Disorders. American Psychiatric Association, London, England.
Asvestopoulou, T., Manousaki, V., Psistakis, A., Smyrnakis, I., Andreadakis, V.,
Aslanides, I. M., and Papadopouli, M. (2019). Dyslexml: Screening tool for dyslexia
using machine learning.
British Dyslexia Association (2018). Dyslexia Style Guide 2018.
Catts, H. W., McIlraith, A., Bridges, M. S., and Nielsen, D. C. (2017). Viewing a
phonological deﬁcit within a multifactorial model of dyslexia. Reading and Writing,
30(3):613–629.
De Zubicaray, G. and Schiller, N. O. (2018). The Oxford handbook of neurolinguistics.
Oxford University Press, New York, NY.
Paulesu, E., Danelli, L., and Berlingeri, M. (2014). Reading the dyslexic brain: multiple
dysfunctional routes revealed by a new meta-analysis of PET and fMRI activation
studies. Frontiers in human neuroscience, 8:830.
Paz-Alonso, P. M., Oliver, M., Lerma-Usabiaga, G., Caballero-Gaudes, C., Quiñones, I.,
Suárez-Coalla, P., Duñabeitia, J. A., Cuetos, F., and Carreiras, M. (2018). Neural
correlates of phonological, orthographic and semantic reading processing in dyslexia.
NeuroImage. Clinical, 20:433–447.
Rauschenberger, M., Füchsel, S., Rello, L., Bayarri, C., and Thomaschewski, J. (2015).
Exercises for German-speaking children with dyslexia. In Human-Computer
Interaction–INTERACT 2015, pages 445–452, Bamberg, Germany.

References
Rauschenberger, M., Füchsel, S., Rello, L., and Thomaschewski, J. (2017). DysList
German resource: A language resource of German errors written by children with
dyslexia. https://zenodo.org/record/809801#.XOVWRFMzYWo. [Online, accessed
06-June-2019].
Rauschenberger, M., Rello, L., and Baeza-Yates, R. (2019a). Predicting Dyslexia
through a Web-Game using Language-Independent Content (in progress. In in
progress, N.N. in progress.
Rauschenberger, M., Rello, L., and Baeza-Yates, R. (2019b). Technologies for Dyslexia.
In Yesilada, Y. and Harper, S., editors, Web Accessibility Book, volume 1, pages
603–627. Springer-Verlag London, London, 2 edition.
Rauschenberger, M., Rello, L., Baeza-Yates, R., and Bigham, J. P. (2018). Towards
language independent detection of dyslexia with a web-based game. In W4A ’18: The
Internet of Accessible Things, pages 4–6, Lyon, France. ACM.
Rauschenberger, M., Rello, L., Füchsel, S., and Thomaschewski, J. (2016). A language
resource of German errors written by children with dyslexia. In Proceedings of the
Tenth International Conference on Language Resources and Evaluation (LREC 2016),
Paris, France. European Language Resources Association (ELRA).
Rello, L. and Baeza-Yates, R. (2016). The Eﬀect of Font Type on Screen Readability by
People with Dyslexia. ACM Transactions on Accessible Computing, 8(4):1–33.
Rello, L. and Baeza-Yates, R. (2017). How to present more readable text for people with
dyslexia. Universal Access in the Information Society, 16(1):29–49.

References
Rello, L., Baeza-Yates, R., and Llisterri, J. (2016). A resource of errors written in
Spanish by people with dyslexia and its linguistic, phonetic and visual analysis.
Language Resources and Evaluation, 51(2):1–30.
Reynolds, L. and Wu, S. (2018). "I’m Never Happy with What I Write": Challenges and
Strategies of People with Dyslexia on Social Media. In Proceedings of the 12th
International Conferenceon Web and Social Media, page 280. The AAAI Press, Palo
Alto, California USA.
Tamboer, P., Vorst, H. C. M., Ghebreab, S., and Scholte, H. S. (2016). Machine
learning and dyslexia: Classiﬁcation of individual structural neuro-imaging scans of
students with and without dyslexia. NeuroImage. Clinical, 11:508–514.

Technologies for Dyslexia

More Related Content

Similar to Technologies for Dyslexia

Recently uploaded

Technologies for Dyslexia