A study that shows significantly-improved comprehension, by people with intellectual disabilities, of Web text simplified with operationalized plain-language standards. This work has significant promise for training artificial intelligence how to create simplified text. Presented at the CSUN Assistive Technology Conference 2019.

1. Creating Simple Web Text for
People with ID and to Train AI
John Rochford
Director, INDEX Program
Eunice Kennedy Shriver Center
Invited Expert
World Wide Web Consortium

2. INDEX Staff: Representing the Diversity of our Community

3. DisabilityInfo.org Home Page

4. Cognitive Disabilities
• Clinical Examples
•intellectual disabilities, Autism/ASD, learning disabilities,
TBI, Schizophrenia, Alzheimer’s, Depression, aging, etc.
•Functional Examples
•cannot understand typical text
•are easily distracted
•are confused by inconsistent interfaces
•do not recover easily from errors

5. Cognitive Web Accessibility Examples, 1 to 3
Deliver content in more than one way.
Give guided help.
Work well with assistive technology.

6. Cognitive Web Accessibility Examples, 4 to 6
Focus attention.
Augment text with relevant images.
Use simple language.

7. Web Text Simplification Goal
Make Web text so
simple it can be
understood the first
time it is read.

8. Simplified Text Example
Typical: “When the process of freeing a vehicle that has
been stuck results in ruts or holes, the operator will fill
the rut or hole created by such activity before removing
the vehicle from the immediate area.”
Simple: “If you make a hole while freeing a stuck vehicle,
you must fill the hole before you drive away.”
Source

9. Simple Web Text Examples
Simple Wikipedia
Easy To Read News

10. 2 Ways We Are Simplifying Text
Short Term: Operationalize
plain language guidelines.
Long Term: Develop
Artificial Intelligence
capabilities.

11. Simplifying Web Text: Plain Language
Make plain-language guidelines
easy to follow to simplify text.
Make
Test if simplified text is more
understandable than typical text.
Test
Publish techniques for people to
simplify text easily and reliably.
Publish

12. Operationalize Plain Language Guidelines: Steps 1 & 2
Gathered 33 plain language guidelines.
Narrowed to 19 parsable by algorithms.

13. Operationalize Plain Language Guidelines: Steps 3 & 4
Refined to 6 that consistently reduced
reading grade levels for 4 experimenters.
Conducted trials to determine the order
they should be applied.

14. Plain Language Guidelines In Order: 1st Set
1. Use short, simple words.
2. Write short sentences.
3. Avoid using acronyms and abbreviations.

15. Plain Language Guidelines In Order: 2nd Set
4. Use an active voice in the present tense.
5. Use correct grammar and spelling.
6. Remove proper nouns.

16. Example Operationalization
Rule 1: Use short, simple words.
Replace 3+ syllable words with words that:
• have 1 or 2 syllables; and
• are commonly used.

17. Easy Text Study Advertisement

18. Study Participants
25 adults with ID
Tested at 3rd to 5th grade
17 to 63 years of age
Paid for their time

19. Text Passages, Questions, Answers
• 8 Collected Text Passages
• Simplified 4 Text Passages
• 2 Literal Questions
• 2 Inferential Questions
• 3 multiple-choice answers

20. Sample Typical Passage Webpage
Lux Level is a luxurious, in-theatre dining
experience at select theatres. Movie-goers can
indulge themselves with premium reserved-
seating, in-seat dining throughout the show, as well
as other special amenities. Each seat is equipped
with a server call button so your server is always
there when you need them. This truly is the finest
movie-going experience available today.

21. Sample Typical Passage, Literal Question Webpage
Lux Level is a luxurious, in-theatre
dining experience at select theatres.
Movie-goers can indulge themselves
with premium reserved-seating, in-
seat dining throughout the show, as
well as other special amenities. Each
seat is equipped with a server call
button so your server is always there
when you need them. This truly is
the finest movie-going experience
available today.
Where can Lux Level be
found?
1. At all theatres.
2. At a few theatres.
3. At no theatres yet.

22. Sample Passage To Be Simplified
On Miniclip you can play games to compile games
statistics such as high scores, and rankings on your
own player page and can receive awards for playing
games. You can also create your own Miniclip avatar
with clothing and accessories to represent you in
our fun Miniclip games. You can play single player
games or play advanced massive multiplayer games
with other players from around the world.

23. Sample Simplified Passage Webpage
On Miniclip, you can play games to collect game stats.
These can be high scores and rankings on your own
player page. You can get awards for playing games. You
can also create your own Miniclip avatar with clothing
and extras. It can be you in our fun Miniclip games. You
can play single player games. Or you can play advanced,
huge games with many players. Those players live
around the world.

24. Sample Simple Passage, Inferential Question Webpage
On Miniclip, you can play games to
collect game stats. These can be high
scores and rankings on your own player
page. You can get awards for playing
games. You can also create your own
Miniclip avatar with clothing and extras.
It can be you in our fun Miniclip games.
You can play single player games. Or you
can play advanced, huge games with
many players. Those players live around
the world.
How can you play with people
who live far away?
1. Pick an advanced game.
2. Pick a single player game.
3. Pick an award.

25. Study Results: 3 Subgroups
60%
70%
80%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Very Poor Poor to Fair Good to Very Good
% Correct Answers – Typical Text Passages

26. Study Results: Simplified Text By Subgroup

27. Study Results: All Participants: Answers
13%
23%
Inferential Questions
Literal Questions
% Improvement of Simple- Over
Typical-Text Passage Answers

28. Study Results: Overall Comprehension Improvement
18%

29. Reason # 1 Why Study Result Is Important
Reasonable assumption: people with higher
levels of executive functioning will also
benefit from text simplification.
Examples: people with learning disabilities,
functional illiteracy, non-native language.

30. Reason # 2 Why Study Result Is Important
Parallel between people with ID and AI: both have a
very-limited understanding of the world.
Despite that, operationalized plain language guidelines
help people with ID understand simplified text.
We build them into algorithms to help AI “understand”
simplified text sufficiently to recognize and create it.

31. Knowledge Representation And Reasoning
A knowledge base has facts about the world.
AI can reason about those facts.
AI can deduce new facts.

32. AI Training Data Sources And Preprocessing
Sentence Pairs (Typical and Simple)
•manually matched from Wikipedia and Simple Wikipedia;
•automatically matched with algorithms; and/or
•matched or created by crowdsourcing.
Data Preprocessing
•Open-source tools with sample data sets.
•Examples: Google word2vec, Stanford NLTK.

33. Artificial Intelligence
Pattern Recognition
Picture Source

34. This Person Does Not Exist Dot Com

35. We Must Not Teach AI
Our Biases
1 in 4 U.S. adults have a disability.
• Accurate distribution in data sets
• Self-driving cars example (source: Jutta)
Example Barriers to Self Identification
• Insurance higher cost or denial
• Employment and housing discrimination
IBM AI Fairness Toolkit on GitHub Image Source

36. Partners
•Amazon Machine
Learning Research
Awards
• IBM Watson
•UMass Boston
•Worcester
Polytechnic
University

37. Thank You!
•John Rochford
•John.Rochford@umassmed.edu
•Twitter: @ClearHelper
•Slide Deck: bit.ly/CSUN2019