Eulexia A Wearable Aid For Spell-checking

Eulexia: A Wearable Aid For Spell-checking
SEBASTIAN CHEAH, BUSHENG LOU, MICHELLE WU, PHOEBE GAO, and ANDREW DU,
University of California, San Diego
With Eulexia, we wanted to create an application that would help close the learning gap between dyslexic and non dyslexic
individuals. Word processors can to spell checking, but often we find ourselves in situations (homework assignments, white
boarding during company meetings, etc) that require handwriting. We envision Eulexia as a solution to this issue by providing
a ubiquitous form of spell checking for hand written documents. Our prototype is currently limited to accurate processing of
typed text and neat handwritten text, but access to a better OCR engine would allow us to easily achieve our vision.
Additional Key Words and Phrases: Dyslexia, Ubiquitous Computing, Optical Character Recognition (OCR), Dictation, Spell-
check and Suggestions, Levenshtein Distance
ACM Reference Format:
Sebastian Cheah, Busheng Lou, Michelle Wu, Phoeba Gao, and Andrew Du. 2015. Eulexia: A Wearable Aid For Spell-checking.
ACM Trans. Appl. Percept. 0, 0, Article 0 ( 2015), 15 pages.
DOI: 0000001.0000001
1. INTRODUCTION
With 3-7% of the population diagnosedand 20% exhibiting degrees of symptomsdyslexia continues to
be one of the most common learning disabilities in America. Characterized by inaccurate word recog-
nition during reading and writing despite the student possessing normal intelligence, dyslexia often
correlates to dysgraphia as well. Dysgraphia mainly entails trouble with written expression, and often
people misspell words when writing by hand [Peterson and Pennington 2012]. Due to this cognitive
disorder, misspellings in the classroom have become commonplace. Existing non-ubiquitous applica-
tions to aid dyslexic students with language and visual processing include spellchecking features in
Word processors and voice assistants such as Amazon Echo. Our ubiquitous approach is to implement
a wearable solution for correcting spelling via a Google Glass application.
The general idea is to offer the wearer a simple and easy-to-use spellchecker that can be utilized in
daily classroom tasks such as when taking notes or completing classwork. By integrating our appli-
cation into a wearable, we provide a method for the user to correct his or her own spelling through
simple steps: tapping to take a picture and swiping between spelling suggestions. People who suffer
from dyslexia do not have issues with audio processing, so we also provide a text-to-speech feature that
enables the user to listen to the spelling of each suggestion.
In section 2, we will describe the motivations and background behind our work.
In section 3, we will outline how we approached designing features for dyslexic users.
In section 4, we will discuss the system development of our application, specifically the architecture
design, the technologies we used, and the features we implemented.
In section 5, we will detail how we tested and evaluated our application.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others
than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. Request permissions from permissions@acm.org.
c 2015 ACM. 1544-3558/2015/-ART0 $15.00
DOI: 0000001.0000001
ACM Transactions on Applied Perception, Vol. 0, No. 0, Article 0, Publication date: 2015.

0:2 • S. Cheah, B. Lou, M. Wu, P. Gao, and A. Du
In section 6, we will cover our collaboration, namely our team structure and how we solved issues.
In section 7, we will conclude our work and discuss future directions for our research.
2. MOTIVATION AND BACKGROUND
2.1 The Problem
Dyslexia is a condition that affects how the brain processes written and spoken language, which means
it specifically has an impact on writing, spelling, and speaking; however listening is not affected. Due
to this, students with dyslexia require methods of learning alternative to what conventionally exists
for the general population (i.e. listening to audio versions of a text since reading comprehension is
difficult).
A major side effect of this invisible illness is its tendency to affect an individual’s chances of educa-
tional success due to early experiences with the condition. Take, for example, two students, Jack and
Jill. Jack suffers from a mild form of dyslexia and was never diagnosed. Jill is not affected by dyslexia.
They both go through early elementary school, Jack struggling to keep up with other kids due to his
impaired reading and writing abilities, not really knowing why he is unable to learn as well as the
other students. Jill excels in all subjects, with a natural affinity for reading and obtaining new knowl-
edge. An important note is that Jack possesses intelligence akin to Jill, and simply becomes deficient
because he does not have the resources that he needs accessible in order to learn in an alternative way.
As the years go by, Jack begins to believe he simply is not as smart as the other students and gives
up trying on his assignments-slacking off and eventually dropping out of high school. Jill continues
to succeed in school-earning straight A’s, acceptance into a top university, and landing a competitive
position at a top company. This opportunistic gap is the motivation behind our application, Eulexia.
The technical problem-or the reason why this problem does not have a trivial solution-is the fact that
dyslexia cannot be cured. Individuals suffering from this condition simply learn to deal with dyslexia
in their own ways over time; however far too often, this happens too late in the game. We need to
target weaknesses caused by dyslexia in earlier stages of development so that effects do not grow
exponentially. Each individual with dyslexia is also unique, which is another reason why this problem
requires a ubiquitous solution, one that modifies itself continuously to fit each user’s needs.
2.2 Previous Solutions
2.2.1 Spell checking on computers. Modern word processors have built-in spell-checking features
such as Microsoft Word and Google Docs, and the latter also comes with a custom dictionary feature
which the user can use to add their commonly misspelled words. Though this feature is useful for
older dyslexics (and are currently used by them), none of these word processors process handwrit-
ing. Younger students may not regularly complete schoolwork on computers and are often assigned
worksheets in the classroom, and they are limited in resources to verify their spelling.
2.2.2 Text-to-Speech. There are multiple existing text-to-speech applications for documents on the
computers, which is useful to dyslexics because they have no issues with processing audio. DCODIA
is a mobile iOS text-to-speech application built for dyslexic students [Hall 2014]. A student takes a
picture of printed text and the application then dictates the words in the text aloud from the image.
While this approach is viable, the user must take out their mobile phone in order to position the camera
and take a picture, and this only works for printed text, not handwriting.
2.2.3 Voice assistants. Amazon Echo, a wireless speaker and voice command device from Ama-
zon.com, and Apple’s Siri provide another alternative solution for dyslexic students to spell-check.
Users can simply query these devices for the correct spelling of a word (i.e. ”Siri, how do you spell

Eulexia: A Wearable Aid For Proofreading • 0:3
dyslexia?”) [Amazon 2015] [Sheehan 2012]. However, this method becomes reliant on user pronunci-
ation and there is no secondary option if multiple words have the same pronunciation but different
spellings. As solely voice assistants, these devices do not come with a camera feature and cannot pro-
cess written text. Constantly asking for correct spelling also becomes tedious and inefficient for those
with dyslexia because the problem is not that they do not know how to spell a specific word, it is that
they do not realize when they have misspelled one.
3. DESIGN
3.1 Design Idea
Before we could begin to design our application, we needed to familiarize ourselves with the target
user’s needs. To do so, we interviewed Gini Keating, a UX designer with dyslexia who works at Qual-
comm Inc. Through this interview, we gathered the following information on dyslexia, as it pertains to
her case:
—Spelling is still a challenge for her and does not improve over time.
—She spells phonetically, so she often misspells the same word in numerous different ways. However,
she spells Spanish words correctly.
—Text-to-speech feature to spell out loud is very helpful (she sometimes uses Amazon Echo). She also
uses the speech-to-text feature on her smartphone.
—She uses a custom dictionary on her word processor to give her spelling suggestions.
—During company meetings, her coworker writes on the whiteboard for her because one of her autistic
coworkers will be too distracted by misspelled words.
—She would like to either see a rectangle around misspelled words or blur/grey out the other text.
We then had Keating try on Google Glass to locate any problems she has with its current user
interface design. She could read words easily and had no issues with the font or default white color.
With this gathered information, we decided to keep the existing Google Glass font (Roboto) as well
as its white color with the slight shadow behind it. Since a high contrast leads to easier readability, we
set the background of each slide to black. In order to achieve a real-time effect, we intended to leave
the camera in live preview mode while the user writes. When a misspelled word is detected, a red box
would appear around the word and the user can then tap to view spelling suggestions in list format,
with the incorrect spelling in red color (Fig.1.). We also decided to include a text-to-speech feature for
the user to listen to the different spelling suggestions as well as a custom dictionary that prioritizes
which suggestions to display first based on previous selections.
3.2 Design Prototypes
Our prototype consists of two specific features. First, we would like to provide an image overlaid with
bounding boxes over the misspelled words. Second, using a misspelled word as reference, provide sug-
gestions for the correct spelling. Figure 1 shows an example of how we visualized the Google Glass to
work.
3.3 Design Challenges
3.3.1 User Interface Design. One of the main challenges we faced when designing the user inter-
face was adopting existing design standards with Android and Google Glass applications. Because our
team had limited experience developing for Android, we had to initially spend some time to learn ba-
sic Android application flow. In addition, Google Glass has it’s own design and user flows that built
off of existing Android libraries that we had to learn. For example, the card view was one of Google

Fig. 1. Initial Prototype Designs
Glasss main drivers to getting user gestures, navigating between different parts of the application,
and passing information between activities.
Another challenge for the design, involved the limitations of Google Glass’s screen size. Because the
screen was only large enough to fit part of the user’s view, we had to make sure that we didn’t overload
the user with information that made the application difficult to understand.
We overcame this by prototyping some card views that were simple to understand, provided short
instructions to guide the user in the application flow, and offered simple signifiers – such as a scrolling
bar to indicate that they can navigate through a list – to assist them they in easily discovering new
parts of the application.
3.3.2 Word Recognition. Word Recognition, also known as optical character recognition (OCR), will
be the key component in the system design. There are many different kinds of OCR methods, like
online OCR or local OCR (i.e., using Tesseract library). We initially started by trying out local OCR
through (OpenCV and Tesseract), but because Glass has limited support and uses an outdated Android
SDK, we spent a good amount of time trying to fix build issues and overall OCR library integration
in our application. After we finally had local OCR working, we found that the results were extremely
poor. We tried doing basic image manipulation to improve the quality of the OCR, but still faced the
same issues that we initially ran into.
Our next option was using A9T9’s Online OCR API which did extremely well in recognizing printed
text, but did not do so well with handwritten text. We detail this challenge more in depth in later
sections (along with its trade-offs), but overall given the time constraints of the course, we decided to
proceed with this library.
3.3.3 Spell checking and suggestions. For spell checking, there are also many methods. For ex-
ample, there is already a library called spelling checker in Android, which will need internet to run
successfully. We need pay more attention to the decision for using a internet method or a local method,
as well as how and when to display the suggestion.
3.4 Evolution of Our Idea
Our idea evolved over time both from setbacks and also from valuable feedback. Initially, we envisioned
Eulexia as an augmented reality application, with incorrect spellings being detected in real time and
providing a list of word suggestions. After interviewing Gini Keating, we decided to add in dictation for
the suggested spellings. We also initially aimed for our application to conduct OCR locally so that it is
independent of an Internet connection. However, the Tesseract library did not produce usable results,
so we opted for an online API, which made our word recognition much more accurate.

Fig. 2. Timeline
Our original UI design also evolved to work cohesively with the UX. Prior to development of Eu-
lexia, we were unfamiliar with the Google Glass UI and believed that a list of spellings on one card
view would be sufficient. However, Glasss native card views show that lists on one card view are only
accessible via voice command, so for easy usability we switched to have each card view contain a single
spelling suggestion, and the user simply presses on the side of the Glass to hear the spelling out loud.
We also moved away from our original plan to have an augmented reality application because Google
Glass does not posses the amount of processing power needed to achieve this. Ideally, we would prefer
the live camera preview to be on as the user writes and then box misspelled words, but the Glass
overheats after a period of time, which currently limits this feature.
3.5 Timeline
Our timeline mainly featured work regarding two important functional modules: OCR and spell-check.
Without either one, our application would be unable to implement the features we seek to provide. Our
first priority as a group was to implement each module independently, and then combine the two, tak-
ing extracted text from OCR and passing it to the spell checking module. Our second priority then
consisted of features useful to the user, such as user navigation, image capture aids, and audio dicta-
tion. The timeline, that we set out to complete also involved user feedback from dyslexic individuals.
Overall, we have followed the timeline, illustrated in Figure 2, fairly closely. A mishap with our
initial OCR solution lead us to scrap most of our work in progress, forcing us to transition to other
OCR libraries. On the other hand, spell checking and suggestions was implemented and integrated
seamlessly. Unfortunately, we were not able to demonstrate the application to dyslexic individuals for
feedback due to time constraints.
3.6 Final Design
For our final design, we changed many aspects from our initial prototype. Instead of a card view con-
taining both the misspelled word and the spelling suggestions below it, we changed the views to con-
tain one word per card. This design choice to switch to large lettering was due to Google Glass’s limited
screen real estate. The incorrectly spelled word as well as each spelling suggestion is displayed in the
center of the screen and a simple single line instruction indicating the possible gesture as a caption
on the bottom (Figure 3). The user can also swipe down to return to any previous card view, which
is one of Google Glass’s native gestures. The scroll bar at the bottom of the spelling suggestions card
indicates that there are more suggestions and that the user may swipe right and left to swap between
suggestions.

Fig. 3. Reading and Correcting a misspelled word
Fig. 4. Module view of architecture
4. SYSTEM DEVELOPMENT
4.1 Architecture
Eulexia is designed as an Android application that uses the Glass Development Kit (GDK) preview
release add-on. With the GDK, Eulexia can be implemented as Glassware for use with the Google
Glass Explorer Edition. Also, we use additional libraries and APIs to implement certain features to be
discussed later.
The architecture design must support the following functions:
(1) Take an image
(2) Process image for text
(3) Spell-check input text
(4) Provide suggestions for misspelled words
(5) Audio feature for spelling of suggestions
(6) Save commonly misspelled words
The architecture is split into functional modules or domains in relation to the above list. To illus-
trate, Figure 4 shows each module with their respective transitions. Typically, each module receives an
input and provides an output to a user or module. This way, changes are isolated within each module,

Fig. 5. Comparison between Online OCR Service with different images
allowing for easier debugging efforts. The transitions between each module is aided with the use of the
GDK, allowing us to create transitions and read gestures from the user.
4.2 Technology used
4.2.1 Google Glass Explorer Edition. The application is created for use with the Google Glass Ex-
plorer Edition. We note that the Google glass allows us to work with its audio speaker, gesture com-
mands, and camera. These all will play a role in incorporating features for our Glassware.
The gesture commands are they key feature for the user to interact with our application. The user
would be prompted for certain gestures to navigate in the application. As for the audio speaker, it
allows us to provide a dictation aid when providing suggestions.
The camera on the Google Glass has some limiting features we have to consider when implementing
Eulexia. First, the camera takes pictures up to 5 megapixels uncompressed. This will affect the time it
will take to process the image during the OCR (Optical Character Recognition) process. Compression
can help speed up the processing times, but leads to degradation of quality and accuracy of results.
With this in mind, we consider accuracy as a top priority for the application when communicating
results to the user. No amount of compression is utilized, leading to the best possible image quality
at the cost of processing times. On a different note, users may have difﬁculty positioning the camera
when taking a picture of text. To counteract the problem with the camera capturing a small range of
the users vision, we provide a live camera preview feature. We make this feature readily available,
but optional in order to alleviate possible overheating problems. Additionally, the camera hardware is
limited to a single focus, leading to some low quality text in images. Nevertheless, before processing
the image with OCR, we provide an image preview of the image taken for the user to verify that the
picture taken contains the text they want to spell check.
4.2.2 OCR Service. An accurate OCR service is necessary to provide precise results for spell check-
ing. If this feature mistakenly extracts text from images, a spell checker would lose its ability to help
and inform the user. We decided upon the OCR online API from a9t9 (Autonomous Technology). This
online API is basically an extension of the Microsoft OCR library, which is easier to use and gives bet-
ter results than Tesseract, previously our ﬁrst choice for an OCR library [Technology 2015]. We also
consider other services, but research into other OCR online services yielded lesser results most of the
time. In our research, we note results comparing many services, shown in Figure 5. Comparison be-
tween Tesseract Online (a variant of Tesseract) and Abby Cloud SDK (which is similar to a9t9’s Online
OCR) show a clear difference in accuracy. Results were based on the Levenshtein distance between

Table I. Edit distance calculation
parameters
Parameter Cost
Remove a character 95
Insert a character 95
Swap adjoining characters 90
Substituting a character 100
Change case of a character 10
Suggestion Cost Threshold 140
Fig. 6. Application Flow Simplified
Fig. 7. Capturing an image with the viewfinder
output and truth input strings (this will be discussed later). Our test cases involve a low quality image
of text (ideally close to a low quality scan of a text document), and we prefer an OCR solution capable of
providing accurate results. Despite this, we note that there is a limitation with current OCR solutions
used with images taken from a camera, as they do not consistently provide accurate results. Also to
consider, since this is an online service, it may take some time to upload the image using the Internet.
To alleviate these processing times, loading bars are used to provide feedback to the user.
4.2.3 Spell-check and Suggestions. For spell checking and suggestions, the Jazzy library is used.
Jazzy is an open source spell checker, providing a set of APIs relevant to our project. Basically, Jazzy
takes as input a dictionary or word list with one word per line, to populate a dictionary hash map.
We check against this hash map when determining whether or not a word is misspelled. As for sug-
gestions, the Jazzy library utilizes the suggestion strategy from Aspell (the standard spell checker for
the GNU operating system). Basically, suggestions are based on thresholding an edit distance value
based on a variant of the Levenshtein distance between two strings [Atkinson 2004]. The edit distance
between two strings is defined as the number of operations, involving interchanging two adjacent let-
ters, changing one letter, deleting a letter, or adding a letter, to transform one string into the other.
To further clarify the default edit distance algorithm, please refer to Table I. Each operation has a
cost associated with it. Suggestions are generated and threshold-ed with costs less than the indicated
threshold value. The default configuration follows closely with Aspells suggestion strategy of returning
sounds-like suggestions up to two edit distances away.

4.3 Features
The first feature we show the user in the application flow is the camera view to assist them in taking
pictures shown in Figure 7. We provide a short instruction on the homescreen to signify that the
user is able to switch back and forth between camera view and the home screen. Once the user is
on the camera view they are able to see a live preview through the Google Glass’s camera. We chose
this feature because as we started testing the application, we realized that taking pictures without a
preview was much more difficult than we initially thought. The images were sometimes out of view
and we would have to then take multiple pictures because words were cut off. With the live preview, it
shows the user the actual image they would be capturing, so they are able to adjust to include/exclude
words that they want to check for spelling.
The next core feature of our application is the A9T9 Online OCR API based off of Microsoft’s OCR
engine. We use this API to process images that are taken with the Google Glass. We first start by
saving the image and uploading it to A9T9. After, we then parse the JSON response back and continue
with our application flow. One of the trade-offs of using an online library was that our application was
now dependent on having an internet connection. This caused testing and future development of the
application to be much more difficult as we had to make sure that Glass had proper internet connection
with each run of the application. Google Glass did not have capability of using on-campus WiFi as it
required secondary authentication which Glass did not support at all, so we relied heavily on mobile
hotspots. This was a problem in itself as we then had to schedule times to meet to develop with Glass
(see more in the Challenges section). And though this new dependency slowed development process,
this same feature can also be seen as a benefit for the application overall. One of the major stress points
of Glass is that it overheats when it does heavy computation, and it does overheat very frequently. So
having an intensive computational process such as OCR on the server-side (as opposed to local OCR)
means that we can avoid having Glass overheat at that point in the application flow.
Another point that should be taken note of is that this is an OCR library and not an ICR (Intelligent
Character Recognition) library, so A9T9 and our application mostly works on just printed characters.
Our results have been fairly good with using this library, but for handwritten text it fails because it
simply wasn’t designed to support it. Though this is definitely an issue for our application, we see it
more as an trade-off because we looked into many libraries apart from A9T9 and most seemed to be
not as reliable as the current one we were using. In addition, due to the time constraints of the course,
integrating and testing the pros/cons of all of them did not fit with the timeline of our project, so we
had to make a decision on the library we were going to use and then move from there. We stand by this
feature decision because it there were other features that needed just as much attention as OCR, and
in the future, we could definitely come back and swap out the library for an ICR library easily because
our application was designed to have loosely coupled modules.
After character detection is complete and processed back into our application flow, we then check
for any misspelled words. The user is prompted with card views of words that they misspelled and
they are then able to swipe forward and back to view all the words that were misspelled in the image
captured (as shown in Figure 8. We chose card views because it matches the existing design standards
of Google Glass application and it provides the proper discoverability, signifiers, and gestures to allow
the user to easily figure out how to navigate the card views given the minimal screen space of the
Google Glass. From this view, we offer a small instruction at the bottom of the view to tell users that
they are then able to tap on a misspelled word card view which will then prompt them with suggested
correct word spellings (as shown in Figure 9). We decided to use the open-source library Jazzy to detect
incorrectly spelled words and offer their respective correct suggestions (see more in the Jazzy section of

Fig. 8. Misspelled Words Found
Fig. 9. Suggestions for the misspelled word ”wayst”
the article). One major feature that Jazzy provides is that it allows us to use a custom dictionary so that
it allows us to account for words that aren’t in normal dictionaries such as slang and abbreviations.
The next major feature that Eulexia provides is that it offers text-to-speech spellings of words. This
feature was inspired through our interview with a person with dyslexia. She mentioned that she had
trouble with homonyms such as (”Waste” vs. ”Waist”), and that to overcome this barrier she would con-
stantly use Amazon Echo to ask Alexa how to spell the given word in the context of a sentence. Because
Amazon Echo needs to be stationed somewhere, it can’t be used on the go and so our application builds
off this feature by porting it to Google Glass.
In the suggested words card view the user is able to swipe back and forward to see scroll through
word suggestions for their misspelled word. We then provide a small instruction to the user that they
are able to long press on a word to hear the suggested word being spelled out character by character, as
seen in Figure 9. Through our interviews and research, we found that this feature was key to include
in our application because it allows one with dyslexia to draw their focus away from their eyes and
instead allow them to use hearing to be able to interpret and then spell the word correctly.
Finally the last major feature of our application is that it memorizes words that have been misspelled
frequently. We store misspelled words in a map and then use a priority queue to have that same word
offered higher in the suggested words list the next time a user misspells that word. We draw this
feature from our research because it builds off the fact that people with dyslexia still have a hard time
spelling words that they commonly misspell. In fact, in one of our interviews with a dyslexic person, we
noticed that the user actually misspelled her own name (even though she has probably wrote it down

Fig. 10. Testing process flow
countless times throughout her whole life). Her name wasn’t a common one, so it wouldn’t appear
in a normal dictionary, but in combination with Jazzy’s custom dictionary capabilities and our word
memorization feature, our application could detect whenever she incorrectly spells her name and then
offer her the corrected name spelling as the first suggestion provided – which would be therefore be
extremely customized to the user’s experience.
5. TESTING AND EVALUATION
5.1 Testing Process
The testing process follows closely with the flow of the application, which is illustrated in Figure 10.
First, the application begins with the title screen, which has some instructions and an application
logo. The user then simply taps to enter a live camera preview with the viewfinder. After capturing
and accepting the picture, the application will run the online OCR module, with a loading screen
indicating progress with saving the image and receiving the OCR response. With the OCR response,
we would get a string a words extracted from the image with or without misspelled words. The next
module for testing would be the spell check. The application will send the response from OCR to the
spell check module. The spell check should return a array of misspelled words, which can be used to
check whether the results are expected. Then for each misspelled words, the user can tap and go to the
suggestion module, with a list of suggested words related to each misspelled words.
5.2 White-box Testing
White-box testing, also known as clear box testing, is used to test internal structures (i.e., each module).
For the main module, we create possible inputs and send them to the special module and then check
whether we get the desired outputs.
For the OCR module, we write some neat handwriting on the whiteboard, consisting of some correct
words and misspelled words. Then we use the Google Glass to take a picture, and send the picture
to the OCR module. In the debug mode, we can easily print out the response from OCR and check
whether it detect the words in the whiteboard correctly. One interesting we find in the testing is that,
the a9t9 OCR will detect the new lines as escape characters such as ”/r/n”, which is not expected. To
solve this, we remove all the escape characters ”/r/n”.
For the spell check module, a string of words are used as input to the spell check module. Tests
include several misspelled words in the string. The expected results should be a hashmap with mis-
spelled words as the key and maps with a list of suggestion words. To avoid too many suggestion words
slowing down the speed of the application, we only capture the suggestions up to two edit distances
away, as previously mentioned in our system design.

5.3 Black-box Testing
Black-box testing is used to examine the functionality of the application without peering into its inter-
nal structures or workings. In the testing process, we do some role playing to implement the Black-box
testing. Behaving as a user, we tested our application with writing from a whiteboard and typed text
on a laptop screen. In this part of testing, we mainly focus on the correctness and the processing times
of our application. Also we evaluated the user experience of our application (i.e., how long it will take
users to get used to our application and what difﬁculties they may meet when running) and the poten-
tial issues, such as Google Glass overheating and limited battery life with high power consumption.
5.4 Evaluation
5.4.1 Performance. The recognition works near-perfect on print type words, the correction rate can
be more than 95% when testing with 10 words. For neat handwriting, the OCR also works satisfactorily.
The spell check module works well and we can update our dictionary for future use. The text-to-speech
works great; we can modify the speech speed to make it more user-friendly. Also in the processing of
running the application, we provide operation instructions to help the user navigate forward within
our application. Since we built our application in a clear and concise way, users would soon learn how
to operate the application properly. Also, we provide loading screen when processing OCR and spelling
checking to inform the users of the current running status and the current application progress (such
as saving pictures or receiving OCR responses).
5.4.2 Limitations. The limitation of the application mainly consists of the OCR results. The speed
of OCR is still an issue, currently it may need 10-20 seconds to ﬁnishing OCR and spell check. We need
fast speed and less time running OCR if we want the application be used in real world applications.
Also, only print type words and neat handwriting words can be recognized. Detecting messy hand-
writing will easily result in mistakes, citing a need for more powerful and accurate OCR methods. For
hardware and device limitations, the Google Glass itself has some weaknesses. It will generate a lot of
heat, causing the device to overheat in some cases. Additionally, wearing the Google Glass for too long
will cause discomfort with the right ear, because the battery and other hardware situated on the right
side makes it heavier than typical eyewear. The battery can also be a problem: with full battery, the
google glass can last for around 2 hours. Finally, another limitation is the fact that the application has
to be connected to the Internet to function fully. Without it, the OCR module will be unable to process
images.
6. COLLABORATION
6.1 Contributions
6.1.1 Application Contributions
—Phoebe Gao: Native OCR, result caching, user interface design, gestures, camera
—Michelle Wu: User interface design, testing/debugging, interview
—Andrew Du: Native OCR, live preview, interview
—Sebastian Cheah: Online OCR, spellcheck, suggestions
—Busheng Lou: Online Spelling check,testing/debugging, evaluation
6.1.2 Report Contributions
—Phoebe Gao: Executive summary, motivation and background, evolution of ideas, screenshots
—Michelle Wu: Introduction, previous solutions, design, collaboration, conclusion

—Andrew Du: Features, conclusion
—Sebastian Cheah: System development, Document formatting
—Busheng Lou: Design challenges, testing and evaluation, future work
6.2 Collaboration Tools and Problems
Our team split into smaller subgroups to work on different aspects of our application. For OCR imple-
mentation, Phoebe, Andrew, and Sebastian worked together and for the user interface design, Phoebe
and Michelle collaborated on user experience and gestures. Michelle and Andrew also made up the
sub-team that scheduled an interview with the dyslexic UI researcher. Busheng tested and debugged
the written code. During our weekly meetings, we worked together to brainstorm application ideas, so-
lutions to current implementation problems, as well as troubleshoot. The largest problem we face was
schedule coordination, which we overcame by constant online communication via Facebook Messenger
and pairing up with team members with similar schedules. Github was our repository host of choice,
and each team member working on a different part of the application created a different branch from
our main repository. Once a feature was fully functional, the responsible team member created a pull
request where it was reviewed before merging his or her code with the master branch. For final testing,
we went through the whole application flow, and each time we got bugs, we isolate the conditions to
reproduce it, and then review the problem through general debugging techniques. Some bugs were too
complicated to debug, so we refer to the sources who worked on similar functionalities to help solve the
bugs.
7. CONCLUSION AND FUTURE WORK
7.1 Conclusion
We first started this project by expanding out problem spaces that we noticed throughout our lives.
Given the technologies we were able to choose from and our course goal to help others through ubiq-
uitous computing, we tackled the problem space of assisting users with dyslexia. This space in itself
is extremely challenging to address because the dyslexia is a disability that is rarely discussed even
if one was diagnosed with it or even if one has shown symptoms with it. In one of our user interviews
with a person with dyslexia, she mentioned that even though she and her colleagues knew that she
had this disability, it was not a comfortable subject to bring up. In fact, she argued that dyslexia is a
type of disability that one would try to hide, and if a person had dyslexia and was somewhat conscious
of it, they would not be willing to get themselves tested or diagnosed with it mostly due to the neg-
ative connotations that revolve around the disability. So with that said, one of our major challenges
was designing for a group of users that we had no prior experience of working with. It was extremely
difficult to design in a space that could not be simulated easily. We couldn’t address the basic design
questions such as: What was it like to read/write with dyslexia? What were their pain points? How
did they feel about existing solutions? What problems haven’t been addressed yet? And are there any
underlying problems that stem from reading/writing with dyslexia? And even trying to find people that
were dyslexic to interview/test our application was one pitfall that we encountered.
One pitfall that we initially tried to avoid in the beginning of the project was being too solution
driven. We tried to stray away from this common engineering misstep by collaborative brainstorming
uses cases, user flows, challenged points in the process of writing, and conducting user interviews.
However, as the project dragged on throughout the course, we ran into issues with technical imple-
mentations of libraries with Google Glass and even the limitations on the hardware with Glass itself.
We spent a good amount of time trying to overcome these issues to get back on track, but in doing so,
we derailed ourselves from constantly involving the customer to having the general thought of them in

mind. Though this may seem as a poor design decision, we believe it was the right one to make because
it allowed us to push out the essentials of our application and build from there. We made this choice
due to the time constraints of the course, and a tradeoff of this decision is that we weren’t able to test
the user experience of the application thoroughly. But this problem isn’t too big of an issue because we
could then test users in the future, and having that foundation of good design process in the beginning
of the project allows us to be confident that the decisions that we make now are geared towards the
user.
One of the core features of Eulexia was character recognition. We addressed this problem in the
current application version by utilizing an online OCR library. Though it works great for printed text,
it doesn’t work nearly as well for handwritten text so a major future direction of the project would
be to instead use an ICR library to detect handwritten text. In addition, the interface of displaying
misspelled words and providing more context to the user about where the misspelled words located,
and how we should then go about informing the user of correct spellings could also be improved.
Eulexia does well in providing initial steps in ubiquitous computing towards helping users with
dyslexia. Though there were hardware and software limitations with Google Glass, we tackled each
problem by keeping the user in mind and made design decisions and tradeoffs based off of our initial
research with our customers. And overall, we are pleased with the progress and insights that we able
to provide to others that choose to address this problem in the future with our product.
7.2 Future work
Ideally, our vision is for Eulexia to become a fully ubiquitous application. Future work can be done
in terms of offering the user different modes of spell checking to choose from. Once Google Glass
improves its processing power, one feasible mode can be real-time spell checking, in which the live
preview feature of Glass is left on for the duration of the user’s note taking. Once the user misspells a
word, the Glass will beep, indicating the error and the user can then stop writing and perform a spell-
check to make a correction, all achieved in real-time. The benefit of this feature is that it is similar to
a non-dyslexic person changing his or her mind and returning to correct their mistake. The mode we
currently have implemented is to be used when a user has a long trail of thought and does not want to
be disturbed until he or she is finished writing. Once done writing, he or she performs the spell-check
to receive all the misspelled words displayed at one time, and can then jump from word to word.
Improving our currently existing mode entails more customization towards the user’s needs. Dyslexia
varies in different degrees among people, so a multi-sensory approach is key to developing the optimal
spell checking application. This means catering to the user in both visual and auditory cases, and ap-
proaches to achieving this include having customization options with different font choices and colors,
as well as allowing users to choose between highlighting words, displaying them in different colors, or
boxing/circling words. Offering a definition or sentence usage feature for each suggestion is another
area where customization can be key. An audio spelling may not be enough for the user to decide
which spelling he or she intended, so pulling up an example usage can give more context. We can also
look into examining the context of the sentence that the user wrote the misspelled word in. A machine
learning algorithm that can adjust to the user’s writing style is a more efficient way of predicting which
spelling suggestion the user will choose.
Another core feature, spell-checking and suggestion generation, can be improved upon. First, some
words can be detected as misspelled despite the user disapproving of the application’s conclusion. It is
possible to modify the dictionary word list in memory and on disk to help with this use case. Jazzy sup-
ports this partially, with word additions to the dictionary, but missing the opposite: deletion support.
With no way of reverting an addition without manually modifying the dictionary word list directly, we
did not include this in the initial application. In the future, we can modify the Jazzy library, and recom-

Eulexia: A Wearable Aid For Proofreading • 15
pile a new version with this functionality in place. If we do this, we must consider how to support such
features with the existing gestures available and remaining to us in our implementation of a Google
Glass app. Second, the loading of a dictionary word list into memory versus disk is an ongoing issue.
We can look into newer algorithms with a suitable balance for use with the system specifications of
the Google Glass. Such algorithms may have an effect on dictionary modification support, as we have
seen implementations utilize specifically formatted dictionaries generated using a fixed word list. It
can lead to the loss of addition and deletion operations on word lists. Nevertheless, we can extend upon
the base classes already in Jazzy, providing our own implementations in the future.
REFERENCES
Amazon. 2015. Ask Alexa. (2015). https://www.amazon.com/gp/help/customer/display.html?nodeId=201549800
Kevin Atkinson. 2004. Aspell Suggestion Strategy. (2004). http://aspell.net/man-html/Aspell-Suggestion-Strategy.html
Joshua Hall. 2014. DCODIA app helps student with dyslexia read, saves families thousands. (2014). http://techpoint.org/2014/
08/dyslexia-technology-software/
Robert L. Peterson and Bruce F. Pennington. 2012. Developmental dyslexia. (2012). http://pediatrics.uchicago.edu/chiefs/DBP/
documents/reading%20pdf/Dyslexia.Peterson.pdf
Michael Sheehan. 2012. IOS 5 TIP: USING SIRI AS A SPELL CHECKER AND SPELLING
ASSISTANT SPELLING TEST RESULTS! (2012). http://www.hightechdad.com/2012/01/23/
ios-5-tip-using-siri-as-a-spell-checker-and-spelling-assistant-spelling-test-results/
Autonomous Technology. 2015. ”Hosted Microsoft OCR library”: Free OCR API. (2015). http://blog.a9t9.com/2015/09/ocr-api.
html

Online Appendix to:
Eulexia: A Wearable Aid For Spell-checking
SEBASTIAN CHEAH, BUSHENG LOU, MICHELLE WU, PHOEBE GAO, and ANDREW DU,
University of California, San Diego
A. KEY PROBLEMS ADDRESSED
A.1 Difficulties with writing for individuals with dyslexia
This obstacle can hinder performance and confidence in school, or the workplace, putting dyslexic in-
dividuals at a great disadvantage. Eulexia would be able to help balance out the playing field and even
prevent young dyslexic students from falling behind in school and losing faith in their own abilities.
A.2 Differentiation between correctly spelled and misspelled words
For a dyslexic, one issue with trying to learn the correct spellings of words is that even though they
might be looking at the correct spelling of a word, they would still not be aware of how to spell it
because they might not be able to process the characters like everyone else. Eulexia addresses this
issue by using the microphone on the Google Glass to spell out the word. In one interview, we found
that the interviewee found it useful to look at words as shapes rather than individual characters.
This could be a potential future direction to investigate different ways in which we can help dyslexic
individuals get a better handle on both reading and writing.
A.3 Tendency for repeat mistakes
This problem exists not only within the dyslexic population, but it exists within the entire human pop-
ulation as well. Eulexia addresses this issue by keeping track of the user’s most frequently misspelled
words so that it makes the learning process smoother and more organized.
B. DESIGN IDEAS
B.1 Ubiquity
We wanted our application to integrate flawlessly into the workflow of dyslexic individuals, so we
originally came up with the idea to use augmented reality to highlight the user’s misspelled words in
real time. However, we were not able to achieve this due to the current technological limitations of the
Google glass. Instead, we came up with the idea of allowing the user to manually take a picture of the
text in order to check for misspelled words. We reasoned that this might actually be a better option in
the end, since it’s a lot less obtrusive and it allows the user to be in control of when he or she wants to
verify his/her work.
B.2 Personalization
We really liked the idea of allowing personalization within Eulexia, so we came up with the idea of
keeping track of the users’ most commonly misspelled words and providing the user with different
mediums of presentation (such as text or audio) for spelling suggestions.
c 2015 ACM. 1544-3558/2015/-ART0 $15.00
DOI: 0000001.0000001

App–2 • S. Cheah, B. Lou, M. Wu, P. Gao, and A. Du
C. IMPLEMENTATION
C.1 OCR
For OCR, we used a9t9’s (Autonomous Technology) online OCR API to process images that are captured
with the Google Glass’ built in camera.
C.2 Spell-check
For spell-check, we used the Jazzy library and downloaded a dictionary from Jazzys website. With
greater computational power on the Glass, we would be able to use larger, more advanced dictionaries.
C.3 Word Caching
We used local storage (on the Google Glass) of a hash map to keep track of most commonly misspelled
words for the user.
D. FEATURES
(1) Detect misspelled words
(2) Spelling suggestions
(3) Audio playback of spellings
(4) Word caching
E. EVALUATION
Our prototype looks very promising, but because of current hardware limitations and time constraints,
we were unable to implement all of our design ideas. We are extremely glad to say that the OCR
for our prototype is working on neatly handwritten text, and we believe Eulexia has the potential to
make a big difference in the lives of those who are affected by dyslexia. In particular, its features of
audio playback and word caching will allow Eulexia to cater to the dyslexic user and personalize their
learning experience.
F. FUTURE DIRECTIONS
Future directions include ﬁnding a better OCR engine that is more adept at recognizing handwriting.
Perhaps even a machine learning model would be appropriate in order to allow personalization of
the character recognition. We also hope to do more research on different types of dyslexia and offer
different modes to better suit users with different needs. If Google glass ever improves its processing
power, we can also look into providing a real time mode.

Eulexia A Wearable Aid For Spell-checking

More Related Content

What's hot

Viewers also liked

Similar to Eulexia A Wearable Aid For Spell-checking

Eulexia A Wearable Aid For Spell-checking