This an overview of ELSA's proprietary speech recognition technology. Find out how voice technology can help with language learning, and what differentiates ELSA's model from the others.

1. SPEECH RECOGNITION
FOR LANGUAGE LEARNING

2. In the age of globalization, English fluency has become the pivotal skill for any professional or
student to develop,with special focus on the ability to confidently communicate.In recent years,
language education has constantly been in the spotlight for development.Novel methodologies
are extensively researched and applied in order to deliver the best learning experience, with
advancement in automation navigating the wave. In the current context of COVID-19 disabling
traditional schooling in many ways, technology-aided language education is growing at an
unprecedented speed.
One of the most popular technologies applied in English teaching is Automatic Speech
Recognition (ASR). ASR converts the audio speech input into written text, based on big
data-searching language corpora and finding matching patterns (Carrier, 2017). This
technology will support the development of listening and speaking – the two foundational pillars
of English communication. The application of ASR in language pedagogy offers all the benefits
of interactive learning in a regular classroom setup, as well as several prevailing characteristics
making it the ideal solution for the 4.0 digitalization of education.
The ELSA Speak App is equipped with our own spoken language
assessment technology, which will be discussed in detail in this
paper along with the benefits of its application in English learning
through real success stories with our partners.

3. The ELSA ASR technology is developed entirely in-house - we have a dedicated research
department responsible for creating and evolving all of our speech assessment algorithms. Our
technology inherits a lot from statistical signal processing techniques traditionally used in
speech recognition. In this respect, we use deep neural networks to learn what the correct
pronunciation is,trained from thousands of hours of native speech in American English as well as
other machine learning algorithms to classify intonation, rhythm, etc. Our pipeline also includes
novel signal processing to extract information from the speech signal that is then converted to
features to be classified into correct, almost correct, and incorrect classes.
The main factor differentiating our technology from speech recognition research (and especially
from APIs from major cloud computing services) is what we target to identify. While speech
recognition research aims at identifying what words were spoken regardless of the speaker’s
accent, we focus on distinguishing between correct and incorrect pronunciations and
benchmark the speech against actual native speaker standards. Traditionally, as speech
recognition systems get more training data from non-native speakers,they also lose the ability to
tell a user that his accent is not correct, and are even more unlikely to be able to identify why. In
our case, as we gather more data from users and evolve the system, we will improve our
pronunciation detection algorithms and the quality of our feedback
1. TECHNICAL OVERVIEW:

4. To perform effective communication training, in our
analysis of the student’s speaking proficiency, we
independently analyze the following dimensions:
Pronunciation: This skill evaluates the correct generation of American
English sounds, identifying when the user speaks with a non-native accent
and offering concrete actionable feedback to reduce it.
Word stress: This skill measures how well the student stresses the right
syllable in multi-syllable words so that the word is correctly understood by the
listener, namely in the case of homonyms.
Fluency: This skill measures the ability of the student to speak with the right
rhythm and apply pauses where necessary.This is a suprasegmental skill that
is fundamental to bringing students to a good conversational level with English
natives.
Sentence intonation: This skill evaluates 2 different aspects of intonation.
On one hand,it checks whether there is adequate pitch variation to make the
speech pleasant and not monotonous. On the other hand, it inspects
whether the student is adding enough emphasis to the prominent words in
the sentence, conveying the expected message to the listener.
Listening: This skill refers to the ability of a student to differentiate among
the different sounds in English, which many times convey different
meanings. This is currently done via the use of minimal pairs (i.e. words
whose only difference is the tested phoneme).

5. In the ELSA app, we combine our proprietary speech analysis
technology with a unique scoring system and gamified learning
experience to evaluate the user’s input and give accurate and
actionable feedback, simultaneously designing engaging and
interactive lessons.
When the student starts using the application they are invited to take a placement test that
evaluates their overall English proficiency. The assessment test highlights to the student what
English mistakes they are making and creates a learning path to improve proficiency. The ELSA
proficiency score shown to the user is a combination of the scores obtained in each of the skills
outlined above. The ELSA interface is also designed to support the learning journey. On the
home screen are planets created as part of the core user experience. These planets represent
the phonemes of English that will help build their pronunciation at the atomic level.
For example,the assessment test evaluates users’ pronunciation of all American English sounds
and gives a score for each planet at the end. Once the result is generated, the planets will be
reordered on the home screen with the highest score at the top and the lowest at the bottom.
This experience allows users to easily access the sounds that they want to practice first. Finally,
ELSA offers multiple types of lessons in the form of games that train the different skills above.
There are games that focus on a single skill (e.g. listening and word stress games) and others
that train multiple skills, like the conversation games, which evaluate pronunciation, intonation,
and fluency. This gamified approach can boost learners’ performance by up to 90%, according
to a study by ScienceDirect in 2020.

6. 2.1 Interactive learning
One-on-one interaction is known to be the best way for language learners to improve their
conversational skills (Strik, Neri and Cucchiarini, 2011). This practice gives them instant
corrective feedback, not only keeping the knowledge fresh but also developing a natural flow
and vocabulary suitable for real-life interaction.
However,organizing one-on-one tutoring with trained instructors is complicated for students as
well as institutions. The cost and logistics will be substantial over time, not to mention the
demand to customize the curriculum to each student’s need and may even compromise the
teaching quality.
The ELSA Speak App may offer a new approach to personal tutoring. Our ASR will ensure
immediate feedback to speech input. The built-in library of 120+ topics of daily conversation
helps students develop vocab naturally. All learning curricula is tailored for individual needs by
the ELSA A.I coach.
2.2 Pronunciation improvement
Recent studies have shown that, although ignored in the past, pronunciation plays an important
role in effective communication and can improve the cognitive level of learners (Strik,2009).Our
pronunciation scoring is measured against the standardized IELTS speaking band and the app
produces a native-accent version of the sample sentence. The student thus can learn with
quality similar to a teacher’s lesson, or better, to a conversation with the natives. Pronunciation
does not simply mean one can correctly produce each phoneme, it also covers listening
comprehension. Many students report a common challenge being interpretation of foreign
accents (Fathi Sidig Sidgi and Jelani Shaari, 2017). As illustrated in section 1, ELSA technology
can detect, identify accents, and design training respectively.
An independent study conducted at the University of Yogyakarta (Indonesia) among students of
the English Department saw a 30% increase in average pronunciation scoring from before and
after 3 rounds of practicing with the ELSA Speak App. The research also reflected positive
feedbacks from users on the learning experience (Kholis, 2021)
2. THE BENEFITS OF ASR

7. 2.3 Confidence boost & Experience enhancement
No longer bound to the classroom setting and schedule, the student can now learn anytime,
anywhere at their own pace, with continual access to other additive learning materials, such as
visualizations and recordings.According to Golonka et al.(2012),learners often report favorable
experiences with A.I-assisted language training and more motivation to use English in real life.
Bodnar et al. (2017) even demonstrated that receiving corrections from an ASR-based system
had no negative impact on the learners' enjoyment,willingness to practice,or self-efficacy.All in
all, the technology offers a private, stress-free learning environment that ultimately improves
learner’s confidence and overall learning experience.
The study mentioned above by Kholis and a similar research conducted by Darsih and partners
(2021) both reported over 85% happy feedback on the ELSA app.
A 2-month pilot program with our EdTech partner saw
that attendance rate increased 4x, total engagement is
8x compared to current stats. The organization saved a
total of 359 hours of teachers’ time.
Assuming a $20-40 per hour salary, ELSA is projected
to help save $400,000 - $800,000 per year per 1,000
students.
Improved confidence Rating maintained on
the App stores.

8. REQUEST A DEMO:
https://elsaspeak.com/en/english-for-companies/demo
A 2021 report from J’son & Partners Consulting estimates that language learning constitutes ¼
of the EdTech market, and English takes up 80% of this ratio. According to data from the
mentioned report, the global EdTech market size is 268 billion USD in 2021, implying 44 billion
USD for the English segment. On the supply side, nearly 20 billion USD of private funds was
poured into learning technology companies during the first half of 2021, according to Metaari’s
Chief Researcher. This corresponds well to a reported 4.1 billion invested artificial intelligence
development for the sector per the A.I Index Report 2021 by Stanford University.
That is to say, EdTech for English has been and will continue to be the target of revolution and
investment. This paper has discussed how automated spoken language technology will
certainly be the key in the development of communicative language training. Educational
institutions and EdTech companies alike are all looking for ways of integrating spoken language
technology into their existing program to enhance training effectiveness and learner’s
experience.
CONCLUSION
Proudly equipped with our AI-powered language assessment
technology, ELSA is the next step to take your solution to the next
level. Book a consultation today to learn how we can help you.