For a summary of the research and link to published research paper: https://www.wallstreetenglish.com/blog/speech-recognition-for-improving-pronunciation-skills/

1. How effective is speech
recognition software for
improving pronunciation skills?
Bindi Clements, PhD. IATEFL conference 2021
@bindi_clements
linkedin.com/in/bindiclements/

2. 03
Study Aims &
Research
Questions
06
Results &
Discussion
01
ASR &
Pronunciation
Instruction
02
05
Beliefs and
Perceptions
Participants &
Instruments
07
Conclusion
04
ASR
Pronunciation
Activities
08
References

3. 1. ASR and
Pronunciation
Instruction

4. How is ASR used for Pronunciation Instruction?
• ASR = Automatic Speech Recognition
• Uses NLP to transform speech into text
• Dictation software (e.g. Dragon Dictation)
• E.g. Students read aloud and notice which
words are not recognised.
• Virtual assistants (e.g. Alexa, Siri, Google
Assistant)
• E.g. Students practice asking questions and
get responses
• Automatic pronunciation assessment (ASA) used
in CALL (e.g. Speechace, Wall Street English,
ELSA, etc.)
• E.g. Students record turns in a dialogue and
are given feedback on their pronunciation

5. Why use ASR for pronunciation training?
• Develop speaking proficiency
• Opportunities for more and different speaking practice in and out of the
classroom.
• Fill a gap
• Some teachers do not have the time, confidence, or training to give
pronunciation feedback.
• Objective feedback
• Learners find it difficult to hear pronunciation errors without feedback
• Promote learner autonomy
• Pronunciation is a difficult skill to learning on your own. With ASR feedback,
students can practice on their own without a teacher
• Low-anxiety environment
• Learners do not feel judged when practising speaking
• Personalised learning
• Students can go at their own pace and practice according to their needs
Chen (2011)
Golonka et al. (2014)
Levis (2007; 2018)
McCrocklin (2016; 2019)
Perez Castillejo (2021)
Thomson and Derwing (2015)
Underwood (2021)

6. Challenges when using ASR
• Inaccuracy of feedback (Levis 2007;
Golonka et al. 2014)
• Pedagogy is often traditional (drill-
oriented) (Levis 2007)
• Lack of actionable feedback for
learner to improve (Levis 2007)
• Native-like pronunciation is often
assumed to be the model (Thomson
and Derwing 2014)
• Content in apps is often ‘fixed’ and
may not be flexible to learner needs.
• Depends on the technology. ASA more
accurate than off-the-shelf ASR
• This depends on instructional design of
activities.
• More of an issue for off-the-shelf ASR
rather than for ASA technology
• Can be mitigated with instructional
design and feedback (and not focusing
on ‘perfect’ scores)
• Can be mitigated when used as part of a
comprehensive learning programme

7. Does ASR work for improving pronunciation ?
Key questions (Levis 2007):
• Does the instruction lead to
improvements on specific
features trained?
• Does the learning last over
time?
• Does learning in one
pronunciation topic lead to
improvement in other areas?
Evidence:
• Yes, there is strong evidence of this
(Levis 2007; Thomson and Derwing,
2015; Golonka et al., 2014)
• Few studies include delayed post-
testing (Levis 2007)
• Some evidence (Levis, 2007, Golonka et
al., 2014), improves learner autonomy
(McCrocklin 2016)

8. 2. Beliefs and
Perceptions

9. Beliefs & Perceptions for Pronunciation Skills
• Emotion (anxiety) and motivation can affect the acquisition of second
language speech (Saito, Dewaele, Abe & In’name, 2018)
• Perception of difficulty of pronunciation skills linked to L1/nationality (cf.
Cenoz & Lecumberri, 1999; Simon & Taverniers, 2011).
• Some beliefs regarding effective learning strategies linked to L1/nationality
(Nowacka, 2012).
• Belief that adult learners cannot achieve ‘native-like’ pronunciation
(Marinova-Todd, Marshall, & Snow, 2000).
• Beliefs and perceptions about ASR-CALL can be influenced (Borges, 2014).

10. 3. Study Aims &
Research
Questions

11. Study Aims
• To understand student beliefs about the
effectiveness of ASR-CALL activities for
improving pronunciation.
• To identify differences in feature usage of
ASR-CALL activities according to (a) age and
(b) L1/nationality.
• To identify differences in student beliefs and
perceptions about CALL-ASR according to (a)
age and (b) L1/nationality.

12. Research Questions
1. Do students think pronunciation activities with
ASR help them improve their pronunciation?
2. Do students in four countries (China, Vietnam,
Italy and Saudi Arabia) make different use of
ASR activity features?
3. Are there differences between age groups and
L1/nationality in students’ beliefs and
perceptions on learning pronunciation using
ASR?

13. 4. ASR
Pronunciation
Activities

14. 5. Wall Street English study cycle
1. Listen. (key new language through
video).
2. Pronounce. (listen and repeat ‘chunks’
with ASR feedback).
3. Practice. (online, various, some ASR
feedback).
4. Speak. (guided production, ASR
feedback at lower levels) .
5. Confirm. (assessment with
remediation).
6. Read & Write. (online ‘workbooks’)
7. Encounter. (classroom activation,
validation, assessment)

15. Pronounce: Repeat from Audio
• Listen to model audio
• Listen to own recording
• Maximum 3 attempts

16. Practice: Read and Record
• No model audio
• Listen to own recording
• Maximum 3 attempts

17. Speak: Conversation
• Listen to model audio
• Listen to own recording
• Maximum 4 attempts

18. 5. Participants
& Instruments

19. Participants: Unique Students
China
Beijing x2
Vietnam
Ho-Chi-Minh x2
Saudi Arabia
Jeddah x1
Italy
Rome x1
1,153 1,192 171
Total
2,867
351

20. Participants: Surveys
China
Beijing x2
Saudi Arabia
Jeddah x1
Italy
Rome x1
153 175 86
Total
485
71
Vietnam
Ho-Chi-Minh x2

21. Distribution of Survey Participants
1%
11%
50% 30%
7%
1%
0%
12%
39%
23%
17%
5%
3%
0%
48%
14%
25% 11%
2%
0%
0%
7%
4%
23% 15%
30%
11%
10%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
16-18 19-22 23-30 31-40 41-50 51-60 60+
China Vietnam Saudi Arabia Italy

22. RQ1 Survey Question on Effectiveness
RQ1: Do students think pronunciation
activities with ASR help them improve
their pronunciation?
• I think having pronunciation feedback
helps me improve my pronunciation.
Yes/no, if no, why?

23. RQ2 Survey Questions on Feature Use
RQ2: Do students in four countries
(China, Vietnam, Italy and Saudi Arabia)
make different use of ASR activity
features?
• Did you listen to the model answer?
Always / Sometimes / Never
• Did you listen to your own recording?
Always / Sometimes / Never

24. RQ3 Survey Questions on Learning L2 Pronunciation
Source/Adapted from Statement
Kaypak & Ortaçtepe (2014), Borges (2014),
Horwitz (1985), Yang (1999), Borges (2014)
I believe that I will eventually be able to speak English very well.
New item I can use technology to help me improve my pronunciation.
Meerleer (2012), Rieger (2009), Borges
(2014)
I feel at ease when I have to speak English.
Meerleer (2012), Rieger (2009), Borges
(2014)
I feel insecure about my pronunciation.
Horwitz (1985), Yang (1999), Nowacka
(2012), Moyer (1999), Sobkowiak (2002),
Borges (2014)
It is important for me to speak English with an excellent English
pronunciation.
Meerleer (2012), Rieger (2009), Borges
(2014)
I am happy with my pronunciation as long as people can
understand me.
RQ3: Are there differences between age groups and L1/nationality in
students’ beliefs and perceptions on learning pronunciation using ASR?

25. 6. Results &
Discussion

26. RQ1 Do students think pronunciation activities with
ASR help them improve their pronunciation?
91.5% China
95.2% Saudi Arabia
98.8% Vietnam
98.5% Italy
95% Global
I think having pronunciation feedback helps me improve my pronunciation.

27. RQ1 Do students think pronunciation activities with
ASR help them improve their pronunciation?
• Students who have positive beliefs and attitudes towards learning L2
pronunciation are expected to engage more in feature use.
• Students perceptions and beliefs about L2 language learning seem to
have a powerful impact on L2 acquisition (Borges, 2014)
• Dearth of research on learner beliefs and pronunciation; shift towards
looking at the socio-cultural environment rather than to the cognitive
aspects of SLA (Li & Ruan, 2015)

28. Value China Vietnam Saudi Arabia Italy
N 39,009 60,229 21,833 9,558
Mean 1.86 1.79 1.43 1.43
Median 1.00 2.00 1.00 1.00
Maximum 46 43 21 15
# of values >10 47 21 5 5
% of abnormal
attempts
3.13% 1.75% 1.57% 1.11%
RQ2 Do students in four countries make different use
of ASR activity features?
Feature Usage: # of Attempts
We configured 2 retries, so Students had 3
attempts.
As we can see from the actual data in our
servers, in all countries the average
number of attempts was below 2, even if
there was a % of abnormal behaviour by
which a fraction of Students attempted
more times than they were allowed to (4
attempts or higher)

29. RQ2 Do students in four countries make different use
of ASR activity features?
82% 82%
57%
41%
73%
15% 15%
30%
37%
22%
1% 2%
11%
4% 4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
China Vietnam Saudi Arabia Italy Total
Always
Sometimes
Never
Feature Usage: Listening to Model Audios

30. 66%
80%
61%
57%
71%
30%
18%
30%
21%
25%
3%
0%
8%
4% 3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
China Vietnam Saudi Arabia Italy Total
Always
Sometimes
Never
Feature Usage: Listening to Own Audios
RQ2 Do students in four countries make different use
of ASR activity features?

31. RQ2 Do students in four countries (China, Vietnam,
Italy and Saudi Arabia) make different use of ASR
activity features?
• Attempts: Students using number of attempts as expected. Confirms
expectations that students with L1 further from English use more
attempts (e.g. China more than Italy). Students will use the necessary
number of attempts – they will not use all attempts if they do need to.
• Listening to model audios: Remarkable differences (e.g. Italy far
less than China). May be related to distance of L1 from target
language or beliefs about learning strategies for pronunciation (cf.
Nowacka 2012).
• Listening to own recordings: Fewer differences (but e.g. Italy more
listen to their own than the model recordings). May be related again to
beliefs about learning strategies.

32. RQ3 Are there differences between age groups and
L1/nationality in students’ beliefs and perceptions on
learning pronunciation using ASR?
35%
32%
34%
16-22 23-30 31-60+
66% of participants
were young adults
Age Groups

33. Kruskall-Wallis test results for beliefs and perceptions about L2 pronunciation and age

34. Kruskall-Wallis test results for beliefs and perceptions about L2 pronunciation and age

35. RQ3 Age & Beliefs and Perceptions about L2
Pronunciation: Age
• “I believe that I will eventually be able to speak very well” – only item with a
medium effect size favourable to the youngest age group.
• Age-related declines in perception and production of L2 sounds (Flege,
1991; Ioup et al., 1994; Flege & MacKay, 2004; Moyer, 2014).
• Negative impact of age because of other factors that co-vary with age:
sociopsychological factors (Marinova-Todd et al., 2000; Moyer, 2007), affect
(Dörnyei & Skehan, 2003), motivation, formal instruction, opportunities for
authentic input (Moyer, 2007).

36. RQ3 Are there differences between age groups
and L1/nationality in students’ beliefs and
perceptions on learning pronunciation using ASR?
32%
36%
18%
15%
Chinese Vietnamese Arabic Italian
L1/Nationality

37. Kruskall-Wallis test results for beliefs and perceptions about L2 pronunciation and L1s/nationality

38. Kruskall-Wallis test results for beliefs and perceptions about L2 pronunciation and L1s/nationality

39. RQ3 Age & Beliefs and Perceptions about L2
Pronunciation: L1/Nationality
• Large differences in pronunciation learning goal: intelligibility or
excellence? (Saudi and Vietnamese)
• Medium differences in self-efficacy (Saudi, Vietnamese) and feeling at
ease when speaking (Vietnamese, Saudi).
• Research emphasizes the need to treat some affective variables as
learning context-dependent variables, such as motivation (Ryan, 2009) or
emotion (Mendez & Fabela, 2014)
• Japanese learners display low levels of motivation (Nakata, 2006; Kubota,
2002), high levels of anxiety in Chinese learners (Jiang & Dewaele, 2019)
• Other studies have not found differences (Nowacka, 2012) in beliefs of L1
Polish, Italian & Spanish. Easier for speakers of Indo-European languages
to learn English (Hakuta, Butler & Witt, 2000).

40. 7. Conclusions

41. • Learners were overwhelmingly positive towards ASR-CALL activities in terms
of helping improve their pronunciation.
• Differences in feature usage were observed between students of different
L1s/nationalities.
• These differences may be explained by differences in learner beliefs.
• When designing ASR-CALL activities, learner beliefs may impact feature
usage.
• Differences in language learning perceptions and beliefs were observed
between students of different age groups and L1s/nationalities. More
apparent when L1s/nationalities are not Indo-European.
• Ultimately, learner attitudes toward L2 pronunciation contribute to learning
goals and outcomes (Setter & Jenkins, 2005) and motivation and emotion can
have a direct impact on acquisition (Saito et al., 2018), pronunciation beliefs
can be influenced to improve effective learning (Borges, 2014).
Conclusions

42. 8. References

43. References
Artieda, G.; Clements, B. (2019). A comparison of learner characteristics, beliefs, and usage of ASR-CALL systems. In Meunier,
Fanny; Van de Vyver, Julie; Bradley, Linda; Thouësny, Sylvie (Eds), CALL and complexity – short papers from EUROCALL
2019 (pp. 19-25). Research-publishing.net. Available at: https://doi.org/10.14705/rpnet.2019.38.980
Borges, L. (2014). Pronunciation beliefs and other predictors of phonological performance: a study with Brazilian ESL learners.
Unpublished Masters dissertation. Available at:
https://englishagenda.britishcouncil.org/sites/default/files/filefield_paths/pronunciation_beliefs_and_other_predictors_of_phonologic
al_performance_v2_0.pdf University College London
Cenoz, J. & Lecumberri, M. (1999). The Acquisition of English Pronunciation: Learners’ Views. International Journal of Applied
Linguistics, 9(1), 3-17.
https://doi-org.sire.ub.edu/10.1111/j.1473-4192.1999.tb00157.x
Chen, H. (2011) ‘Developing and evaluating an orals skills training website supported by automatic speech recognition technology’
ReCALL 23(1) pp59-7
Dörnyei, Z. & Skehan, P. (2003). Individual Differences in Second Language Learning. In The Handbook of Second Language
Acquisition. C. Doughty & M. Long (eds.) pp 589-630.
https://doi.org/10.1002/9780470756492.ch18
Flege, J. (1991) Perception and Production: the relevance of phonetic input to L2 phonological learning. Cross Currents in Second
Language Acquisition and Linguistic Theory. T. Huebner and C. Ferguson (eds.) pp. 249-289.

44. References
Flege, I. & MacKay, J. (2004) Effects of the age of second language learning on the duration of first and second language
sentences: The role of suppression. Applied Psycholinguistics, 25, 373–396. https://doi.org/10.1017/S014271640400117
Golonka , E., Bowles, A, Frank, V., Richardson D., & Freynik, S. (2014). Technologies for foreign language learning: a review of
technology types and their effectiveness. Computer Assisted Language Learning, 27(1), 70-105.
http://dx.doi.org.sire.ub.edu/10.1080/09588221.2012.700315
Hakuta, K., Butler, Y.G., and Witt, D. (2000). How long does it take English language learners to develop oral proficiency and
academic proficiency in English? Stanford, CA: University of California Linguistic Minority Research Institute.
Horwitz, E. (1985). Using Student Beliefs About Language Learning and Teaching in the Foreign Language Methods Course.
Foreign Language Annals, 18(4), 333-340. https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1944-9720.1985.tb01811.x
Ioup,G., Boustague, E., El Tigi, M., Moselle, M. (1994). Reexamining the Critical Period Hypothesis: A Case Study of Successful
Adult SLA in a Naturalistic Environment. Studies in Second Language Acquisition, 16(1), 73-98.
https://doi.org/10.1017/S0272263100012596
Jiang, Y. & Dewaele, J. (2019). How unique is the foreign language classroom enjoyment and
anxiety of Chinese EFL learners? System, 82, 13-25. https://doi.org/10.1016/j.system.2019.02.017

45. References
Kaypak, E. & Ortaçtepe, D. (2014). Language learner beliefs and study abroad: a study on English as a lingua franca
(ELF). System, 42, 355-367. https://doi.org/10.1016/j.system.2014.01.005
Kubota (2002) The impact of globalization on language teaching in Japan. Globalization and Language Teaching. D. Block & D.
Cameron (eds) pp13-29. https://doi.org/10.4324/9780203193679
Levis, J. (2007). Computer technology in teaching and researching pronunciation. Annual Review of Applied Linguistics, 27, 184-
202. http://search.ebscohost.com.sire.ub.edu/login.aspx?direct=true&db=edb&AN=31935503&lang=es&site=eds-live
Li, C. & Ruan, Z. (2015). Changes in beliefs about language learning among Chinese EAP learners in an EMI context in Mainland
China: A socio-cultural perspective. System, 55, 43-52. https://doi.org/10.1016/j.system.2015.08.010
Marinova-Todd, S., Marshall, D., & Snow, C. (2000). Three Misconceptions about Age and L2 Learning. TESOL Quarterly, 34(1), 9-
34. https://doi-org.sire.ub.edu/10.2307/3588095
McCrocklin, S. (2016). Pronunciation learner autonomy: The potential of Automatic
Speech Recognition. System, 57, 25-42. https://doi.org/10.1016/j.system.2015.12.013
McCrocklin, S. (2019) ‘Learners’ Feedback Regarding ASR-based Dictation Practice for Pronunciation Learning’ Calico Journal
36(2) pp119-137

46. References
Meerleer, M. (2012). Beliefs and attitudes towards English as a Lingua Franca:
Native and non native pronunciation. A Flemish and Walloon perspective. Unpublished Masters
dissertation. Ghent University.
Mendez Lopez, M. & Fabela, M. (2014). Emotions and their effects in a language learning Mexican context. System 42, 298-307.
https://doi.org/10.1016/j.system.2013.12.006
Moyer, A. (2004). What’s age got to do with it? Accounting for individual factors in second language accent. Studies in Second
Language Learning and Teaching, 4(3), 443-464. https://doi.org/10.14746/ssllt.2014.4.3.4
Moyer, A. (2007). Do Language Attitudes Determine Accent? A Study of Bilinguals in the USA. Journal of multilingual and
multicultural development, 28(6), 502-518.
Nakata, Y. (2006). Motivation and Experience in Foreign Language Learning. Peter Lang: Oxford.
Nowacka, M. (2012). Questionnaire-Based Pronunciation Studies: Italian, Spanish and Polish Students’ Views on their English
Pronunciation. Research in Language, 10(1), 43-61. https://doi-org.sire.ub.edu/10.2478/v10015-011-0048-3
Pérez Castillejo, Susana. (2021). Automatic speech recognition: can you understand me?. In Beaven, Tita; Rosell-Aguilar,
Fernando (Eds), Innovative language pedagogy report (pp. 121-126). Research-publishing.net.
https://doi.org/10.14705/rpnet.2021.50.1246

47. References
Rieger, B. (2009). Gender and target language effect on beliefs about language learning. Practice and Theory in Systems of
Education, 4(3-4), pp.101-114.
Ryan, S. (2009). Self and Identity in L2 Motivation in Japan: The Ideal L2 Self and Japanese Learners of English. In Z. Dornyei, &
E. Ushioda (Eds.), Motivation, Language Identity and the L2 Self (pp. 120-143). Bristol: Multilingual Matters.
Saito, K., Dewaele, JM, Abe, M., In’name, Y. (2018). Motivation, Emotion, Learning Experience, and Second Language
Comprehensibility Development in Classroom Settings: A Cross‐Sectional and Longitudinal Study. Language Learning, 68(3), 709-
743. https://doi.org/10.1111/lang.12297
Setter, J. & Jenkins, J. (2005). Pronunciation. State-of-the-art Review Article. Language Teaching, 38, 1-17.
Simon, E. & Taverniers, M. (2011). Advanced EFL Learners’ Beliefs about Language Learning and Teaching: A Comparison
Between Grammar, Pronunciation, and Vocabulary. English Studies, 92(8), 896-922. https://www-jstor-
org.sire.ub.edu/stable/3588095
Sobkowiak, W. (2002). English speech in Polish eyes: What university students think about English pronunciation teaching and
learning. Accents and speech in teaching English phonetics and phonology, 177-193.

48. References
Thomson, R and Derwing, T. (2015) ‘The Effectiveness of L2 Pronunciation Instruction: A Narrative Review’ Applied Linguistics
36(3) pp326-344
Underwood, J. (2021). Speaking to machines: motivating speaking through oral interaction with intelligent assistants. In T. Beaven
& F. Rosell-Aguilar (Eds), Innovative language pedagogy report (pp. 127-132). Research-publishing.net.
https://doi.org/10.14705/ rpnet.2021.50.1247
Yang, N.D. (1999). The relationship between EFL learners' beliefs and learning strategy use. System, 27(4), 515-535.
https://doi.org/10.1016/S0346-251X(99)00048-2

49. gemma.artieda@wallstreetenglish.com | bindi.clements@wallstreetenglish.com
@bindi_clements
linkedin.com/in/bindiclements/