- The document summarizes a presentation given by Gillian Lord comparing the effectiveness of Rosetta Stone for learning Spanish as a second language. - Lord collected data from university students learning Spanish through Rosetta Stone, in a classroom, or both, and assessed their linguistic outcomes, attitudes, and perceptions of the materials. - Based on her data and criteria including professional assessments, empirical studies, and student feedback, Lord found both benefits and limitations to the Rosetta Stone approach for language learning.

1. Rosetta Stone for Language Learning:
YAY or NAY?
Gillian Lord
University of Florida
glord@ufl.edu

2. • How many have heard of Rosetta Stone?
• How many have used Rosetta Stone?
• How many work for Rosetta Stone?

3. Why this topic?
• Powerful, omnipresent marketing
– Asserts that Rosetta Stone is the
only way to learn a language
• K-12 and higher education programs feeling
threatened
“Rosetta Stone …
spent $98.5 million on
advertising in 2011,
up from $70.5 million
in 2010, according to
Kantar Media”
www.nytimes.com/2012/06/20/bus
iness/media/rosetta-stone-ads-emphasize-
fun-not-efficiency.html

4. GOAL OF THIS PRESENTATION:
to provide a global understanding, using
multiple data sources, of the advantages
and disadvantages of the Rosetta Stone
program for learning (Spanish as) a
second/foreign language

5. How to assess such a program?
1. Professional assessments
2. (Previous) Empirical studies
3. Student attitudes (a priori) *
4. Perceived quality of materials *
5. Affective factors *
6. Outcomes
– The “Does it work?” question, considering
learning in terms of:
a) Self-perceived communicative abilities *
b) Quantitative measures of communicative abilities *
c) Qualitative measures of communicative abilities *
* = my data

6. My data – Participants
• University of Florida students enrolled in Beginning
Spanish 1 (avg. age = 20)
– L1 English
– No other L2 proficiency (beyond h.s. requirement)
– Course designed for those with NO prior Spanish
instruction
• 3 environments:
– Classroom (C): N=4
– Rosetta Stone (RS): N=4
– Classroom+Rosetta Stone (RS+C): N=4
Original population had
20-25 participants in
each of the three groups.

7. My data – Learning Environments
Classroom
• In-tact section
• Followed regular
syllabus with
standard materials
Rosetta Stone
• Self-selected (required by
IRB)
• Not required to attend
any regular class
• Used RS package
(“Conversational
Spanish”):
– 16-week course designed
to cover material
comparable to a face-to-face
beginning class
– 6 units of Rosetta Stone®
Version 4 TOTALe®
Spanish, each has 4
lessons [Level 1, half of
Level 2]
– Minimum of 6 Rosetta
StudioTM sessions
– Minimum of 8 hours in
Rosetta WorldTM
– Monitoring of program
access and time on task
RS + Class
• In-tact section of
Beginning Spanish
class
• Same instructor as
control group
• Used Rosetta Stone
materials as their
textbook
– (including all
features
described for RS
group)

8. Data used here (1)
Quantitative Data
• Portion of Spanish CLEP test (grammatical competence)
• Versant Proficiency Test in Spanish (oral proficiency)
• Attitude survey (Likert, style, pre- and post)
• Acoustic analysis of vowel production in Spanish
interviews
– 10 tokens of /e/ extracted from each participant’s
interview.
• Tokens separated for male and female
• Each formant measured at midpoint using Praat
• F1 and F2 extracted using a script and compared to standard
native values

9. Data used here (2)
Qualitative Data
• Fluency analysis of Spanish interviews
• Total number of words spoken
• Number of Spanish words; Number of English words
• Number of dysfluencies
• Lexical density (number of unique Spanish words)
• Number of fillers/non-lexical items
• Content analysis of English interviews
• Affective factors
• Reactions to instructional materials
• Student perceived learning
• Language communication and use
• Other specific language problem or comment

10. Criterion 1:
PROFESSIONAL ASSESSMENTS OF
THE ROSETTA STONE PROGRAM

11. Professional Assessments
• Godwin-Jones (2007, 2009)
– Traditional computer-based training programs often informed by technology specialists,
not SLA specialists
– Rosetta Stone provides visualization feedback for specific sounds (pronunciation); mostly
accurate
• Lafford, Lafford & Sykes (2010)
– Evaluate if programs provide the tools necessary for effective language learning, based on
features that research has shown to be important (interaction, relevant contextualization
of language, etc.)
– “… these products do not incorporate a number of the [necessary] research-based
insights (e.g., the need for culturally authentic, task-based activities) that informed SLA
scholars might have given them.”
• Santos (2011)
– Lack of context
– General inability to respond to spontaneous student speech
– What Rosetta Stone calls interaction is “a rather poor and limited version of what one
would encounter in a real-life conversation”
• DeWaard (2013)
– Based on personal experience, professional reactions
– “Not a viable replacement of current instruction at the postsecondary level”

12. Criterion 2:
EMPIRICAL STUDIES INTO THE
OUTCOMES OF THE ROSETTA STONE
PROGRAM

13. Empirical studies
• Vesselinov (2009)
– Commissioned by Rosetta Stone; RS beginning users
demonstrated increased knowledge of the language after 55
hours of use.
• Nielson (2011)
– Self-study programs in workplace; some success among
committed users, but extreme attrition.
• Stevenson & Liu (2010)
– Use of social interaction in web-based language learning tools;
lack of ability to engage learners in true interaction; users do
not take advantage of network opportunities.

14. Criterion 3:
STUDENT ATTITUDES TOWARDS
THE ROSETTA STONE PROGRAM

15. Background questionnaire
• Why did you volunteer for the Rosetta Stone
class-replacement option?
– I heard a lot of good things about RS and wanted to try it.
– Can better manage my time and schedule and move more
at my own pace without dealing with class.
– Sounded more beneficial.
– I was going to use my own to supplement education
anyway.

16. Criterion 4:
PERCEIVED QUALITY OF THE
ROSETTA STONE MATERIALS

17. Student Impressions of Materials
• Comments from English interviews coded for
mentions of the Rosetta Stone materials
– 42/181 comments in RS group
– 80/124 comments in RS+C group
– Difficult to classify as categorically positive or
negative
• Main themes:
1. Technology [Flexibility, Ease of use, Glitches]
2. Approach to language learning

18. Sample student comments
(Flexibility)
• Like you’re able to kind of
do it like on your own
time, you know, I’m not
like restricted.
• It’s nice not to go to class.
• I have always preferred to
learn language, like, on
my own.
• It’s more flexible with my
schedule.

19. Sample student comments
(Ease of Use)
• Because it’s a lot of visual stuff,
and I feel like I’m a visual learner.
• I like Rosetta Stone a lot. It’s
pretty easy to use.
• World is good, I do the games.
• Oh, and you can do it on your
iPad … so I’d do that a lot, like
take it to people’s houses, and
they’ll all be like watching TV, and
I would be like, doing my Rosetta
Stone, so it’s very versatile.
• I like how it’s like uh, more like a
game, so I’m more willing to
actually do it…
• It was just kind of a lot harder [to
use] than I expected.
• I just didn’t show what words I
needed to use before it.
• I [don’t like] the lack of human
interaction.
• Sometimes it’ll show the person
speaking, and sometimes it’ll say
like he or she, and sometimes it’ll
be I. And I couldn’t tell the
difference.
• …but it is tedious, a lot of it.

20. Sample student comments
(Tech glitches)
• I’m still having problems
with the computer.
• I can’t get the
microphone to work…
• I was doing my Studio
session and… I had no
audio, like, I could hear
them, but they couldn’t
hear me the whole
time.

21. Student Impressions
(Language learning in general)
• I feel like it’s more like how you
naturally learn the language instead
of like, “These are … your vocabulary
words this week”.
• I think the Studio session was the
most interesting thing.
• Speaking it for sure [is hardest],
because when you see it on paper it’s
a little bit… but when it gives you
four options, it’s easier to be like, it’s
one of these four.
• The program is really good with like
teaching like vocabulary.
• With like vocabulary, it’s like really
good, and you get by.
• It’s just like the grammar, and how to like,
put it together.
• You can’t ask questions.
• You don’t get any writing, and then all of a
sudden there’s one writing thing.
• You have that whole grammar and
conjugation issue on Rosetta, because
they don’t really explain it.
• You really need to have communication
with a real person.
• I would enjoy getting more grammar
lessons … just to get a foundation of
knowledge, then building up on that.
• But I’m glad to be in the class, too,
because it’s really hard by itself.
• I think it’s helpful to have a teacher to
explain like, why things are.
• Rosetta Stone doesn’t like, I don’t know,
hammer it down for you like they would in
a normal grammar class.

22. Criterion 5:
AFFECTIVE FACTORS AND
EMOTIONAL REACTIONS TO THE
ROSETTA STONE PROGRAM

23. Affective factors
• Comments from English interviews coded for
any mention of affect/emotion
– Related to using the materials, learning, etc. and
how the learners felt in that respect
• 33/181 comments in RS group
– 8 positive towards RS, 8 negative, 17 unrelated
• 24/123 comments in RS+C group
– 2 positive towards RS, 9 negative, 13 unrelated

24. Student reactions/emotions
• I don’t have a problem
with staying focused on
my computer at all.
• I like not having a
textbook to worry
about.
• I like it a lot.
• I’m kind of struggling.
• I didn’t like the stories.
• I’m always just
frustrated because I’m
like, I don’t really
understand it.
• I feel like it should be
more structured.

25. Criterion 6:
LINGUISTIC OUTCOMES OF THE
ROSETTA STONE PROGRAM
a) Self-perceived communicative abilities

26. Perceived learning
• Comments from English interviews coded for
any mention of perceived learning or
outcomes
• 34/181 comments in RS group
– 9 positive assessments, 22 negative, 3 unrelated
to materials
• 14/123 comments in RS+C group
– 8 positive assessments, 3 negative, 3 unrelated

27. Perceived learning
• I understood like the vocab.
• I feel like I’m slowly learning
and progressing to learn the
language.
• I do like the studio sessions…
When I do do them, like I feel
like it helps me more.
• I can understand almost
always what is being said.
• Learning a language just like
on your computer as opposed
to like in person can actually
be effective.
• Rosetta Stone doesn’t give you
too much, like, actual
instruction so you don’t learn.
• I don’t know how to use words
in Spanish.
• I’m just kind of like nervous
about going back to class.
• I don’t feel as confident as I
did at that last meeting,
honestly.
• We didn’t ever have to like
memorize the conjugations, so
I don’t know them, so I can’t
say that it was super effective.
• It was just kind of a lot harder
than I expected.

28. Perceived learning
(pre-post changes on relevant
items from attitude survey)
– Significant decrease among RS on item #11:
• “Interacting via chat or telephone is
comparable to interacting face-to-face.”
– Significant increase among RS and RS+C on item
#19:
• “I would prefer to learn a language on my own
time and at my own pace than in a group or
classroom setting.”

29. Criterion 6:
LINGUISTIC OUTCOMES OF THE
ROSETTA STONE PROGRAM
b) Empirical measurements of communicative abilities

30. CLEP test
Average scores (converted to %)
p = 0.165
38.65 39.17
47.50
100.00
90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
Classroom Rosetta Stone RS+class

31. Versant test
Average scores (converted to %)
p = 0.620
27.08 26.25
20.00
100.00
90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
Classroom Rosetta Stone RS + Class

32. Pronunciation
Acoustic analysis of vowel /e/
• No significant difference
between groups or test
times.
– Control males shows slight
trend toward more native-like
F1 values in final
interview.
RS = RS+Class = Classroom?

33. Criterion 6:
LINGUISTIC OUTCOMES OF THE
ROSETTA STONE PROGRAM
c) Discourse analysis reflecting communicative abilities

34. Final interview - Classroom
INTERVIEWER: Cuéntame, ¿qué te gusta hacer en tu tiempo libre, o los fines de semana?
SL: Repitas, please.
INTERVIEWER: ¿Qué te gusta hacer?
SL: Qué te gusta hacer…
INTERVIEWER: ¿Te gusta ir a películas? ¿Te gusta escuchar música?
SL: Uh, ¿fin de semana?
INTERVIEWER: Sí.
SL: Uh, sí. En fin de semana, yo… yo estudio, uh, mucho.
INTERVIEWER: ¿Sí?
SL: Uh, para mis exámenes. Sí. Yo tengo muchos examines en química orgánica, biología, y laboratorio. Uh,
sí. Mucho, uh… no, muy ocupado. So, no películas, no, uh, deportes.
INTERVIEWER: ¿Cuál fue la última película que viste?
SL: Cuál te…
INTERVIEWER: La última vez, the last time, que viste una película.
SL: Phew… Hmm. Let’s see… dos menses.
INTERVIEWER: Meses, mhm.
SL: Meses. Ago. ¿Cómo se dice “ago”?
INTERVIEWER: Hace. Hace dos meses.
SL: Hace, sí.
INTERVIEWER: Wow.
SL: Yo no… yo no veo muchas películas en Gainesville.
INTERVIEWER: ¿Qué película fue esa, hace dos meses? ¿Cómo se llamaba?
SL: Uh, el pelí—la película… ¿cómo se dice “was”?
INTERVIEWER: Era, o fue.
SL: Era. La película era…
INTERVIEWER: ¿No te acuerdas?
SL: Yo no… sí.

35. Final interview – Rosetta Stone
INTERVIEWER: Mhm, ¿y qué haces en Gainesville?
SH: Um… you’re going to have to forgive me, my mind’s like blown… Um, yo estoy estudiar.
INTERVIEWER: ¿Tú estudias? ¿Y qué más?
SH: Yo trabajo en un restaurante de Dragonfly.
INTERVIEWER: Y, ¿con mucha frecuencia, vas de compras?
SH: Yo no entendí, repetirlo, por favor.
INTERVIEWER: ¿Con mucha frecuencia, vas de compras? “Ir de compras” significa go shopping.
SH: All right, say that one more time, please.
INTERVIEWER: ¿Con mucha frecuencia, vas de compras?
SH: Uh, no, uh, no voy a… what did you, how did you say “to go shopping”?
INTERVIEWER: Ir de compras.
SH: No voy de compras.
INTERVIEWER: Y, ¿qué vas a hacer este verano?
SH: Este verano, yo voy a visitar Brazil.
INTERVIEWER: Vas a visitar Brazil, y ¿vas a estudiar en Brazil?
SH: No, um, yo voy a trabajar en Brazil.
INTERVIEWER: Y, em, ¿qué más vas a hacer en Brazil? ¿Vas a leer, vas a jugar deportes?
SH: What am I going to do in Brazil? I thought I just answered that.
INTERVIEWER: ¿Solo trabajar?
SH: I don’t know, I’m going on a missions trip, I don’t know how to express that in Spanish,
but…
INTERVIEWER: Pues, buena suerte, muchas gracias.

36. Ratio of L1/L2 words
0.26
0.83
0.68
1.00
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
Ratio of English-to-Spanish words used, by group
Control Average
RS + class Average
Rosetta Stone Average
0 = no English words produced
1 = 1 English word produced for every Spanish word

37. Assistance requests
Average # of clarification / assistance requests by group
2.06
1.03
0.11
1.75
0.61
3.24
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
# Clarification requests in Spanish # Clarification requests in English
Control Average RS + class Average Rosetta Stone Average

38. SUMMARY

39. Yay or Nay?
1. Professional assessments
2. Empirical studies
3. Student attitudes (a priori)
4. Perceived quality of materials
5. Affective factors
6. Outcomes
a) Self-perceived communicative abilities
b) Empirical measurements of communicative
abilities
c) Discourse analysis reflecting communicative
abilities

40. What about overall time on task?
GROUP
Completion
Rate
Average
Score
Total Course
Usage (hours)
Total Class
Time (hours)
Classroom 96.99% 90.77% 70.00 39.00
RS+C 93.67% 98.63% 32.81 37.25
RS 97.67% 95.88% 30.69 NA

41. But… ?

42. IN CONCLUSION…

43. 1. More research is needed.
• Further professional assessments of the
programs, as new features are added
• Continuing analysis of current data
• Empirical studies of outcomes and
effectiveness
– Attitudes and reactions
– Linguistic outcomes
– Larger, more diverse populations

44. 2. Rosetta Stone can be effective.
• In certain circumstances:
– Introduction and exposure to new languages
– Refresh skills previously acquired
– Excellent vocabulary presentation and practice
– Flexible for varied populations
– Highly motivated/diligent learners more apt to
make progress
Undoubtedly better than nothing!

45. 3. BUT…
• Based on multiple data sources, Rosetta Stone
does not appear to be, or have the potential to be
more effective,
faster,
easier
or
(necessarily) more enjoyable
than other methods,
such as common classroom-based approaches.

46.  Keep an open mind about RS and other
similar products.
– CALL continues to expand and develop.
– Rosetta Stone has acquired new tools and
capabilities.
– Knowledge of another language is always
valuable.
 But don’t believe everything the ads
promise.
– Claims are largely unfounded so far.
– Immersion or classroom experiences remain
most promising methods of acquiring FL
proficiency.

47. Thank you.
glord@ufl.edu
Special thanks to:
• UF College of Liberal Arts &
Sciences
• UF CLAS Humanities Scholarship
Enhancement Fund
• Carlos Enrique Ibarra (statistics)
• Caroline Reist, Brandon Shufelt,
Keegan Storrs, Diana Wade (RAs)
• Laura Bradley, Lisa Frumkes
(Rosetta Stone)

48. Works Cited
• DeWaard, L. (2013). “Is Rosetta Stone a viable option for L2 learning?” Forthcoming in ADFL
Bulletin.
• Godwin-Jones, R. (2007). “Emerging technologies; Tools and trends in self-paced language
instruction. Language Learning and Technology,” 11(2), 10-17. Retrieved 26 September 2012
from http://llt.msu.edu/vol11num2/emerging/
• Godwin-Jones, R. (2009). “Emerging technologies: Speech tools and technologies. Language
Learning and Technology,” 13(3), 4-11. Retrieved 26 September 2012 from
http://llt.msu.edu/vol13num3/emerging.pdf
• Lafford, B., Lafford, P. & Sykes, J. (2007). “Entre dicho y hecho …: An assessment of the
application of research from second language acquisition and related fields to the creation of
Spanish CALL materials for lexical acquisition.” CALICO Journal, 24(3), 427-529.
• Nielson, K. B. (2011). “Self-study with language learning software in the workplace.” Language
Learning and Technology, 15(3), 110-129. Retrieved 26 September 2012 from
http://llt.msu.edu/issues/october2011/nielson.pdf
• Santos, V. (2011). “Review of Rosetta Stone Portuguese (Brazil) levels 1, 2, & 3.”CALICO Journal,
29(1), 177-194.
• Stevenson, M. P. & Liu, M. (2010). “Learning a language with web 2.0: Exploring the use of social
networking features of foreign language learning websites.” CALICO Journal, 27(2), 233-259
• Vesselinov, Roumen. Measuring the Effectiveness of Rosetta Stone.
http://resources.rosettastone.com/CDN/us/pdfs/Measuring_the_Effectiveness_RS-5.pdf.

50. Sample CLEP test items

51. Versant proficiency test

52. Versant proficiency test
• Instrument has been tested for reliability and validity

53. Rosetta Stone interface

54. Rosetta Stone interface
(vocabulary)

55. Rosetta Stone interface
(grammar)

56. Rosetta Stone interface
(pronunciation)

57. Rosetta Stone interface
(World – “play”)

58. Rosetta Stone interface
(World – “talk”)

59. Rosetta Stone interface
(World – “explore”)

60. Rosetta Stone interface
(Studio)

61. Attitude survey