• Save
Calico2013_GLord_Software-vs-Teacher
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Calico2013_GLord_Software-vs-Teacher

on

  • 1,012 views

 

Statistics

Views

Total Views
1,012
Views on SlideShare
1,010
Embed Views
2

Actions

Likes
0
Downloads
0
Comments
1

1 Embed 2

https://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Felicidades Carlos, saludos !!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • ROSETTA = The Fastest Way To Learn A Language. Guaranteed. AD: “"Rosetta Stone is, in my opinion, the ultimate method, the only method, there is to learn a language.”PIMSLEUR = “learn any language in just 10 days”
  • Using 575 native Spanish speakers (from various countries) and 564 Spanish learners, who all took the test; human raters (ACTFL OPI; government-certified SPT; Common European Framework; ILR ) compared ratings across samples to ensure that native speakers score high, and learners score across a range; and to ensure that test results will be consistent for same test-taker if no change in proficiency
  • Need for a larger scale study – more participants, more levels, longer duration…?

Calico2013_GLord_Software-vs-Teacher Presentation Transcript

  • 1. a work inprogress
  • 2. Motivation for studyNothing can replace the teacher!BUT…0 Increasing desire among educational institutions to cutcosts:0Infrastructure costs0Personnel costs0 Increasing pressure to embrace digital technologies:0 Technology offers teachers and learners documentedbenefits0 Stand-alone programs are gaining popularity:0 Among public in general0 Among administrators(if not among informed language teachers themselves!)
  • 3. Digital softwarelanguage programs0 Examples:0 Rosetta Stone0 Tell Me More (Auralog)0 Learn Spanish Now (Transparent Language)0 Pimsleur Approach0 Claim to be as effective or more effective than classroomlearning0 “fastest” or “only” way to learn a language??0 “learn any language in just 10 days”??0 Marketing heavily in recent years, especially in K-12 andhigher education arenasRosetta Stone is best seller and themost complete (Godwin-Jones, 2007, 2009)
  • 4. Rosetta Stone0Informed by the Natural Approach to languagelearning (Krashen & Terrell, 1983):0 Works under the premise that adults learn a secondlanguage in the same way that a child learns a first language0However… research has shown that adult secondlanguage learning is “fundamentally different”0Bley-Vromans [1988] Fundamental DifferenceHypothesis…0Children learn in a highly social, affective, and rich setting0Children lack the cognitive advantage of their linguisticbackground that adults have
  • 5. Reviews of these programs0 Lafford, Lafford & Sykes (2010)0 Evaluate if programs provide the tools necessary for effectivelanguage learning, based on features that research has shown to beimportant0 Interaction, relevant contextualization of language, etc.0 “… these products do not incorporate a number of the [necessary]research-based insights (e.g., the need for culturally authentic, task-based activities) that informed SLA scholars might have giventhem.”0 Santos (2011)0 Lack of context0 General inability to respond to spontaneous student speech0 What Rosetta Stone calls interaction is “a rather poor and limitedversion of what one would encounter in a real-life conversation”0 DeWaard (forthcoming 2013)0 “Not a viable replacement of current instruction at thepostsecondary level”
  • 6. Studies on effectivenessof Rosetta Stone and other programs0 Few empirical studies(Aside from Rosetta Stone’s own commissioned work, e.g.Vesselinov, 2009)0 Nielson (2011) – self-study programs in workplace; somesuccess but remarkable attrition; lack of community(e.g., Rovai, 2002)?0 Stevenson & Liu (2010) – lack of ability to engage learnersin true interaction; users do not take advantage of Web 2.0tools to network.Overall lack of evidence to support the notion that suchprograms can replace the classroom or canoffer anything that a classroom program cannot
  • 7. So….The time has come to put theclaims made by Rosetta Stone(and by extension other programs)to the test!This research was conductedwith Rosetta Stone’sknowledge but they were notinvolved in the design, datacollection or analysis.
  • 8. Research Questions1. How does (Spanish) learner achievement compareacross environments in terms of0lexical items?0structural/grammatical content?0writing proficiency?0listening proficiency?0pronunciation?0oral communicative proficiency?0cultural awareness?2. How do student attitudes toward the Spanishlearning experience differ across the learningenvironments?
  • 9. Research Questions1. How does (Spanish) learner achievement compareacross environments in terms of0lexical items?0structural/grammatical content?0writing proficiency?0listening proficiency?0pronunciation?0oral communicative proficiency?0cultural awareness?2. How do student attitudes toward the Spanishlearning experience differ across the learningenvironments?
  • 10. Learning Environments0Participants are University of Florida studentsenrolled in Beginning Spanish 1 (avg. age = 20)0L1 English0No other L2s0No prior Spanish instruction0Participants belong to one of 3 environments:0Classroom (C): (N=13)0Rosetta Stone (RS): (N=20)0Classroom+Rosetta Stone (RS+C): (N=14)
  • 11. ParticipantsControl group (C)0N=130In-tact section of Beginning Spanish section0Follow traditional syllabus with standardmaterials0Carry out standard classroom assessmentmaterials0Meet with researcher 3x during semester
  • 12. ParticipantsRosetta Stone group (RS)0 N=200 Not required to attend any regular class0 Rosetta Stone package (“Conversational Spanish”)0 16-week course designed to cover material comparable to a face-to-face beginning class0 6 units of Rosetta Stone® Version 4 TOTALe® Spanish, eachcontains 4 lessons [Level 1, half of Level 2]0 At least 6 Rosetta StudioTM sessions0 At least 8 hours in Rosetta WorldTM0 Monitoring of program access and time on task0 Follow predetermined deadlines in progressing through thematerial0 Meet with researcher 3x during semester
  • 13. ParticipantsClassroom + Rosetta Stone group (RS+C)0N=140In-tact section of Beginning Spanish class0Same instructor as control group0Using Rosetta Stone materials as the textbook(including all features described for RS group)0Meet with researcher 3x during semester
  • 14. Rosetta Stone interface
  • 15. Rosetta Stone interface(vocabulary)
  • 16. Rosetta Stone interface(grammar)
  • 17. Rosetta Stone interface(pronunciation)
  • 18. Rosetta Stone interface(World – “play”)
  • 19. Rosetta Stone interface(World – “talk”)
  • 20. Rosetta Stone interface(World – “explore”)
  • 21. Rosetta Stone interface(Studio)
  • 22. Data collected0Assessment of attitudes0January, May0Discussion of experiences0January, March, May0General oral and written proficiency and skills0January, March, May0General language skills – partial CLEP test (30 items)0 May
  • 23. Sample CLEP test items
  • 24. Data collected0Assessment of attitudes0January, May0Discussion of experiences0January, March, May0General oral and written proficiency and skills0January, March, May0General language skills – partial CLEP test (30 items)0 May0 Oral proficiency – Versant test0 May
  • 25. Versant proficiency test
  • 26. Versant proficiency test0 Instrument has been tested for reliability and validity
  • 27. Resultsand preliminary observations
  • 28. Results: CLEP testAverage scores (%)10.0025.0040.0055.0070.00CTRL RS RS_CBoxPlotGroupPercentageinCLEPtest41.05 36.4944.00
  • 29. Results: CLEP testStatistical analysisSource Term DF Sum ofSquaresMeanSquareF-Ratiop-valueBetween groups 2 486.4109 243.2055 2.77 0.074158Within groups 43 3781.544 87.94288Total (Adjusted) 45 4267.9550 F(2,43) = 2.77, p = 0.0740 No significant interactions at alpha = 0.050 Groups performed statistically similarly on CLEP test.
  • 30. Results: Versant testComponent scores (20-80)SentencemasteryVocabulary Fluency PronunciationOVERALLPROFICIENCYControl 35.55 23.73 35.73 42.00 34.64Rosetta Stone 31.25 27.00 40.20 43.90 35.20Rosetta Stone + class 29.42 24.83 37.25 42.58 32.7520.0030.0040.0050.0060.0070.0080.00
  • 31. Results: Versant testOverall average scores (20-80)20.0031.2542.5053.7565.00CTRL RS RSCBoxPlotGroupPercentageinOverallVersanttest
  • 32. Results: Versant testStatistical analysisSource Term DF Sum ofSquaresMeanSquareF-Ratiop-valueBetween groups 2 36.97765 18.48883 0.27 0.762965Within groups 40 2646.802 67.86672Total (Adjusted) 42 2683.780 F(2,40) = 0.27, p = 0.7630 No significant interactions at alpha = 0.050 Groups performed statistically similarly on Versant test.
  • 33. Does this mean Rosetta Stoneis just as good as theclassroom?Not so fast…There are other issues to consider!
  • 34. Issue 1: Level tested0Stages of SLA0 Beginning levels markedby lexical acquisition inmuch greaterproportion tomorpho/syntacticacquisition0 Krashen and Terrell,(e.g., the NaturalApproach, 1983)recognized this as well0RS does vocabulary well!SOLUTION: Longer treatment period, encompassing greater acquisition;also follow up beyond the treatment into next level(s)
  • 35. Issue 2: (Unrelated) student attitudes0Control group did not respond well to instructor’steaching style0 Felt his expectations were too high0 Emphasis on quizzes to demonstrate preparation0 Perceived as disrespectful towards students0 Affected control RS+class group also, but the negativestudent attitude was much more contagious anddetrimental in the control group.SOLUTION: ???
  • 36. Issue 3: Assessment materials0Goal was an objective test, not dependentupon curriculum but sensitive to level and notvulnerable to ceiling effects0CLEP test0May have picked too difficult an assessment!0Versant test0Although very good at low-level discrimination,overall scores may not best reflect abilities0 For example 
  • 37. Sample VERSANT answer (ctrl)0Muchos niños pasan horas cada díapracticando deportes. En su opinión, ¿esbueno alentar a los niños a hacer deporte? Porfavor, aclare por qué si o por qué no.(Many children spend hours each day playingsports. Is it good to encourage children to playsports? Why or why not?)Sentence Vocab Fluency Pronunciation Overall40 23 47 54 41
  • 38. Sample VERSANT answer (RS)0¿Cómo prefiere usted enterarse de lasnoticias, por la radio o por el periódico? Porfavor explique su respuesta.(When getting news information, would you ratherhear the information on a radio program, or do youprefer to read the information in a newspaper?Please explain why.)Sentence Vocab Fluency Pronunciation Overall20 27 31 32 24
  • 39. Issue 3: Assessment materials0Goal was an objective test, not dependentupon curriculum but sensitive to level and notvulnerable to ceiling effects0CLEP test0May have picked too difficult an assessment!0Versant test0Although very good at low-level discrimination,overall scores may not best reflect abilities0 For example SOLUTION: Investigate other objective means of assessment acrossgroups, suitable for lowest levels of acquisition.
  • 40. Issue 4: Many more data!0Next phase of analysis1. Analysis of additional texts0Written: patterns of errors, lexical use0Audio: patterns of errors; lexical use; acousticanalysis2. Analysis of discussion of experiences, attitude data
  • 41. Preliminary observationsfrom monthly meetings and discussions0 Linguistic observations0 All learners exhibit typical morphosyntactic errors commonto this level0Number/gender agreement0Verb conjugation0Etc.0 All learners evidence English-influenced phonology0 Communicative skills are minimal across all groups0But conversation is virtually impossible with RS-only group0RS+class group comparable to control group1
  • 42. Preliminary observationsfrom monthly meetings and discussions0 Attitudes and behavior0 RS participants greatly enjoyed the program0Flexibility, no need to attend class (RS only)0 No problems with digital-only materials0 Emerging concerns about continuing in Spanish nextsemester (2nd half of two-semester sequence to fill thelanguage requirement)0 RS students consistently do activities until they achieve100% - this is not the pattern we see in our online workbookin traditional classes. Why?There are definitely lessons to belearned as we develop digitalmaterials and activities…!2
  • 43. Preliminary observationsfrom monthly meetings and discussions0 RS+Class students enjoyed the materials far less than theRS-only group:0 Frustrated by lack of grammar explanations in RS0 Sensed a lack of structure in materials0 Instructor spent excessive class time on grammarexplanations, and trying to contextualize the RS material tomake it appropriate for conversational interaction…In essence, they un-flipped the classroom!3
  • 44. Issue 4: Many more data!0Next phase of analysis1. Analysis of additional texts0Written: patterns of errors, lexical use0Audio: patterns of errors; lexical use; acousticanalysis2. Analysis of discussion of experiences, attitude data3. How to assess cultural awareness?SOLUTION 1: Keep analyzing! Follow up with these participants nextsemester. Case studies as well as group trends.SOLUTION 2: There is a need for a larger scale study –more participants at different institutions, more levels ofRosetta Stone program, longer duration, etc.
  • 45. Thank you.glord@ufl.edu0Special thanks to:0 UF College of LiberalArts and Sciences0 HumanitiesScholarshipEnhancement fund0 Carlos Enrique Ibarra(instructor, statistician)0 Caroline Reist andKeegan Storrs (RAs)0 Laura Bradley (RosettaStone)
  • 46. Works Cited0 Bley-Vroman, R. (1988). “The fundamental character of foreign language learning.” In W. Rutherford & M.Sharwood Smith (Eds.), Grammar and second language teaching (pp. 19-30). Rowley, MA: Newbury House.0 DeWaard, L. (2013). “Is Rosetta Stone a viable option for L2 learning?” Forthcoming in ADFL Bulletin.0 Godwin-Jones, R. (2007). “Emerging technologies; Tools and trends in self-paced language instruction.Language Learning and Technology,” 11(2), 10-17. Retrieved 26 September 2012 fromhttp://llt.msu.edu/vol11num2/emerging/0 Godwin-Jones, R. (2009). “Emerging technologies: Speech tools and technologies. Language Learning andTechnology,” 13(3), 4-11. Retrieved 26 September 2012 from http://llt.msu.edu/vol13num3/emerging.pdf0 Krashen, S. D. & Terrell, T. D. (1983). The Natural Approach: Language acquisition in the classroom. Hayward,CA: Alemany Press.0 Lafford, B., Lafford, P. & Sykes, J. (2007). “Entre dicho y hecho …: An assessment of the application ofresearch from second language acquisition and related fields to the creation of Spanish CALL materials forlexical acquisition.” CALICO Journal, 24(3), 427-529.0 Nielson, K. B. (2011). “Self-study with language learning software in the workplace: What happens”Language Learning and Technology, 15(3), 110-129. Retrieved 26 September 2012 fromhttp://llt.msu.edu/issues/october2011/nielson.pdf0 Rovai, A. P. (2002). “Development of an instrument to measure classroom community.” The Internet andHigher Education, 5, 197-211.0 Santos, V. (2011). “Review of Rosetta Stone Portuguese (Brazil) levels 1, 2, & 3.”CALICO Journal, 29(1), 177-194.0 Stevenson, M. P. & Liu, M. (2010). “Learning a language with web 2.0: Exploring the use of social networkingfeatures of foreign language learning websites.” CALICO Journal, 27(2), 233-2590 Vesselinov, Roumen. Measuring the Effectiveness of Rosetta Stone.http://resources.rosettastone.com/CDN/us/pdfs/Measuring_the_Effectiveness_RS-5.pdf