Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Measuring Linguistic Gains in Immersion 
Settings: Empirical Studies from French and 
Spanish as a Second Language 
Dennis...
Wofford College 
• Four-year, private, liberals arts college 
• Founded in 1854 
• Methodist affiliated 
• Phi Beta Kappa ...
National Survey of Student Engagement
National Survey of Student Engagement
National Survey of Student Engagement
National Survey of Student Engagement
Linguistic Gains: High Impact 
Practices 
All French or Spanish students in these data 
have engaged in at least one of th...
Methodology 
• OPIc 
• Immediately upon return from study abroad, 
set / proctored 
• French students 
– Paris, Nantes, Re...
SPAIN 
LAT. AMER. 
SPANISH_STUDY_ABROAD_2007_'13 
0 20 40 60 80 100 120 140 
LAT. AMER. SPAIN 
Series1 118 77
ACTFL_OPI_DID_NOT_PARTICIPATE_# 
0 20 40 60 80 100 120 140 
SPAIN DNT 
SPAIN 
LAT. AMER DNT 
LAT. AMER. 
LAT. AMER. LAT. A...
SPANISH_LAT._AMER._ACTFL_2007_2013_# 
0 5 10 15 20 25 30 35 40 45 50 
DNT 
IL 
IM 
IH 
AL 
AM 
AH 
AH AM AL IH IM IL DNT 
...
SPANISH_SPAIN_ACTFL_2007_2013_# 
0 5 10 15 20 25 30 35 
DNT 
IL 
IM 
IH 
AL 
AM 
AH 
AH AM AL IH IM IL DNT 
Series1 1 7 25...
SPANISH_LAT._AMER._ACTFL_2007_2013_% 
0 5 10 15 20 25 30 35 40 
DNT 
IL 
IM 
IH 
AL 
AM 
AH 
AH AM AL IH IM IL DNT 
Series...
SPANISH_SPAIN_ACTFL_2007_2013_% 
0 5 10 15 20 25 30 35 40 
DNT 
IL 
IM 
IH 
AL 
AM 
AM AL IH IM IL DNT 
Series1 1.3 9.1 32...
French_ACTFL_2007_2013 
AH, 3 
AM, 6 
AL, 21 
IH, 16 
IM, 12 
IL 
IM 
IH 
AL 
AM 
AH 
0 5 10 15 20 25
French_ACTFL_’09_’13 
AH, 3 
AM, 4 
AL, 19 
IH, 13 
IM, 7 
IL 
IM 
IH 
AL 
AM 
AH 
0 2 4 6 8 10 12 14 16 18 20
French_ACTFL_’11_’13 
AH, 3 
AM, 2 
AL, 11 
IH, 10 
IM, 4 
IL 
IM 
IH 
AL 
AM 
AH 
0 2 4 6 8 10 12
French_ACTFL_’12_’13 
AH, 3 
AM, 1 
AL, 7 
IH, 6 
IL 
IM 
IH 
AL 
AM 
AH 
0 1 2 3 4 5 6 7 8
French_ACTFL_2013 
AH, 2 
AM, 1 
AL, 6 
IH, 1 
IL 
IM 
IH 
AL 
AM 
AH 
0 1 2 3 4 5 6 7
French_ACTFL_Advanced_2007_ 
2013 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 
2013 
2012_2013 
2011_2013 
2009_2013 
200...
Spanish_ACTFL_Advanced_2007 
_2013 
0% 5% 10% 15% 20% 25% 30% 35% 
2013 
2012_2013 
2011_2013 
2009_2013 
2007_2013
What is COPI telling us about Spanish 
improvement in the study abroad 
context? 
CIEE Conference 
Baltimore, November 201...
CONTENTS 
• Describe the process of training, 
implementation and correction of the exam; 
• Analyze the progress made by ...
What is COPI (Computer based Oral Proficiency Interview)? 
3 
• The COPI exam is a computerized 
semi-adaptive exam aimed ...
Evaluation Levels 
4 
1. Superior: The candidate gives opinions, hypothesizes, and uses abstract language in unfamiliar 
l...
Administration of the Exam 
The exam has the following components: 
• A CD for the administration of the COPI exam 
• An a...
Interview Evaluation and Grading and Final Report of the Evaluation 
of the level reached by the candidate. 
6 
• The prog...
Exam and Certification of COPI Evaluators 
• Evaluators are trained by using a kit provided by CAL 
(Center of applied lin...
8
>197 %
10
11
Why COPI implementation represents a challenge? 
12 
• BECAUSE it implies working with 
isolated language samples that 
mu...
Effects of International 
Service-Learning on the 
Development of Spanish Oral 
Proficiency 
CIEE Conference: Breaking Bar...
Overview of presentation 
① Goal 
② Community Service-Learning 
a) General notions 
b) SL in higher education 
c) SL in th...
GOAL 
¡ Gain further insight into the effects of context 
(i.e., service-learning in an international 
setting) on the L2...
SERVICE-LEARNING (SL): 
GENERAL NOTIONS 
¡ SL? “Experiential education that engages students in 
activities that address ...
SL IN HIGHER EDUCATION 
¡ Based on statistics from 2009-2010, ½ + of 
community colleges and approximately ¼ of all 
univ...
SL IN THE SPANISH CLASSROOM 
¡ The incorporation of SL in Spanish curricula follows 
similar trends observed in other dis...
SL AT-HOME: L2 GAINS 
¡ Offer students authentic communicative contexts in 
the target language 
¡ Communicative compete...
¡ SL as an effective way enhance student’ language 
skills, knowledge of the host country, intercultural 
sensitivity, an...
RESEARCH QUESTIONS 
① What is the effect of a community-enhanced 
Spanish for Specific Purpose course on the oral 
profici...
INSTRUMENT 
10
PARTICIPANTS (CONT’D) 
Experimental group 
22 students (14 f / 8 m) 
19-21 yrs old 
3rd year Spanish 
Spanish minors/major...
PARTICIPANTS 
Experimental group Control group 
12
13 
BLOGGING & SPANISH FOR THE 
WORKING ENVIRONMENT
BLOGGING & SPANISH FOR THE 
WORKING ENVIRONMENT (CONT’D) 
14
BLOGGING & SPANISH FOR THE 
WORKING ENVIRONMENT 
15
RESULTS (CONT’D) 
ü Pre & Post Score Comparison (Experimental Group, N=22) 
16
RESULTS (CONT’D) 
ü Pre & Post Score Comparison (Control Group, N=18) 
17
RESULTS 
Post 
Mean 
p-value 
Mastery sentence 43.10 50.00 0.0003*** 
Experimental Vocabulary 36.77 51.22 <0.0001*** 
Grou...
DISCUSSION 
① What is the effect of a community-enhanced Spanish 
for Specific Purpose course on the oral proficiency at 
...
CONCLUSIONS 
① The incorporation of the SL component into a Spanish 
for Specific Purposes course in an international sett...
CHALLENGES & 
RECOMMENDATIONS 
q Challenges: 
§ Time and effort: More so than the ‘normal’ course; 
§ Technological pro...
ACKNOWLEDGEMENTS 
² Department of Hispanic Studies, U of Kentucky 
² Service-Learning Council, U of Kentucky 
² Educati...
THANK YOU!
QUESTIONS? COMMENTS?
Upcoming SlideShare
Loading in …5
×

Measuring Linguistic Gains in Immersion Settings: Empirical Studies from French and Spanish as a Second Language

450 views

Published on

Study abroad provides students with a unique opportunity to develop foreign language skills alongside native speakers. Or does it? This panel will look at different assessment methods and tools used to measure student linguistic gains after a semester studying abroad in France, Spain, and Argentina. Special emphasis will be placed on “high-impact” practices that contribute to positive gains in language learning during study abroad experiences including homestays, direct enrollment, and community engagement.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Measuring Linguistic Gains in Immersion Settings: Empirical Studies from French and Spanish as a Second Language

  1. 1. Measuring Linguistic Gains in Immersion Settings: Empirical Studies from French and Spanish as a Second Language Dennis M. Wiseman Council for International Education Exchange Annual Conference Breaking Barriers: Cost, Curriculum, Culture Baltimore, Md November 19-22, 2014
  2. 2. Wofford College • Four-year, private, liberals arts college • Founded in 1854 • Methodist affiliated • Phi Beta Kappa • # 4 in the Open Doors survey • 1600 students
  3. 3. National Survey of Student Engagement
  4. 4. National Survey of Student Engagement
  5. 5. National Survey of Student Engagement
  6. 6. National Survey of Student Engagement
  7. 7. Linguistic Gains: High Impact Practices All French or Spanish students in these data have engaged in at least one of the following activities in study abroad. – Home stay – Language course work – Internship – Volunteer or service learning
  8. 8. Methodology • OPIc • Immediately upon return from study abroad, set / proctored • French students – Paris, Nantes, Rennes, Bordeaux, Aix-en- Provence, Dijon, Arles • Spanish students – Spain, Argentina, Panama, Chile, Dominican Republic, Peru, Ecuador, Bolivia, Honduras, Nicaragua
  9. 9. SPAIN LAT. AMER. SPANISH_STUDY_ABROAD_2007_'13 0 20 40 60 80 100 120 140 LAT. AMER. SPAIN Series1 118 77
  10. 10. ACTFL_OPI_DID_NOT_PARTICIPATE_# 0 20 40 60 80 100 120 140 SPAIN DNT SPAIN LAT. AMER DNT LAT. AMER. LAT. AMER. LAT. AMER DNT SPAIN SPAIN DNT Series1 118 16 77 29
  11. 11. SPANISH_LAT._AMER._ACTFL_2007_2013_# 0 5 10 15 20 25 30 35 40 45 50 DNT IL IM IH AL AM AH AH AM AL IH IM IL DNT Series1 1 6 31 44 18 2 16
  12. 12. SPANISH_SPAIN_ACTFL_2007_2013_# 0 5 10 15 20 25 30 35 DNT IL IM IH AL AM AH AH AM AL IH IM IL DNT Series1 1 7 25 15 29
  13. 13. SPANISH_LAT._AMER._ACTFL_2007_2013_% 0 5 10 15 20 25 30 35 40 DNT IL IM IH AL AM AH AH AM AL IH IM IL DNT Series1 0.8 5.17 26.27 37.28 15.25 1.6 13.56
  14. 14. SPANISH_SPAIN_ACTFL_2007_2013_% 0 5 10 15 20 25 30 35 40 DNT IL IM IH AL AM AM AL IH IM IL DNT Series1 1.3 9.1 32.47 19.48 37.66
  15. 15. French_ACTFL_2007_2013 AH, 3 AM, 6 AL, 21 IH, 16 IM, 12 IL IM IH AL AM AH 0 5 10 15 20 25
  16. 16. French_ACTFL_’09_’13 AH, 3 AM, 4 AL, 19 IH, 13 IM, 7 IL IM IH AL AM AH 0 2 4 6 8 10 12 14 16 18 20
  17. 17. French_ACTFL_’11_’13 AH, 3 AM, 2 AL, 11 IH, 10 IM, 4 IL IM IH AL AM AH 0 2 4 6 8 10 12
  18. 18. French_ACTFL_’12_’13 AH, 3 AM, 1 AL, 7 IH, 6 IL IM IH AL AM AH 0 1 2 3 4 5 6 7 8
  19. 19. French_ACTFL_2013 AH, 2 AM, 1 AL, 6 IH, 1 IL IM IH AL AM AH 0 1 2 3 4 5 6 7
  20. 20. French_ACTFL_Advanced_2007_ 2013 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2013 2012_2013 2011_2013 2009_2013 2007_2013
  21. 21. Spanish_ACTFL_Advanced_2007 _2013 0% 5% 10% 15% 20% 25% 30% 35% 2013 2012_2013 2011_2013 2009_2013 2007_2013
  22. 22. What is COPI telling us about Spanish improvement in the study abroad context? CIEE Conference Baltimore, November 2014 Vera Cerqueiras, BA Spanish Coordinator
  23. 23. CONTENTS • Describe the process of training, implementation and correction of the exam; • Analyze the progress made by students between their entrance COPI and their exit COPI; • Show some of the challenges and the subsequent effects (washback) resulting from this evaluation. 2
  24. 24. What is COPI (Computer based Oral Proficiency Interview)? 3 • The COPI exam is a computerized semi-adaptive exam aimed at determining the ability for using oral spanish language at a specific moment taking into account the ACTFL Guidelines. • It is intended for high school students, college students, and professionals whose native language is English or who have an advanced English level • Standarized exam • Measuring linguistic production holistically • Providing feedback to both the test takers and their professors. • Exam results can be used to show the progress of a student after a semester of study. • It cannot be used to make decisions to pass or fail Spanish courses Characteristics
  25. 25. Evaluation Levels 4 1. Superior: The candidate gives opinions, hypothesizes, and uses abstract language in unfamiliar language situations 2. Advanced: The candidate narrates and describes situations with some level of complication 3. Intermediate: The candidate demonstrates creative use of the language in simple and everyday situations 4. Novice: The candidate uses fixed expressions, short phrases and lists of memorized words The three latter levels are subdivided into 3 sublevels: High: The candidate does not have the ability to reach and maintain the next level Medium: Represents the prototypical level Low: The candidate has difficulty in meeting the level requirements
  26. 26. Administration of the Exam The exam has the following components: • A CD for the administration of the COPI exam • An application for the certification of the exam • Lab with PC`s ,Computers, headset with microphone and Speaker, keyboards • The candidates must select their Spanish level taking in account the summary description of each level given by the program • The program selects from 7 to 11 tasks • The candidates must complete the exam by recording their answers • Teachers / Raters should be in the computer lab to assist. • The exam lasts between 45/60 minutes 5
  27. 27. Interview Evaluation and Grading and Final Report of the Evaluation of the level reached by the candidate. 6 • The program saves the candidate’s activities. • A double assessment is performed by two certified professors who correct all the activities completed by each candidate. • The evaluator assigns a corresponding level for each answer he/she listens to, taking into account the level assigned by the student or the program • The evaluator has the option to write comments about the activity, which the program then saves as part of the evaluation. • The program establishes an overall average after the evaluator assigns a level for each of the activities performed by the candidates. • A final report with the certification of each candidate’s obtained level is generated by the program including the overall average and a description of the proficiency
  28. 28. Exam and Certification of COPI Evaluators • Evaluators are trained by using a kit provided by CAL (Center of applied linguistics) • They have to pass an exam according to ACTFL Guidelines • This exam consists of 15 comprehensive COPI exams that should be rated • Evaluators approve the exam if at least 60% (9/15) scores coincide with those indicated by the CAL experts. 7
  29. 29. 8
  30. 30. >197 %
  31. 31. 10
  32. 32. 11
  33. 33. Why COPI implementation represents a challenge? 12 • BECAUSE it implies working with isolated language samples that must be analyzed without consideration of neither its production context nor its acquisition process. • BECAUSE the evaluator's work needs permanent training and peer feedback in order to adjust and perfect the administration, correction and considerations of the exam. • The internalization of the practices incorporated by giving the COPI exam allow us to focus on the elaboration of evaluation instructions and on the importance that the "knowledge of the world" has towards the successful achievement of that instruction. • The progress data between entry exams and exit exams tests the results obtained by the very same students in the evaluations of the Spanish courses they take as part of their immersion program studies. Until now they are consistent. In other words, those students who advance the most are the same students who achieve the highest grades in their Spanish courses. Washback Effects
  34. 34. Effects of International Service-Learning on the Development of Spanish Oral Proficiency CIEE Conference: Breaking Barriers November 19-22, 2014, Baltimore Francisco Salgado-Robles, Ph.D. The City University of New York
  35. 35. Overview of presentation ① Goal ② Community Service-Learning a) General notions b) SL in higher education c) SL in the Spanish classroom d) SL at-home: Linguistic gains e) SL abroad: Linguistic gains ③ Research Questions ④ Instrument ⑤ Participants ⑥ Blogging & Service-Learning ⑦ Results ⑧ Discussion ⑨ Conclusions ⑩ Challenges & Recommendations
  36. 36. GOAL ¡ Gain further insight into the effects of context (i.e., service-learning in an international setting) on the L2 acquisition process (i.e., the development of Spanish oral proficiency) through a case-control study analysis of participants’ experiences enrolled in a Language for Specific Purposes (LSP) course (i.e., Spanish for the working environment) during a summer semester abroad. 3
  37. 37. SERVICE-LEARNING (SL): GENERAL NOTIONS ¡ SL? “Experiential education that engages students in activities that address human and community needs together with structured opportunities intentionally designed to promote student learning and development” (Jacoby, 1996) ¡ Students’ service experiences are tied to the academic content of an on-campus course or curriculum (Giles, Honnet, & Migliore, 1991) ¡ From the tenets of experiential education and constructivist theories that advocate for learning through first-hand discovery (Furco, 2001) ¡ “One of the most effective approaches for engaging students and boosting learning” (Learn and Serve America, 2011) ¡ Shape learners’ knowledge and cultural beliefs, professional aspirations, and community engagement. 4
  38. 38. SL IN HIGHER EDUCATION ¡ Based on statistics from 2009-2010, ½ + of community colleges and approximately ¼ of all universities integrate SL in different academic curricula (Learn and Serve America, 2011) ¡ SL pedagogy has been adopted not only due to the overwhelming amount of research in support of this type of educational experience, but also because it has been witnessed how life-changing these opportunities can be for their students (Jeandron & Robinson, 2010) 5
  39. 39. SL IN THE SPANISH CLASSROOM ¡ The incorporation of SL in Spanish curricula follows similar trends observed in other disciplines in higher education ¡ Community engagement has been a “growing presence in Spanish classes” and that there is presently “significant activity and interest within our ranks (Hellebrandt & Jorge, 2013) ¡ Rapidly expanding Hispanic population in the United States (Rabin, 2009; Zlotkowski, 1999) ¡ Convincing evidence of the positive impact SL can have on students taking Spanish as a second or heritage language (Petrov, 2013) 6
  40. 40. SL AT-HOME: L2 GAINS ¡ Offer students authentic communicative contexts in the target language ¡ Communicative competence improved from regular interaction with NSs after participating in SL (Barreneche, 2011; Navarro, 2012; Olazagasti-Segovia, 2004; Salgado-Robles, 2014) ¡ Facilitate linguistic improvements in the L2 (Hale, 1999; Mullaney, 1999; Overfield, 1997) ¡ Spanish SL students scored higher on L2 reading, speaking, and writing tasks (Malkin, 2010) ¡ Students increased their use of Spanish outside the classroom and felt more confident speaking in Spanish (Pellettieri, 2011) 7
  41. 41. ¡ SL as an effective way enhance student’ language skills, knowledge of the host country, intercultural sensitivity, and global competence (Kiely, 2011) ¡ Students gained confidence in their Spanish skills, developed intercultural skills (Kopischke Smith & Moreno-López, 2012) ¡ Facilitate linguistic improvements in the L2 (Cubillos, 2013) ¡ Students perceived a positive relationship between their participation in SL projects and the development of their linguistic and cultural proficiency (Brown & Purmensky, 2014) 8 SL ABROAD: L2 GAINS
  42. 42. RESEARCH QUESTIONS ① What is the effect of a community-enhanced Spanish for Specific Purpose course on the oral proficiency at the Intermediate-level? ② What aspects of oral proficiency are most and least influenced by the community engagement experience? 9
  43. 43. INSTRUMENT 10
  44. 44. PARTICIPANTS (CONT’D) Experimental group 22 students (14 f / 8 m) 19-21 yrs old 3rd year Spanish Spanish minors/majors Spanish for the Professions Control group 18 students (11 f / 7 m)* 19-23 yrs old 3rd year Spanish Spanish minors/majors Spanish for the Professions * 1 heritage speaker removed 11
  45. 45. PARTICIPANTS Experimental group Control group 12
  46. 46. 13 BLOGGING & SPANISH FOR THE WORKING ENVIRONMENT
  47. 47. BLOGGING & SPANISH FOR THE WORKING ENVIRONMENT (CONT’D) 14
  48. 48. BLOGGING & SPANISH FOR THE WORKING ENVIRONMENT 15
  49. 49. RESULTS (CONT’D) ü Pre & Post Score Comparison (Experimental Group, N=22) 16
  50. 50. RESULTS (CONT’D) ü Pre & Post Score Comparison (Control Group, N=18) 17
  51. 51. RESULTS Post Mean p-value Mastery sentence 43.10 50.00 0.0003*** Experimental Vocabulary 36.77 51.22 <0.0001*** Group Fluency 45.52 54.19 <0.0001*** (+SL) Pronunciation 46.18 57.30 0.0003*** Mastery sentence 43.94 46.04 0.0002*** Control Vocabulary 39.75 52.76 <0.0001*** Group Fluency 47.00 49.15 0.1320 (-SL) Pronunciation 44.60 48.63 0.0302 *** High Statistical Significance Pre Mean ü Paired t-tests run to determine significance of trends over time 18
  52. 52. DISCUSSION ① What is the effect of a community-enhanced Spanish for Specific Purpose course on the oral proficiency at the Intermediate-level? ü A statistically significant impact on the proficiency level of Intermediate-level learners. ü Just one participant experienced L2 gains towards the advanced-low proficiency level in this experiment. ② What aspects of oral proficiency are most and least influenced by the community engagement experience? ü SL had an observable and statistically significant impact on all sub-skills of oral proficiency measured by the Versant Test. 19
  53. 53. CONCLUSIONS ① The incorporation of the SL component into a Spanish for Specific Purposes course in an international setting had a significant effect on oral proficiency; ② The learning conditions provided by the community engagement facilitated oral skill development across all sub-skills; ③ Overall, the SL pedagogy worked successfully for this LSP course and, thus, should be further promoted, since: ü Provide more opportunities for L2 use in meaningful ways; ü Practice different L2 skills: speaking, reading, writing, listening, cultural understanding, etc.; ü Debrief and reflect on SL experience with others; ü Introduction to the professional field in a hands-on way; ü Give more confidence to students when they are in their services. 20
  54. 54. CHALLENGES & RECOMMENDATIONS q Challenges: § Time and effort: More so than the ‘normal’ course; § Technological problems and conquering the learning curve; § Lots of smaller tasks (visit report via Google Docs; blog entry and comments; in-class led discussions); some students had a hard time keeping up with due dates. q Recommendations: § Cooperative efforts among researchers, students, instructors, administrators, and community supervisors are plainly needed; § The development and dissemination of SL in LSP pedagogic materials are warranted; § More large-scale studies and longitudinal assessment of L2 acquisition (both qualitative and quantitative) within the SL (and LSP) contexts would help to further understand a new strand of experiential learning (Kolb, 1984). 21
  55. 55. ACKNOWLEDGEMENTS ² Department of Hispanic Studies, U of Kentucky ² Service-Learning Council, U of Kentucky ² Education Abroad Office, U of Kentucky ² Spanish Studies Abroad, Seville, Spain ² Non-profit organizations ² 40 students
  56. 56. THANK YOU!
  57. 57. QUESTIONS? COMMENTS?

×