Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using mobile phones for
the assessment of and for
oral skills development in
secondary education.
Ton Koenraad
Hogeschool ...
Anglia Network Europe
www.anglianetwork.eu
Step by step
- 28 January
- 15 April
- 20 May
- 24 June
- Speaking Tests
Examinations
English for Kids Foundation
“Voice for children”
A, not -for-profit organisation
EFKF projects:
India
Nicaragua
Very Young
Learners
South Africa
Elandsdoorn
Kenia Teacher
support
Surinam
EDS
Gobabis
Nami...
Why this Project?
 Anglia: flexibility / assessment of
- sharing innovative speaking practice
materials
- delivery of for...
Issues to be researched
 general organisation
 task and test design,
 teacher competences
 face & content validity asp...
How it works
Teachers:
Personal online workspace to set spoken
questions/tasks.
Questions are easy to set online by using ...
How it works
Students:
Connect using:
Mobile phones
iPod Touch
Skype
Landline
Computer
Access spoken exercises & leave voi...
Pilot
 2 secondary ed. EFL teachers
 2 Anglia member schools
 Volunteer students (n= 20)
 Assessment: asynchronous, in...
Pre-Questionnaire:
Learner Profiles (2)
Aspect Group A Group B
Speaking Skills:
(Self reported)
Fairly good Good
Like spea...
Pre-Questionnaire:
Learner perceptions:
L2 in class & IRL
Aspect Group A Group B
L2 in lessons Once in 3
lessons
Practical...
Assessment of Oral skills
Aspect Group A Group B
Assessed Tasks in
2010
All: 1 All: 3.5
School reports: Oral
skills includ...
Topics
 Introduction/warming up
 Your holiday this year
 A good school
 Social networks
 A million euros
 The climate
Post-Questionnaire (1)
Aspect Group A Group B
Technically OK? Yes So, so
Read
Instructions
Yes Sure
Different from
expecta...
Pupils’ Comments
 Time constraint is unnatural
 Was interrupted: new session needed
 Retries: worries about costs
 Que...
Teacher Perceptions
 System usability
-System: fairly user-friendly
 Topics
- More alignment with pupils’ interests migh...
Reviewing work online
22
Teacher Perceptions
 Teacher competencies:
- Knowledge of CEFR -> difficult, training
/ practice needed
- Evaluation cate...
Teacher perceptions
 Implementation
- Use as practice material and preparation for
speaking test. Actual testing: rather ...
Conclusions / next steps
 Improve briefing (demo, online tutorial) +
raise awareness implications of re-tries
 Redesign ...
Conclusions
 Pupils, teachers & management have
concerns about costs
 Also found in other projects:
[…] cost to the end ...
Thank you for
your attention.
Comments, Questions?
www.koenraad.info
Ton.Koenraad@gmail.com
Annex
Literature Selection
Research data
Learnosity
Literature selection
 Collins, T. (2005). ‘English Class on the air: Mobile
Language Learning with CellPhones’, Proceedin...
Literature selection (2)
 Naismith, L., Lonsdale, P., Vavoula, G. & Sharples, M.
(2004). ‘Literature Review in Mobile Tec...
Pre-Questionnaire:
Learner Profiles (1)
Aspect Group A Group B
Group size 12 8
Girls 5 4
Boys 7 4
Years of English 3 3
Pos...
Pre-Questionnaire:
Learner Profiles (2)
Aspect Group A Group B
Like speaking in
class
2.67
STD: 0.78
3.78
0.44
Actual spea...
Pre-Questionnaire:
Learner perceptions: Oral L2 in
class
Aspect Group A Group B
L2 in lessons 2.1
STD: 0.50
5.0
0.0
Answer...
Assessment of Oral skills
Aspect Group A Group B
Assessed Tasks in
2010
All: 1 All: 3.5
Formal reports: Oral
skills includ...
Post-Questionnaire (1)
Aspect Group A Group B
Likert scale Disagree 1 Agree 4
Technically OK? 2.43
STD: 1.13
1.8
0.84
Read...
Post-Questionnaire (2)
Aspect Group A Group B
Likert scale Disagree 1 Agree 4
Answer time left 3.4
STD: 0.5
3.2
1.1
Expect...
www.learnosity.com
Twitter @learnosity
Speak. Listen. Learn.
Using phones for & of oral skills development
Using phones for & of oral skills development
Using phones for & of oral skills development
Upcoming SlideShare
Loading in …5
×

Using phones for & of oral skills development

662 views

Published on

Presentation of an exploratory pilot project on using mobile devices for assessment of and for oral EFL skills at the PechaKucha session of the TISLID 2014 conference in Avila, Spain

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Using phones for & of oral skills development

  1. 1. Using mobile phones for the assessment of and for oral skills development in secondary education. Ton Koenraad Hogeschool Utrecht, University of Applied Sciences, Faculty of Education TELLConsult GLoCALL 2010
  2. 2. Anglia Network Europe www.anglianetwork.eu
  3. 3. Step by step
  4. 4. - 28 January - 15 April - 20 May - 24 June - Speaking Tests Examinations
  5. 5. English for Kids Foundation “Voice for children” A, not -for-profit organisation
  6. 6. EFKF projects: India Nicaragua Very Young Learners South Africa Elandsdoorn Kenia Teacher support Surinam EDS Gobabis Namibia Cambodia
  7. 7. Why this Project?  Anglia: flexibility / assessment of - sharing innovative speaking practice materials - delivery of formal language assessments English for Kids: mobile as infrastructure, & schools assessment for: washback effect of testing
  8. 8. Issues to be researched  general organisation  task and test design,  teacher competences  face & content validity aspects  system usability  Research design / instrument development
  9. 9. How it works Teachers: Personal online workspace to set spoken questions/tasks. Questions are easy to set online by using a microphone or uploading audio files. Audio player allows teachers to review spoken work and leave feedback. Over time teachers establish a digital portfolio of student work. 12
  10. 10. How it works Students: Connect using: Mobile phones iPod Touch Skype Landline Computer Access spoken exercises & leave voice responses. Connect with other students for role play. Personal online workspace to store work, listen & receive feedback. Listen to exemplar and sample questions posted by teachers.13
  11. 11. Pilot  2 secondary ed. EFL teachers  2 Anglia member schools  Volunteer students (n= 20)  Assessment: asynchronous, interview format  Oral presentation skills
  12. 12. Pre-Questionnaire: Learner Profiles (2) Aspect Group A Group B Speaking Skills: (Self reported) Fairly good Good Like speaking in class So, so Definitely Actual speaking hardly Very frequently Telecollaboration at school n/a Slightly more than once Tel. Experience in projects n/a Very occasionally Tel. Experience IRL seldom seldom
  13. 13. Pre-Questionnaire: Learner perceptions: L2 in class & IRL Aspect Group A Group B L2 in lessons Once in 3 lessons Practically every lesson Answers /Discussion Only now and then (very) frequently Pairwork Hardly ever sometimes Use of English IRL 1. Chat in games 2. Holidays 3. Skype 1. Holidays, 2. Chat in games 3. Txt chat
  14. 14. Assessment of Oral skills Aspect Group A Group B Assessed Tasks in 2010 All: 1 All: 3.5 School reports: Oral skills included? 35% ? 70% ? Expectations: Is tele-testing valid? Yes: 35% ?: 55% Yes: 60% ?: 40 % Post: Valid Yes: 60% 40%
  15. 15. Topics  Introduction/warming up  Your holiday this year  A good school  Social networks  A million euros  The climate
  16. 16. Post-Questionnaire (1) Aspect Group A Group B Technically OK? Yes So, so Read Instructions Yes Sure Different from expectation Yes Yes Questions: complexity, speed, loudness, Hard to remember, Speed bit fast, Not loud enough Idem, but Speed OK
  17. 17. Pupils’ Comments  Time constraint is unnatural  Was interrupted: new session needed  Retries: worries about costs  Questions could be louder  Questions: peer voice is more inviting
  18. 18. Teacher Perceptions  System usability -System: fairly user-friendly  Topics - More alignment with pupils’ interests might be needed  Validity -Content measured in time is less suitable as criterion when no interaction is possible - computer-based testing, as such, not perceived as unusual or unfriendly. - Retry option?
  19. 19. Reviewing work online 22
  20. 20. Teacher Perceptions  Teacher competencies: - Knowledge of CEFR -> difficult, training / practice needed - Evaluation categories (content, accuracy, complexity, fluency.) useful; scoring doable in one session. But …would prefer a grading scale that results in a CEF-level: better match to Dutch current grade system
  21. 21. Teacher perceptions  Implementation - Use as practice material and preparation for speaking test. Actual testing: rather face-to- face - Gives students the opportunity to practice outside the classroom, extra practice - Chances for providing individualised feedback - May help reduce anxiety of insecure & shy students:
  22. 22. Conclusions / next steps  Improve briefing (demo, online tutorial) + raise awareness implications of re-tries  Redesign questions (granularity)  Try-out alternatives: - system access (landlines, computers) - content aligned to syllabus / textbook
  23. 23. Conclusions  Pupils, teachers & management have concerns about costs  Also found in other projects: […] cost to the end user is a major consideration and can be a barrier to successful uptake when using mobile devices (Kukulska-Hulme & Shield, 2007)
  24. 24. Thank you for your attention. Comments, Questions? www.koenraad.info Ton.Koenraad@gmail.com
  25. 25. Annex Literature Selection Research data Learnosity
  26. 26. Literature selection  Collins, T. (2005). ‘English Class on the air: Mobile Language Learning with CellPhones’, Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT’05).  http://csdl2.computer.org/comp/proceedings/icalt/2005/23 38/00/23380402.pdf  Fallahkair, S., Pemberton, L. & Griffiths, R. 2007. ‘Development of a cross-platform ubiquitous language learning service via mobile phone and interactive television’. Journal of Computer Assisted Learning, 23 (4), 312-325.  Kukulska-Hulme, Agnes; Sharples, Mike; Milrad, Marcelo; Arnedillo-Sanchez, Inmaculada and Vavoula, Giasemi (2009). Innovation in Mobile Learning: A European Perspective. International Journal of Mobile and Blended Learning, 1(1), pp. 13–35.
  27. 27. Literature selection (2)  Naismith, L., Lonsdale, P., Vavoula, G. & Sharples, M. (2004). ‘Literature Review in Mobile Technologies and Learning’. FutureLab Report 11. http://www.futurelab.org.uk/resources/documents/lit_revie ws/Mobile_Review.pdf.  Sharples, M. (Ed.). (2006). Big issues in mobile learning. Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative, University of Nottingham, UK.  Shield, Lesley and Kukulska-Hulme, Agnes (2008). Special issue of ReCALL on Mobile Assisted Language Learning. Cambridge University Press.  Thornton, P. & Houser, C. (2005). ‘Using mobile phones in English education in Japan’. Journal of Computer Assisted Learning, 21, (3): 217-228.
  28. 28. Pre-Questionnaire: Learner Profiles (1) Aspect Group A Group B Group size 12 8 Girls 5 4 Boys 7 4 Years of English 3 3 Positive Attitude to Learning English 2.42 STD: 0.51 3.11 STD: 0.33 Average score at Secondary 6.37 STD: 0.9 7.6 STD: 1.7 Speaking Skills: (Self reported) 6.58 STD: 0.9 7.55 STD 0.68
  29. 29. Pre-Questionnaire: Learner Profiles (2) Aspect Group A Group B Like speaking in class 2.67 STD: 0.78 3.78 0.44 Actual speaking 1.5 STD: 0.52 3.56 STD: 0.53 Telecollaboration at school n/a 2.0 0.0 Tel. Experience in projects n/a 1.38 STD: 1.0 Tel. Experience IRL 1.83 STD: 1.19 1.44 STD: 0.73
  30. 30. Pre-Questionnaire: Learner perceptions: Oral L2 in class Aspect Group A Group B L2 in lessons 2.1 STD: 0.50 5.0 0.0 Answers /Discussion 2.2 / 1.6 3.9 / 3.2 Pairwork 1.6 1.9 English IRL 1. Chat in games 2. Holidays 3. Skype 1. Holidays, 2. Chat in games 3. Txt chat
  31. 31. Assessment of Oral skills Aspect Group A Group B Assessed Tasks in 2010 All: 1 All: 3.5 Formal reports: Oral skills included? 35% ? 70% ? Is tele-testing valid? Yes: 35% ? : 55% Yes: 60% ?: 40 % Post: Valid Yes: 60% 40%
  32. 32. Post-Questionnaire (1) Aspect Group A Group B Likert scale Disagree 1 Agree 4 Technically OK? 2.43 STD: 1.13 1.8 0.84 Read Instructions 2.8 STD: 0.7 3.0 0.7 Different from expectation 2.7 STD: 0.76 2.8 1.3 Questions: complex, speed, loudness, Not loud enough Hard to remember idem Speed OK
  33. 33. Post-Questionnaire (2) Aspect Group A Group B Likert scale Disagree 1 Agree 4 Answer time left 3.4 STD: 0.5 3.2 1.1 Expected Mark Just sufficient STD: 1.4 O.K 1.1 Problem Topics Networks; 1M Euros Good School; Climate Fun to do 2.8 STD: 0.4 2.4 0.9 More pleasant without teacher 2.07 STD: 0.6 2.0 1.0
  34. 34. www.learnosity.com Twitter @learnosity Speak. Listen. Learn.

×