Presentation made in the third Technological Ecosystems for Enhancing Multiculturality conference (TEEM15), Porto (Portugal)

1. Identifying foreign language learning profiles in
game-based environments by using data mining
Manuel Palomo-Duarte
Anke Berns
Andrés Yañez Escolano
Juan Manuel Dodero

2. Contents
● Introduction
● Design and architecture
● Case Study
– Settings
– Analysis
● Conclusions

3. Introduction
● Foreign language courses (ECTS)
– few hours of language practice in class
– many hours of independent learning
●
We propose using a 3D virtual world to:
– Encourage independent learning
– Improve communication skills
– Provide a fun and “natural” learning environment
● But How do students benefit from the experience?
– Do all benefit equally?

4. Design and architecture
● We used the Opensim Virtual world engine
– Well-known reliable open source
– Client-server approach

5. Game design
● Level 1: Memory game
– Pre-requisite for level 2
– Single player-mode
– Focus on changing
prepositions:
● Combination with the
accusative or dative
● And proper gender
– 30+ minutes

6. Game design
● Level 2: Hidden room
– Collaborative game, two players, ~35 minutes
– Reinforce of the previously introduced prepositions
– Game aim: tidying a room using a text-chat
Student1 knows where to collocate the
objects
Student2 must collocate the objects
in the right place

7. Case study: settings
● German as a second foreign language
undergraduate course, Univ. of Cadiz (Spain)
– Beginner level (A1.2 CEFR), 6 ECTS
● 48 hours of face-to-face teaching
● 102 hours of independent learning
– Students passed a previous course (A1.1 CEFR)
– 99 students took part in the experiment

8. Case study: settings
● Experiment steps:
1.Pre-test
2.Playing the game taking one role
3.Post-test 1
4.Playing the game taking the other role
5.Post-test 2
6.Technology Acceptance Model (TAM) questionnaire

9. Case study: analysis
● Anonymous TAM-based questionnaire [−2,+2]
– Students confirm that they benefited from the game
● Interacting with others (average 1.64)
● For fostering vocabulary (average 1.42)
● Increasing grammatical knowledge (average 1.53)
– Students felt comfortable with the experience
● Few issues using technology

10. Case study: analysis
● Clustering (k-means algorithm)
– A well-known data mining technique
– Summarizes global behaviour in groups with similar values
in the considered attributes
– Attributes of tests used for clustering:
● Number of words written in a text describing the position of
different objects on a picture
● Ratio of lexical errors by written word
● Ratio of grammatical errors by written word
● Additionally, checked grades in previous A1.1 course

11. Case study: analysis
● Lexical clustering:
– General constant
improvement
– C0 and C1:
● Similar pre- level
● Very good students
● C1 students usually
wrote more words and
→? improved more

12. Case study: analysis
● Lexical clustering:
– C2: ~half of students
● Good previous oral skills
● Great improvement in
post-test2
– C3: ¼ of students
● Wrote few words
● Best previous writing
grades
● Did not learn so much
with the game?

13. Case study: analysis
● Lexical clustering:
– General constant
improvement
– C0 and C1:
● Similar pre- level
● Very good students
● C1 students usually
wrote more words and
→? improved more

14. Case study: analysis
● Grammatical clustering:
– General improvement only after post-test2
– C2: ~half of students, good at participation
● Outstanding final results
– C0, C1 and C3: same final level
● C0 and C3 had same poor pre-test level
– C3 students wrote always an average number of words and they
started improving in post-test1
– C0 students only wrote a reasonable number of words in post-test1
and improved only after post-test2. Needed a second round?
● C1: students between C0 and C1, good previous participation
grades →? not comfortable with the game

15. Case study: analysis
● Lexical and grammatical clustering (5 groups):
– C3, 40% of students: very good pre-test results and
the best final results (more words than average)
– The other clusters have similar number of students:
● C0 and C4: good at previous tasks
– C0 good at oral tasks and average writing skills in post-test2
– C4 made effort to write more, but worst post-test2
● C1 and C2: similar previous performance
– C1 poor in vocabulary, good at grammar
– C2 the opposite

16. Conclusions
● Students felt conformable with the game
– Although some decreased grades from previous semester
● Particular insights:
– Previous oral skills ~ influence in vocabulary
– Good (but not brilliant) student in previous course + good
participation got best results in grammar
– Scarce relation to previous grades in mixed analysis
– Some students needed two rounds to improve
● Future work:
– Analysis including game-chat logs
– Applying other clustering techniques or association rules

17. Thank you for your attention!
Questions?
Source code available at
https://code.google.com/p/aprendizaje-colaborativo-preposiciones-aleman-en-opensim/