Presentation at GALA Conf 2018 of the full paper "Improving serious games analyzing learning analytics data: lessons learned"

1. Improving serious games analyzing
learning analytics data:
lessons learned
Cristina Alonso-Fernández, Iván Pérez-Colado, Manuel Freire Morán,
Iván Martínez-Ortiz and Baltasar Fernández-Manjón
Games and Alliance Conference (GALA Conf)
December 7th 2018, Palermo, Italy

2. Introduction
Serious Games
● applied successfully in multiple fields
● however, poor uptake in formal education

3. Introduction
Serious Games
● applied successfully in multiple fields
● however, poor uptake in formal education
Why?
● high development cost of new games
● difficulty to assess acquired learning
● few serious games formally evaluated and with
limited number of users
Learning Analytics data can provide insight into students’ learning and
improve development and deployment of Serious Games.

4. Uses of Learning Analytics data
● Open data can be shared for research
purposes
● Real-time feedback for teachers to control
their classroom when applying games
● With enough data gathered, data mining to
improve design, evaluation and deployment
allow stakeholders to benefit from enhanced
feedback or stealth assessment

5. In-game user interaction data
● Anonymization: to ensure no personal details are attached to student data
● Collection: non-intrusive and transparent
● Storage: capability for large amounts of data and handle format

6. In-game user interaction data
● Anonymization: to ensure no personal details are attached to student data
Using randomly generated codes
● Collection: non-intrusive and transparent
Using standard tracking model (xAPI-SG) to simplify and standardize
● Storage: capability for large amounts of data and handle format
GLA System to handle data in standard format xAPI-SG

7. Experience API - Serious Games Profile
https://adlnet.gov/news/a-serious-games-profile-for-xapi
https://xapi.e-ucm.es/vocab/seriousgames
Ángel Serrano-Laguna, Iván Martínez-Ortiz, Jason Haag, Damon Regan, Andy Johnson, Baltasar Fernández-Manjón (2017):
Applying standards to systematize learning analytics in serious games. Computer Standards & Interfaces 50 (2017) 116–123,
http://dx.doi.org/10.1016/j.csi.2016.09.014

8. Learning Analytics Framework (LAF)
Ref: Chatti, M. A., Lukarov, V., Thüs, H., Muslim, A., Yousef, A. M. F., Wahid, U., … Schroeder, U. (2015). Learning Analytics: Challenges and
Future Research Directions. E-Learning and Education (Eleed), (10). Retrieved from http://eleed.campussource.de/archive/10/4035

9. Adapted version of the LAF

10. Real-time applications: teachers
Games in a classroom environment
Visual analytics provide information of students’
interactions:
a) correct and incorrect alternatives selected
b) total session players
c) maximum progress of players per
completable
d) games started and completed

11. Real-time applications: teachers
Dashboards provide general
information
However, specific situations
may require immediate actions
for teachers (e.g. player inactive
for too long)
Alerts and warnings can be
triggered in those situations

12. Real-time applications: students
Students can receive information to easily
assess strengths and weakness
Common solutions: compare results of
students with their peers
Should promote knowledge mastery
instead of competition
Adaptive learning experiences in
response to players’ in-game performance

13. Offline data analysis
Data mining processes can provide richer information for all stakeholders.
● evidence-based
decisions: quantify to
what extent games
benefit learning
Game developer
or designer
Student
Educational institution
and administrator
● feedback to improve
game and learning
design
● categorize players creating
profiles for targeted feedback
● improve assessment without
pre-post experiments
● extract patterns of use to
improve the game for
future deploymentsTeacher

14. Offline data analysis
Evaluation of students with pre-post
Our alternative proposal, applying machine
learning on learning analytics data:
● Create prediction models at validation
stage
● Predict students post knowledge based on
interaction data
● Therefore, avoid carrying out post-test
Knowledge
before playing
Players’
interactions
Knowledge
after playing

15. Conclusions
Great variety of applications of Learning Analytics data to Serious Games:
● Improve all steps of full games lifecycle
● Benefits for all stakeholders involved
Both in real-time scenarios & after enough data has been collected (mining)
Increase and improve adoption of games in schools with a general game LA system
that standardizes data tracking, collection, analysis and visualizations.
Data-driven solutions that use game LA are key to guide the future of SGs.

16. Improving serious games analyzing
learning analytics data:
lessons learned
Cristina Alonso-Fernández, Iván Pérez-Colado, Manuel Freire Morán,
Iván Martínez-Ortiz and Baltasar Fernández-Manjón
Games and Alliance Conference (GALA Conf)
December 7th 2018, Palermo, Italy
calonsofernandez@ucm.es