SlideShare a Scribd company logo
1 of 30
Possibilities of game learning
analytics in Coding4Learning
games
Baltasar Fernandez-Manjon
balta@fdi.ucm.es @baltafm
Is our serious game effective?
Is the game contributing to change the students
perception of programming? -> Girls?
Are the students learning computational concepts by
playing the game?
Most common methodology are pre-post questionaries in experiments:
Formal validation of serious games
Is there a significant
difference between
pre-questionary and
post-questionary
results?
Pre and post questionaries should
have been previously validated
Game Learning Analytics
breaking the game black box
model to obtain information
while students play.
Manuel Freire, Ángel Serrano-Laguna, Borja Manero, Iván Martínez-Ortiz, Pablo Moreno-Ger, Baltasar Fernández-Manjón (2016): Game
Learning Analytics: Learning Analytics for Serious Games. In Learning, Design, and Technology (pp. 1–29). Cham: Springer International
Publishing. http://doi.org/10.1007/978-3-319-17727-4_21-1.
●GLA is learning analytics applied to serious games
○collect, analyze and visualize data from learners’
interactions with SGs
Game Learning Analytics (GLA)
Using game learning analytics to improve SG lifecycle
Game learning analytics: Learning Analytics + Game Analytics
● LA improve the learning process
● GA improve the game lifecycle
○ Long time used in the game industry
● GLA improves the serious game lifecycle
○ Focuses on the learning process
○ Measures the impact of the game
○ Provides feedback for the learning process
○ Detects game issues (e.g. students getting stuck)
.
● GLA is costly and complex to implement
Analysis & Visualizations
Game Learning Analytics process
When?
● Real-time
● Off-line
Stakeholders
● Developers - Designers
● Researchers
● Teachers
● Students
play
collection & analysis
reports
information
& feedback
Game Learning Analytics is Not Informagic!!
● Educational design should guide both game design
and analytics design → Game Learning Analytics Model
○ the interaction data to be collected (the more the merrier)
○ the analyses to perform on the interaction data
■ from click-stream to educational insight
○ visualizations/reports/dashboards
No meaningful information can be extracted without defining
the required data early on the design process!
Game Learning Analytics Model
Ivan Perez-Colado, Cristina Alonso-Fernández, Manuel Freire, Iván Martínez-Ortiz, Baltasar Fernández-Manjón (2018): Game Learning Analytics is
not informagic!. IEEE Global Engineering Education Conference (EDUCON), April 18-20, 2018, Santa Cruz de Tenerife, Canary Islands, Spain.
Learning Analytics Model (LAM)
1. Design-implementation-data cycle: Allows linking specific educational
objectives to game design with game interaction data.
2. Relates through data that the educational objectives correspond to
the objectives in the game.
3. Specifies the traces to be sent, necessary information to be recorded
and sent by the game so that it can be identified that the game
objectives are met
4. The analysis model, how the traces should be analysed and
interpreted.
5. May include specific analysis and visualisations for the different
stakeholders involved
9
Game Learning Analytics Model
What data is to be
collected from the game
and how it relates to
learning goals
Which specific statements (e.g.
in xAPI format) are to be
tracked from the game
containing that information
How the statements collected
are to be analyzed and what
information is to be reported
and/or visualized
Minimun Game Requirements for GLA
●Most of games are black boxes.
○No access to what is going on during game play
●We need access to game “guts”
○User interactions
○Changes of the game state or game variables
●Or the game must communicate with the outside world
○Using some logging framework
●What is the meaning of that data?
●Ethics: adequate experimental design and setting
○Are users informed?
○Anonymization of data could be required
○Fair data exploitation for all stakeholders?
Experience API for Serious Games: xAPI-SG Profile
Experience API (xAPI) is a new IEEE standard standard that
enables the capture of data about human performance and its
context.
https://adlnet.gov/news/2021/10/20/IEEE-to-Standardize-xAPI-v2.0-as-an-International-Standard/
The e-UCM Research Group in collaboration with ADL created
the Experience API for Serious Games Profile (xAPI-SG), a
xAPI profile for the specific domain of Serious Games.
The xAPI-SG Profile defines a set of verbs, activity types and
extensions, that allows tracking of all in-game interactions as
xAPI traces (e.g. level started or completed)
https://xapi.e-ucm.es/vocab/seriousgames
Ángel Serrano-Laguna, Iván Martínez-Ortiz, Jason Haag, Damon Regan, Andy Johnson, Baltasar Fernández-Manjón (2017):
Applying standards to systematize learning analytics in serious games. Computer Standards & Interfaces 50 (2017) 116–
xAPI-SG Profile
The xAPI-SG Profile is the result of the implementation of an interactions
model for Serious Games in xAPI.
The types of interactions that can be performed in a Serious Game, and
are included in the profile, can be grouped based on the type of
interactions and game objects that the interaction is performed over.
These common interactions and game objects related with them, are
classified in general categories of xAPI-SG statements.
● completables
● accessibles
● alternatives
● GameObjects
Java
xApi
Tracker
Unity
xApi
Tracker
C#
xApi
Tracker
xAPI Game trackers as open code
https://github.com/e-ucm
Analysis and visualizations with
xAPI-SG
The generic analyses for games and their
visualisation are simplified
A: degree of a participant's success in
applying knowledge
G: shows the scores achieved in each task
H: average times per task
D, E, and I: identify the degree of progress
15
Cristina Alonso-Fernández, Antonio Calvo-Morata, Manuel Freire, Iván Martínez-Ortiz, Baltasar Fernández-Manjón (2022): Game
Learning Analytics: Blending visual and data mining techniques to improve serious games and to better understand players
learning. Journal of Learning Analytics. Volume 9(3), 32–49. https://doi.org/10.18608/jla.2022.7633, https://learning-
analytics.info/index.php/JLA/article/view/7633/7669.
Articoding
●Educational game to promote and
improve programming learning in
schools
●Block programming similar to
Scratch
●The player has to pass levels by
solving problems posed in a board-
like scenario.
●The game is aimed at students
between 12 and 16 years old.
●Experimental design support system
of evaluation
○ Quizzes + analytics
Articoding Game validation: Escolapias Puerta de Hierro
School
Formative evaluation
Participants– junio 2021
●134 students
●100 min game (2 class slots)
Questionaries
●Pre y post
●Videogame experience and use
●Previous programming knowledge
●Students opinion
Data interaction collected in xAPI
Interactions:
●Level in/out
●Level completed
●Stars obtained
●Completed intent
●Failed intent
●add/remove instruction
●…
Analytics in Articoding validation and evaluation
Main goals:
1. to verify that players learn and find out any difficulties that they may
have
2. to validate that the game design is adequate to meet its educational
objectives.
Research questions:
● RQ1. How many levels can players complete in two 50-minutes sessions?
● RQ2. Do players learn and apply the expected programming concepts?
● RQ3. Do players have problems completing game levels? Are the levels well designed?
● RQ4. Can we measure engagement using the collected interaction data?
● RQ5. Do players like the game?
Articoding: gender differences
Different number of interactions
●boys interact more
●girls think more their interactions
No significant difference in the number of
levels completed
Articoding levels: well designed? progression?
Use of interaction data to study the appropriateness of levels and
progression in game difficulty
Is the game working?
● Positive results
● 65.8% of students would continue to play.
● Boys seem to have liked it more 81.36% vs. 46.16%.
● Greater indifference among girls 5.08% vs. 32.69%.
Still an open question to
evaluate the learning with the
game compared with an
standardized method
Gaming4Coding: GLA challenges and open issues
● Mobile game
○ Android + Apple IOS
○ Apps stores
○ pre-post questionnaires with more limitations (limited screen and writing)
● BYOD - game will be played on the students mobile and/or school
tablets
● how to convey to the user the quality of their solution
○ programming allows multiple possible solutions
● Multiplayer game - maintaining the security
○ competition with other users to promote user interest
○ analytics are more complex
● Evaluation of the actual programming learning?
Prepare for the unexpected in GLA experiments
● not so hi-tech environment (e.g. schools with poor or unreliable wifi)
● dual tracker (local storage of GLA data)
● BYOD - always complex to deploy in students devices
Teachers should be provided with some extra support
● guide about how to use the game in schools - educational scenarios and uses
Why deploy your GLA when there are commercial services (some “free”)?
● Data ownership?
● Your GLA can be implemented with some standar modules (e.g. LRS)
Ethics should drive all the GLA process
● Informing the user about the data collection
● All participant should benefit from the GLA
Other aspects to consider
● Game Learning Analytics has a great potential for improving SGs
○ Evidence based serious games
○ Games as assessments (better “Stealth” student evaluation)
○ Games as powerful research environments
● GLA should be systematized by a Learning analytics model that relates
educational design with game implementation and analytics (data
collected)
○ Analytics should be applied to the whole game lifecycle
● All the Gaming4Coding project results will be open code
● Experimental design: Ethics should drive the GLA process and comply
with legal requirements (e.g. EU GDPR)
Conclusions

More Related Content

Similar to eMadrid Gaming4Coding - Possibilities of game learning analytics for coding learning games.pptx

Multi-level game learning analytics for serious games - VSGames 2018
Multi-level game learning analytics for serious games - VSGames 2018Multi-level game learning analytics for serious games - VSGames 2018
Multi-level game learning analytics for serious games - VSGames 2018
Iván Pérez Colado
 

Similar to eMadrid Gaming4Coding - Possibilities of game learning analytics for coding learning games.pptx (20)

Learning analytics for improving educational games jcsg2017
Learning analytics for improving educational games jcsg2017Learning analytics for improving educational games jcsg2017
Learning analytics for improving educational games jcsg2017
 
Multi-level game learning analytics for serious games - VSGames 2018
Multi-level game learning analytics for serious games - VSGames 2018Multi-level game learning analytics for serious games - VSGames 2018
Multi-level game learning analytics for serious games - VSGames 2018
 
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
 
Using Learning Analytics to support a more scientific approach to Serious Gam...
Using Learning Analytics to support a more scientific approach to Serious Gam...Using Learning Analytics to support a more scientific approach to Serious Gam...
Using Learning Analytics to support a more scientific approach to Serious Gam...
 
Downtown, A Subway Adventure: Using Learning Analytics to Improve the Develop...
Downtown, A Subway Adventure: Using Learning Analytics to Improve the Develop...Downtown, A Subway Adventure: Using Learning Analytics to Improve the Develop...
Downtown, A Subway Adventure: Using Learning Analytics to Improve the Develop...
 
Applying learning analytics in serious games
Applying learning analytics in serious games Applying learning analytics in serious games
Applying learning analytics in serious games
 
Serious games: current uses and emergent trends
Serious games: current uses and emergent trends Serious games: current uses and emergent trends
Serious games: current uses and emergent trends
 
Using Simva to evaluate serious games and collect game learning analytics dat...
Using Simva to evaluate serious games and collect game learning analytics dat...Using Simva to evaluate serious games and collect game learning analytics dat...
Using Simva to evaluate serious games and collect game learning analytics dat...
 
Game learning analytics dashboards teacher understanding icwl18
Game learning analytics dashboards teacher understanding icwl18Game learning analytics dashboards teacher understanding icwl18
Game learning analytics dashboards teacher understanding icwl18
 
LSAC 2018 Presentation
LSAC 2018 PresentationLSAC 2018 Presentation
LSAC 2018 Presentation
 
Integrating learning analytics into a game authoring tool
Integrating learning analytics into a game authoring toolIntegrating learning analytics into a game authoring tool
Integrating learning analytics into a game authoring tool
 
Teaching video game development panel FDG2014
Teaching video game development panel FDG2014Teaching video game development panel FDG2014
Teaching video game development panel FDG2014
 
Game Design as an Intro to Computer Science (Meaningful Play 2014)
Game Design as an Intro to Computer Science (Meaningful Play 2014)Game Design as an Intro to Computer Science (Meaningful Play 2014)
Game Design as an Intro to Computer Science (Meaningful Play 2014)
 
Fernandez manjon
Fernandez manjonFernandez manjon
Fernandez manjon
 
E adventure serious games as open reusable content ocwg
E adventure   serious games as open reusable content ocwgE adventure   serious games as open reusable content ocwg
E adventure serious games as open reusable content ocwg
 
2022_03_22 EDUCON 2022 “Using e-Learning Standards to Improve Serious Game De...
2022_03_22 EDUCON 2022 “Using e-Learning Standards to Improve Serious Game De...2022_03_22 EDUCON 2022 “Using e-Learning Standards to Improve Serious Game De...
2022_03_22 EDUCON 2022 “Using e-Learning Standards to Improve Serious Game De...
 
Data science meets standardized game learning analytics
Data science meets standardized game learning analyticsData science meets standardized game learning analytics
Data science meets standardized game learning analytics
 
EDUCON 2021_Data Science meets standardized game learning analytics
EDUCON 2021_Data Science meets standardized game learning analyticsEDUCON 2021_Data Science meets standardized game learning analytics
EDUCON 2021_Data Science meets standardized game learning analytics
 
Gaming Learning Analytics Real Colegio Complutense Harvard
Gaming Learning Analytics Real Colegio Complutense HarvardGaming Learning Analytics Real Colegio Complutense Harvard
Gaming Learning Analytics Real Colegio Complutense Harvard
 
Game learning analtytics is not informagic educon 2018
Game learning analtytics is not informagic educon 2018Game learning analtytics is not informagic educon 2018
Game learning analtytics is not informagic educon 2018
 

More from eMadrid network

More from eMadrid network (20)

Recognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo Tovar
Recognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo TovarRecognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo Tovar
Recognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo Tovar
 
A study about the impact of rewards on student's engagement with the flipped ...
A study about the impact of rewards on student's engagement with the flipped ...A study about the impact of rewards on student's engagement with the flipped ...
A study about the impact of rewards on student's engagement with the flipped ...
 
Assessment and recognition in technical massive open on-line courses with and...
Assessment and recognition in technical massive open on-line courses with and...Assessment and recognition in technical massive open on-line courses with and...
Assessment and recognition in technical massive open on-line courses with and...
 
Recognition of learning: Status, experiences and challenges - Carlos Delgado ...
Recognition of learning: Status, experiences and challenges - Carlos Delgado ...Recognition of learning: Status, experiences and challenges - Carlos Delgado ...
Recognition of learning: Status, experiences and challenges - Carlos Delgado ...
 
Bootstrapping serious games to assess learning through analytics - Baltasar F...
Bootstrapping serious games to assess learning through analytics - Baltasar F...Bootstrapping serious games to assess learning through analytics - Baltasar F...
Bootstrapping serious games to assess learning through analytics - Baltasar F...
 
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth CobosMeta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
 
Best paper Award - Miguel Castro
Best paper Award - Miguel CastroBest paper Award - Miguel Castro
Best paper Award - Miguel Castro
 
Seminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdf
Seminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdfSeminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdf
Seminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdf
 
eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...
eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...
eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...
 
Open_principles_and_co-creation_for_digital_competences_for_students.pdf
Open_principles_and_co-creation_for_digital_competences_for_students.pdfOpen_principles_and_co-creation_for_digital_competences_for_students.pdf
Open_principles_and_co-creation_for_digital_competences_for_students.pdf
 
Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...
Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...
Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...
 
eMadrid_KatjaAssaf_DigiCred.pdf
eMadrid_KatjaAssaf_DigiCred.pdfeMadrid_KatjaAssaf_DigiCred.pdf
eMadrid_KatjaAssaf_DigiCred.pdf
 
Presentazione E-Madrid - 12-01-2023 Ruth Kerr.pdf
Presentazione E-Madrid - 12-01-2023 Ruth Kerr.pdfPresentazione E-Madrid - 12-01-2023 Ruth Kerr.pdf
Presentazione E-Madrid - 12-01-2023 Ruth Kerr.pdf
 
EDC-eMadrid_20230113 Ildikó Mázár.pdf
EDC-eMadrid_20230113 Ildikó Mázár.pdfEDC-eMadrid_20230113 Ildikó Mázár.pdf
EDC-eMadrid_20230113 Ildikó Mázár.pdf
 
2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...
2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...
2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...
 
2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»
2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»
2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»
 
2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...
2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...
2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...
 
2022_11_11 «AI and ML methods for Multimodal Learning Analytics»
2022_11_11 «AI and ML methods for Multimodal Learning Analytics»2022_11_11 «AI and ML methods for Multimodal Learning Analytics»
2022_11_11 «AI and ML methods for Multimodal Learning Analytics»
 
2022_11_11 «The promise and challenges of Multimodal Learning Analytics»
2022_11_11 «The promise and challenges of Multimodal Learning Analytics»2022_11_11 «The promise and challenges of Multimodal Learning Analytics»
2022_11_11 «The promise and challenges of Multimodal Learning Analytics»
 
2022_11_11 «Biometrics and Behavior Understanding Technologies for e-Learning...
2022_11_11 «Biometrics and Behavior Understanding Technologies for e-Learning...2022_11_11 «Biometrics and Behavior Understanding Technologies for e-Learning...
2022_11_11 «Biometrics and Behavior Understanding Technologies for e-Learning...
 

Recently uploaded

Personalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaPersonalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
EADTU
 
SPLICE Working Group: Reusable Code Examples
SPLICE Working Group:Reusable Code ExamplesSPLICE Working Group:Reusable Code Examples
SPLICE Working Group: Reusable Code Examples
Peter Brusilovsky
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project research
CaitlinCummins3
 

Recently uploaded (20)

Stl Algorithms in C++ jjjjjjjjjjjjjjjjjj
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjjStl Algorithms in C++ jjjjjjjjjjjjjjjjjj
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjj
 
An Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge AppAn Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge App
 
Book Review of Run For Your Life Powerpoint
Book Review of Run For Your Life PowerpointBook Review of Run For Your Life Powerpoint
Book Review of Run For Your Life Powerpoint
 
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaPersonalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
 
diagnosting testing bsc 2nd sem.pptx....
diagnosting testing bsc 2nd sem.pptx....diagnosting testing bsc 2nd sem.pptx....
diagnosting testing bsc 2nd sem.pptx....
 
Sternal Fractures & Dislocations - EMGuidewire Radiology Reading Room
Sternal Fractures & Dislocations - EMGuidewire Radiology Reading RoomSternal Fractures & Dislocations - EMGuidewire Radiology Reading Room
Sternal Fractures & Dislocations - EMGuidewire Radiology Reading Room
 
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of TransportBasic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
 
SPLICE Working Group: Reusable Code Examples
SPLICE Working Group:Reusable Code ExamplesSPLICE Working Group:Reusable Code Examples
SPLICE Working Group: Reusable Code Examples
 
How To Create Editable Tree View in Odoo 17
How To Create Editable Tree View in Odoo 17How To Create Editable Tree View in Odoo 17
How To Create Editable Tree View in Odoo 17
 
Đề tieng anh thpt 2024 danh cho cac ban hoc sinh
Đề tieng anh thpt 2024 danh cho cac ban hoc sinhĐề tieng anh thpt 2024 danh cho cac ban hoc sinh
Đề tieng anh thpt 2024 danh cho cac ban hoc sinh
 
OSCM Unit 2_Operations Processes & Systems
OSCM Unit 2_Operations Processes & SystemsOSCM Unit 2_Operations Processes & Systems
OSCM Unit 2_Operations Processes & Systems
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project research
 
VAMOS CUIDAR DO NOSSO PLANETA! .
VAMOS CUIDAR DO NOSSO PLANETA!                    .VAMOS CUIDAR DO NOSSO PLANETA!                    .
VAMOS CUIDAR DO NOSSO PLANETA! .
 
The Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDFThe Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDF
 
Major project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategiesMajor project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategies
 
Improved Approval Flow in Odoo 17 Studio App
Improved Approval Flow in Odoo 17 Studio AppImproved Approval Flow in Odoo 17 Studio App
Improved Approval Flow in Odoo 17 Studio App
 
Including Mental Health Support in Project Delivery, 14 May.pdf
Including Mental Health Support in Project Delivery, 14 May.pdfIncluding Mental Health Support in Project Delivery, 14 May.pdf
Including Mental Health Support in Project Delivery, 14 May.pdf
 
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptxAnalyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
 
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
 
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
 

eMadrid Gaming4Coding - Possibilities of game learning analytics for coding learning games.pptx

  • 1. Possibilities of game learning analytics in Coding4Learning games Baltasar Fernandez-Manjon balta@fdi.ucm.es @baltafm
  • 2. Is our serious game effective? Is the game contributing to change the students perception of programming? -> Girls? Are the students learning computational concepts by playing the game?
  • 3.
  • 4. Most common methodology are pre-post questionaries in experiments: Formal validation of serious games Is there a significant difference between pre-questionary and post-questionary results? Pre and post questionaries should have been previously validated
  • 5. Game Learning Analytics breaking the game black box model to obtain information while students play. Manuel Freire, Ángel Serrano-Laguna, Borja Manero, Iván Martínez-Ortiz, Pablo Moreno-Ger, Baltasar Fernández-Manjón (2016): Game Learning Analytics: Learning Analytics for Serious Games. In Learning, Design, and Technology (pp. 1–29). Cham: Springer International Publishing. http://doi.org/10.1007/978-3-319-17727-4_21-1. ●GLA is learning analytics applied to serious games ○collect, analyze and visualize data from learners’ interactions with SGs Game Learning Analytics (GLA)
  • 6. Using game learning analytics to improve SG lifecycle Game learning analytics: Learning Analytics + Game Analytics ● LA improve the learning process ● GA improve the game lifecycle ○ Long time used in the game industry ● GLA improves the serious game lifecycle ○ Focuses on the learning process ○ Measures the impact of the game ○ Provides feedback for the learning process ○ Detects game issues (e.g. students getting stuck) . ● GLA is costly and complex to implement Analysis & Visualizations
  • 7. Game Learning Analytics process When? ● Real-time ● Off-line Stakeholders ● Developers - Designers ● Researchers ● Teachers ● Students play collection & analysis reports information & feedback
  • 8. Game Learning Analytics is Not Informagic!! ● Educational design should guide both game design and analytics design → Game Learning Analytics Model ○ the interaction data to be collected (the more the merrier) ○ the analyses to perform on the interaction data ■ from click-stream to educational insight ○ visualizations/reports/dashboards No meaningful information can be extracted without defining the required data early on the design process! Game Learning Analytics Model Ivan Perez-Colado, Cristina Alonso-Fernández, Manuel Freire, Iván Martínez-Ortiz, Baltasar Fernández-Manjón (2018): Game Learning Analytics is not informagic!. IEEE Global Engineering Education Conference (EDUCON), April 18-20, 2018, Santa Cruz de Tenerife, Canary Islands, Spain.
  • 9. Learning Analytics Model (LAM) 1. Design-implementation-data cycle: Allows linking specific educational objectives to game design with game interaction data. 2. Relates through data that the educational objectives correspond to the objectives in the game. 3. Specifies the traces to be sent, necessary information to be recorded and sent by the game so that it can be identified that the game objectives are met 4. The analysis model, how the traces should be analysed and interpreted. 5. May include specific analysis and visualisations for the different stakeholders involved 9
  • 10. Game Learning Analytics Model What data is to be collected from the game and how it relates to learning goals Which specific statements (e.g. in xAPI format) are to be tracked from the game containing that information How the statements collected are to be analyzed and what information is to be reported and/or visualized
  • 11. Minimun Game Requirements for GLA ●Most of games are black boxes. ○No access to what is going on during game play ●We need access to game “guts” ○User interactions ○Changes of the game state or game variables ●Or the game must communicate with the outside world ○Using some logging framework ●What is the meaning of that data? ●Ethics: adequate experimental design and setting ○Are users informed? ○Anonymization of data could be required ○Fair data exploitation for all stakeholders?
  • 12. Experience API for Serious Games: xAPI-SG Profile Experience API (xAPI) is a new IEEE standard standard that enables the capture of data about human performance and its context. https://adlnet.gov/news/2021/10/20/IEEE-to-Standardize-xAPI-v2.0-as-an-International-Standard/ The e-UCM Research Group in collaboration with ADL created the Experience API for Serious Games Profile (xAPI-SG), a xAPI profile for the specific domain of Serious Games. The xAPI-SG Profile defines a set of verbs, activity types and extensions, that allows tracking of all in-game interactions as xAPI traces (e.g. level started or completed) https://xapi.e-ucm.es/vocab/seriousgames Ángel Serrano-Laguna, Iván Martínez-Ortiz, Jason Haag, Damon Regan, Andy Johnson, Baltasar Fernández-Manjón (2017): Applying standards to systematize learning analytics in serious games. Computer Standards & Interfaces 50 (2017) 116–
  • 13. xAPI-SG Profile The xAPI-SG Profile is the result of the implementation of an interactions model for Serious Games in xAPI. The types of interactions that can be performed in a Serious Game, and are included in the profile, can be grouped based on the type of interactions and game objects that the interaction is performed over. These common interactions and game objects related with them, are classified in general categories of xAPI-SG statements. ● completables ● accessibles ● alternatives ● GameObjects
  • 15. Analysis and visualizations with xAPI-SG The generic analyses for games and their visualisation are simplified A: degree of a participant's success in applying knowledge G: shows the scores achieved in each task H: average times per task D, E, and I: identify the degree of progress 15 Cristina Alonso-Fernández, Antonio Calvo-Morata, Manuel Freire, Iván Martínez-Ortiz, Baltasar Fernández-Manjón (2022): Game Learning Analytics: Blending visual and data mining techniques to improve serious games and to better understand players learning. Journal of Learning Analytics. Volume 9(3), 32–49. https://doi.org/10.18608/jla.2022.7633, https://learning- analytics.info/index.php/JLA/article/view/7633/7669.
  • 16. Articoding ●Educational game to promote and improve programming learning in schools ●Block programming similar to Scratch ●The player has to pass levels by solving problems posed in a board- like scenario. ●The game is aimed at students between 12 and 16 years old. ●Experimental design support system of evaluation ○ Quizzes + analytics
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22. Articoding Game validation: Escolapias Puerta de Hierro School Formative evaluation Participants– junio 2021 ●134 students ●100 min game (2 class slots) Questionaries ●Pre y post ●Videogame experience and use ●Previous programming knowledge ●Students opinion
  • 23. Data interaction collected in xAPI Interactions: ●Level in/out ●Level completed ●Stars obtained ●Completed intent ●Failed intent ●add/remove instruction ●…
  • 24. Analytics in Articoding validation and evaluation Main goals: 1. to verify that players learn and find out any difficulties that they may have 2. to validate that the game design is adequate to meet its educational objectives. Research questions: ● RQ1. How many levels can players complete in two 50-minutes sessions? ● RQ2. Do players learn and apply the expected programming concepts? ● RQ3. Do players have problems completing game levels? Are the levels well designed? ● RQ4. Can we measure engagement using the collected interaction data? ● RQ5. Do players like the game?
  • 25. Articoding: gender differences Different number of interactions ●boys interact more ●girls think more their interactions No significant difference in the number of levels completed
  • 26. Articoding levels: well designed? progression? Use of interaction data to study the appropriateness of levels and progression in game difficulty
  • 27. Is the game working? ● Positive results ● 65.8% of students would continue to play. ● Boys seem to have liked it more 81.36% vs. 46.16%. ● Greater indifference among girls 5.08% vs. 32.69%. Still an open question to evaluate the learning with the game compared with an standardized method
  • 28. Gaming4Coding: GLA challenges and open issues ● Mobile game ○ Android + Apple IOS ○ Apps stores ○ pre-post questionnaires with more limitations (limited screen and writing) ● BYOD - game will be played on the students mobile and/or school tablets ● how to convey to the user the quality of their solution ○ programming allows multiple possible solutions ● Multiplayer game - maintaining the security ○ competition with other users to promote user interest ○ analytics are more complex ● Evaluation of the actual programming learning?
  • 29. Prepare for the unexpected in GLA experiments ● not so hi-tech environment (e.g. schools with poor or unreliable wifi) ● dual tracker (local storage of GLA data) ● BYOD - always complex to deploy in students devices Teachers should be provided with some extra support ● guide about how to use the game in schools - educational scenarios and uses Why deploy your GLA when there are commercial services (some “free”)? ● Data ownership? ● Your GLA can be implemented with some standar modules (e.g. LRS) Ethics should drive all the GLA process ● Informing the user about the data collection ● All participant should benefit from the GLA Other aspects to consider
  • 30. ● Game Learning Analytics has a great potential for improving SGs ○ Evidence based serious games ○ Games as assessments (better “Stealth” student evaluation) ○ Games as powerful research environments ● GLA should be systematized by a Learning analytics model that relates educational design with game implementation and analytics (data collected) ○ Analytics should be applied to the whole game lifecycle ● All the Gaming4Coding project results will be open code ● Experimental design: Ethics should drive the GLA process and comply with legal requirements (e.g. EU GDPR) Conclusions