Uploaded byeMadrid network
Bootstrapping serious games to assess learning through analytics - Baltasar Fernández-Manjón
This document summarizes research on using serious games to assess learning through analytics. It discusses how games can be validated using pre-post tests to ensure effectiveness and provide training data for machine learning models. Interaction data from validated games can then train models to predict learning from gameplay without exams. The researchers developed tools like a game analytics tracker, validation tool, and analysis tool to facilitate collecting interaction data, validating games, and analyzing results. Their authoring tool integrates these analytics capabilities. Future work will integrate machine learning models into the validation tool to directly provide assessment scores based on interaction data. The goal is to close the assessment loop for serious games.
More Related Content
Similar to Bootstrapping serious games to assess learning through analytics - Baltasar Fernández-Manjón
Bootstrapping serious games to assess learning through analytics - Baltasar Fernández-Manjón
- 1. Bootstrapping serious gamesto assess learning through analytics Freire-Morán, Manuel; Calvo-Morata, Antonio; Martinez-Ortiz, Ivan; Fernández-Manjón, Baltasar balta@fdi.ucm.es Dept. of Software Engineering and Artificial Intelligence Complutense University of Madrid EDUCON 2023 | May 1-4 2023 | Salmiya, Kuwait
- 2. Serious games forassessment Serious Games are effective at teaching, creating awareness, changing behaviours, … What obstacles remain to make them more useful for assessment? Development high costs must design for both fun & learning Testing & Analytics waterfall testing much worse than agile require infrastructure Deployment much harder to produce than other LMS activities must communicate SG, analytics & LMS Assessment intrusive exams… or validated games, analytics, & Machine Learning
- 3. ● Validating agame: ○ ensures effectiveness at stated goal ○ typically via pre-post experiments ○ provides training data for assessment ○ can also collect player interactions, so that ● ML trained on validation data can predict learning from interactions ● Predicted learning without exam = stealth assessment ○ non-intrusive ○ does not break player immersion Serious games to assess learning Improving evidence-based assessment of players using serious games Telematics and Informatics Volume 60, July 2021, 101583 https://doi.org/10.1016/j.tele.2021.101583
- 4. 1. validation via pre-posttests 2. interaction data + target Once model is trained: from interactions to predictions, can be used for assessment 3. train ML model to predict target from interactions proving evidence-based assessment of players using serious games Telematics and Informatics Volume 60, July 2021, 101583 https://doi.org/10.1016/j.tele.2021.101583
- 5. Deploying assessment-friendly SGs ●Analytics require player ids ○ interactions from a given player should be stored & analyzed together ○ not necessary to know real player identity, as long as id is consistent ● Simplest possible analytics-enabled deployment = a link from LMS ○ link must support launch + communication to & from LMS ○ a mediator tool solves many deployment issues, can be integrated with server-side analytics Using e-Learning Standards to Improve Serious Game Deployment and Evaluation EDUCON 2022, 10.1109/EDUCON52537.2022.9766573
- 6. Using e-Learning Standardsto Improve Serious Game Deployment and Evaluation EDUCON 2022, 10.1109/EDUCON52537.2022.9766573 What the teacher sees 1. generate LTI link from authoring tool 2. paste into LTI widget in LMS course 3. see how students are playing / have played What really happens see illustration right What students see 1. click link 2. play game
- 7. Democratizing Game LearningAnalytics for Serious Games GALA 2021 https://doi.org/10.1007/978-3-030-92182-8_16
- 8. ● Analytics aspart of design ○ incremental improvement is better than waterfall models, but improvement is hard to track without analytics! ○ requires developers/designers to choose in-game interactions to report, and tracker component within each game to report those interactions ○ requires tools to analyze results ● Analytics infrastructure ○ server-side component to validate, store & retrieve interactions ○ analytics dashboards provide the insights into collected interactions ○ infrastructure also for deployment, validation, and finally assessment Testing and improving an assessment-friendly SG e-UCM Game Learning Analytics components, based on xAPI
- 9. e-UCM xAPI tracker ●C#, JS and Unity. ● High-level interface for devs. ● Compatible with xAPI for SGs. SIMVA validation tool ● Orchestrates LA activities and LTI. ● Manages users, groups (A/B), pre-post, analysis and trace storage. ● Unity plugin for easy setup with OAuth2.0. TxMon analysis tool ● Analyzes xAPI traces. ● Default visualizations.
- 10. Developing assessment-friendly games ●Game development is expensive ○ Technical expertise to code ○ Art assets may require talented artists ○ Game design (for the fun) + domain experience (for the education) ● Assessment requires data ○ from interactions (or it will not be stealth) ○ collected via an analytics infrastructure uAdventure, our authoring environment built on Unity, integrates analytics
- 11. uAdventure authoring tool ●Narrative and geolocated gameplay styles ● Easy to use by non-experts ● GLA out-of-the-box ● Exports native and WebGL ● Includes e-UCM Tracker
- 12.
- 13. Next steps ● SeriousGames can be a powerful method for student assessment in complex issues (i.e. knowledge, awareness) and it is open to include AI techniques ● Still complex but open software and standards could help in the process ● Assessment not yet fully integrated into Simva ● Building new module to ○ train a simple ML models from validation data ○ use trained models to provide predictions based on gameplay interactions ○ upload results as activity scores via LTI, making them available in host LMS ● Would close SG lifecycle, from authoring to evaluation ● All the technical solutions should take into account the ethical aspects and comply with the privacy regulations (e.g. EU GDPR)
- 14. Thanks! Contact: balta@fdi.ucm.es This workhas been partially funded by the Regional Government of Madrid (eMadrid S2018/TCS-4307, co-funded by the European Structural Funds FSE and FEDER), by the Ministry of Education (PID2020-119620RB-I00) and by the Telefónica-Complutense Chair on Digital Education and Serious Games. https://github.com/e-ucm/uAdventure https://github.com/e-ucm/simva Our publications: https://www.e-ucm.es/publications/all