Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Baltasar Fernández-Manjón, profile picture
Uploaded byBaltasar Fernández-Manjón
PPTX, PDF3,975 views

Learning analytics applied to serious games, invited talk at ECGBL 2013 Porto, Portugal

This document discusses applying learning analytics to serious games. It begins by introducing the speaker and their research group on learning technologies and serious games. It then discusses how learning analytics and serious games are both emerging technologies that could be combined. The document outlines how learning analytics could be used to better understand player behavior and learning within serious games, moving beyond only tracking basic completion data. It proposes methods for capturing detailed game and player data, defining relevant variables and goals, and using this intensive data set to provide feedback and improve the learning experience.

Education◦Technology◦Entertainment & Humor◦
Related topics:
The Serious Games Overview• Learning Analytics•
Downloaded 41 times
Learning Analytics Applied to Serious Games European Conference in Game-Based Learning (ECGBL 2013), Porto, Portugal Baltasar Fernandez-Manjon, balta@fdi.ucm.es , @BaltaFM e-UCM research group, www.e-ucm.es
About me and context ‣ CS Professor at Complutense U. • Director of e-UCM ‣ e-UCM research group about Learning technologies • 15 researchers • Serious games - Application to the medical domain • European projects - GALA - SEGAN - CHERMUG • www.e-ucm.es 2
Learning analytics and SG? The NMC Horizon Report: 2013 Higher Education Edition ‣ new and emerging technologies on teaching, learning, and research Time-to-Adoption Horizon: Two to Three Years ‣ Games and Gamification ‣ Learning Analytics 3 http://www.nmc.org/publications/2013-horizon-report-higher-ed
Serious Games use? ‣ Serious Games have probed to be educationally effective in several domains • Medicine, military, business, corporate training ‣ But still is a low adoption of Serious Games • Cost? ROI? ‣ Serious Games considered usually as a complementary content • Mainly used for motivational purposes • No actual impact on the final mark ‣ Difficult to include Serious Games in the learning curriculum • Assessment of acquired learning? 4
Black box model ‣ Games as independent pieces of content ‣ No information about what is happening during the in-game play ‣ Or very simple • Completed or not completed • Time used 5 del Blanco et al (2013). Using e-Learning standards in educational video games. Computer Standards & Interfaces 36 (1) pp. 178–187
Learning Analytics ‣ Improving education based on data analysis • Data driven ‣ Evidence-based education ‣ Related with … • Educational data mining • Business intelligence • Visual analytics 6 www.ed.gov/edblogs/technology/files/2012/03/edm-la-brief.pdf
Different uses ‣ Learning analytics • Data analysis that helps students improve learning outcomes. ‣ Academic/program analytics • Data analysis that provides information of what is happening in a specific program and how to plug holes or otherwise adjust. ‣ Institutional analytics • Data analysis that helps make decisions about how to improve at the institutional level. 7 Learning Impact Blog, Big data: Cool; Small data: Cooler http://www.imsglobal.org/blog/?p=258
When data is analyzed? ‣ Off-line • Analyzing data after use • Discovering patterns of use • Allows to improve the experience for future ‣ Real time • Analyzing data while the system is in use to improve/adapt the current learning experience • Allows to also use it in actual presential classes ‣ Mixed approach 8
Extensive Data ‣ Large number of participants ‣ Relatively limited number of variables ‣ Usually very little demographic information ‣ Relatively few observations for each user ‣ Wide but shallow data set 9 Adapted from: Learning Analytics and Educational Data Mining Workshop New York University – CREATE Lab April 4–5, 2013
Intensive Data ‣ Relatively low number of participants ‣ Large number of observations for each variable ‣ Large number of variables for each participant, such as • User actions, In-Game Events • Survey responses; Extensive demographic information • Video Observations • Biometric Data (HR, RESP, GSR, EEG, EKG) • Eye-tracking ‣ Narrow but deep data set ‣ Correlations among different data? 10 Adapted from : Learning Analytics and Educational Data Mining Workshop New York University – CREATE Lab April 4–5, 2013
But there is a parallel world … 11 http://us.battle.net/wow/en/media/screenshots/races?keywords=&view#/goblins04
Game Analytics ‣ Application of analytics to game development and research ‣ Telemetry • Data obtained over distance • Mobile games, MMOG ‣ Game metrics • Interpretable measures of data related to games 12
Game metrics ‣ User metrics • Customer • Player ‣ Performance metrics • Technical infrastructure ‣ Process metrics • Development of the game ‣ User metrics • Generics metrics • Genre specific metrics • Game specific metrics 13
Game metrics 14 Springer, Game Analytics Maximizing the Value of Player Data, pag 22
Game requirements for LA ‣ Most of games are black boxes. • No access to what is going on during game play. ‣ We need access to game “guts” • Game state, game variables ‣ Or the game must communicate with the outside world • Using some logging framework • Not applicable to COTS games (yet) ‣ Mozila Open Badges? http://openbadges.org/ 15
SESSION VARIABLES PLAYER GAME VARIABLES GAME VARIABLES TRACES GOALS From game data to educational information PLAYER VARIABLES User Information Educational System Information SERIOUS GAME
Sessions GAME PLAYER SESSIONS+ = A player plays a game. Produces a game session. One game session is a set of traces of ONE specific player in ONE specific game. SESSIONS = TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E player started phase 1 player scored 200 in phase 1 player clicked in Help button
VARIABLES GENERATO R SESSION Analyzing game sessions TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E TRAC E VAR VAR VAR VAR VAR VAR VAR VAR VAR VAR GOAL RESULT GOAL RESULT GOAL RESULT GOAL RESULT GOAL RESULT GOALS GENERATO R VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N GOAL DEFINITIO N GOAL DEFINITIO N GOAL DEFINITIO N GOAL DEFINITIO N
Variable generator: User sessions VAR DEFINITION VAR+ = TRACE TRACE varName:‘beatgame‘, operation: { type:'trace_present‘ ,traceType: ‘logic', trace: { event:‘end_game’ } } event: ‘end_game', timeStamp: 1380181610150 usersession.beatgame = true;+ =
Goal Generators: User sessions VAR VAR GOAL DEFINITION GOAL RESULT= usersession: { beatgame: true, score: 4747 } id: ‘finalResult’ result: ‘beatgame && score > 3000’ } usersession: { beatgame: true, score: 4747, goals: { finalResult: true } } = + +
PLAYER * GAME Analyzing game results SESSION SESSION VAR VAR VAR VAR GOAL RESULT GOAL RESULT SESSION SESSION VAR VAR VAR VAR GOAL RESULT GOAL RESULT VARIABLES GENERATO R GOALS GENERATO R VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N VAR DEFINITIO N GOAL DEFINITIO N GOAL DEFINITIO N GOAL DEFINITIO N GOAL DEFINITIO N
eAdventure game platform Open code authoring environment for the production of point-and-click adventure games & immersive learning simulations Easy to include Learning Analytics in eAdventure games
http://www.chermug.eu With MGH-Harvard Universityhttp://first-aid-game.e-ucm.es With ONT, educ@ONT
eAdventure + Learning Analytics Game Engine Communication API { type: 'input', timeStamp: some_timestamp, device: 'some_device', action: 'some_action', target: 'target_id', data: { key1: value, ...} } { type: 'logic', timeStamp: some_timestamp, event: 'some_event', target: 'some_id', data: { key1: value, ...} } Learning Analytics Database
GLEANER ‣ GLEANER: Game Learning Analytics for education research • Open code framework to capture game traces 25 Reference model in the EU NoE GALA
GLEANER Collector ‣ Implemented: Nodejs + Mongodb ‣ A web server ready to receive traces ‣ Cleans and sorts the data
GLEANER Analysis ‣ Reporter has access to the database, and presents its data through reports • graphics, heat-maps, relational tables… ‣ Evaluator has access to the database and checks the educational defined goals in the assessment model
Example: A game to learn XML http://gleaner.e-ucm.es/lostinspace/play/index.html
About the game... ‣ A basic platform game to acquire familiarity with XML documents • Learn the syntax • Understand nesting and attributes • Gain agility writing and reading XML documents ‣ Designed as a complementary activity in a Web Programming Course • For undergraduate computer science students ‣ Not developed with eAdventure
Understanding the game… Goal Main character Power-ups (new syntax elements) Write XML snippets here to move the main character to the goal
Educational goals ‣ Basic understanding of XML Syntax • Markup syntax • Basic documents • Attributes • Complex documents ‣ Basic programing skills • Sequencing • Loops
LA perspective ‣ What does GLEANER trace? • Higher level events • Generic traces & game specific traces • Only those relevant for our learning objectives ‣ The aggregator will filter and transmit: • Level completion events • Each XML fragment submitted by the player
Main dashboard (GLEANER Report) 33
Progress report 34 User Session Phases completed Current score Learning achievements Achievements are updated and highlighted in real time
Detailed traces 35 Filters XML snipets The fragments are updated in real time as submitted by the student. The instructor can filter to see only mistakes, or all the fragments submitted by a specific students.
La Dama Boba : The Game Based on The Foolish Lady by Lope de Vega The game is available at http://damaboba.e-ucm.es/ (in Spanish)
Experiment game vs class ‣ 757 students ‣ From 8 middle and high schools in Madrid region ‣ Control group and experimental group
Formal evaluation pre-post ‣ Similar results with Learning Analytics than with pre-post test? 39
learning analytics outcomes
41
Other issues in LA ‣ Ethical and legal aspects ‣ Security model ‣ Ownership of information • Informing the user ‣ Anonymization of information ‣ Aspects specially relevant if you are working with kids! 42
But there are new especifications and develpments that could sistematize the work 43
ADL eXperience API (xAPI) ‣ Result of Project Tin Can ‣ Tracks experiences, informal learning, real- world experiences (not just completions) ‣ Allows data storage AND retrieval (ex. 3rd party reporting and analytics tools) ‣ Enables tracking mobile, games, ITS, and virtual worlds experiences ‣ Developed by open source community 44 From Damon Regan (ADL) presentation at SINTICE2013
45 Activity Streams ‣ http://activitystrea.ms ‣ Collaboration between Google, Facebook, Microsoft and others ‣ Allows reporting of experiences, not just completions ‣ Format: <Actor> <Verb> <Object> (I did this): Simple Statement: I (actor) watched (verb) a video on protecting employee data (object) Complex: in the context of [information assurance certification training] with result [timestamp:2013-0618T18:30:32.360Z ]. From Damon Regan (ADL) presentation at SINTICE2013
Reporting Systems Assessme nt Services Semantic Analysis Statistical Services xAPI Learning Record Store (LRS) 46 From Damon Regan (ADL) presentation at SINTICE2013
‣ Raw data can feed several systems • An LRS • A Learning Analytics System eAdventure + LA with xAPI Raw data LRS Learning Analytics System Statements Analyzer Statements Analyzer EXPERIENCE API EXPERIENCE API
IMS Global Learning Analytics Interoperability Framework http://www.imsglobal.org/IMSLearningAnalyticsWP.pdf
Conclusions ‣ LA in Serious Games has a great potential from the application and research perspective • Simplify more complex and complete experiments ‣ LA in Serious Games should benefits from Games Analytics experience and work ‣ Still complex to implement LA in SG • Increases the (already high) cost of the games ‣ Frameworks and new standards specifications could greatly simplify LA implementation and adoption 50
Thank you! 51 balta@fdi.ucm.es @BaltaFM
Our current projects Increasing patient safety using games Modelling/teaching medical procedures at National Transplant Organization