SERIOUS GAMES ASSESSMENT Not many serious games include in-game evaluationSerious games with integrated assessment usually rely in Q&A structures … but games produce a lot of data that can be analyzed with educational/assessment purposes
LEARNING ANALYTICS + VIDEOGAMES Web Analytics Business Intelligence Game Analytics
WHAT DATA CAN WE TRACK?INTERACTION TRACES LOGIC TRACES Low level events High level events GUI interactions Player score Mouse clicks Phases changes Keystrokes Completed missionsJoystick movements Player "deaths"
START GAME Whenever a student launches the gameHow many students played the game, who they were and when they played.
END GAME Whenever a student successfully the game.Who accomplished the goals established for the game Does the optimal goal attain?
QUIT GAMEWhenever a student quits the game, before finishing itWho abandoned the game before finishing it, and with the appropriate context, where he quitted.
PHASE CHANGES In an educational videogame, these phases can mark several educational sub-goals. Identifying most time-consuming phasesUnderstand how each part of the game is being accessed (if the phase exploration sequence is not linear)
SIGNIFICANT VARIABLES Games rely on variables to represent their state Some of those variables can be relevant for the assessmentTrack when and with which values these variables are updated
USER INTERACTIONRaw user interaction (mouse clicks, screen touches, keys pressed…) can be used to retrieve some useful information
SOME REQUIREMENTS Most of games are black boxes. No access to what is going on during game play. We need access to game “guts”Or… the game must communicate with the outside world, using some logging framework
GLEANERGAME LEARNING ANALYTICS FOR EDUCATIONAL RESEARCH
GAME DESIGN FOR LEARNING ANALYTICS1. What do we exactly want to teach in the game? Is a skill, is a concept?2. Can we measure if the students learned it?3. If so, how are we going to measure it? Direct methods: Q&A Indrect methods: accomplish a mission that requires the skill/concept
CAN WE RELATE THESE RESULTS WITH TRACES COLLECTED IN THE GAME?In-game metrics: correct and incorrect xml documents Students with best performance in post-tests had less incorrect documents They also had a smaller number of documents sent
CONCLUSIONSOur "universal" traces model is useful to extract web-analytics style statistics.Game design is key to enable effective learninganalyticsWe need more concrete analysis tools to measure thelearning process Simple approaches to measure simple things Generalization and abstraction