Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Simva: Simplifying the scientific validation of serious games icalt2019


Published on

Simva: Simplifying the scientific validation of serious games

Published in: Education
  • MADE $30 ON MY FIRST DAY! Being a fresh graduate and having lots of free time, I stumbled upon your site when I was searching for work at home opportunities, good thing I did! Just on my first day of joining I already made $30! Now I'm averaging close to $80 a day just for filling out surveys! ●●●
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Simva: Simplifying the scientific validation of serious games icalt2019

  1. 1. Simva: Simplifying the scientific validation of serious games Baltasar Fernández-Manjón Ivan J. Pérez-Colado, Antonio Calvo-Morata, Cristina Alonso-Fernández, Manuel Freire, Iván Martínez-Ortiz @baltaFM IEEE ICALT 2019 Brazil, 15-18 July 2019
  2. 2. Validation of Serious Games Serious games effective tool applied in multiple fields. However, still need for: ● Evidences of the effectiveness of the games ● Validation of the serious game design Methods: ● Pre-Post questionnaires ● Survey ● Interaction data (Game and Learning Analytics) Very complex process: time, cost, human errors...
  3. 3. Pre-Post Questionnaires Issues: Collection Paper-based questionnaires: ● Waste of time in preparation, printing before each experiment and distributing them to participants ● Transcribe responses later for analysis (arduous task, error prone) Computer-based questionnaires: ● Internet access is mandatory for online questionnaires ● If questionnaires are offline, need to collect them (tedious task)
  4. 4. Pre-Post Questionnaires Issues: Storage Paper-based questionnaires: ● Huge accumulations of paper ● How, when and where destroy the data Computer-based questionnaires: ● Need backup not to lose data
  5. 5. Pre-Post Questionnaires Issues: Relate data How to relate all the data of the same user? Different information collected from users: ● Pre-post questionnaires ● Interaction data How to identify the users? ● Name, National Document Non-anonymous data ● Random User Identifier Anonymous data
  6. 6. Simva Simva is a tool for scientific validation of serious games. Goal: to simplify the previously-identified issues: ● Connection with surveys system and learning analytics system ● Management of participants and surveys ● Data storage: questionnaires responses and traces of interactions ● Control while experiments are in play: questionnaires finished, data sent
  7. 7. Simva: Surveys and Learning Analytics Simva is connected with: ● LimeSurvey ○ Create and manage surveys ○ Link users and surveys (access control) ○ Export and download responses ● e-UCM Analytics (Game Learning Analytics System) ○ Connection with game tracker ○ Collect and download user traces
  8. 8. Simva: Participants management Users with the same identifier in LimeSurvey and e-UCM Analytics Participants management: creation of classes & users Pseudonymization Anonymous 4-letter tokens automatically provided: same identifier for pre-test, post-test and game traces PDF with identifiers to distribute them in the experiment: allow anonymization while teachers can relate user and token
  9. 9. Simva: Data Storage Store questionnaires responses and interaction data Researcher or session managers can easily know the users (given by token) that completed all steps Download all data in the same format (CSV) to analyse it in batch
  10. 10. Simva: Control during the experiments The game can check Simva to see if user has completed some survey and the game ● surveys status: started, finished, not found Game will only allow users to continue if they have completed the previous step (e.g. only access game after completing pre-test)
  11. 11. Simva: Additional metadata Allow additional metadata to be stored in Simva to complement experiments For instance, to conduct A/B experiments ● In the same experiment, distribute two different questionnaires for each half class
  12. 12. Conclusions ● Validation of serious games is a complex, error-prone process ● Simva tool aims to simplify all the possible issues ○ Before the experiments: ■ Managing users & surveys ■ Providing anonymous identifiers to users ○ During the experiments: ■ Collecting and storing all data collected ■ Relating different data from users ■ Allowing additional metadata ○ After the experiments: ■ Simplifying downloading of all data collected
  13. 13. THANKS!! Baltasar Fernández-Manjón @baltaFM