Successfully reported this slideshow.
Your SlideShare is downloading. ×

2020_11_13 «Aplicando analíticas de aprendizaje en un juego serio de puzles geométricos» - José A. Ruipérez

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 35 Ad
Advertisement

More Related Content

Similar to 2020_11_13 «Aplicando analíticas de aprendizaje en un juego serio de puzles geométricos» - José A. Ruipérez (20)

More from eMadrid network (20)

Advertisement

Recently uploaded (20)

2020_11_13 «Aplicando analíticas de aprendizaje en un juego serio de puzles geométricos» - José A. Ruipérez

  1. 1. LOGO Applying learning analytics to a serious geometric puzzle game José A. Ruipérez Valiente – @JoseARuiperez
  2. 2. Agenda vIntroductions vLearning analytics and games for learning vOverview of Shadowspect vCase studies § Assessment of geometry competencies § Visualization dashboard § Evaluation of experimental study José A. Ruipérez Valiente— http://joseruiperez.me/Applying learning analytics to a serious geometric puzzle game 2
  3. 3. INTRODUCTIONS
  4. 4. Who am I? v Training: § BEng Telecommunication Systems (UCAM), MEng Telecomunications, MSc and PhD Telematics (UC3M), Postdoc (MIT) v Research in edtech: § Learning analytics in different contexts, learning environments and applications v Focus on MOOCs and GBA, but expanding to new areas J v Currently, Juan de la Cierva Researcher at UMU and research affiliate at MIT Applying learning analytics to a serious geometric puzzle game 4 José A. Ruipérez Valiente— http://joseruiperez.me/
  5. 5. Acknowledgements v Yoon Jeon Kim – Executive Director Playful Journey Lab (MIT) § https://playful.mit.edu/ v Manuel J. Gómez and Pedro A. Martínez – MSc students at the CyberDataLab (UMU) § https://cyberdatalab.um.es/ v Many others have collaborated in the project v Funding comes from several projects Applying learning analytics to a serious geometric puzzle game 5 José A. Ruipérez Valiente— http://joseruiperez.me/
  6. 6. LEARNING ANALYTICS AND GAMES FOR LEARNING 6
  7. 7. The implementation process of learning analytics Data collection Data tidying and feature engineering Visualization dashboard Recommendation System Educational reports Iterative process Which data needs to be collected and how to store it? Which is the objective application and the targeted user? Which features are necessary and how to generate them? Technology, learning sciences, privacy, institutions and educational policy Exploration, clustering, prediction, causality… Learning environments The application and objetive should have a direct effect on the learning environments to close the loop Adaptive contents What is the context and who are the users? What analysis and models need to be implemented? Applying learning analytics to a serious geometric puzzle game 7 José A. Ruipérez Valiente— http://joseruiperez.me/
  8. 8. Game definition 8 A game is a voluntary interactive activity, in which one or more players follow rules that constrain their behavior, enacting an artificial conflict that ends in a quantifiable outcome. ~Eric Zimmerman (2004) Applying learning analytics to a serious geometric puzzle game 8 José A. Ruipérez Valiente— http://joseruiperez.me/
  9. 9. Why Games? 9 v Games are “flexible enough for players to inhabit and explore through meaningful play” (Salen & Zimmerman) § Deep learning § Multiple pathways to solutions v Majority of children grow up playing games v Multiple strengths that favour their use for assessment purposes § Engaging and motivating § Stealth assessment Applying learning analytics to a serious geometric puzzle game José A. Ruipérez Valiente— http://joseruiperez.me/
  10. 10. Metaphor Applying learning analytics to a serious geometric puzzle game 10 José A. Ruipérez Valiente— http://joseruiperez.me/
  11. 11. OVERVIEW OF SHADOWSPECT
  12. 12. Design, Development and Evaluation Process Applying learning analytics to a serious geometric puzzle game 12 José A. Ruipérez Valiente— http://joseruiperez.me/
  13. 13. Shadowspect Applying learning analytics to a serious geometric puzzle game 13 ● Educational game that targets common core standards of geometry and spatial reasoning ● Play it at https://shadowspect.org/play/ ● Link to video José A. Ruipérez Valiente— http://joseruiperez.me/
  14. 14. The power of learning analytics Applying learning analytics to a serious geometric puzzle game 14 José A. Ruipérez Valiente— http://joseruiperez.me/
  15. 15. ASSESSMENT OF GEOMETRY COMPETENCIES Case Studies
  16. 16. Context Applying learning analytics to a serious geometric puzzle game 16 José A. Ruipérez Valiente— http://joseruiperez.me/ v Motivation: Using games a formative assessment v Objective: Design and develop tasks in games to measure common core geometry standards v Method: Evidence-centered design and knowledge inference
  17. 17. Evidence-centered Design framework Applying learning analytics to a serious geometric puzzle game 17 José A. Ruipérez Valiente— http://joseruiperez.me/
  18. 18. Common Core Geometry Standards Applying learning analytics to a serious geometric puzzle game 18 José A. Ruipérez Valiente— http://joseruiperez.me/ v Competency model: We focus on the common core geometry standards o MG.A.1: Use geometric shapes, their measures, and their properties to describe objects (e.g., modeling a tree trunk or a human torso as a cylinder) o GMD.B.4: Identify the shapes of two-dimensional cross-sections of three- dimensional objects, and identify three-dimensional objects generated by rotations of two-dimensional objects o CO.A.5: Given a geometric figure and a rotation, reflection, or translation, draw the transformed figure o CO.B.6: Use geometric descriptions of rigid motions to transform figures and to predict the effect of a given rigid motion on a given figure
  19. 19. Applying learning analytics to a serious geometric puzzle game 19 José A. Ruipérez Valiente— http://joseruiperez.me/ ECD Summary for Geometry Common Standards Assessment Puzzle MG.A.1 GMD.B.4 … Puzzle 1 Weak Weak … Puzzle 2 None None … … … … … Student Puzzle 1 Puzzle 2 … Student 1 OK, # 1 attempt OK, # 3 attempts … Student 1 NA Fail, # 5 attempt … … … … … Collaboration between geometry specialist, game designer and assessment designer § Evidence model: We generate puzzles that generate evidence from the Geometry Common Standards § Task model: We map the relationship (none, weak or strong) of each puzzle with the common standard § Assembly model: We put all the evidence from a student together to assess their content knowledge § Presentation & Delivery model: Reports and dashboards by student/standard. Difficulty by exercise
  20. 20. Applying learning analytics to a serious geometric puzzle game 20 José A. Ruipérez Valiente— http://joseruiperez.me/ ECD Summary for Geometry Common Standards Assessment v Constraints of our case scenario: § We can have more than one knowledge component per puzzle § We assume that knowledge evolves (students get better) v Based on this constraints, we implement Multivariate ELO § Originally used for game ranking (such as chess, league of legend, and so on) § The rationale is that the update in the score is proportional to the ‘surprise’ of the result v Steps of the algorithm § We assign initial scores § We compute the probably of solving a puzzle correctly based the competency § Update the difficulty of the puzzle § Update the competency for that student
  21. 21. VISUALIZATION DASHBOARD Case Studies
  22. 22. Applying learning analytics to a serious geometric puzzle game 22 José A. Ruipérez Valiente— http://joseruiperez.me/ Context v Motivation: Teachers do not have a good sense of what students are doing with the game v Objective: A teacher-centered dashboard with metrics to support them v Method: A live and interactive visualization dashboard
  23. 23. Applying learning analytics to a serious geometric puzzle game 23 José A. Ruipérez Valiente— http://joseruiperez.me/ Metric ideation process and definition v We followed the next steps for the ideation process 1. Define a multidisciplinary working group 2. Initial ideation process 3. Initial proposal of metrics 4. Prioritization of metrics 5. Initial metric definition v Final output of this process:
  24. 24. Applying learning analytics to a serious geometric puzzle game 24 José A. Ruipérez Valiente— http://joseruiperez.me/ Dashboard ideation process v Collaborative work between three teams: § Learning analytics: Technical implementation of metrics § Vizards team: Developing digital and paper viz prototypes § Co-design team: Working with a cohort of school teachers v Example of Sequences Within Puzzle metric
  25. 25. Applying learning analytics to a serious geometric puzzle game 25 José A. Ruipérez Valiente— http://joseruiperez.me/ Final overview of our dashboard vFinal product is a live dashboard with 7 metrics using Shiny R framework
  26. 26. EVALUATION OF EXPERIMENTAL STUDY Case Studies
  27. 27. Applying learning analytics to a serious geometric puzzle game 27 José A. Ruipérez Valiente— http://joseruiperez.me/ Context v Motivation: Guidelines regarding how to use games in primary and secondary education v Objective: What is the effect of having students playing alone or collaboratively in a class v Method: Experiment in a high school of Massachusetts and evaluation through learning analytics
  28. 28. Applying learning analytics to a serious geometric puzzle game 28 José A. Ruipérez Valiente— http://joseruiperez.me/ Experimental Design v Experimental methodology with a 2-level factor v We separate groups by condition (individual vs collaborative) § Presentation of Shadowspect (5 minutes) § Interaction with Shadowspect (50 minutes) • We recommend to solve first tutorial, but the game does not enforce it v Evaluation: We compare a number of metrics by condition v Levels completed, level pathways and within-game engagement metrics v Final sample of 45 individual students and 29 couples
  29. 29. Applying learning analytics to a serious geometric puzzle game 29 José A. Ruipérez Valiente— http://joseruiperez.me/ Pictures of the context
  30. 30. Applying learning analytics to a serious geometric puzzle game 30 José A. Ruipérez Valiente— http://joseruiperez.me/ Completed levels
  31. 31. Applying learning analytics to a serious geometric puzzle game 31 José A. Ruipérez Valiente— http://joseruiperez.me/ Level pathways Individual play Collaborative play
  32. 32. Applying learning analytics to a serious geometric puzzle game 32 José A. Ruipérez Valiente— http://joseruiperez.me/ Within-game engagement metrics
  33. 33. Applying learning analytics to a serious geometric puzzle game 33 José A. Ruipérez Valiente— http://joseruiperez.me/ Conclusions v Three examples that show potential to help the teacher understand and assess through games v Emerging challenges like teacher data and assessment literacy, and trustworthiness v Highly challenging and multidisciplinary field, problems: § Cost, scalability and generalization across GBA tools, model validity, trustworthiness, and teacher literacy v Some companies are already using GBA as part pre-hiring
  34. 34. Applying learning analytics to a serious geometric puzzle game 34 José A. Ruipérez Valiente— http://joseruiperez.me/ References § Ruipérez-Valiente, J. A. & Kim, Y. J. Effects of Solo vs. Collaborative Play in a Digital Learning Game on Geometry: Results from a K12 Experiment (2020). Computers & Education, 159 – Link § Martinez, P. A., Gomez, M. J., Ruipérez-Valiente, J. A., Martínez Pérez, G., & Kim, Y. J. (2020). Visualizing Educational Game Data: A Case Study of Visualizations to Support Teachers. In Learning Analytics Summer Institute Spain. CEUR Workshop Proceedings, Vol 2671, 97-111 – Link § Kim Y.J., & Ruipérez-Valiente, J. A. 2020. Data-driven Game Design: The Case of Difficulty in Educational Games. Proceedings of the 15th European Conference on Technology Enhanced Learning (pp. 449--454) – Link § Kim Y.J., Ruipérez-Valiente, J. A., Tan P., Rosenheck L. & Klopfer E. Towards a Process to Integrate Learning Analytics and Evidence-Centered Design for Game-based Assessment. In Companion Proceedings of the 9th International Learning Analytics and Knowledge Conference (LAK’19) – Link § Ruipérez-Valiente, J. A., Gaydos, M., Rosenheck L., Kim Y.J., & Klopfer E. (2020). Patterns of Engagement in an Educational Massive Multiplayer Online Game: A Multidimensional View. IEEE Transactions on Learning Technologies – Link § Gomez, M., Ruipérez-Valiente, J. A., Martinez, P. A., & Kim, Y. J. (2020). Exploring the Affordances of Sequence Mining in Educational Games. In Proceedings of the Eight International Conference on Technological Ecosystem for Enhancing Multiculturality
  35. 35. LOGO José A. Ruipérez Valiente – @JoseARuiperez Applying learning analytics to a serious geometric puzzle game José A. Ruipérez Valiente— http://joseruiperez.me/

×