Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Pablo Moreno Ger
pablom@fdi.ucm.es
Pori
July 22nd, 2014
GALA Summer School
Pablo Moreno Ger (et al.)
Learning Analytics in...
Hello!
 Let’s start with a story…
 An online course on how to use Excel
 With online delivery of exercises
 Delivering...
The story develops…
 A few days before the exam, I receive an email:
I have been following the course, reading the materi...
A quick look at the Moodle log
2013/02/27-18:35:43 - NameOmitted - Course View - Main
*** 28 hits ommited ***
2013/03/07-1...
A slow look at the Moodle log
Course opens: February 21st
2013/02/27-18:35:43 - NameOmitted - Course View – Main
***
28 hi...
A slow look at the Moodle log
March 10th – The exercise is corrected and an email is sent
2013/03/11-08:55:43 - NameOmitte...
A slow look at the Moodle log
May 21st (continued)
*** Student checks the two pending exercises (past their deadline) ***
...
In other words
Data can tell stories…
The story begins…
Analytics, Analytics, Analytics!
Web
Analytics
Web Analytics
What we can do with Web Analytics
 Web Analytics for Dummies
 How are my keywords working?
 Which are my landing pages?...
What we can do with Web Analytics
 Web Analytics for Pros
 Which ads are generating more traffic?
 Which ads are genera...
Facebook
That’s ok but…
OK, enough corporate,
let’s talk about learning!
Analytics, Analytics, Analytics!
Web
Analytics
Learning
Analytics
And now it’s 2014
 LA was an obscure term in 2007
 Today:
 Learning Analytics is featured in H2020-ICT-20 as a key
lear...
From business to (TEL)-research
What happened?
A perfect storm
No, really, what happened?
 2006-2010 steady increase of “Learning Analytics”
 In 2009, “Big Data” explodes
 In 2010, G...
What we can do with Learning Analytics
 Learning Analytics for dummies
 Most accessed contents
 Least accessed contents...
What we can do with Learning Analytics
 Learning Analytics for pros
 Changes in usual patterns (potential issues)
 Stud...
Dimensions of Learning Analytics
 The what we measure dimension
 Activity on a virtual campus (e- or b-learning)
 Usage...
Dimensions of Learning Analytics
 The why we measure dimension
 Assessment of learning effectiveness
 Assessment of the...
Dimensions of Learning Analytics
 The where we measure dimension (or scope)
 Individual analytics
 Classroom analytics
...
LA is here to stay
But there’s more!
And now for something completely different…
Analytics, Analytics, Analytics!
Web
Analytics
Game
Analytics
Learning
Analytics
Game Analytics
Game Analytics
 Game Analytics for Dummies
 Time spent on each level
 Barriers and game issues
 Dead scenes
 Usabilit...
Candy Crush Saga
Enough about money…
Can we get back to
learning?
And here, we, go!
Analytics, Analytics, Analytics!
Web
Analytics
Game
Analytics
Game Analytics for
LearningLearning
Analytics
Stop and think
 HTML files vs. Game
 Quiz vs. In-game performance
 Forum vs. MMORPG
Stop and think
Single player in a single
gameplay
Stop and think
An entire school playing
the same game
Stop and think
A 1-million student
MOOC with a game
Stop and think
An edX-like platform
filled with games
Stop and think
All schools in Europe
playing games
So much power in our hands!
But we have no clue on how to use it
What we (GALA) do
ANGEL SERRANOLAGUNA
LEARNING
ANALYTICS
AND
SERIOUS
GAMES
GLEANER
"The gleaners", Jean-François Millet
GLEANER
Stuff we can do
 Generic traces
 Mouse clicks (left/right/middle)
 Mouse movement (free movement, drag)
 Key presses (...
Stuff we can do
 Engine-specific traces (for eAdventure game engine)
 Actions performed
 Speech bubbles
 Answers in mu...
Case Study: The Big Party
GLEANER traces
100,000 lines for a 40-minute playthrough
Heatmaps
Case study 2: The Foolish Lady
Analytics-based assessment
Case Study 3: Lost in XML Space
Case Study 3: Lost in XML Space
Realtime Dashboard
Real Time dashboard
REALTIME DASHBOARD
But this is not about what WE do…
What you can do
 Figure out how to use generic traces in your games
 Figure out new traces specific to your games
 Try ...
Upcoming SlideShare
Loading in …5
×

Learning Analytics in Serious Games

715 views

Published on

Lecture by Pablo Moreno Ger on the potential and implications at the crossroads of Learning Analytics and Serious Games

Published in: Education
  • Be the first to comment

Learning Analytics in Serious Games

  1. 1. Pablo Moreno Ger pablom@fdi.ucm.es Pori July 22nd, 2014 GALA Summer School Pablo Moreno Ger (et al.) Learning Analytics in Serious Games
  2. 2. Hello!  Let’s start with a story…  An online course on how to use Excel  With online delivery of exercises  Delivering the exercises on time is a requisite for attending the final face-to-face exam
  3. 3. The story develops…  A few days before the exam, I receive an email: I have been following the course, reading the materials carefully and delivering my exercises. The last two exercises have been really difficult, and I have been trying different solutions and seeking help from some friends. In the end I managed to complete them and tried to upload them last night. But the website kept showing me errors and I have been unable to upload them. Can I send them via email and still be eligible for the exam?
  4. 4. A quick look at the Moodle log 2013/02/27-18:35:43 - NameOmitted - Course View - Main *** 28 hits ommited *** 2013/03/07-18:35:43 - NameOmitted - Course View - Main 2013/03/07-18:37:45 - NameOmitted - Resource View – Exercise 1 2013/03/07-19:01:19 - NameOmitted - Assignment View - Exercise 1 2013/03/07-19:01:38 - NameOmitted - Assignment View - Exercise 1 2013/03/07-19:01:43 - NameOmitted - Course View - Main 2013/03/11-08:55:43 - NameOmitted - Course View – Main 2013/03/11-08:55:51 - NameOmitted - email_list view_all_mail 2013/03/11-08:55:55 - NameOmitted - email_list view_110652 2013/03/11-09:01:19 - NameOmitted - Assignment View - Exercise 1 2013/05/21-22:27:06 - NameOmitted - Course View - Main 2013/05/21-22:27:11 - NameOmitted - email_list view_all_mail 2013/05/21-22:27:13 - NameOmitted - email_list view_123657 2013/05/21-22:35:43 - NameOmitted - Course View - Main 2013/05/21-22:37:45 - NameOmitted - Resource View - Exercise 2 2013/05/21-22:38:01 - NameOmitted - Course View - Main 2013/05/21-22:38:05 - NameOmitted - Resource View - Exercise 3 2013/05/21-22:41:01 - NameOmitted - Course View – Main 2013/05/21-22:41:05 - NameOmitted - Resource View – Chapter 1 2013/05/21-22:41:06 - NameOmitted - Resource View – Chapter 2 2013/05/21-22:41:06 - NameOmitted - Resource View – Chapter 3 2013/05/21-22:44:41 - NameOmitted - Resource View - Exercise 2 2013/05/21-22:45:58 - NameOmitted - email_list view_all_mail 2013/05/21-22:46:01 - NameOmitted - email_list compose 2013/05/21-22:59:36 - NameOmitted - email_list view_all_mail
  5. 5. A slow look at the Moodle log Course opens: February 21st 2013/02/27-18:35:43 - NameOmitted - Course View – Main *** 28 hits in 30 minutes Chapter 1, Chapter 2, forums, exercise delivery tools, etc. *** March 7th 2013/03/07-18:35:43 - NameOmitted - Course View - Main 2013/03/07-18:37:45 - NameOmitted - Resource View – Exercise 1 2013/03/07-19:01:19 - NameOmitted - Assignment View - Exercise 1 2013/03/07-19:01:38 - NameOmitted - Assignment View - Exercise 1 2013/03/07-19:01:43 - NameOmitted - Course View – Main This is a typical pattern: - Study the exercise (he does not open the course materials) - Complete it (24 minutes) - Submission (two steps, Moodle does not differentiate)
  6. 6. A slow look at the Moodle log March 10th – The exercise is corrected and an email is sent 2013/03/11-08:55:43 - NameOmitted - Course View - Main 2013/03/11-08:55:51 - NameOmitted - email_list view_all_mail 2013/03/11-08:55:55 - NameOmitted - email_list view_110652 2013/03/11-09:01:19 - NameOmitted - Assignment View – Exercise 1 May 19th – Instructor sends mass email notifying exam dates May 21st – First student log in in two months 2013/05/21-22:27:06 - NameOmitted - Course View – Main 2013/05/21-22:27:11 - NameOmitted - email_list view_all_mail 2013/05/21-22:27:13 - NameOmitted - email_list view_123657 Another typical pattern: - Log in - Read mail - Check grade & comments (from link in email)
  7. 7. A slow look at the Moodle log May 21st (continued) *** Student checks the two pending exercises (past their deadline) *** 2013/05/21-22:35:43 - NameOmitted - Course View - Main 2013/05/21-22:37:45 - NameOmitted - Resource View - Exercise 2 2013/05/21-22:38:01 - NameOmitted - Course View - Main 2013/05/21-22:38:05 - NameOmitted - Resource View - Exercise 3 *** Quick look at the materiales (using tabbed browsing) *** 2013/05/21-22:41:01 - NameOmitted - Course View – Main 2013/05/21-22:41:05 - NameOmitted - Resource View – Chapter 1 2013/05/21-22:41:06 - NameOmitted - Resource View – Chapter 2 2013/05/21-22:41:06 - NameOmitted - Resource View – Chapter 3 2013/05/21-22:44:41 - NameOmitted - Resource View - Exercise 2 *** Student decides to write. Writing time: 13 minutes *** 2013/05/21-22:45:58 - NameOmitted - email_list view_all_mail 2013/05/21-22:46:01 - NameOmitted - email_list compose 2013/05/21-22:59:36 - NameOmitted - email_list view_all_mail
  8. 8. In other words Data can tell stories…
  9. 9. The story begins…
  10. 10. Analytics, Analytics, Analytics! Web Analytics
  11. 11. Web Analytics
  12. 12. What we can do with Web Analytics  Web Analytics for Dummies  How are my keywords working?  Which are my landing pages?  Where are my customers coming from?  Which days/hours have more traffic?
  13. 13. What we can do with Web Analytics  Web Analytics for Pros  Which ads are generating more traffic?  Which ads are generating more revenue?  From which pages are my users departing?  And the really advanced stuff:  Cycles  Dead ends  Losses of revenue  Dead pages
  14. 14. Facebook
  15. 15. That’s ok but… OK, enough corporate, let’s talk about learning!
  16. 16. Analytics, Analytics, Analytics! Web Analytics Learning Analytics
  17. 17. And now it’s 2014  LA was an obscure term in 2007  Today:  Learning Analytics is featured in H2020-ICT-20 as a key learning technology  Most TEL conferences include a track on LA  And some summer schools, a lecture…  Special issues on LA in major TEL journals
  18. 18. From business to (TEL)-research What happened?
  19. 19. A perfect storm
  20. 20. No, really, what happened?  2006-2010 steady increase of “Learning Analytics”  In 2009, “Big Data” explodes  In 2010, GALA starts  In 2010, MOOCs happen  Learning Analytics become a “Big Data” problem.
  21. 21. What we can do with Learning Analytics  Learning Analytics for dummies  Most accessed contents  Least accessed contents  Time spent in each resource  Average grades in quizzes  Easy/hard questions on quizzes  Students that drop out  Trends and timelines
  22. 22. What we can do with Learning Analytics  Learning Analytics for pros  Changes in usual patterns (potential issues)  Study of the impact of changes (see Facebook)  Local / Regional / National data aggregation  Big data problems (if your population is large enough  And the really advanced stuff:  Predict student dropout  Predict grades  Automatic adaptation
  23. 23. Dimensions of Learning Analytics  The what we measure dimension  Activity on a virtual campus (e- or b-learning)  Usage patterns by a spefic students  Usage patterns by groups of students  Detailed assessment (per question, per answer)  Forum participation  Time spent on each resource  Access frequency
  24. 24. Dimensions of Learning Analytics  The why we measure dimension  Assessment of learning effectiveness  Assessment of the learning process  Assessment for learning  Assessment of the e-learning platform  And of course…  Predictive assessment  Usability  Validation
  25. 25. Dimensions of Learning Analytics  The where we measure dimension (or scope)  Individual analytics  Classroom analytics  School / Institution analytics  Regional analytics  National / International analytics
  26. 26. LA is here to stay But there’s more!
  27. 27. And now for something completely different…
  28. 28. Analytics, Analytics, Analytics! Web Analytics Game Analytics Learning Analytics
  29. 29. Game Analytics
  30. 30. Game Analytics  Game Analytics for Dummies  Time spent on each level  Barriers and game issues  Dead scenes  Usability Assessment  Game Analytics for Pros  Monetization
  31. 31. Candy Crush Saga
  32. 32. Enough about money… Can we get back to learning?
  33. 33. And here, we, go!
  34. 34. Analytics, Analytics, Analytics! Web Analytics Game Analytics Game Analytics for LearningLearning Analytics
  35. 35. Stop and think  HTML files vs. Game  Quiz vs. In-game performance  Forum vs. MMORPG
  36. 36. Stop and think Single player in a single gameplay
  37. 37. Stop and think An entire school playing the same game
  38. 38. Stop and think A 1-million student MOOC with a game
  39. 39. Stop and think An edX-like platform filled with games
  40. 40. Stop and think All schools in Europe playing games
  41. 41. So much power in our hands!
  42. 42. But we have no clue on how to use it
  43. 43. What we (GALA) do ANGEL SERRANOLAGUNA LEARNING ANALYTICS AND SERIOUS GAMES GLEANER "The gleaners", Jean-François Millet
  44. 44. GLEANER
  45. 45. Stuff we can do  Generic traces  Mouse clicks (left/right/middle)  Mouse movement (free movement, drag)  Key presses (up / down / press)
  46. 46. Stuff we can do  Engine-specific traces (for eAdventure game engine)  Actions performed  Speech bubbles  Answers in multiple choice questions  Scene transitions  Changes in variable values
  47. 47. Case Study: The Big Party
  48. 48. GLEANER traces 100,000 lines for a 40-minute playthrough
  49. 49. Heatmaps
  50. 50. Case study 2: The Foolish Lady
  51. 51. Analytics-based assessment
  52. 52. Case Study 3: Lost in XML Space
  53. 53. Case Study 3: Lost in XML Space
  54. 54. Realtime Dashboard
  55. 55. Real Time dashboard REALTIME DASHBOARD
  56. 56. But this is not about what WE do…
  57. 57. What you can do  Figure out how to use generic traces in your games  Figure out new traces specific to your games  Try to standardize and share  GLEANER or your own approach  There is standard stuff to exchange data (e.g. xAPI)

×