Learning Analytics in Serious Games

Pablo Moreno Ger
pablom@fdi.ucm.es
Pori
July 22nd, 2014
GALA Summer School
Pablo Moreno Ger (et al.)
Learning Analytics in Serious
Games

Hello!
 Let’s start with a story…
 An online course on how to use Excel
 With online delivery of exercises
 Delivering the exercises on time is a requisite for
attending the final face-to-face exam

The story develops…
 A few days before the exam, I receive an email:
I have been following the course, reading the materials
carefully and delivering my exercises. The last two
exercises have been really difficult, and I have been
trying different solutions and seeking help from some
friends.
In the end I managed to complete them and tried to
upload them last night. But the website kept showing me
errors and I have been unable to upload them. Can I send
them via email and still be eligible for the exam?

A quick look at the Moodle log
2013/02/27-18:35:43 - NameOmitted - Course View - Main
*** 28 hits ommited ***
2013/03/07-18:37:45 - NameOmitted - Resource View – Exercise 1
2013/03/07-19:01:19 - NameOmitted - Assignment View - Exercise 1
2013/03/11-08:55:43 - NameOmitted - Course View – Main
2013/03/11-08:55:51 - NameOmitted - email_list view_all_mail
2013/03/11-08:55:55 - NameOmitted - email_list view_110652
2013/05/21-22:37:45 - NameOmitted - Resource View - Exercise 2
2013/05/21-22:41:05 - NameOmitted - Resource View – Chapter 1
2013/05/21-22:46:01 - NameOmitted - email_list compose

A slow look at the Moodle log
Course opens: February 21st
***
28 hits in 30 minutes
Chapter 1, Chapter 2, forums, exercise delivery tools, etc.
***
March 7th
2013/03/07-18:37:45 - NameOmitted - Resource View – Exercise 1
This is a typical pattern:
- Study the exercise (he does not open the course materials)
- Complete it (24 minutes)
- Submission (two steps, Moodle does not differentiate)

March 10th – The exercise is corrected and an email is sent
2013/03/11-09:01:19 - NameOmitted - Assignment View – Exercise 1
May 19th – Instructor sends mass email notifying exam dates
May 21st – First student log in in two months
Another typical pattern:
- Log in
- Read mail
- Check grade & comments (from link in email)

May 21st (continued)
*** Student checks the two pending exercises (past their deadline) ***
*** Quick look at the materiales (using tabbed browsing) ***
*** Student decides to write. Writing time: 13 minutes ***
2013/05/21-22:46:01 - NameOmitted - email_list compose

In other words
Data can tell stories…

Analytics, Analytics, Analytics!
Web
Analytics

What we can do with Web Analytics
 Web Analytics for Dummies
 How are my keywords working?
 Which are my landing pages?
 Where are my customers coming from?
 Which days/hours have more traffic?

What we can do with Web Analytics
 Web Analytics for Pros
 Which ads are generating more traffic?
 Which ads are generating more revenue?
 From which pages are my users departing?
 And the really advanced stuff:
 Cycles
 Dead ends
 Losses of revenue
 Dead pages

That’s ok but…
OK, enough corporate,
let’s talk about learning!

Web
Analytics
Learning
Analytics

And now it’s 2014
 LA was an obscure term in 2007
 Today:
 Learning Analytics is featured in H2020-ICT-20 as a key
learning technology
 Most TEL conferences include a track on LA
 And some summer schools, a lecture…
 Special issues on LA in major TEL journals

From business to (TEL)-research
What happened?

No, really, what happened?
 2006-2010 steady increase of “Learning Analytics”
 In 2009, “Big Data” explodes
 In 2010, GALA starts
 In 2010, MOOCs happen
 Learning Analytics become a “Big Data” problem.

What we can do with Learning Analytics
 Learning Analytics for dummies
 Most accessed contents
 Least accessed contents
 Time spent in each resource
 Average grades in quizzes
 Easy/hard questions on quizzes
 Students that drop out
 Trends and timelines

What we can do with Learning Analytics
 Learning Analytics for pros
 Changes in usual patterns (potential issues)
 Study of the impact of changes (see Facebook)
 Local / Regional / National data aggregation
 Big data problems (if your population is large enough
 And the really advanced stuff:
 Predict student dropout
 Predict grades
 Automatic adaptation

Dimensions of Learning Analytics
 The what we measure dimension
 Activity on a virtual campus (e- or b-learning)
 Usage patterns by a spefic students
 Usage patterns by groups of students
 Detailed assessment (per question, per answer)
 Forum participation
 Time spent on each resource
 Access frequency

 The why we measure dimension
 Assessment of learning effectiveness
 Assessment of the learning process
 Assessment for learning
 Assessment of the e-learning platform
 And of course…
 Predictive assessment
 Usability
 Validation

 The where we measure dimension (or scope)
 Individual analytics
 Classroom analytics
 School / Institution analytics
 Regional analytics
 National / International analytics

LA is here to stay
But there’s more!

And now for something completely different…

Web
Analytics
Game
Analytics
Learning
Analytics

Game Analytics
 Game Analytics for Dummies
 Time spent on each level
 Barriers and game issues
 Dead scenes
 Usability Assessment
 Game Analytics for Pros
 Monetization

Enough about money…
Can we get back to
learning?

Web
Analytics
Game
Analytics
Game Analytics for
LearningLearning
Analytics

Stop and think
 HTML files vs. Game
 Quiz vs. In-game performance
 Forum vs. MMORPG

Stop and think
Single player in a single
gameplay

Stop and think
An entire school playing
the same game

Stop and think
A 1-million student
MOOC with a game

Stop and think
An edX-like platform
filled with games

Stop and think
All schools in Europe
playing games

But we have no clue on how to use it

What we (GALA) do
ANGEL SERRANOLAGUNA
LEARNING
ANALYTICS
AND
SERIOUS
GAMES
GLEANER
"The gleaners", Jean-François Millet

Stuff we can do
 Generic traces
 Mouse clicks (left/right/middle)
 Mouse movement (free movement, drag)
 Key presses (up / down / press)

Stuff we can do
 Engine-specific traces (for eAdventure game engine)
 Actions performed
 Speech bubbles
 Answers in multiple choice questions
 Scene transitions
 Changes in variable values

GLEANER traces
100,000 lines for a 40-minute playthrough

Case study 2: The Foolish Lady

Case Study 3: Lost in XML Space

Real Time dashboard
REALTIME DASHBOARD

But this is not about what WE do…

What you can do
 Figure out how to use generic traces in your games
 Figure out new traces specific to your games
 Try to standardize and share
 GLEANER or your own approach
 There is standard stuff to exchange data (e.g. xAPI)

Learning Analytics in Serious Games

Recommended

Recommended

More Related Content

Similar to Learning Analytics in Serious Games

Similar to Learning Analytics in Serious Games (20)

Recently uploaded

Recently uploaded (20)

Learning Analytics in Serious Games