Introduction to learning analytics and approaches to learner engagement to raise awareness and set the seen for upcoming projects and advice for supported learning providers.

1. Learning Analytics
What? How? Why?
James Ballard
JamesBallard2
@jameslballard
jameslballard

2. Overview
What are Learning Analytics?
Learner Engagement - a metric for learning
Preparing institutions – tools and skills
Infinite Rooms

3. Activity 1 - Introduction
Open Discussion
What are learning analytics?
Who are they for?

4. Learning Analytics
What are they?

5. Not quite big data
In 2012 we created 2,500,000,000,000,000,000 (2.5 quintillion) bytes of
data every day
Annual Moodle log data
5Gb

6. Learning Analytics
Type of Analytics
Level or Object of Analysis
Who benefits
Learning Analytics
Course-level: social networks,
conceptual development, discourse
analysis, intelligent curriculum
Learner, faculty
Departmental: predictive
modelling, patterns of
success/failure
Learners, faculty
Institutional: learner profiles,
performance of academics,
knowledge flow
Administrators, funders, marketing
Regional (state/provincial):
comparisons between systems
Funders, administrators
National and International
National governments, education
authorities
Academic Analytics
Siemens and Long (2011)

7. Common focus
Retention
Performance
• Identifying learners at-risk of drop-out from the course
• Identifying momentum/crisis points
• Predicting final exam success
• Predicting future performance (e.g. school -> university)
Activity
• Quantitative views of activity
• What are learners doing
Course
• Usually linked to a bench-marking of staff performance
• Learning design patterns
Engagement
• What types of things are learners doing
• Learner engagement as a metric/proxy

8. Activity 2 - Examples
Small groups
List some examples of what learning
providers are measuring or might want to
measure.
Retention
Performance
Activity
Course
Engagement

9. Learner Engagement
A metric for learning

10. Engagement
Common activities such as checking announcements, viewing grades and uploading
assignments represent little time investment from the user and may not be useful indicators of
engagement.

11. Engagement
Engagement has emerged as an
alternative view of the learner experience
that can enrich the often reductionist
language of performance, skills and
competence.
HEA, Trowler and Trowler (2010)

12. Engagement Process
Engagement is the new metric that supersedes previous linear metaphors, through
a developmental process of discovery, evaluation, use, and affinity.
Haven (2007)

13. Activity 2 - Examples
Small groups
Tag previous examples within the
engagement process.
Involvement
• The presence of a
learner within the
institution including
data such as
physical or virtual
visits
Interaction
• Provides a depth of
understanding:
where involvement
measures
touches, interaction
measures actions.
Intimacy
• Helps understand
sentiment and
affection; the most
common way to
collect this type of
data is through
interviews or
surveys.
Influence
• Determines the
likelihood of the
individual
recommending
learning to others
and contributing to
local culture(s).

14. Involvement
The presence of a learner
within the institution including
data such as physical or virtual
visits.
 Overall Activity
 Locations
 Time of day

15. Involvement
The presence of a learner
within the institution including
data such as physical or virtual
visits.
 Overall Activity
 Locations
 Time of day

16. Interaction
Provides a depth of
understanding: where
involvement measures
touches, interaction measures
actions.
 Activity types
 Action analysis
 Connectivity maps
Conole (2007)

17. Interaction
Provides a depth of
understanding: where
involvement measures
touches, interaction measures
actions.
 Activity types
 Action analysis
 Connectivity maps

18. Intimacy
Helps understand sentiment
and or affection; the most
common way to collect this
type of data is through
interviews or surveys.
 Learning Power
 Self-theory
 Motivated Strategies for
Learning
Questionnaire, MSLQ
 Self-determination theory
Deakin Crick, Broadfoot, and Claxton (2004)

19. MSLQ
6
5
Intimacy
4
Helps understand sentiment
and or affection; the most
common way to collect this
type of data is through
interviews or surveys.
3
2
 Learning Power
 Self-theory
1
 Motivated Strategies for
Learning
Questionnaire, MSLQ
0
Rehearsal
Elaboration
Organisation
Pre
Self-Regulation
Critical Thinking
Post
Pintrich (1990)
 Self-determination theory

20. Influence
Determines the likelihood of the
individual recommending
learning to others and
contributing to local culture(s).
 Social Network Analysis
 Distributed Cognition
 Collective Intelligence
 Pathway of Participation
Dawson (2010)

21. Influence
Determines the likelihood of the
individual recommending
learning to others and
contributing to local culture(s).
 Social Network Analysis
 Distributed Cognition
 Collective Intelligence
 Pathway of Participation
School Leader Network
Harré (1983)

22. Preparing Institutions
Empowering environments for learning

23. Metrics are based on the data that is
easiest to extract/access, and what you
don‟t measure is lost.
Anything you measure will impel a person
to optimize his score on that metric.
Don‟t be surprised if people find ingenious
and destructive ways in how they get
there.
For example, standardised assessment
produce kids who perform well on these
tests but can falter when asked to
demonstrate their knowledge of the same
material in a different way
You are what
you measure
„Incremental change is not
enough. You have to drive
large-scale change by
changing the environment in
which people work‟
– Kevin Bonnett, Deputy Vice
Chancellor Student Experience
JISC Report
MMU Review

24. Activity 1 - Introduction
Open Discussion
What types of skills are required by elearning teams?
Do they already exist?

25. Analytics Process
Collection
Storage
Cleaning
Integration
Analysis
Presentation
CETIS Analytics Series (2012)

26. MySQL / PostgreSQL
Apache Hadoop
HP Vertica
Pentaho
Rapid Miner
Gephi
Google Visualisation
d3.js
InfoVis Toolkit
Open Source
Tools
Investing in staff
experimentation with low cost
components from a range of
traditions may be a more
prudent initial move, even if the
most effective tool
subsequently turns out to be a
ready-made suite.

27. Algorithm
Usage
Purpose
Step regression
Used for binary
classification (0,1)
• Select a parameter
• Assign a weight
• Calculate value
Predicts simple binary
results such as is a
student at-risk?
Logistic regression
Same as above but
more conservative
J48/C4.5 Decision
trees (Quinlan, 1993)
Tries to find optimal
split in variables
Good when data splits
into groups
JRip Decision rules
Find the “best” path
Good when multi-level
and make this a rule
interaction are
until no sensible paths common
are left and set these
to otherwise.
K* Instance based
classifiers
Predicts data based
on neighbouring
points.
Good when data is
very divergent
Data Mining
Classification is used when one
wants to predict something
(label) which is categorical and
not a number.

28. Infinite Rooms
Learner enhanced technology

29. Web dashboards based on engagement
process accessing a data warehouse
model developed from Activity Theory.
Utilises new and existing analytics and
supports multiple learning design
approaches.
1. How can student activity help identify
and promote effective teaching
practices?
2. Understand the role that analytics can
play in learning design, feedback and
assessment.
Research
Project
If patterns of nonparticipation
(disengagement) are to be
disrupted an improved
conceptual framework may be
necessary.

30. Activity
Analysis
Engeström‟s (1987, 1999)
approach allows us to
overcome oppositions between
activity and communication and
highlight subject-community
relations.
Modelling pedagogy with
Activity Theory
Stevenson (2008)
http://goo.gl/vOuiqp

31. Exposing
Activity
The intention of this is to reveal
the nature of the
system, allowing designers
(e.g. teachers) to evaluate the
system in the wider context of
their teaching and learning
practice.

32. Dimension
Fact
Action
Post to forum
Tool
Forum
Instance
Discussion topic
User
Oliver Twist
Role
Student
Course
Introduction to English
Date
02/10/2013
Time
9:45
System
Moodle
Data Model
Enables multi-dimensional
tagging to explore data from
different perspectives.

33. Actions
Things a learner does
• Submissions
• Quiz attempts
• Forum posts
Interventions
Feedback to the learner
• Targets
• Grades
• Assignment feedback
Achievements Recognising learning
• Course completions
• Badges
• Certificates
Surveys
How learning is perceived
• Attitudes to learning/technology
• Satisfaction survey
Data Capture
What types of things can we
capture.

34. Coding
One can then begin to
distinguish the possible actions
that are generated through the
use of tools from the operations
needed to access them and
code these via learning design
theories.

35. Activity Types
Apply learning design models
to learner data.
Conole (2007)

36. Visualisation
Explore different visualisations
of the same data set for
different insights.