Learning Analytics
What? How? Why?
James Ballard
JamesBallard2
@jameslballard
jameslballard
Overview
What are Learning Analytics?

Learner Engagement - a metric for learning
Preparing institutions – tools and skill...
Activity 1 - Introduction
Open Discussion
What are learning analytics?
Who are they for?
Learning Analytics
What are they?
Not quite big data
In 2012 we created 2,500,000,000,000,000,000 (2.5 quintillion) bytes of
data every day

Annual Moodle l...
Learning Analytics
Type of Analytics

Level or Object of Analysis

Who benefits

Learning Analytics

Course-level: social ...
Common focus
Retention
Performance

• Identifying learners at-risk of drop-out from the course
• Identifying momentum/cris...
Activity 2 - Examples
Small groups
List some examples of what learning
providers are measuring or might want to
measure.
...
Learner Engagement
A metric for learning
Engagement

Common activities such as checking announcements, viewing grades and uploading
assignments represent little ti...
Engagement
Engagement has emerged as an
alternative view of the learner experience
that can enrich the often reductionist
...
Engagement Process

Engagement is the new metric that supersedes previous linear metaphors, through
a developmental proces...
Activity 2 - Examples
Small groups
Tag previous examples within the
engagement process.
Involvement
• The presence of a
l...
Involvement
The presence of a learner
within the institution including
data such as physical or virtual
visits.
 Overall ...
Involvement
The presence of a learner
within the institution including
data such as physical or virtual
visits.
 Overall ...
Interaction
Provides a depth of
understanding: where
involvement measures
touches, interaction measures
actions.

 Activi...
Interaction
Provides a depth of
understanding: where
involvement measures
touches, interaction measures
actions.

 Activi...
Intimacy
Helps understand sentiment
and or affection; the most
common way to collect this
type of data is through
intervie...
MSLQ
6

5

Intimacy

4

Helps understand sentiment
and or affection; the most
common way to collect this
type of data is t...
Influence
Determines the likelihood of the
individual recommending
learning to others and
contributing to local culture(s)...
Influence
Determines the likelihood of the
individual recommending
learning to others and
contributing to local culture(s)...
Preparing Institutions
Empowering environments for learning
Metrics are based on the data that is
easiest to extract/access, and what you
don‟t measure is lost.
Anything you measure ...
Activity 1 - Introduction
Open Discussion
What types of skills are required by elearning teams?
Do they already exist?
Analytics Process
Collection

Storage

Cleaning

Integration

Analysis

Presentation

CETIS Analytics Series (2012)
MySQL / PostgreSQL
Apache Hadoop
HP Vertica
Pentaho
Rapid Miner
Gephi
Google Visualisation
d3.js
InfoVis Toolkit

Open Sou...
Algorithm

Usage

Purpose

Step regression

Used for binary
classification (0,1)
• Select a parameter
• Assign a weight
• ...
Infinite Rooms
Learner enhanced technology
Web dashboards based on engagement
process accessing a data warehouse
model developed from Activity Theory.
Utilises new a...
Activity
Analysis
Engeström‟s (1987, 1999)
approach allows us to
overcome oppositions between
activity and communication a...
Exposing
Activity
The intention of this is to reveal
the nature of the
system, allowing designers
(e.g. teachers) to evalu...
Dimension

Fact

Action

Post to forum

Tool

Forum

Instance

Discussion topic

User

Oliver Twist

Role

Student

Course...
Actions

Things a learner does

• Submissions
• Quiz attempts
• Forum posts

Interventions

Feedback to the learner

• Tar...
Coding
One can then begin to
distinguish the possible actions
that are generated through the
use of tools from the operati...
Activity Types
Apply learning design models
to learner data.

Conole (2007)
Visualisation
Explore different visualisations
of the same data set for
different insights.
Upcoming SlideShare
Loading in …5
×

JISC RSC London Workshop - Learner analytics

498 views

Published on

Introduction to learning analytics and approaches to learner engagement to raise awareness and set the seen for upcoming projects and advice for supported learning providers.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
498
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Refine topic summaries
  • JISC RSC London Workshop - Learner analytics

    1. 1. Learning Analytics What? How? Why? James Ballard JamesBallard2 @jameslballard jameslballard
    2. 2. Overview What are Learning Analytics? Learner Engagement - a metric for learning Preparing institutions – tools and skills Infinite Rooms
    3. 3. Activity 1 - Introduction Open Discussion What are learning analytics? Who are they for?
    4. 4. Learning Analytics What are they?
    5. 5. Not quite big data In 2012 we created 2,500,000,000,000,000,000 (2.5 quintillion) bytes of data every day Annual Moodle log data 5Gb
    6. 6. Learning Analytics Type of Analytics Level or Object of Analysis Who benefits Learning Analytics Course-level: social networks, conceptual development, discourse analysis, intelligent curriculum Learner, faculty Departmental: predictive modelling, patterns of success/failure Learners, faculty Institutional: learner profiles, performance of academics, knowledge flow Administrators, funders, marketing Regional (state/provincial): comparisons between systems Funders, administrators National and International National governments, education authorities Academic Analytics Siemens and Long (2011)
    7. 7. Common focus Retention Performance • Identifying learners at-risk of drop-out from the course • Identifying momentum/crisis points • Predicting final exam success • Predicting future performance (e.g. school -> university) Activity • Quantitative views of activity • What are learners doing Course • Usually linked to a bench-marking of staff performance • Learning design patterns Engagement • What types of things are learners doing • Learner engagement as a metric/proxy
    8. 8. Activity 2 - Examples Small groups List some examples of what learning providers are measuring or might want to measure. Retention Performance Activity Course Engagement
    9. 9. Learner Engagement A metric for learning
    10. 10. Engagement Common activities such as checking announcements, viewing grades and uploading assignments represent little time investment from the user and may not be useful indicators of engagement.
    11. 11. Engagement Engagement has emerged as an alternative view of the learner experience that can enrich the often reductionist language of performance, skills and competence. HEA, Trowler and Trowler (2010)
    12. 12. Engagement Process Engagement is the new metric that supersedes previous linear metaphors, through a developmental process of discovery, evaluation, use, and affinity. Haven (2007)
    13. 13. Activity 2 - Examples Small groups Tag previous examples within the engagement process. Involvement • The presence of a learner within the institution including data such as physical or virtual visits Interaction • Provides a depth of understanding: where involvement measures touches, interaction measures actions. Intimacy • Helps understand sentiment and affection; the most common way to collect this type of data is through interviews or surveys. Influence • Determines the likelihood of the individual recommending learning to others and contributing to local culture(s).
    14. 14. Involvement The presence of a learner within the institution including data such as physical or virtual visits.  Overall Activity  Locations  Time of day
    15. 15. Involvement The presence of a learner within the institution including data such as physical or virtual visits.  Overall Activity  Locations  Time of day
    16. 16. Interaction Provides a depth of understanding: where involvement measures touches, interaction measures actions.  Activity types  Action analysis  Connectivity maps Conole (2007)
    17. 17. Interaction Provides a depth of understanding: where involvement measures touches, interaction measures actions.  Activity types  Action analysis  Connectivity maps
    18. 18. Intimacy Helps understand sentiment and or affection; the most common way to collect this type of data is through interviews or surveys.  Learning Power  Self-theory  Motivated Strategies for Learning Questionnaire, MSLQ  Self-determination theory Deakin Crick, Broadfoot, and Claxton (2004)
    19. 19. MSLQ 6 5 Intimacy 4 Helps understand sentiment and or affection; the most common way to collect this type of data is through interviews or surveys. 3 2  Learning Power  Self-theory 1  Motivated Strategies for Learning Questionnaire, MSLQ 0 Rehearsal Elaboration Organisation Pre Self-Regulation Critical Thinking Post Pintrich (1990)  Self-determination theory
    20. 20. Influence Determines the likelihood of the individual recommending learning to others and contributing to local culture(s).  Social Network Analysis  Distributed Cognition  Collective Intelligence  Pathway of Participation Dawson (2010)
    21. 21. Influence Determines the likelihood of the individual recommending learning to others and contributing to local culture(s).  Social Network Analysis  Distributed Cognition  Collective Intelligence  Pathway of Participation School Leader Network Harré (1983)
    22. 22. Preparing Institutions Empowering environments for learning
    23. 23. Metrics are based on the data that is easiest to extract/access, and what you don‟t measure is lost. Anything you measure will impel a person to optimize his score on that metric. Don‟t be surprised if people find ingenious and destructive ways in how they get there. For example, standardised assessment produce kids who perform well on these tests but can falter when asked to demonstrate their knowledge of the same material in a different way You are what you measure „Incremental change is not enough. You have to drive large-scale change by changing the environment in which people work‟ – Kevin Bonnett, Deputy Vice Chancellor Student Experience JISC Report MMU Review
    24. 24. Activity 1 - Introduction Open Discussion What types of skills are required by elearning teams? Do they already exist?
    25. 25. Analytics Process Collection Storage Cleaning Integration Analysis Presentation CETIS Analytics Series (2012)
    26. 26. MySQL / PostgreSQL Apache Hadoop HP Vertica Pentaho Rapid Miner Gephi Google Visualisation d3.js InfoVis Toolkit Open Source Tools Investing in staff experimentation with low cost components from a range of traditions may be a more prudent initial move, even if the most effective tool subsequently turns out to be a ready-made suite.
    27. 27. Algorithm Usage Purpose Step regression Used for binary classification (0,1) • Select a parameter • Assign a weight • Calculate value Predicts simple binary results such as is a student at-risk? Logistic regression Same as above but more conservative J48/C4.5 Decision trees (Quinlan, 1993) Tries to find optimal split in variables Good when data splits into groups JRip Decision rules Find the “best” path Good when multi-level and make this a rule interaction are until no sensible paths common are left and set these to otherwise. K* Instance based classifiers Predicts data based on neighbouring points. Good when data is very divergent Data Mining Classification is used when one wants to predict something (label) which is categorical and not a number.
    28. 28. Infinite Rooms Learner enhanced technology
    29. 29. Web dashboards based on engagement process accessing a data warehouse model developed from Activity Theory. Utilises new and existing analytics and supports multiple learning design approaches. 1. How can student activity help identify and promote effective teaching practices? 2. Understand the role that analytics can play in learning design, feedback and assessment. Research Project If patterns of nonparticipation (disengagement) are to be disrupted an improved conceptual framework may be necessary.
    30. 30. Activity Analysis Engeström‟s (1987, 1999) approach allows us to overcome oppositions between activity and communication and highlight subject-community relations. Modelling pedagogy with Activity Theory Stevenson (2008) http://goo.gl/vOuiqp
    31. 31. Exposing Activity The intention of this is to reveal the nature of the system, allowing designers (e.g. teachers) to evaluate the system in the wider context of their teaching and learning practice.
    32. 32. Dimension Fact Action Post to forum Tool Forum Instance Discussion topic User Oliver Twist Role Student Course Introduction to English Date 02/10/2013 Time 9:45 System Moodle Data Model Enables multi-dimensional tagging to explore data from different perspectives.
    33. 33. Actions Things a learner does • Submissions • Quiz attempts • Forum posts Interventions Feedback to the learner • Targets • Grades • Assignment feedback Achievements Recognising learning • Course completions • Badges • Certificates Surveys How learning is perceived • Attitudes to learning/technology • Satisfaction survey Data Capture What types of things can we capture.
    34. 34. Coding One can then begin to distinguish the possible actions that are generated through the use of tools from the operations needed to access them and code these via learning design theories.
    35. 35. Activity Types Apply learning design models to learner data. Conole (2007)
    36. 36. Visualisation Explore different visualisations of the same data set for different insights.

    ×