Getting Started with
Learning Analytics
Lori Lockyer
Tim Rogers
Shane Dawson
What about today?
•
•
•
•
•
•
•

Introductions and background
From base camp to summit
Data – it seems important
Analytics...
Learning Analytics
…is the collection, collation, analysis and
reporting of data about learners and their
contexts, for th...
Learning Analytics

Ed theory, Ed practice, SNA, Data
mining, Machine learning, semantic,
data visualisations, sense-makin...
Examine large data sets – trends/ patterns or
anomalies.
What do patterns indicate and what do
changes in habit indicate?
Current State
Higher education:
• Lots of isolated work targeting attrition. Very few
large enterprise egs.
• Commercial –...
Education Examples
Education Examples
Education Examples
Education - Purdue
Education - UMBC
Education – UniSA
Current Focus

Pass/Fail, Retention
Concept understanding

Kentucky: 1.3% - 80 stds approx 400k
Where next?
Beyond dashboards
Predictive and recommender states
Future
Learner control
NLP – video annotations
Future
Emotions/ face tracking
Frustrated

Confusion

Engaged

Activity
modified

Continuous
state of
challenge
First steps – the why,
what and how of data
Improving feedback in mass higher education
Data, data,
everywhere…
• Where data is accessible it is usually
lagged, scattered, indecipherable,
requires manipulating,...
…but not a digit of
use

• Currently, despite all the data,
• Students often don’t know how they are
going

• Academics do...
Outline of data
thinking process
• What is the purpose for the data?
• What data is needed (and who ‘owns’
it)

• How to w...
Data for what
purpose?

• Student level support (success and
retention)

• Educator needs – improving teaching
and learnin...
Who owns the data…
• …aka where do you get it? IT,
Business Intelligence, Admin?

• And others, e.g. class rolls, library ...
Working with data
• All data will need various degrees of
extraction and transformation

• All data needs contextualisatio...
42
42

Student
42

Student

Test score
/100
42

Student

Test score 10%
/100
course
mark
42

Student

Test score 10%
Degree:
/100
course Chemical
mark
Engineering
42

Student

Test score 10%
Degree:
Course:
/100
course Chemical
Shakesp
mark
Engineering eare and
Society
42

Student

Test score 10%
Degree:
Course: Class
/100
course Chemical
Shakesp Position
mark
Engineering eare and : 1/36
S...
Making data
actionable
• Visualising the data for summary and
exception highlighting

• Trends, key junctures, cumulative ...
Visualising critical
metrics
The work of Stephen Few
Context is everything
Data and analytics to support
learning design and implementation
Learning Analytics:
… measurement, collection, analysis and reporting
of data about learners and their contexts, for
purpo...
Where and how does learning
occur in HE in Australia?
• Within courses/units
• which are designed predominately by
teacher...
How might a university teacher
use data and analytics?
• Analytics to inform design decisions
• Just-in-time analytics to ...
What can data help us with?
• Moe than…
– retention/attrition
– “… and they liked it”

• To are they…
–
–
–
–
–

doing wha...
Learning analytics can only help us
answer these questions if they are:
- specific to the learning outcomes of
the unit
- ...
In other words…

… the teaching and learning context
matters.
Mapping a design
Case Based Learning Design adapted from Bennett (2002) available at
http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
LMS log: Student log in;
access case

Network diagram: even
pattern of participation

Network diagram: teachercentred patt...
LMS log: Student log in;
access case

Network diagram: even
pattern of participation

Network diagram: teachercentred patt...
LMS log: Student log in;
access case

Network diagram: even
pattern of participation

Network diagram: teachercentred patt...
Now it is your turn:
Sample design or
Your learning design?
•
•
•
•
•
•

What are the learning outcomes?
What does the des...
Summary
• We are already capturing a lot of data
• There’s a lot of information we are not
systematically capturing
• Curr...
A-LASI Getting started in learning analytics (Lockyer, Rogers and Dawson)
A-LASI Getting started in learning analytics (Lockyer, Rogers and Dawson)
A-LASI Getting started in learning analytics (Lockyer, Rogers and Dawson)
A-LASI Getting started in learning analytics (Lockyer, Rogers and Dawson)
Upcoming SlideShare
Loading in …5
×

A-LASI Getting started in learning analytics (Lockyer, Rogers and Dawson)

1,678 views

Published on

Workshop held at the Australian Learning Analytics Summer Institute (A-LASI) run by Lori Lockyer, Tim Rogers and Shane Dawson

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,678
On SlideShare
0
From Embeds
0
Number of Embeds
735
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Focused on institutional reporting, indicators of attrition and student learning support
  • Focused on institutional reporting, indicators of attrition and student learning support
  • A-LASI Getting started in learning analytics (Lockyer, Rogers and Dawson)

    1. 1. Getting Started with Learning Analytics Lori Lockyer Tim Rogers Shane Dawson
    2. 2. What about today? • • • • • • • Introductions and background From base camp to summit Data – it seems important Analytics for teachers Wrap up Beer and cookies Questions, concerns or issues
    3. 3. Learning Analytics …is the collection, collation, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning
    4. 4. Learning Analytics Ed theory, Ed practice, SNA, Data mining, Machine learning, semantic, data visualisations, sense-making, psychology (social, cognitive, organisational), learning sciences
    5. 5. Examine large data sets – trends/ patterns or anomalies. What do patterns indicate and what do changes in habit indicate?
    6. 6. Current State Higher education: • Lots of isolated work targeting attrition. Very few large enterprise egs. • Commercial – IBM, SAS, Hobsons, D2L Insight, BB analytics
    7. 7. Education Examples
    8. 8. Education Examples
    9. 9. Education Examples
    10. 10. Education - Purdue
    11. 11. Education - UMBC
    12. 12. Education – UniSA
    13. 13. Current Focus Pass/Fail, Retention Concept understanding Kentucky: 1.3% - 80 stds approx 400k
    14. 14. Where next? Beyond dashboards Predictive and recommender states
    15. 15. Future
    16. 16. Learner control
    17. 17. NLP – video annotations
    18. 18. Future Emotions/ face tracking Frustrated Confusion Engaged Activity modified Continuous state of challenge
    19. 19. First steps – the why, what and how of data Improving feedback in mass higher education
    20. 20. Data, data, everywhere… • Where data is accessible it is usually lagged, scattered, indecipherable, requires manipulating, lacks context… • Yes, there are BI reports, but they are mostly for the converted and don’t flag exceptions
    21. 21. …but not a digit of use • Currently, despite all the data, • Students often don’t know how they are going • Academics don’t know if their teaching is effective • Program/degree owners don’t know how their students navigate their way through • Management don’t know if the Uni is on track
    22. 22. Outline of data thinking process • What is the purpose for the data? • What data is needed (and who ‘owns’ it) • How to work with the data? • How to make the data actionable?
    23. 23. Data for what purpose? • Student level support (success and retention) • Educator needs – improving teaching and learning • Program designer/owner needs – curriculum flows • Management/QA requirements – are courses/subjects meeting standards and improving?
    24. 24. Who owns the data… • …aka where do you get it? IT, Business Intelligence, Admin? • And others, e.g. class rolls, library data, orientation attendance, in-class formative and summative assessments etc
    25. 25. Working with data • All data will need various degrees of extraction and transformation • All data needs contextualisation, and a decision about how fine-grained that needs to be • For example, is this a problem?…
    26. 26. 42
    27. 27. 42 Student
    28. 28. 42 Student Test score /100
    29. 29. 42 Student Test score 10% /100 course mark
    30. 30. 42 Student Test score 10% Degree: /100 course Chemical mark Engineering
    31. 31. 42 Student Test score 10% Degree: Course: /100 course Chemical Shakesp mark Engineering eare and Society
    32. 32. 42 Student Test score 10% Degree: Course: Class /100 course Chemical Shakesp Position mark Engineering eare and : 1/36 Society
    33. 33. Making data actionable • Visualising the data for summary and exception highlighting • Trends, key junctures, cumulative risk • Tools for action, e.g. CRMs, and business processes
    34. 34. Visualising critical metrics The work of Stephen Few Context is everything
    35. 35. Data and analytics to support learning design and implementation
    36. 36. Learning Analytics: … measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (LAK11 - https://tekri.athabascau.ca/analytics/)
    37. 37. Where and how does learning occur in HE in Australia? • Within courses/units • which are designed predominately by teachers (not instructional designers) • who interact with students as they are learning • who can, may, may not, intervene in the learning process.
    38. 38. How might a university teacher use data and analytics? • Analytics to inform design decisions • Just-in-time analytics to understand learner activity and experience during implementation • Recommendations for learner action • Analytics for post-implementation reflection and revision • Support scholarship of teaching
    39. 39. What can data help us with? • Moe than… – retention/attrition – “… and they liked it” • To are they… – – – – – doing what you intended? understanding the task? on-task/off-task? motivated, engaged? actually learning anything?
    40. 40. Learning analytics can only help us answer these questions if they are: - specific to the learning outcomes of the unit - related to how we think learning occurs for such outcomes, in the discipline… - relevant to the learning design we have put in place
    41. 41. In other words… … the teaching and learning context matters.
    42. 42. Mapping a design
    43. 43. Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
    44. 44. LMS log: Student log in; access case Network diagram: even pattern of participation Network diagram: teachercentred pattern Network diagram: even pattern of participation Document sharing logs of contribution LMS log: access to teacher feedback LMS log: submission of reflection template Content analysis: depth of reflection Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
    45. 45. LMS log: Student log in; access case Network diagram: even pattern of participation Network diagram: teachercentred pattern Network diagram: even pattern of participation Document sharing logs of contribution LMS log: access to teacher feedback LMS log: submission of reflection template Content analysis: depth of reflection Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
    46. 46. LMS log: Student log in; access case Network diagram: even pattern of participation Network diagram: teachercentred pattern Network diagram: even pattern of participation Document sharing logs of contribution LMS log: access to teacher feedback LMS log: submission of reflection template Content analysis: depth of reflection Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
    47. 47. Now it is your turn: Sample design or Your learning design? • • • • • • What are the learning outcomes? What does the design look like? Map it? What do you want to know? What data will inform these? What patterns do you anticipate? What can you do about it?
    48. 48. Summary • We are already capturing a lot of data • There’s a lot of information we are not systematically capturing • Current or possible answer might answer our questions • First we have to have relevant questions and know what we are prepared to do with the answers

    ×