Invited presen

1. JISC CETIS 2013 Conference: Analytics and Institutional Capabilities
Building Analytics
Capability @open.edu
Simon Buckingham Shum
Professor of Learning Informatics
Knowledge Media Institute, The Open University, UK
http://simon.buckinghamshum.net
@sbskmi

2. bit.ly/OULAprof

3. Same
outcomes,
but
higher
scores?
Learning
Analy=cs
as
Evolu&onary
Technology.
Same
training
+
educa=onal
paradigms
•
more
engaging
•
beBer
assessed
•
beBer
outcomes
3
•
deliverable
at
scale

4. Learning
dynamics
we
couldn’t
assess
before?
Learning
Analy=cs
as
Revolu&onary
Technology.
A
vehicle
for
paradigm
shiF?
•
interpersonal
learning
networks
•
quality
of
discourse
+
wri=ng
•
lifelong
learning
disposi=ons
•
problem
solving
strategies
•
lifewide
learning

5. open.edu
BI perspective
5

6. OU
data
warehouse
(in
progress)
IT corral key
institutional data in the IT provide data
central warehouse 1 2 dictionary
IT provide data
Business data
users propose 5 Data
3 marts and cubes
action Warehouse
for commonly used
data sets
“Data Wranglers”
assist staff in
understanding BI 4 OU Analytics
Board
Explore the challenge/issue/problem/
opportunity/question using SAS/preferred tool

7. open.edu
VLE
perspective
7

8. VLE
Analy;cs
@
the
OU
Virtual
Learning
Environment
Data
Warehouse
Usage
sta;s;cs
at
system,
faculty
and
‘Par;cipa;on
Tracking’
func;on
to
track
module
level
–
general
paCerns
individual
students’
interac;on
with
specific
online
learning
ac;vi;es
In
pilot
2012/13

9. VLE
Analy;cs
@
the
OU
Virtual
Learning
Environment
Data
Warehouse
Usage
sta;s;cs
at
system,
faculty
and
‘Par;cipa;on
Tracking’
func;on
to
track
module
level
–
general
paCerns
individual
students’
interac;on
with
specific
online
learning
ac;vi;es
In
pilot
2012/13

10. VLE
Analy;cs
@
the
OU
Virtual
Learning
Environment
Data
Warehouse
Usage
sta;s;cs
at
system,
faculty
and
‘Par;cipa;on
Tracking’
func;on
to
track
module
level
–
general
paCerns
individual
students’
interac;on
with
specific
online
learning
ac;vi;es
In
pilot
2012/13

11. open.edu
predictive
modelling
11

12. Predictive analytics
Demo-­‐ VLE
graphics
interac=on
?
Registra=on
Library
PaBern
interac=on
How early can we predict
likelihood of dropout, formal
CRM
OpenLearn
withdrawal, failure?
contact
interac=on
Now exploring conventional
Assignment
Futurelearn
statistics, machine learning
grades
interac=on
and growing datasets
OU
track
Social
App
X
New fees regime may well
record
interac=on
change student behaviour…

13. OU Analytics: Predictive modelling
§ Probability models help us to
identify patterns of success that vary
between:
§ student groups / areas of
curriculum / study methods Best predictors of
§ Benefits future success:
previous OU study
§ provide a more robust comparison of data – quantity
module pass rates and results
§ support the institution in identifying
aspects of good performance that
can be shared, and aspects where
improvement could be realised
OU Student Statistics & Surveys Team, Institute of Educational Technology 13

14. Improving student retention with
predictive analytics
4 predictive models:
final result (pass/fail)
Demo- final numerical score
graphics drop in the next TMA
score of the next TMA
Previous
results
VLE
activity
A.L. Wolff and Z. Zdrahal (2012). Improving Retention by Identifying and Supporting “At-risk” Students. EDUCAUSE Review Online, July-
August 2012. http://www.educause.edu/ero/article/improving-retention-identifying-and-supporting-risk-students

15. open.edu
Library
perspective
15

16. Learning Analytics – the Library dimension
Student achievement
Recommender services
Library use
‘Students who looked at this article also
looked at this article’
‘Students on your course are looking at
these articles’
Library Impact Data Project
– Huddersfield University
http://www.flickr.com/photos/davepattern/6928727645/sizes/o/in/photostream/

17. open.edu
Research
perspective
17

18. Visualizing
and
filtering
social
;es
in
SocialLearn
by
topic
and
type
Schreurs,
B.,
Teplovs,
C.,
Ferguson,
R.,
De
Laat,
M.
and
Buckingham
Shum,
S.,
Visualizing
Social
Learning
Ties
by
Type
and
Topic:
Ra;onale
and
Concept
Demonstrator.
In:
Proc.
3rd
Interna6onal
Conference
on
Learning
Analy6cs
&
Knowledge
(Leuven,
BE,
8-­‐12
April,
2013).
ACM
hCps://dl.dropbox.com/u/15264330/papers/Schreurs-­‐etal-­‐LAK2013.pdf

19. Discourse analytics on webinar textchat
Given a 2.5 hour webinar, where in the
live textchat were the most effective
learning conversations?
Sheffield, UK not as
sunny as yesterday - still Not at the start and end of a webinar, but See you!
warm
Greetings from Hong
if we zoom in on a peak… bye for now!
Kong bye, and thank you
80
Morning from Wiltshire, Bye all for now
sunny here!
60
40
20
0
9:28
9:32
10:13
11:48
12:00
12:05
12:04
9:36
9:40
9:41
9:46
9:50
9:53
9:56
10:00
10:05
10:07
10:07
10:09
10:17
10:23
10:27
10:31
10:35
10:40
10:45
10:52
10:55
11:04
11:08
11:11
11:17
11:20
11:24
11:26
11:28
11:31
11:32
11:35
11:36
11:38
11:39
11:41
11:44
11:46
11:52
11:54
12:03
-20
-40
Average Exploratory
-60
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions.
In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664

20. Discourse analytics on webinar textchat
Given a 2.5 hour
webinar, where in
the live textchat
were the most
effective learning
Classified
conversations? as
“exploratory
talk”
100 (more
substantive
50 for learning)
0
9:28
“non-
9:40
9:50
10:00
10:07
10:17
10:31
10:45
11:04
11:17
11:26
11:32
11:38
11:44
11:52
12:03
-50 exploratory
Averag
”
-100
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions.
In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664

21. Discourse analytics on webinar textchat
Visualizing by individual user. The gradient of the threshold line is
adjusted to every 5 posts in 6 classified as “Exploratory Talk”
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions.
In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664

22. Analytics for “21st Century
Competencies & Learning Dispositions”
Different social
Questioning and network patterns in
challenging may load different contexts
onto Critical Curiosity may load onto
Learning
Relationships
Repeated attempts to
pass an online test
Sharing relevant may load onto
resources from other Resilience
contexts may load
onto Meaning Making
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and
Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823

23. open.edu
coming
soon…
23

24. On the horizon…
MOOCs + Analytics…
Educ Research
at SCALE
Partnerships/
Collab
What Research
Data? Biz Models
‘vs’ Open Ethics
http://people.kmi.open.ac.uk/sbs/2013/01/emerging-mooc-data-analytics-ecosystem

25. On the horizon…
Educational Data Scientists