Overview of the Jisc Learning Analytics Project

1. Overview
Jisc Learning Analytics

2. Learning Analytics
Background
Jisc Learning Analytics Services
User views of products
Timeline and next steps
Student App (additional slides)
How data is included (additional slides)
Learning Analytics - Initial Meetings 2015 2
Outline of session

3. About Jisc…
Learning Analytics - Initial Meetings 2015 3
It operates shared digital
infrastructure and services,
negotiates sector-wide deals
with IT vendors and commercial
publishers, and provides trusted
advice and practical assistance
for universities and colleges.
Jisc is the UK higher and
further education sectors’
not-for-profit
organisation for digital
service and solutions.

4. What does Jisc do?
Does 4 things…
Providing and developing a
network infrastructure and
related services that meet the
needs of the UK research and
education communities
Supporting the procurement
of digital content for UK
education and research
Our network of national and
regional teams provide local
engagement, advice and
support to help you get the
most out of our service offer
Our R&D work, paid for entirely
by our major funders, identifies
emerging technologies and
develops them around your
particular needs

5. Co-design challenges
Research at risk (R@R)
Prospect to alumnus (P2A) Learning analytics
Digital learning & capabilitiesImplementing FELTAG
Business intelligence
Hosting platform Hosting platform

6. About Learning Analytics…
6Learning Analytics - Initial Meetings 2015

7. Effective Learning Analytics Challenge
Rationale
Universities and colleges don't have enough useful data about students and how they are learning.
What they have they don’t analyse and interpret.They are missing opportunities to use technology to
provide feedback to students.They need to support staff who could be using analytics and a standard
set of tools and technologies to monitor and intervene.
Who it affects and how
Students are missing out on the possibility of an improved experience, better retention, and better
achievement.
Staff are missing the opportunity to develop skills to use analytics to improve support, teaching and
curriculum design.
Timescale
Pilot tools and metrics 1-2 years
Impact on retention, achievement and progression 3-4 years.
Learning Analytics - Initial Meetings 2015 7

8. What do we mean by Learning Analytics?
The application of big data techniques such as machine based learning
and data mining to help learners and institutions meet their goals:
For our project:
» Improve retention (current project)
» Improve achievement (current project)
» Improve employability (current project)
» Personalised learning (future project)
8Learning Analytics - Initial Meetings 2015

9. About Learning Analytics…
Products and Solutions
9Learning Analytics - Initial Meetings 2015

10. Jisc’s Learning Analytics Project
10
Three core strands:
Learning
Analytics Service
Toolkit Community
Jisc Learning Analytics
Learning Analytics - Initial Meetings 2015

11. Jisc’s Learning Analytics Service
Walkthrough
11
Learning
Analytics Service
Learning Analytics - Initial Meetings 2015

12. Learning Analytics - Initial Meetings 2015 12
Jisc Learning Analytics Architecture

13. Learning Analytics - Initial Meetings 2015 13

14. Learning Analytics - Initial Meetings 2015 14
Data collection
About the student Activity data
TinCan
(xAPI)ETL

15. A user perspective…
15Learning Analytics - Initial Meetings 2015

16. Dashboards
16
Visual tools to allow lecturers, module leaders,
senior staff and support staff to view:
» Student engagement
» Cohort comparisons
» etc…
Based on either commercial tools from Tribal
(Student Insight) or open source tools from
Unicon/Marist (OpenDashBoard)
Learning Analytics - Initial Meetings 2015

17. 17Learning Analytics - Initial Meetings 2015

18. Learning Analytics - Initial Meetings 2015 18

19. First version will include:
» Overall engagement
» Comparisons
» Self declared data
» Consent management
Bespoke development by Therapy Box
19
Student App
Learning Analytics - Initial Meetings 2015

20. 20Learning Analytics - Initial Meetings 2015

21. Alert and Intervention System
Tools to allow management of interactions with students
once risk has been identified:
» Case management
» Intervention management
» Data fed back into model
» etc…
Based on open source tools from Unicon/Marist
(Student Success Plan)
21Learning Analytics - Initial Meetings 2015

22. 22Learning Analytics - Initial Meetings 2015

23. Timeline and next steps…
23Learning Analytics - Initial Meetings 2015

24. Jisc Project in numbers
Expressions of interested: 70
Engaged in activity: 33
Discovery to Sept 16: agreed (25), completed (11), reported (6)
Over 1 millions records collected in real-time
Moodle Historic DataTransformation:
41 millions records transformed from Moodle Log files to xAPI
Blackboard DataTransformation:
12 millions blackboard records transformed from Blackboard Log to xAPI
Learning Analytics - Initial Meetings 2015 24

25. Learning Analytics - Initial Meetings 2015 25
Profile Aim Activity Data Sources
Russell Group Retention of widening participation +
support for students to achieve 2.1 or
better
Discovery +Tribal Insight +
Learning Locker
Moodle + Student
Records
Research led
University
Retention, improve teaching,
empowering students
Discovery + OpenSource
Suite + StudentApp
Moodle +
Attendance+
Student Records
Teaching led
University with WP
mission
Retention - requirement to make
identifying students more efficient so
they can focus on interventions
Tribal Insight + Learning
Locker
Blackboard +
Attendance +
Student Records
Research led
University
Student engagement Discovery + Student app +
Learning Locker
Moodle + Student
Records
Teaching Lead Understanding of how Learning
Analytics can be used
Discovery +Technical
Integration
Moodle

26. Example HE Activity

27. Learning Analytics - Initial Meetings 2015 27
Phase 1&2
Sep 15 – Apr 16
Phase 2&3
Jan – Sept 16
Transition to
Service
Sept 16 – July 17
Jisc Learning
Analytics Service
Sept 2017

28. 28
Jisc/Unicon
Discovery
Jisc Learning
Analytics
Implementation
Wish to
explore
readiness
and
products
Know you
are ready
and what
you want
Want to
get
involved in
tech work
first
Blackboard
Discovery
Unicon/Marist pre-
implementation
Tribal pre-
implementation
Other pre-
implementation
Blackboard
Trial
MoodleTrial
Other Learning
Analytics
Implementation
TechTrials Discovery Pre-implementation Implementation
Learning Analytics - Initial Meetings 2015

29. Next Steps
Review Legal and Ethical issues – Code of Practice
Discovery Stage – See offers from Blackboard and Unicon
Technical Overview – Register and enrol https://courses.alpha.jisc.ac.uk
Technical Trials – Set up Learner Records Warehouse, install VLE plugin, identify and
share, look data sets for student information
Technical Implementation – chose preferred analytics solution (Tribal Student Insights or
Unicon processor and dashboard). Jan 2016 onwards for implementation.
Learning Analytics - Initial Meetings 2015 29

30. Learning Analytics - Initial Meetings 2015 30
https://courses.alpha.jisc.ac.uk

31. The student app…
31Learning Analytics - Initial Meetings 2015

32. Student Learning Analytics App 32
When first logging in the
student is able to select their
institution from a pre-populated
lists of UK universities. If the
students’ institution is using
other parts of Jisc’s learning
analytics architecture, in
particular the learning analytics
warehouse, then much more
data will be available to the app.

33. Student Learning Analytics App 33
The screen will be an activity
feed or timeline, we plan to
integrate this dynamic and
engaging concept, so essential to
applications such asTwitter and
Facebook.
Photos or badges could be
included next to the text.

34. Student Learning Analytics App 34
Stats – Provides an engagement
and attainment overview and
drilling down to gives
comparative activity graphs.
Log – Allows you to log time
spent on specified activities e.g.
reading for an assignment
Target – Allows you set personal
targets to improve your
engagement e.g. study for 10
hours this week

35. Student Learning Analytics App 35
The engagement and
attainment overview mirrors
what many fitness apps do: it
provides an overview of your
“performance” to date. Critically
here we show how you compare
to others.This will be based on
data about you and others held
in the learning analytics
warehouse.

36. Student Learning Analytics App 36
In the activity comparison
screen you’ll see a graph of your
engagement over time and how
it compares with that of others.
You can select a particular
module or look at your whole
course.
You can compare yourself with
people on my course, people on
this module and top 20% of
performers (based on grades).
Comparing yourself to prior
cohorts of students on a module
might be of interest in the future
too.

37. Student Learning Analytics App 37
Starting an activity allows you
to select the module on which
you’re working, choose an
activity type from a drop-down
list such as reading a book,
writing an essay, or attending a
lab, and select a time period you
want to spend on the activity and
whether you want a notification
when that period is up.
A timer is displayed in the image
box and you can hit the Stop
button when you’ve
finished. The timer will continue
even if you navigate away from
the app.

38. Student Learning Analytics App 38
Setting a target is the final bit of
functionality we want to include
in the app at this stage.Again
this is building on the success of
fitness tracking apps where you
set yourself targets as a way of
motivating yourself.

39. Student Learning Analytics App 39
Setting a target involves
selecting a learning activity from
a pre-populated list and
specifying how long you want to
be spending on it.
We added a “because” free text
box so that learners can make it
clear (to themselves) why they
want to carry out the activity e.g.
I want to pass the exam, tutor told
me I’m not reading enough).
Users may be more likely to
select a reason from a pre-
populated list than to fill in a text
field but we’ll monitor this to see
whether it’s being used.

40. Jisc Learning AnalyticsToolkit
40
Toolkit
Learning Analytics - Initial Meetings 2015

41. Discovery …
The learning analytics discovery service is a way of
investigating your institution’s readiness for learning
analytics. The process will investigate strategic,
technical, process and data readiness, providing
recommendations for action before moving on to deploy
a learning analytics solution.
41Learning Analytics - Initial Meetings 2015

42. http://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics
Code of Practice
Learning Analytics - Initial Meetings 2015 42

43. Deeper Dive
http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf
Literature review – basis
for the code of practice
Learning Analytics - Initial Meetings 2015 43

44. Code of Practice
Privacy
Validity
Responsibility
Access
Enabling positive
interventions
Minimising adverse impacts
Transparency and consent
Learning Analytics - Initial Meetings 2015 44

45. 45
Community
Community
Learning Analytics - Initial Meetings 2015

46. Project Blog, mailing list and
network events
Blog: http://analytics.jiscinvolve.org
Mailing: analytics@jiscmail.ac.uk
46Learning Analytics - Initial Meetings 2015

47. How can institutions get involved…
47
Toolkit
Learning Analytics - Initial Meetings 2015

48. Learning Analytics - Initial Meetings 2015 48
Phase 1&2
Sep 15 – Apr 16
Phase 2&3
Jan – Sept 16
Transition to
Service
Sept 16 – July 17
Jisc Learning
Analytics Service
Sept 2017

49. Timeline
Jun 15 Sep 15 Jan 16 Apr 16
3. Trial
Integration pt1
x 2
1. Jisc complete
contracts
2. Jisc Sandbox
4. Phase 1
Discovery
5. Phase 1
implementation
x 6
8. Phase 2
implementation
x 6 - 12
6. Trial
Integration pt2
x 2
7. Phase 2
Discovery x 6-
12
Learning Analytics - Initial Meetings 2015 49

50. 50
Jisc/Unicon
Discovery
Jisc Learning
Analytics
Implementation
Wish to
explore
readiness
and
products
Know you
are ready
and what
you want
Want to
get
involved in
tech work
first
Blackboard
Discovery
Unicon/Marist pre-
implementation
Tribal pre-
implementation
Other pre-
implementation
Blackboard
Trial
MoodleTrial
Other Learning
Analytics
Implementation
TechTrials Discovery Pre-implementation Implementation
Learning Analytics - Initial Meetings 2015

51. michael.webb@jisc.ac.uk
One Castlepark Tower Hill Bristol BS2 0JA
T 020 3697 5800
info@jisc.ac.uk jisc.ac.uk
Michael Webb
Director ofTechnology and Analytics
51

52. How’s the data collected?
52Learning Analytics - Initial Meetings 2015

53. Learning Analytics - Initial Meetings 2015 53
About the student Activity data
TinCan
(xAPI)ETL
Data collection

54. About the student’ data
Personal (demographic) data
Birthdate, gender etc.
Course data
mode of study, level etc.
Grade data
Assignment, module etc.
(aligned with HESA data)
54Learning Analytics - Initial Meetings 2015

55. Activity data viaTin Can API
• People learn from interactions with other
people, content, and beyond.
• These actions can happen anywhere and signal
an event where learning could occur.
• When an activity needs to be recorded, the
application sends secure statements in the
form of “Actor, verb, object” or “I did this” to
the Learning Record Store (LRS.)
from: http://tincanapi.com/
55Learning Analytics - Initial Meetings 2015

56. Activity Data (trivial!) examples
56
Actor Action Object Result
Michael Accessed VLE
Sally Completed Basic MathsTest 85.0
Kim Module CommentAdded
https://registry.tincanapi.com
Learning Analytics - Initial Meetings 2015

57. 57Learning Analytics - Initial Meetings 2015

58. ‘Recipes’ are key
• ‘Recipes’ are a shared way of describing
activities..
• So the data from ‘accessing a course’ is the
same whether Moodle or Blackboard is
used.
• The same holds for..
• ‘Attend a lecture’
• ‘Borrow a book’
• …
58Learning Analytics - Initial Meetings 2015