Session aims: • Introduce learning analytics • Describe the development of the NTU Student Dashboard • Discuss potential benefits of learning analytics for personal tutors • Raise some challenges of converting student information to actionable intelligenc

1. Using learning analytics to boost personal tutoring
ABLE Project 2015-1-BE-EPPKA3-PI-FORWARD
STELA Project: 562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD

2. Session aims
• Introduce learning analytics
• Describe the development of the NTU Student Dashboard
• Discuss potential benefits of learning analytics for personal tutors
• Raise some challenges of converting student information to actionable
intelligence

3. Introduction to learning analytics

4. "Analytics is a term used
in business and science
to refer to computational
support for capturing
digital data to help
inform decision-making
… Learning Analytics
appropriates this concept
for education.”
(Buckingham Shum,
2012, p.1)
(Clow, 2012)
Learning Analytics

5. Why NTU is interested in learning analytics?
• Reduce barriers to being known by a tutor
• Space for tutors to make notes & plan
• Students as agents
• Can see own grades, and compare self to
peers
• Feedback
• Tool for academic use
• Potentially promotes virtuous behaviour
• Alerts tutors if students are high risk for
leaving early
• Students can compare their engagement
with their peers
Attainment
Progression
Belonging
Strategic
information
• Insights into groups at risk
• Potentially design and delivery of
courses

6. Development of the NTU Student Dashboard

7. Developmental Cycle
Sept
2013
Sept
2014
Sept
2015
Pilot Phase Phase One Phase
Two
Phase Three
 Personalisation
 Embedding into University
systems
 4 courses
 40 staff
 500 1st years
Willing participants:
very positive staff
feedback, limited
student awareness
 8 of 9 Schools
 Governance
 Problem solving
 Ethics
Near to whole
University roll out:
increased awareness
 All Schools
 New data
sources
 Assessment
view
Whole
University roll
out: increased
awareness
Feb
2016
Increasing resources (e.g.
how to guides),
communication, guidance
Staff and student consultation ongoing throughout developments
Further details of projects at http://www.ableproject.eu/project-outputs/ and
https://eng.kuleuven.be/english/projects/STELAproject/stela

8. The NTU Student Dashboard
• Can be viewed as two products:
Physical
Dashboard
Algorithm
• Staff and students interact with physical dashboard
• Algorithm is the behind the scenes, learning analytics element

9. Raises
alerts!!
What does the Dashboard do?
NTU
Student
Dashboard
Student biographical
info, e.g. enrolment
status
Evidence of student
engagement
• Door swipes
(where appropriate)
• Library books
• NOW use
• Dropbox
submissions
• Attendance data
• Access to e-
books and
journals through
Shibboleth
authentication
Staff
view
Student
view
Compares student
engagement across
the cohort & gives
rating
Can make
comments in
free text box

10. Potential benefits of learning analytics for personal
tutors

11. Data accuracy for algorithm spotting students
at risk
• Two big questions:
1. Can the algorithm correctly identify at risk students?
2. Can it do so on a timescale that allows intervention?

12. Relationship between yearly average engagement &
progression
• Low average (mode) engagement for the year is an indicator of risk

13. Relationship between term one average engagement &
progression
• Low average (mode) engagement for the 1st term is an indicator of risk

14. No engagement alerts
• Any one alert is an indicator of risk
• Students with multiple alerts had lower incidence of progression

15. Tutorial Landing Screen
• Access to tutees in addition to search facilities for other students the
staff member interacts with

16. Links to student’s dashboard Able to sort on headings
Class view
• Designed so staff have easy access to student data.
• Allows staff to quickly identify potentially at risk students

17. Individual student view
Not Fully
Enrolled
Staff and students can
benchmark engagement –
springboard for conversation

18. Notes and referrals
• Notes inputted by
staff only
• Time and date stamp
• Referral to:
 Library Academic
Skills
 Student Support
Services
 Employability
(planned)
Personal tutors can track interactions with students
Personal tutors can make referral whilst in room with student

19. Student profile
• Basic information (ability to
report if this is wrong)
• Engagement summary
• Entry qualification details
• Engagement history (for
previous years)
• Details to help early tutorials

20. • Data drawn
automatically from
the attendance app
• The wheel shows
overall attendance
• The table below
provides more
detailed information
for recent weeks
Attendance
• Springboard for
conversation
(potential context of
School attendance
policy)

21. Assessment & Feedback View
• Only show
assessments and
feedback
submitted
through NOW
(the University’s
VLE)
• Shows
assessments and
feedback for
multiple modules
• Better sight of student performance than only seeing own module results so
tutor can make more informed recommendations

22. Challenges of converting student information to
actionable intelligence

23. How are we using the Dashboard to address
disparities in attainment?
• Two change agents
• Developing learning analytics is challenging
• Institutional change based on learning analytics may be a different
order of magnitude
– We already have data on students at risk, attendance, non-submission of
coursework
– Yet it is often extremely difficult to change student outcomes
Students Staff

24. Staff as Change Agents
• Personal tutor
– Induction activity
– Primary contact for alerts
– Ongoing support
– iPad trial
• Referrals to support services
– Notes and referrals function
• Staff communication
– Briefings and drop-in sessions
– Newsletter term 1 – low engagement
– Newsletter term 2 – alerts

25. Students as change agents
• Dashboard as resource to inform students about their engagement
• Dashboard focuses positively on engagement rather than risk
• Challenges around Success for All
• Focus on communication and development of tool in-line with
student views

26. Feedback from the Student Transition Survey
• The transition survey was conducted with first year students (Feb/March 2016)
• 91% reported using the Dashboard at least once (90% in 2015)
Have you logged into the NTU student Dashboard?
When using the Dashboard, how often have you explored the following?
Base: 515 (2016), 469 (2015)
4%
5%
14%
15%
13%
24%
5%
6%
26%
24%
35%
36%
12%
13%
33%
34%
34%
36%
79%
77%
28%
28%
19%
5%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Spoke to someone providing specialist help (for
example student support services/ library) as a result
of looking at information on the Dashboard
Spoke to your tutor as a result of looking at
information on the Dashboard
Changed your behaviour to raise or maintain your
engagement score (for example made sure that you
swiped to go into a building)
Compared your engagement score with other students
on your course
Increased the amount of time you spend studying
Checked your own engagement score
Very Often Often Sometimes Never

27. Relationship between number of Dashboard
log-ins and engagement classification

28. Thank you for listening.
Any questions?