The NTU Student Dashboard: Implementing a whole institution learning analytics platform to improve student engagement.

1. The NTU Student Dashboard: Implementing a whole institution learning
analytics platform to improve student engagement
Workshop delivered in partnership by the
ABLE Project 2015-1-BE-EPPKA3-PI-FORWARD
STELA Project - 2015-1-UK01-KA203-013767

2. Thinking about Learning Analytics
• Why did NTU develop the Student Dashboard?
• What is it & how does it work?
• How have we managed the development of the resource?
• And has it led to transformational change to the experience of all
students and staff yet?

3. Why was NTU interested in the Dashboard?

4. Why was NTU interested in the Dashboard?
Attainment
Belonging
Progression
• “What Works? Retention & Success”
• Internal audit
• IS department talking to the sector

5. The NTU Student Dashboard
• The Dashboard monitors students’ engagement with their course
– Door swipes, Library use, log ins to the VLE & submission through VLE Drop box
– In 2015-16 added – electronic resources & attendance
• It compares this data to a profile from previous years and assigns
the five ratings:
– High
– Good
– Partial
– Low
– Not fully enrolled (for students who never completed enrolment at the start of
the year, or who have withdrawn)
• Students and staff can see exactly the same view
– Staff have a few additional management screens
• Tutors can also make notes in the notes section
• Staff are also sent an alert if there is no engagement for two weeks
during term time

6. What is it and how does it work?

7. Tutorial Group View

8. Student 1
“High” Engagement
Student 2
“Low” Engagement

9. Student 1
“High” Engagement
Student 2
“Low” Engagement

10. Developmental Journey?

11. Developing the Dashboard
• In the early phases the Dashboard build was an agile process
between a relatively small team of NTU staff and our external
provider DTP Solutionpath (2013-14)
• Regular meetings, sharp exchange of ideas, quick change
• Experts in learning & teaching, student progression and attainment,
equal opportunities & representatives from each of the three Pilot
Schools
• Quick consultation and approval
• Now morphed into a more mature Governance structure

12. Dashboard Governance Group
Dashboard Operations Group
Informal student
group
Ethics Group University Systems Group
Academic Standards & Quality
Committee
University Governance &
Management
Academics
Students
Educational developers
Student Support
Student Planning
Information Systems

13. Is it accurate, has it changed the world yet?

14. Average engagement 2013-14 for all students
(23rd September 2013 – 6th June 2014)
• Every day including weekends and holidays, therefore created a lower
overall average engagement
• Part-time students were more likely to have low engagement
(methodology revised in 2015-16 for this group)

15. Relationship between average engagement & progression
1st year students only 23rd Sept 2013 – 6th June 2014
Low engagement Satisfactory engagement Good engagement High engagement
Other (n=76) 3.1% 1.2% 0.7% 0.2%
Withdrawn (n=310) 18.5% 4.0% 2.8% 4.0%
Transfer (n=143) 5.3% 2.2% 1.1% 1.3%
Repeating (n=364) 12.4% 6.2% 2.0% 2.5%
Academic failure (n=281) 36.5% 3.1% 0.8% 0.4%
Progressed (n=5,836) 24.2% 83.4% 92.5% 91.7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Progression status by engagement rating (year 1 full time UG students)

17. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
low
satisfactory
good
high
Engagement classification
low satisfactory good high
1st Class Honours 13.2% 15.2% 20.9% 28.0%
2nd Class Honours-1st Division 28.6% 47.0% 55.2% 52.5%
2nd Class Honours-2nd Division 28.6% 30.5% 21.5% 17.0%
3rd Class Honours 11.0% 4.9% 1.4% 1.8%
Other - Ordinary Degree 18.7% 2.4% 1.1% 0.6%
Final degree awards by engagement classification
Impact of average engagement on attainment
(Final year students Sept 2013-June 2014)

18. Student Feedback
In 2014, 27% of
students reported
changing their
behaviour in response
to seeing their
engagement in the
Dashboard

19. Student Feedback

20. Yes about 1-2
times
Yes about
monthly
Yes about weekly
Yes whenever I’ve
had an alert email
very useful 2 1 11 4
quite useful 7 26 27 8
undecided 13 11 1 4
Not very useful 7 9 2 2
Not at all useful 1 2 2
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AxisTitle
Relationship between Dashboard use and
perceptions of usefulness (n=140) Survey
conducted summer 2015

21. Lessons from the Dashboard Implementation

22. The nub of the issue for me
• Learning analytics is only as useful as the actions it instigates
For the academic to
jump into the bath
tub, they need
time, training,
motivation, to
have easy access
to data, a room to
talk to the student
or space to email
them.
Just knowing that
they need to jump
is only the first link
in the chain

23. IT infrastructure, management processes & quality
New data sources for the learning analytics model
Further Dashboard developments: NECs, design, use of notes etc.
Student communications
Integrating Dashboard into institutional working practices
2016-17 – trial different aspects of the Dashboard with staff.
The following are under discussion:
- Alerts
- Use of notes pages
- Appointment booking tools
- Reporting tools
- Staff development particularly use in tutorials
- Survey tools & tests
- Links with professional services (CMS type approaches)
Stuff we haven’t considered yet