The NTU Student Dashboard: Developing a learning analytics platform to improve student retention and attainment

1. The NTU Student Dashboard: developing a learning analytics platform to
improve student retention and attainment.
Workshop delivered in partnership with Success for All and the ABLE Project.
Sarah Lawther and Rebecca Edwards

2. Session outline
• Why did NTU develop the Student Dashboard?
• What is it & how does it work?
• How have we managed the development of the resource?
• How does it inform our work to address differences in attainment?
• What are our plans for next phase?

3. Why did NTU develop the Student Dashboard?

4. Student Retention & Success
• Retention
– Internal Audit Report into Student Retention (2012)
• HERE Project – retention and success (Foster et al 2011)
• Identifying ‘at risk’ students (Thomas and Jones 2012), (Quinn et al,
2012)
• What Works – belonging and ‘feeling known’ (Thomas 2012)
• Disparities in Student Attainment (DiSA) (Cousin and Cureton, 2012)
• Across the sector
– Retrospective data – how can we support student retention and success?
– Real time – how can we best support students? (JISC Inform), (Siemens, Dawson,
Lynch, 2013) (Mutton and MacNeill, 2013)

5. What is it and how does it work?

6. The NTU Student Dashboard
• The Dashboard monitors students’ engagement with their course
– Door swipes, Library use, log ins to the VLE & submission through VLE Drop box
– In 2015-16 added – electronic resources & attendance
• It compares this data to a profile from previous years and assigns
the five ratings:
– High
– Good
– Partial
– Low
– Not fully enrolled (for students who never completed enrolment at the start of
the year, or who have withdrawn)
• Students and staff can see exactly the same view
– Staff have a few additional management screens
• Tutors can also make notes in the notes section
• Staff are also sent an alert if there is no engagement for two weeks
during term time

7. What does the student view look like?

9. Opens on click Opens on click
• Scroll down landing page to
find more information.
• Dashboard designed to be
simple at first glance, with the
option to click for extra detail.
• Built in partnership with
Solutionpath.

10. Student profile

11. Help Page

12. What does the staff view look like?
Opens link

13. Links to
student’s
dashboard
Able to sort on headings
Search function
• Designed so staff have easy access to student data.
• Allows staff to quickly identify potentially at risk students

14. Tutorial Group View
Links to group
of students in
engagement
category

15. Any thoughts or questions so far?

16. How have we managed the development of the
resource?

17. NTU Dashboard Pilot 2013-14
• Pilot Aims
– Can the dashboard identify students most at risk of non
progression?
– Can staff and students use the resource?
• Pilot
– Dashboard developed by DTP Solution Path
– Schools of Business, Education and Social Sciences
– Designed around the NTU academic tutor groups
– Four first year courses, over 500 students
– Training
– Student live test

18. Staff and student research 2013
Student Transition Survey (March-May 2013)
• Both doubters (62%) and non-doubters (70%) felt an email from their tutor
inviting them for a meeting would be the most useful support
Staff online survey and interviews (November 2013)
• Variation in use by tutors
• Changed interactions with students in tutor groups
– As I knew which students were not engaging with their …I could contact
them/follow this up. It was useful to have an email prompt re what to do,
and information about how to contact the students. This felt quite
empowering.
• Improvements to the Dashboard
– Link it to attendance
– General work on the glitches
– Lack of information relating to admissions data

19. Implementation (2014 onwards)
• September 2014 – all Schools for undergraduate and postgraduate
students
• Staff briefings and guidance
– Introduction to learning analytics
– Ethics
– Explore good practice

20. Student research 2014 - 15
Student focus groups (2014)
• Familiar with being monitored
• Transparency - “I want to be able to see what is collected on me so I know
what they can see about me”
Student Transition survey (February - March 2015)
• Is there anything NTU could do to encourage you to log into the Dashboard
more frequently?
• Have you done anything differently as a result of using the Dashboard?
– Changing behaviours including those that aren’t monitored
• How could NTU improve the Dashboard?

21. Student consultation
• Student Union
• IS Student Consultation Group
• Student research – surveys, focus groups
• Sept 2015-16 Student Dashboard Marketing Assistant
– Student engagement panel
– Ethics group
– Student introduction to the NTU Dashboard

22. Dashboard Governance Group
Dashboard Operations Group
Informal student
group
Ethics Group University Systems Group
Academic Standards & Quality
Committee
University Governance &
Management
Academics
Students
Educational developers
Student Support
Student Planning
Information Systems

23. Student communication

24. How does it inform our work to address
differences in attainment?

25. Relationship between average engagement & progression
1st year students only 23rd Sept 2013 – 6th June 2014*
Low engagement Satisfactory engagement Good engagement High engagement
Other (n=76) 3.1% 1.2% 0.7% 0.2%
Withdrawn (n=310) 18.5% 4.0% 2.8% 4.0%
Transfer (n=143) 5.3% 2.2% 1.1% 1.3%
Repeating (n=364) 12.4% 6.2% 2.0% 2.5%
Academic failure (n=281) 36.5% 3.1% 0.8% 0.4%
Progressed (n=5,836) 24.2% 83.4% 92.5% 91.7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Progressionstatus by engagement rating (year 1 full time UG students)
* Data provided by Mike Kerrigan, NTU

26. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
low
satisfactory
good
high
Engagementclassification
low satisfactory good high
1st Class Honours 13.2% 15.2% 20.9% 28.0%
2nd Class Honours-1st Division 28.6% 47.0% 55.2% 52.5%
2nd Class Honours-2nd Division 28.6% 30.5% 21.5% 17.0%
3rd Class Honours 11.0% 4.9% 1.4% 1.8%
Other - Ordinary Degree 18.7% 2.4% 1.1% 0.6%
Final degree awardsby engagement classification
Relationship between engagement and attainment
(Final year UG students 2013/14)

27. Implications of research findings
• Low engagement as recorded by the dashboard
correctly identifies students most at risk of
– Withdrawing from study
– Academic failure
– Achieving inferior degree classification
• And low engagers are disproportionately
– Male
– BME
– WP
– BTEC entrants
• Hence, we can target student behaviours, rather
than student characteristics

28. Informing student success provision
• Success for All
– 9 pilot projects developed in 2014/15
– Action research extended to 2015/16 and beyond
– Establish ‘what works’ to address differences in
attainment
• BTEC Champions
– University wide initiative
– Identify, plot and evaluate interventions aimed at
supporting BTEC entrants
– Closely associated with ‘Success for All’
– BTEC Symposium 15 September 2016

29. What are our plans for next phase?

30. Ongoing tasks related to Dashboard
Promotion to
students
Staff training
IT
infrastructure
Management
Processes
and quality
Data analysis

31. Development of the Dashboard based on
feedback
• The Dashboard will be adapted
based on student and staff feedback
- Improve accessibility:
- Improve layout
- Make app version
- Give attendance data higher visibility
- Add to entry qualification reporting
- Build in use of new data sources into
algorithm
- Improve system for students with
non-standard start dates
- Improve mapping of students to
tutors

32. 2016/2017 - Integrating Dashboard into
institutional working practices
• Trial different aspects of the Dashboard with staff from different
schools.
• The following are under discussion:
- Pre-arrival task
- Personal goal setting
- Alerts
- Use of notes pages
- Appointment booking tools
- Monitoring of low engagement
- Reporting tools

33. ABLE Project – Achieving Benefits from
LEarning analytics
Erasmus+ KA2 funded project
ABLE Project Code: 2015-1-UK01-KA203-013767
The ABLE Project will identify, research, and recommend successful
interventions once learning analytics have identified that a university
student may be at risk of early departure or underperforming. Learning
analytics is only as useful as the actions it instigates.
Project partners:
Project website coming soon (August 2016) - www.ABLEProject.eu
Twitter page - @ABLEproject_eu
Find out more:

34. Thank you for listening, any questions?
Sarah Lawther – Success for All
(sarah.lawther@ntu.ac.uk)
Rebecca Edwards – ABLE Project/ Student Dashboard
(rebecca.edwards@ntu.ac.uk)

35. References
• Foster, E., Lawther, S., Keenan, C., Bates, N., Colley, B. and Lefever, R., 2011. The HERE Project Final
Report [online]. Available at: http://www.heacademy.ac.uk/resources/detail/what-works-student-
retention/HERE_Project_What_Works_Final_Report [Accessed 5 March 2014]
• Cousin, G., and Cureton, D., 2012. Disparities in Student Attainment (DiSA). Available at:
https://www.heacademy.ac.uk/sites/default/files/projects/worlverhampton_2010_disa_final_report.pdf
• JISC Inform. How can I use analytics to benefit my students? [online]. Available at:
http://www.jisc.ac.uk/inform/inform36/mobile/QandA.html [Accessed 5 March 2014]
• Mutton, J., and MacNeill, S., 2013. JISC CETIS Analytics Series, Vol. 2, no.1, Case Study: Engaging with
Analytics.
• Quinn, Carole, Bennett, Joanna, Clarke, John A., & Nelson, Karen J., 2012. The evolution of QUT’s
Student Success Program: 20,000 students later, Nuts & Bolts In International First Year in Higher
Education Conference, 26-29 June, 2012, Sofitel Brisbane Central, Brisbane [online]. Available at:
http://eprints.qut.edu.au/53695/2/15A.pdf
• Siemens, G., Dawson, G., Lynch, G., 2013. Improving the Quality and Productivity of the Higher
Education Sector Policy and Strategy for Systems-Level Deployment of Learning Analytics Society for
Learning Analytics Research.
• Thomas and Jones, 2012. Using Data An evidence based approach to improving transition, induction
and retention [online]. Available at: http://www.hestem.ac.uk/sites/default/files/6068_he-
stem_data_evaluation_aw_low_res.pdf [Accessed 5 March 2014]
• Thomas, L., 2012. Building student engagement and belonging in Higher Education at a time of change:
final report from the What Works? Student Retention & Success programme [online]. Available at:
http://www.heacademy.ac.uk/assets/documents/retention/What_works_final_report.pdf