Download as PDF, PPTX
Uploaded byEd Foster
ABLE - NTU Danish visit February 2018
This document provides information about Nottingham Trent University's use of learning analytics and their student dashboard. It discusses: - An overview of the student dashboard which was first piloted in 2013-2014 and fully implemented in 2014-2015. It tracks student engagement using data from various systems. - Research that has shown a strong association between engagement measured in the dashboard and student success. The university is also researching learning analytics through various EU projects. - How both students and staff utilize the dashboard. Students check their engagement and attendance, while staff use it to prepare for tutorials and identify students who may need additional support. - Feedback from students which found the dashboard useful, especially when discussed during tutorials. The university
More Related Content
Similar to ABLE - NTU Danish visit February 2018
ABLE - NTU Danish visit February 2018
- 1. Welcome to Nottingham Trent University David Woolley Headof Schools, Colleges and Community Outreach
- 2.
- 3. • A 21stCentury University o 8 Schools o 2 Further Education providers o 29,000 students o Activity in most major subject areas Nottingham Trent University
- 4. Animal, Rural & Environmental Sciences Architecture,Design & the Built Environment Art & Design Nottingham Business School Nottingham Law School Social Sciences Arts & Humanities Science & Technology Confetti
- 5. • A successfuluniversity. Award-winning expertise in: o Data o Pedagogy o Research Nottingham Trent University
- 6. Creating Opportunity Valuing Ideas EnrichingSociety Connecting Globally Empowering People
- 7. Introduction to SCCO Outreach Teams Volunteering Team CharacterSkills & ‘capitals’ Attainment Schools Liaison Team UG Peer Mentoring Data Research and evaluation Pedagogy Student Engagement Team
- 8. Learning Analytics atNottingham Trent University Ed Foster, Student Engagement Manager, SCCO ABLE Project 2015-1-BE-EPPKA3-PI-FORWARD STELA Project: 562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
- 9. Learning Analytics atNTU • Student Dashboard pilot with Solutionpath (2013-14) • Whole institutional implementation (2014-15) • Last year 28,486 students logged in, as did 2,597 staff • We can show a strong association between engagement measured in the Dashboard and student success • We are actively researching learning analytics in the first year through the ABLE & STELA Erasmus+ Projects (2015- 2018) • We are actively looking for an institutional partner for future Erasmus+ research into learning analytics-driven interventions (2018-2021)
- 10. NTU Student Dashboard Improvinginstitutional data & systems • Cohort insight • Improve University systems • Informing future plans/ strategy Promoting student success • Progression • GPA/ degree attainment • Targeted information for students & staff Improving staff-student working relationships • Engagement information • Info for personalised tutorial discussions Supporting students to manage their own learning • Info to promote reflection • Benchmarking • Developing goal setting Focus on engagement, not risk of failure Specifically, a proxy for engagement using students’ electronic footprint Does not measure socio-economic disadvantage Born out of research into student retention: ‘What Works’
- 11. Student Engagement Student background Socio-economicbackground, expectations of university and personal goals University Environment Transition support, supportive and inspiring teaching, sociability of the course, opportunities to pursue personal interests & participate in C&S etc., clarity of expectations, quality of feedback, course that meets student expectations, provision of further support Student Engagement Personal mission, motivation, time on task, knowing how to learn, learning from others, managing own time, learning from feedback, seeking help where needed, healthy control of own stress and emotions, balancing study, social and family commitments x = Student Outcome pass, fail, repeat, graduate employment etc. Prior educational experience Relative success/ failure +
- 12. Dashboard StaffStudent Metrics Raw data& engagement rating Student engagement with course Metrics & alerts Engagement with students presented to students students act presented to tutors more-informed interactions Fitting the Dashboard into the institutional ecosystem Two agents of change model
- 13. Core Dashboard Process Student data Engagement withlearning Engagement score, Risk alerts Referrals Notes, Sense checking, Referrals Static/ manual information Algorithmic engagement data Attendance E-resources Card swipes VLE log ins Dropbox submissions Library loans !
- 14. Ethics of usinglearning analytics • Privacy & GDPR • Difference between ‘legitimate interest’ & ‘consent’ • Algorithms are not neutral or bias-free • Ethics of not acting • Limits to support available • Free will & the right to ‘mess up’ • Using activity, not demographics • Assessing performativity • Is this empowering, or stress-inducing? • Popular with ‘winners’ • ’Sophisticated stereotyping’ • Isn’t this just about spying on staff anyway?
- 15.
- 16.
- 17. Average engagement for last 7 days Scalable cumulative graph Scalable weekby week graph Engagement for the past 30 days
- 18. Additional resources • Somepersonal information • To inform tutorial conversations & interventions • Resources used • To help see more detail the elements that make up the algorithm • Notes and actions • To improve interventions & support students • Assessments & feedback • Essential for working with students
- 19. Testing the accuracyof the Dashboard
- 20. 24% 83% 93% 92% 16% 82% 93% 96% 9% 81% 92% 95% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low PartialGood High Percentgaeofstudentsprogressing Mode engagement rating for academic year First year progression based on mode engagement rating for the year (FT, UG students) 2013-14 2014-15 2015-16 Relationship between engagement and progression (2013-14 – 2015-16)
- 21. Relationship between engagement& attainment
- 22. 9% Low av. engagement Wholeyear 9% Low av. engagement 1st term 27% Low av. engagement Welcome Week 64% Time x Low Engagement = Risk
- 23. Relationship between alerts& progression
- 24. How usage haschanged
- 25. Staff Dashboard Use 0 2 4 6 8 10 12 14 16 18 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 2014-152015-16 2016-17 Totalusers(blue),totallogins(orange) Academic year NTU Dashboard Use: Staff users, log ins & average log--ins per user 2014/15 2016/17 Staff users Staff log-ins Average log ins per user
- 26. Student Dashboard use 0 5 10 15 20 25 0 100,000 200,000 300,000 400,000 500,000 600,000 2014-152015-16 2016-17 Totaluers(blue),totallog-ins(orange) Academic year NTU Dashboard Use: Student users, log ins & average log- -ins per user 2014/15 2016/17 Student users Student log-ins Average log ins per user
- 27. 46% 72% 78% 81% 84% 86% 90% 0% 20% 40%60% 80% 100% 0 1 2-3 4-6 7-9 10-19 20+ Percentage of students progressing NoofDashboardLogins Proportion of students progressing to second year by number of Dashboard log-ins (1st year, FT UGs in 2015-16) Relationship between Dashboard use & success
- 28.
- 29. Session 2 Implementing learninganalytics Rebecca Edwards
- 30. How do students usethe Dashboard?
- 31. Student log-ins Timeframe Countof students % of students Pre-term 2,955 35.2% Welcome Week 2,157 25.7% Term 1 2,772 33.0% Christmas holiday 29 0.3% Term 2 136 1.6% Easter holiday 3 0.0% Term 3 25 0.3% Summer holiday 15 0.2% No log-ins 296 3.5% Total 8,388 100.0% • Students are using the Dashboard from the start of their time at NTU
- 32. Student Transition Survey •Since 2012-13, over 2,700 first year students have provided feedback on different aspects of the Dashboard • Conducted Feb/ March each year • Response rate = 5-7% of first year • Provides valuable feedback • We use it to test out ideas for improvements
- 33. Exploring the NTUStudent Dashboard When using the Dashboard, how often have you explored the following? Base: 753 5% 5% 13% 17% 49% 8% 9% 28% 28% 34% 18% 22% 33% 29% 16% 70% 63% 26% 26% 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Spoke to someone providing specialist help (for example student support services/ library) as a result of looking at information on the Dashboard Spoke to your tutor Increased the amount of time you spend studying Changed your behaviour to raise or maintain your engagement score (for example made sure that you swiped to go into a building) Checked your attendance Very Often Often Sometimes Never
- 34. Student perceptions oftutors using the Dashboard • 73% said they have a personal tutor • These students have met with their personal tutors in a variety of ways: • 84% In a small group tutorial • 52% In a one to one tutorial • 50% Teaching in a lecture theatre / classroom • 29% (only 8% in 2016) said their tutors had used the Dashboard during one-to-one meetings with them. • Of those, 80% (83% in 2016) found this useful. Do you have a personal tutor? / In which of these situations have you met your personal tutor? / Has your tutor ever used the Dashboard during one-to-one meetings with you? / Did you find it useful? Base: 753
- 35. Student Reactions toDashboard • 64% found the Dashboard to be ‘useful’ or ’very useful’ • Students who found it useful were: • More likely to be enjoying being a student (88% v 81%) • More engaged with their studies (73% v 66%) • More confident about coping with their studies (61% v 54%) • Less likely to have encountered an academic problem (64% v 69%) • Equally likely to have considered leaving (both 27%) • Students wanted to be told that they were at risk of dropping out (94%) or if we could improve their chances of progression (97%) N=753 1st year students, Feb/Mar 2017, (percentages in brackets indicate answered 4 or 5 out of 5 (positive or v. positive))
- 36. This word cloudwas drawn from repertory grid exercise with student interviewees. Students were asked to choose words categorised as positive or negative, and active or passive. As can be seen, most words selected were positive & active Feedback from focus groups
- 37. How do staffuse the Dashboard?
- 38. Using the Dashboardin tutorials 2. Preparing for tutorials 3. Framing the discussion & checking student understanding 4. Supporting the coaching process 5. Action planning & referrals • Staff identified five key uses: 1. Student ‘health check’
- 39. Challenging students self perceptions Realisationthat tutors have access to information Using comparison with peers to challenge perceptions Early warning of problems Referrals Limitations Using the Dashboard in tutorials • Staff identified ways they felt using the Dashboard had helped change student engagement
- 40. Embedding into practice •Pilot activities 2015/16 – iPad pilot, student induction, referrals • Pilot activities 2016/17 – student alerting, staff notes • Mid-term review Social Sciences 2016/17 and 2017/18 • Mid-term review Business School 2017/18 • Supporting other initiatives: Scale-up, Academic Librarians, GRIT training • Internal events/conferences
- 41. Challenges • Dashboard isused regularly by approx. 40% of students • Association between use and higher engagement • In turn to success • Two challenges • Increase student use • & capacity to benefit from it • Increase staff use • & capacity to bring about change
- 42. Barriers to staffusing the Dashboard • Barriers in tutorials: • Frequency and duration of tutorials • Problems with space and time • Topics covered • Broader barriers • Competing priorities • Changing expectations of students and staff in HE • Concerns about methodology
- 43. Overcoming barriers • Communications •Staff • Student • Staff development • Policy • Product development • Partnership working
- 44. Ongoing/future developments • Increasinglycomplete picture of student journey through HE • New data visualisations (individual and group) • Different forms of alerting • Further sophistication
- 45. Five challenges ofimplementing an institutional learning analytics solution Mission & Governance • Defining strategic goals • Implementing effective governance Data • Interacting with institutional tools • Interacting with users of institutional tools • Fundamental challenges with the use of data, e.g. ethics Product & Process Development • Exposing & coping with assumptions • Designing an appropriate tool Communication • Designing communication • Communicating with audience Implementation • Focus on change • Ongoing management • Delivering change not the same as delivering the resource
- 46.
- 47. Session 3 Working withlearning analytics
- 48. Supporting Student Engagement Schoolof Social Sciences
- 49. School Engagement andAttendance Policy • Purpose • Set out clear expectations for attendance and engagement; • Identify and support students who are struggling to engage with their course of study; • Engender a culture of professionalism and regard for others; • Enable course teams to manage student non- attendance consistently and fairly; • Inform course student progression and achievement reviews
- 50.
- 51. Recommendations for 2017-8Reviews • Adjust the ‘at risk’ parameters to include <50 attendance, ‘partial’ engagement, NECs and students who applied through clearing • Flag students who transferred courses for course induction purposes • Standardise the follow up actions by course administrators • Refine and publicise School based support • Set out clear expectations for personal tutors including Dashboard use. • Roll out the approach across the School
- 52. NTU Library andthe Student Dashboard Heather Shaw, Learning and Teaching Team
- 53. Library Learning andTeaching Team Learning and Teaching Team Manager (1) Learning and Teaching Librarians/Advisor (9) Academic Skills Tutors(2) Maths and Stats Academic Skills Tutors (3) Library Student Mentors (40)
- 54. Who are we– Staff?
- 55. What do wedo? • We provide a range of services to support staff with their learning and teaching and students on taught courses (UG/PG) throughout their studies. • Working with others, we support the development of the curriculum, new pedagogy, resources, induction, employability and transferable life long learning academic skills. • We deliver induction and academic skills support to the Academy and local FE/Schools. • We engage in staff student partnerships to co-develop an agenda for change, drive innovative practices and peer review. • We participate in the development of strategy to engage with learning technologies and provide direct support for developing digital literacy skills and online resources. • We contribute to library strategic planning, projects, library resources and professional development opportunities.
- 56.
- 57.
- 58.
- 60.
- 61. Staff view ofbooking form
- 62.
- 63.
- 64.
- 65.