Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Talis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong

292 views

Published on

Published in: Education
  • Login to see the comments

  • Be the first to like this

Talis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong

  1. 1. Finding your way through the mist: Analytics in Learning and Teaching Simon Bedford
  2. 2. UOW @ a glance University of Wollongong, an institution ranking among the top 2% of universities in the world, with an enviable record in teaching and research.
  3. 3. Drive for change @UOW I. Curriculum Transformation Process – 2015 to 2018 1. FYE@UOW 2. Capstones@UOW 3. MyPortfolio@UOW 4. Connections@UOW 5. Hybrid Learning@UOW II. TEL Strategy (DLT’s) & Assessment & Feedback Principles III. 2017-18 TEQSA Re-Registration: • New Higher Education Standards Framework -2017 • Teaching & Assessment Policy Suite (TAPS) -2016 • Assessment Quality Cycle and External Referencing of Standards
  4. 4. HESF TAPS Policy T&A Practice TEQSA Government Institution IMPACT CTP A&FP TEL/DLT Put changes into practice and measure it?
  5. 5. Learning Analytics @ UOW • Motivation is to assist with: – Student retention – Personalising student learning – Continuous improvement of teaching & learning • Narrowed focus 2015@UOW – Near real-time delivery of information…. – …to teachers and students. – Maximising the student learning experience “learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.”
  6. 6. Learning Analytics @ UOW Data Mining Impact of CTP A&FP TEL/DLT Student Learning Inputs Subject Level Course Level
  7. 7. Learning Analytics@UOW
  8. 8. Maximising the teaching and learning opportunities for higher education students – a learning analytics case study within the sciences
  9. 9. Science Medicine & Health – Case Study 1. Chemistry Enabling Science: 5 YR1 subjects, <1000 students 2. Curriculum Transformation (@FYE) 3. Focus was student retention and interventions 4. LA Reports – Week 3, 6, 9, 12, Post Declaration of results 5. Meeting with Subject Coordinator & LA team 6. Interpretation of data – and modification of the model 7. List of actions for the next report. • Bringing together multiple data sources to provide a more holistic picture of student resource utilisation and performance • Analytical insights can inform more tailored student communications; • Caution required when interpreting data to avoid making assumptions; • Learning analytics can serve as a catalyst for deeper understanding of students learning and support needs; • Improvements to data quality (e.g. attendance records) that informs evidence based decision making Multiple dimensions to learning analytics at UOW
  10. 10. SMP – Data Source • Collected by teaching staff on SSHEETS • Amalgamations of marks • Not in real time – completed at the end • Data of little use for interventions • Lacks other inputs e.g. FA or Attendance
  11. 11. Moodle – Data Source • Academic, professional, PT staff added to subject site • Staff Dev to input data into grade book and on time • All activity tracked – e.g attendance, formative & summative, Number of logins, time on site etc.Moodle Logs etc Moodle Data+ Library Data + PASS, + SOLS, Etc… = Student Activity
  12. 12. Outcomes - Early Interventions Students Week 3 Week 6 Students No Moodle?No PASS? Post Intervention
  13. 13. Outcomes – c/w other subjects Science Students Week 6 Week 6 Law Students Predicted FINAL GRADE 78 D 79 D 85 HD 68 C 77 D 86 HD 97 HD 65 C 86 HD 78 D
  14. 14. Outcomes – Detail Report Week 9 Students Week 12 Students 1. Doing FA/Feedback = Did better SA 2. HD  C Drop Off (lots reasons e.g Biology) HD C
  15. 15. Outcomes - Predictors Post-Declaration Students
  16. 16. Outcomes - Predictors >5 PASS Sessions (FA&FFB)1-5 PASS Sessions
  17. 17. Data to Students - LA Dashboard Impact on motivation and moderation of assessment?
  18. 18. Case Study - Conclusions 1. Not all data is useful data – e.g SOLS data not broken down 2. Academic Considerations: Causes data fluctuations 3. No yet able to “see” across all subjects taken in a semester …. 4. … and need to have coordinated approach for interventions. 5. To interpret models you need LA and Subject Specialists. But overall LA has given us a far greater understanding of what students are engaging in as they move through our subjects – and this will be of value in measuring the impact of curriculum transformation in the future.
  19. 19. Data driven decision making for quality assurance purposes “…if you measure something you change it..” Heisenberg's uncertainty principle
  20. 20. Data for Continual Improvement Focused data for Quality Enhancement: 1. Assessment board meetings 2. Subject Evaluation Reports (SC/HoS/ADE) 3. Course annual health check (APD) 4. Course comprehensive reviews (5Yrs) I. External Referencing of Student Attainment to comparable courses of study II. and benchmarking (attrition, retention, pass rates) Data for Review of Teaching (DaRT) Assessment Quality Cycle (AQC)
  21. 21. Subject Results – Current Session - Wollongong Campus Comparison* *The results presented are the latest set of results for each campus / delivery mode for which the subject is taught that has occurred within the last 12 months. 12% 9% 4% 9% 3% 7% 17% 6% 15% 7% 26% 3% 18% 11% 9% 3% 15% 7% 11% 6% 6% 15% 11% 15% 6% 19% 14% 3% 15% 15% 24% 6% 4% 9% 6% 4% 12% 3% 7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% On-Campus Distance On-Campus Bega On-Campus PSB Singapore IPC WD WS WH TF F PS P C D HD Average Mark 62.23 Median Mark 65.35 Highest Mark 92.00 Lowest Mark 30.56 Standard Deviation 15.89 Passed 85% Wollongong Assessment Committee Report SUBJ123; Wollongong, Autumn 2016 – DD/MM/YYYY Student Count: 200 320 280 240 Last Day of Session: 30 June 16 30 June 16 30 June 16 1 Sept 15 3 | P a g e 5% 15% 2% 10% 15% 20% 15% 35% 20% 30% 28% 25% 30% 28% 45% 6% 6% 14% 20% 7% 8% 5% 9% 2% 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Wollongong SIM Regional Campuses IRI Hong Kong SubjectMean GradeDistribution HD D C P PS F TF Mean Student Outcomes – International & Domestic – Grade Distribution Subject Student Type Campus Student Count HD% D% C% P% PS% F% TF% SUBJ123 Domestic Wollongong 3 66.7 33.3 SUBJ123 Inton Wollongong 7 28.6 57.1 14.3 SUBJ123 Intoff Singapore SUBJ123 Total 10 40.0 40.0 20.0 School Total 722 8.7 24.9 35.2 24.2 0.3 6.5 0.1 Faculty Total 9572 8.4 26.5 34.2 24.3 0.9 4.9 0.8 University Total 47057 8.8 24.4 31.2 23.6 1.3 8.2 2.6 Student Outcomes – Comparison by Location Wollongong UOW Singapore Onshore Centres IRI Hong Kong Count Mean Count Mean Count Mean Count Mean SUBJ123 400 62.4 30 73.5 100 72.3 36 69.3 School 3060 67.2 94 71.2 1487 69.7 203 66.8 Faculty 7785 68.0 94 71.2 1490 69.8 203 66.8 University 53259 66.9 3038 63.5 3370 71.1 203 66.8 SUB123,68.4 SUB123,67.3 School,69.5 School,70.3 Faculty,70.2 Faculty,75.2 UOW,72.4 UOW,72.5 Domestic International 62 64 66 68 70 72 74 76 Average Student Results 31% 7% 14% 13% 23% 14% 5% 20% 15% 14% 23% 30% 8% 21% 27% 10% 8% 14% 23% 13% 15% 29% 9% 13% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Assessment 1 Assessment 2 Assessment 3 Assessment 4 Student Outcomes - Across Assessments HD D C P PS F Assessments that assure learning outcomes Student Type Campus Location
  22. 22. Subject Results – Current Session - Wollongong Campus Comparison* *The results presented are the latest set of results for each campus / delivery mode for which the subject is taught that has occurred within the last 12 months. 12% 9% 4% 9% 3% 7% 17% 6% 15% 7% 26% 3% 18% 11% 9% 3% 15% 7% 11% 6% 6% 15% 11% 15% 6% 19% 14% 3% 15% 15% 24% 6% 4% 9% 6% 4% 12% 3% 7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% On-Campus Distance On-Campus Bega On-Campus PSB Singapore IPC WD WS WH TF F PS P C D HD Average Mark 62.23 Median Mark 65.35 Highest Mark 92.00 Lowest Mark 30.56 Standard Deviation 15.89 Passed 85% Wollongong Assessment Committee Report SUBJ123; Wollongong, Autumn 2016 – DD/MM/YYYY Student Count: 200 320 280 240 Last Day of Session: 30 June 16 30 June 16 30 June 16 1 Sept 15
  23. 23. 1 | P a g e This report provides data on student demographics and comparative student outcomes (CSO) for your subject. It is designed to support a self-evaluation and subject-level quality enhancement process. Student Profile 2013 2014 2015 Student Demographics Studying in the Faculty 73% 72% 71% Sex (% Female) 56% 57% 55% Residence (% Illawarra) 60% 62% 61% Domestic Student % 94% 80% 72% Average Age 20.8 19.5 19.5 Average EFTSL 0.8 0.8 0.8 Credits Completed 0 35 28 52 1-48 578 465 420 48-96 21 52 62 96+ 9 16 24 Students Repeating the Subjects Yes 14 10 5 No 629 521 642 Course Enrolments Bachelor of Commerce 250 220 260 Bachelor of Communication and Media Studies 100 150 135 Bachelor of Commerce (Dean’s Scholar) 50 10 60 Bachelor of Arts 20 30 25 Bachelor of Business 5 0 10 Bachelor of Engineering (Honours) 2 25 15 Bachelor of Social Sciences 1 0 0 Yearly Subject Evaluation Report Prototype - SUBJ123 2015 Historical Student Profile Historical Subject Mark Historical % Fails
  24. 24. Student Outcomes - Entry Level Session Enrolments Prior to Census Withdrawn at Census % Change 2015 – Autumn 219 4 -1.82% 2015 – Summer 20 5 -25.00% 2014 – Autumn 250 12 -4.80% 2013 – Autumn 240 18 -7.50% 2013 – Summer 60 10 -16.67% WAM Student Count Average Mark <50 35 45.9 50 to 65 231 58.9 65 to 75 203 67.8 75 to 85 128 82.3 85+ 25 87.1 WAM Unknown 21 65.2 ATAR Student Count Average Mark <50 7 52.4 50 to 64 27 65.8 65 to 74 108 63.2 75 to 84 155 72.6 85+ 137 84.2 UAC Unknown 32 62.5 Direct Entry 177 74.8 IELTS Student Count Average Mark <5.0 7 52.4 5-6 27 65.8 6.5 137 84.2 7 2 62.5 7.5 8 74.8 8+ 5 84.2 Required part of my program 400 Relevant to my career 50 Fitted my personal timetable 20 The reputation of the subject 40 Seemed an interesting subject to do 65 Only subject available 10 Subject 2015 2011 2008 SUBJ123 2.5 1.8 1.7 School Total 2.7 2.7 2.6 Faculty Total 2.6 2.7 2.7 University Total 2.8 2.9 2.7 Reason for Taking the Subject Average Satisfaction Student Feedback – Subject Evaluation Survey Results as at DD/MM/YYYY Please note that this feedback represents the data that was collected the last time the Subject was surveyed. 240 220 215 210 210 205 0 0 0 0 0 20 0 100 200 300 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec StudentEnrolments Subject Enrolment Headcount 2013 2014 2015 Subject Headcount Subject Survey Data
  25. 25. Combined Subjects at Risk Data for ADE/HoS/APD (Faculty of Business) Criteria / Weighting: Enrolments Repeating Students Student Performance Student Type Location Student Satisfaction
  26. 26. Questions

×