1. Finding your way through the mist:
Analytics in Learning and Teaching
Simon Bedford

2. UOW @ a glance
University of Wollongong, an
institution ranking among the
top 2% of universities in the
world, with an enviable record
in teaching and research.

3. Drive for change @UOW
I. Curriculum Transformation Process – 2015 to 2018
1. FYE@UOW
2. Capstones@UOW
3. MyPortfolio@UOW
4. Connections@UOW
5. Hybrid Learning@UOW
II. TEL Strategy (DLT’s) & Assessment & Feedback Principles
III. 2017-18 TEQSA Re-Registration:
• New Higher Education Standards Framework -2017
• Teaching & Assessment Policy Suite (TAPS) -2016
• Assessment Quality Cycle and External Referencing of Standards

4. HESF
TAPS
Policy
T&A
Practice
TEQSA
Government Institution
IMPACT
CTP
A&FP
TEL/DLT
Put changes into practice and measure it?

5. Learning Analytics @ UOW
• Motivation is to assist with:
– Student retention
– Personalising student learning
– Continuous improvement of teaching & learning
• Narrowed focus 2015@UOW
– Near real-time delivery of information….
– …to teachers and students.
– Maximising the student learning experience
“learning analytics is the measurement,
collection, analysis and reporting of data
about learners and their contexts, for
purposes of understanding and optimising
learning and the environments in which it
occurs.”

6. Learning Analytics @ UOW
Data Mining
Impact of
CTP
A&FP
TEL/DLT
Student Learning
Inputs
Subject
Level
Course
Level

7. Learning Analytics@UOW

8. Maximising the teaching and learning
opportunities for higher education students
– a learning analytics case study within the
sciences

9. Science Medicine & Health – Case Study
1. Chemistry Enabling Science: 5 YR1 subjects, <1000 students
2. Curriculum Transformation (@FYE)
3. Focus was student retention and interventions
4. LA Reports – Week 3, 6, 9, 12, Post Declaration of results
5. Meeting with Subject Coordinator & LA team
6. Interpretation of data – and modification of the model
7. List of actions for the next report.
• Bringing together multiple data sources to provide a more holistic picture of student resource
utilisation and performance
• Analytical insights can inform more tailored student communications;
• Caution required when interpreting data to avoid making assumptions;
• Learning analytics can serve as a catalyst for deeper understanding of students learning and
support needs;
• Improvements to data quality (e.g. attendance records) that informs evidence based decision
making
Multiple dimensions
to learning analytics
at UOW

10. SMP – Data Source
• Collected by teaching staff on SSHEETS
• Amalgamations of marks
• Not in real time – completed at the end
• Data of little use for interventions
• Lacks other inputs e.g. FA or Attendance

11. Moodle – Data Source
• Academic, professional, PT staff added to subject site
• Staff Dev to input data into grade book and on time
• All activity tracked – e.g attendance, formative & summative,
Number of logins, time on site etc.Moodle Logs etc
Moodle Data+
Library Data +
PASS, + SOLS,
Etc… =
Student Activity

12. Outcomes - Early Interventions
Students
Week 3 Week 6
Students
No Moodle?No PASS?
Post Intervention

13. Outcomes – c/w other subjects
Science Students
Week 6
Week 6
Law Students
Predicted FINAL
GRADE
78 D
79 D
85 HD
68 C
77 D
86 HD
97 HD
65 C
86 HD
78 D

14. Outcomes – Detail Report
Week 9
Students
Week 12
Students
1. Doing FA/Feedback = Did better SA
2. HD  C Drop Off (lots reasons e.g Biology)
HD
C

15. Outcomes - Predictors
Post-Declaration
Students

16. Outcomes - Predictors
>5 PASS Sessions (FA&FFB)1-5 PASS Sessions

17. Data to Students - LA Dashboard
Impact on motivation
and moderation of
assessment?

18. Case Study - Conclusions
1. Not all data is useful data – e.g SOLS data not broken down
2. Academic Considerations: Causes data fluctuations
3. No yet able to “see” across all subjects taken in a semester ….
4. … and need to have coordinated approach for interventions.
5. To interpret models you need LA and Subject Specialists.
But overall LA has given us a far greater understanding of what students are
engaging in as they move through our subjects – and this will be of value in
measuring the impact of curriculum transformation in the future.

19. Data driven decision making for quality
assurance purposes
“…if you measure
something you
change it..”
Heisenberg's
uncertainty principle

20. Data for Continual Improvement
Focused data for Quality Enhancement:
1. Assessment board meetings
2. Subject Evaluation Reports (SC/HoS/ADE)
3. Course annual health check (APD)
4. Course comprehensive reviews (5Yrs)
I. External Referencing of Student Attainment to comparable courses of study
II. and benchmarking (attrition, retention, pass rates)
Data for Review of Teaching (DaRT)
Assessment Quality
Cycle (AQC)

21. Subject Results – Current Session - Wollongong
Campus Comparison*
*The results presented are the latest set of results for each campus / delivery mode for which the subject is taught that has occurred within the last 12 months.
12% 9%
4%
9%
3%
7%
17%
6%
15%
7%
26%
3%
18%
11%
9%
3%
15%
7%
11%
6%
6%
15%
11%
15%
6%
19%
14%
3%
15% 15%
24%
6% 4%
9%
6%
4%
12%
3%
7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
On-Campus Distance On-Campus
Bega
On-Campus
PSB Singapore
IPC
WD
WS
WH
TF
F
PS
P
C
D
HD
Average Mark 62.23 Median Mark 65.35 Highest Mark 92.00 Lowest Mark 30.56 Standard Deviation 15.89 Passed 85%
Wollongong
Assessment Committee Report
SUBJ123; Wollongong, Autumn 2016 – DD/MM/YYYY
Student Count: 200 320 280 240
Last Day of Session: 30 June 16 30 June 16 30 June 16 1 Sept 15
3 | P a g e
5%
15%
2%
10% 15%
20%
15%
35%
20%
30%
28%
25%
30%
28%
45%
6%
6%
14%
20%
7% 8%
5% 9%
2%
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Wollongong SIM Regional
Campuses
IRI Hong Kong
SubjectMean
GradeDistribution
HD D C P PS F TF Mean
Student Outcomes – International & Domestic – Grade Distribution
Subject
Student
Type
Campus
Student
Count
HD% D% C% P% PS% F% TF%
SUBJ123 Domestic Wollongong 3 66.7 33.3
SUBJ123 Inton Wollongong 7 28.6 57.1 14.3
SUBJ123 Intoff Singapore
SUBJ123 Total 10 40.0 40.0 20.0
School Total 722 8.7 24.9 35.2 24.2 0.3 6.5 0.1
Faculty Total 9572 8.4 26.5 34.2 24.3 0.9 4.9 0.8
University Total 47057 8.8 24.4 31.2 23.6 1.3 8.2 2.6
Student Outcomes – Comparison by Location
Wollongong UOW Singapore Onshore Centres IRI Hong Kong
Count Mean Count Mean Count Mean Count Mean
SUBJ123 400 62.4 30 73.5 100 72.3 36 69.3
School 3060 67.2 94 71.2 1487 69.7 203 66.8
Faculty 7785 68.0 94 71.2 1490 69.8 203 66.8
University 53259 66.9 3038 63.5 3370 71.1 203 66.8
SUB123,68.4
SUB123,67.3
School,69.5
School,70.3
Faculty,70.2
Faculty,75.2
UOW,72.4
UOW,72.5
Domestic International
62
64
66
68
70
72
74
76
Average Student Results
31%
7%
14% 13%
23%
14% 5%
20%
15%
14% 23%
30%
8%
21%
27%
10%
8%
14%
23% 13%
15%
29%
9% 13%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Assessment 1 Assessment 2 Assessment 3 Assessment 4
Student Outcomes - Across Assessments
HD D C P PS F
Assessments that
assure learning
outcomes
Student Type
Campus
Location

22. Subject Results – Current Session - Wollongong
Campus Comparison*
*The results presented are the latest set of results for each campus / delivery mode for which the subject is taught that has occurred within the last 12 months.
12% 9%
4%
9%
3%
7%
17%
6%
15%
7%
26%
3%
18%
11%
9%
3%
15%
7%
11%
6%
6%
15%
11%
15%
6%
19%
14%
3%
15% 15%
24%
6% 4%
9%
6%
4%
12%
3%
7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
On-Campus Distance On-Campus
Bega
On-Campus
PSB Singapore
IPC
WD
WS
WH
TF
F
PS
P
C
D
HD
Average Mark 62.23 Median Mark 65.35 Highest Mark 92.00 Lowest Mark 30.56 Standard Deviation 15.89 Passed 85%
Wollongong
Assessment Committee Report
SUBJ123; Wollongong, Autumn 2016 – DD/MM/YYYY
Student Count: 200 320 280 240
Last Day of Session: 30 June 16 30 June 16 30 June 16 1 Sept 15

23. 1 | P a g e
This report provides data on student demographics and comparative student outcomes (CSO) for your subject. It is designed to support a self-evaluation and
subject-level quality enhancement process.
Student Profile
2013 2014 2015
Student Demographics
Studying in the Faculty 73% 72% 71%
Sex (% Female) 56% 57% 55%
Residence (% Illawarra) 60% 62% 61%
Domestic Student % 94% 80% 72%
Average Age 20.8 19.5 19.5
Average EFTSL 0.8 0.8 0.8
Credits Completed
0 35 28 52
1-48 578 465 420
48-96 21 52 62
96+ 9 16 24
Students Repeating the Subjects
Yes 14 10 5
No 629 521 642
Course Enrolments
Bachelor of Commerce 250 220 260
Bachelor of Communication and Media Studies 100 150 135
Bachelor of Commerce (Dean’s Scholar) 50 10 60
Bachelor of Arts 20 30 25
Bachelor of Business 5 0 10
Bachelor of Engineering (Honours) 2 25 15
Bachelor of Social Sciences 1 0 0
Yearly Subject Evaluation Report Prototype - SUBJ123 2015
Historical Student Profile
Historical Subject Mark
Historical % Fails

24. Student Outcomes - Entry Level
Session Enrolments
Prior to Census
Withdrawn
at Census
% Change
2015 – Autumn 219 4 -1.82%
2015 – Summer 20 5 -25.00%
2014 – Autumn 250 12 -4.80%
2013 – Autumn 240 18 -7.50%
2013 – Summer 60 10 -16.67%
WAM Student Count Average Mark
<50 35 45.9
50 to 65 231 58.9
65 to 75 203 67.8
75 to 85 128 82.3
85+ 25 87.1
WAM Unknown 21 65.2
ATAR Student Count Average Mark
<50 7 52.4
50 to 64 27 65.8
65 to 74 108 63.2
75 to 84 155 72.6
85+ 137 84.2
UAC Unknown 32 62.5
Direct Entry 177 74.8
IELTS Student Count Average Mark
<5.0 7 52.4
5-6 27 65.8
6.5 137 84.2
7 2 62.5
7.5 8 74.8
8+ 5 84.2
Required part of my program 400
Relevant to my career 50
Fitted my personal timetable 20
The reputation of the subject 40
Seemed an interesting subject to do 65
Only subject available 10
Subject 2015 2011 2008
SUBJ123 2.5 1.8 1.7
School Total 2.7 2.7 2.6
Faculty Total 2.6 2.7 2.7
University Total 2.8 2.9 2.7
Reason for Taking the Subject
Average Satisfaction
Student Feedback – Subject Evaluation Survey Results as at DD/MM/YYYY
Please note that this feedback represents the data that was collected the last time the Subject was surveyed.
240 220 215 210 210 205
0 0 0 0 0
20
0
100
200
300
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
StudentEnrolments
Subject Enrolment Headcount
2013 2014 2015
Subject Headcount
Subject Survey Data

25. Combined Subjects at Risk Data for ADE/HoS/APD (Faculty of Business)
Criteria / Weighting:
Enrolments
Repeating Students
Student Performance
Student Type
Location
Student Satisfaction

26. Questions