An institutional study of the influence of ‘onlineness’ on student evaluation of teaching in a dual mode Australian university

1. An institutional study of the influence of
‘onlineness’ on student evaluation of
teaching in a dual mode Australian
university
Stuart Palmer
@s_palm .
Institute of Teaching and Learning
Deakin University
#spalmer

6. Systematic influences on SET data
The literature says…
Class size – negatively correlated;
Year level – positively correlated; and
Discipline area - various
In more recent times, rapid expansion of online learning

7. Student evaluation of teaching & units (SETU)
1. This unit was well taught
2. The course materials in this unit were of high quality
3. The workload in this unit was manageable
4. Requirements for completing the assessment tasks in this unit were clear
5. The teaching staff gave me helpful feedback
6. The library resources met my needs for this unit
7. I would recommend this unit to other students
8. The technologies used to deliver the online content in this unit performed
satisfactorily
9. The on-line teaching and resources in this unit enhanced my learning experience
10. This unit challenged me to learn
n/a; 1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree

8. Data set used in study
Mean SETU rating sets for 1432 units of study
Representing 74498 individual sets of SETU ratings
58.5 % of all units listed in the Deakin University
handbook for the period under consideration

9. SETU items relating to 'onlineness'
5
Mean SETU rating - item 9
4
r2 = 0.73
3
p = 0.0000
2
1
1 2 3 4 5
Mean SETU rating - item 8

10. Class size
4.00
Mean SETU rating - item 8
3.95
F = 0.885
p > 0.41
3.90
3.85
n < 51 50 < n < 101 n > 100
No Sig. Diff.
3.85
Mean SETU rating - item 9
3.80 F = 3.199
3.75
p > 0.041
No Sig. Diff.
3.70
n < 51 50 < n < 101 n > 100

11. Year level
4.10
Mean SETU rating - item 8
4.05
4.00
3.95 F = 8.972
p < 0.0002
3.90
3.85
3.80
Early Later Postgrad
Sig. Diff.
3.90
Mean SETU rating - item 9
3.85
3.80 F = 16.515
p < 1x10-7
3.75
3.70
3.65
Sig. Diff.
3.60
Early Later Postgrad

12. Discipline area
4.10
Mean SETU rating - item 8
4.05
4.00
3.95 F = 11.998
p < 2x10-7
3.90
3.85
3.80
Arts/Ed Bus/Law Sci/Tech Health
Sig. Diff.
4.00
3.95
Mean SETU rating - item 9
3.90
3.85
F = 21.177
p < 4x10-13
3.80
3.75
3.70
3.65 Sig. Diff.
3.60
Arts/Ed Sci/Tech Bus/Law Health

13. Online mode of offer
4.00
Mean SETU rating - item 1
3.90
3.80 F = 18.266
3.70 p < 0.0003
3.60
Conventional Wholly online
Sig. Diff.
4.00
Mean SETU rating - item 7
3.90
3.80 F = 22.350
p < 3x10-6
3.70
3.60
3.50
Sig. Diff.
3.40
Conventional Wholly online

14. SETU and wholly online units
1. *This unit was well taught
2. The course materials in this unit were of high quality
3. The workload in this unit was manageable
4. Requirements for completing the assessment tasks in this unit were clear
5. The teaching staff gave me helpful feedback
6. The library resources met my needs for this unit
7. *I would recommend this unit to other students
8. The technologies used to deliver the online content in this unit performed satisfactorily
9. *The on-line teaching and resources in this unit enhanced my learning experience
10. This unit challenged me to learn
* Reported to university Council

15. Case study – SEB221
4.25
2003
Mean SETU rating
3.75 (n=51, 17%)
2004
(n=73, 42%)
3.25 2005
(n=47, 32%)
2006
2.75 (n=32, 37%)
1 7 9
SETU item

16. Case study – SEB221
4.25
2003
Mean SETU rating
3.75 (n=51, 17%)
2004
(n=73, 42%)
3.25 2005
(n=47, 32%)
2006
2.75 (n=32, 37%)
1 7 9
SETU item

17. Case study – SEB221
4.25
2003
Mean SETU rating
3.75 (n=51, 17%)
2004
(n=73, 42%)
3.25 2005
(n=47, 32%)
2006
2.75 (n=32, 37%)
1 7 9
SETU item

18. Conclusions
Mean ratings for the two ‘online’ SETU items (item 8 – ‘The technologies
used to deliver the online content in this unit performed satisfactorily’ and item 9
– ‘The on-line teaching and resources in this unit enhanced my learning
experience’) are strongly, significantly and positively correlated
Comparing units offered in wholly online mode to units offered in all other
modes, mean ratings for SETU items 1 ‘this unit was well taught’ and 7 ‘I would
recommend this unit to other students’ were both significantly lower for
wholly online units

19. Conclusions
Class size had no significant influence on either SETU item 8 or item 9
Mean ratings for SETU items 8 and 9 are significantly and positively related
to the enrolled year level of the respondent, based on the groupings of ‘early
years’ (first & second years), ‘later years’ (third & later years) and ‘postgraduate’
(programs beyond undergrad level)
Mean ratings for SETU items 8 and 9 are significantly different between
Faculties – with the Faculty of Health having the highest mean rating for both
items

20. Thank you for your time
http://myqr.co/zJrD
Presentation