Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Crossing the divide: Student and 
staff responses to changing an 
institutional learning management 
system 
Stuart Palmer...
… or: 
Part past part (non-)fiction 
Stuart Palmer (@s_palm ) and Dale Holt 
Deakin University
At Deakin University 
WebCT LMS trialled in 2003 
All content migrated in by 2004 
D2L selected as replacement in 2010 
Ph...
Students 2004-2011 
1 
5 
7 
8 
12 
11 10 
5.0 
4.5 
4.0 
3.5 
3.0 
2.5 
2.0 
2.5 3.0 3.5 4.0 4.5 
Importance 
Satisfactio...
Staff 2004-2011 
1 
4 
3 
7 
11 
12 
9 
6 
5.0 
4.5 
4.0 
3.5 
3.0 
2.5 
2.0 
1 Accessing unit guide and other unit inform...
The LMS WILL change, b/c 
Improving online learning and teaching 
Competitive advantage via innovation 
A new direction fo...
Students 2011-2012 
Students 2011 Students 2012 
Sample Population Sample Population 
No. of respondents 1353 22760 1710 2...
Staff 2011-2012 Staff 2011 Staff 2012 
Sample Population Sample Population 
No. of respondents 263 2126 330 2291 
Response...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Students 2011-2012 
1 
2 
3 
4 
5 
7 
10 
9 
14 
15 
16 
17 
18 
19 
5.0 
4.5 
4.0 
3.5 
3.0 
2.5 
3.00 3.25 3.50 3.75 4.0...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Staff 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc. 
...
Staff 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc. 
...
Staff 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc. 
...
Staff 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc. 
...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Students 2011-2012 
1 Accessing unit guide and other unit information 
2 Accessing unit lecture, tutorial or lab notes etc...
Key findings 
Did the new LMS dramatically change the 
student and staff experience of using the 
system? To a large exten...
Other observations 
University LMS users should expect that their 
current system will be retired at some point 
There is ...
In the doledrums 
On the (d)O.L.E. 
Thank for your time 
ow.ly/CiYXm
Upcoming SlideShare
Loading in …5
×

Sp140909

795 views

Published on

Staff and student evaluation of a new learning management system

Published in: Education
  • Be the first to like this

Sp140909

  1. 1. Crossing the divide: Student and staff responses to changing an institutional learning management system Stuart Palmer (@s_palm ) and Dale Holt Deakin University
  2. 2. … or: Part past part (non-)fiction Stuart Palmer (@s_palm ) and Dale Holt Deakin University
  3. 3. At Deakin University WebCT LMS trialled in 2003 All content migrated in by 2004 D2L selected as replacement in 2010 Phased cut-over in 2011 New LMS fully online 2012
  4. 4. Students 2004-2011 1 5 7 8 12 11 10 5.0 4.5 4.0 3.5 3.0 2.5 2.0 2.5 3.0 3.5 4.0 4.5 Importance Satisfaction ♦ 2004 • 2005 2011 2 6 4 3 9 13 5.0 4.5 4.0 3.5 3.0 2.5 2.0 2.5 3.0 3.5 4.0 4.5 Importance Satisfaction ♦ 2004 • 2005 2011 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Contacting teachers via internal unit messaging 6 Contacting students via internal unit messaging 7 Reading contributions to online discussions 8 Contributing to online discussions 9 Completing online quizzes/tests 10 Submitting assignments 11 Receiving feedback on assignments 12 Working collaboratively in a group 13 Reviewing unit progress
  5. 5. Staff 2004-2011 1 4 3 7 11 12 9 6 5.0 4.5 4.0 3.5 3.0 2.5 2.0 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Contacting teachers via internal unit messaging 6 Contacting students via internal unit messaging 7 Reading contributions to online discussions 8 Contributing to online discussions 9 Completing online quizzes/tests 10 Submitting assignments 11 Receiving feedback on assignments 12 Working collaboratively in a group 13 Reviewing unit progress 5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 Importance Satisfaction ♦ 2004 • 2005 2011 2 5 8 10 13 1.5 2.0 2.5 3.0 3.5 4.0 4.5 Importance Satisfaction ♦ 2004 • 2005 2011
  6. 6. The LMS WILL change, b/c Improving online learning and teaching Competitive advantage via innovation A new direction for learning and teaching Adoption of open source IT systems Expiry of current system vendor contract Improving back-office efficiency Consolidating institutional IT systems Acquisition of the current vendor by a competitor
  7. 7. Students 2011-2012 Students 2011 Students 2012 Sample Population Sample Population No. of respondents 1353 22760 1710 24805 Response rate 5.95% — 6.89% — Gender Female 67.3% 59.5% 70.5% 59.9% Male 32.7% 40.5% 29.5% 40.1% Faculty Arts and Education 29.8% 30.4% 30.5% 31.3% Business and Law 28.8% 36.9% 26.6% 35.1% Health 22.8% 19.8% 26.3% 20.1% Science & Technology 17.0% 12.9% 15.6% 13.5% Other 1.6% — 1.1% — Campus Burwood 43.7% 50.3% 42.0% 49.8% Waurn Ponds 14.3% 12.1% 14.3% 12.5% Waterfront 7.4% 5.8% 7.3% 6.0% Warrnambool 3.3% 3.2% 4.4% 3.0% Off-campus 31.3% 28.6% 32.1% 28.7% Mean enrolment duration 2.42 years 2.24 years 2.33 years 2.22 years
  8. 8. Staff 2011-2012 Staff 2011 Staff 2012 Sample Population Sample Population No. of respondents 263 2126 330 2291 Response rate 12.37% — 14.40% — Gender Female 55.9 % 59.8 % 61.8 % 59.8 % Male 44.1 % 40.2 % 38.2 % 40.2 % Faculty Arts and Education 30.3% — 33.0% 24.3% Business and Law 26.3% — 20.3% 20.6% Health 29.2% — 32.4% 37.8% Science & Technology 12.4% — 13.3% 13.9% Other 1.8% — 0.9% 3.3% Age range <25 3.0% 2.8% 1.5% 2.7% 25-29 4.9% 9.1% 7.0% 6.5% 30-34 5.7% 14.5% 9.7% 12.2% 35-39 13.3% 13.3% 9.4% 13.7% 40-44 16.0% 14.8% 15.8% 14.1% 45-49 15.6% 14.4% 19.1% 14.5% 50-54 17.9% 14.3% 14.5% 14.2% 55-59 10.6% 10.0% 11.8% 11.6% 60-64 9.1% 6.0% 5.8% 7.5% >64 3.4% 0.9% 5.5% 2.9% Not indicated 0.4% — — —
  9. 9. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 20 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  10. 10. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 10 9 16 17 18 19 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  11. 11. Students 2011-2012 1 2 3 4 5 7 10 9 14 15 16 17 18 19 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection
  12. 12. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  13. 13. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 20 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  14. 14. Staff 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 21 Deakin's Learning Repository – Equella 22 Respondus online quiz authoring tool 2 1 3 4 5 6 8 9 7 10 11 12 13 14 15 16 17 18 19 20 21 22 5.0 4.5 4.0 3.5 3.0 2.5 2.0 2.75 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  15. 15. Staff 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 21 Deakin's Learning Repository – Equella 22 Respondus online quiz authoring tool 2 1 3 11 8 9 5 12 13 14 15 16 17 18 19 21 22 5.0 4.5 4.0 3.5 3.0 2.5 2.0 2.75 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  16. 16. Staff 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 21 Deakin's Learning Repository – Equella 22 Respondus online quiz authoring tool 2 1 3 5 6 8 9 7 10 11 12 13 14 15 16 17 18 19 21 22 5.0 4.5 4.0 3.5 3.0 2.5 2.0 2.75 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  17. 17. Staff 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 21 Deakin's Learning Repository – Equella 22 Respondus online quiz authoring tool 2 1 3 4 5 6 8 9 7 10 11 12 13 14 15 16 17 18 19 20 21 22 5.0 4.5 4.0 3.5 3.0 2.5 2.0 2.75 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  18. 18. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 20 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  19. 19. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 20 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  20. 20. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 20 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  21. 21. Students 2011-2012 1 Accessing unit guide and other unit information 2 Accessing unit lecture, tutorial or lab notes etc. 3 Interacting with unit learning resources 4 Using the unit calendar 5 Reading unit News announcements 6 Contacting teachers via internal unit messaging 7 Contacting students via internal unit messaging 8 Reading contributions to online discussions 9 Contributing to online discussions 10 Completing online quizzes/tests 11 Submitting assignments 12 Receiving feedback on assignments 13 Working collaboratively in a group 14 Reviewing unit progress 15 Using the e-portfolio 16 eLive synchronous communication session 17 iLecture class lecture recording 18 Other iLecture recording – podcasts, etc. 19 Deakin's social software – MediaWiki, etc 20 Turnitin plagiarism detection 1 2 3 4 5 6 7 8 11 10 9 12 13 14 15 16 17 18 19 20 5.0 4.5 4.0 3.5 3.0 2.5 3.00 3.25 3.50 3.75 4.00 4.25 Importance Satisfaction ♦ No Sig. Difference • 2011 2012
  22. 22. Key findings Did the new LMS dramatically change the student and staff experience of using the system? To a large extent, the answer is, “No.” Some observed significant differences were ‘as expected’
  23. 23. Other observations University LMS users should expect that their current system will be retired at some point There is a need for on-going monitoring It can take many years before significant increases in perceptions of LMS value emerge
  24. 24. In the doledrums On the (d)O.L.E. Thank for your time ow.ly/CiYXm

×