0
Evaluating the Impact of an eLearningStrategy on the Quality of Teaching and               Learning.                  Dr. ...
Overview•   HKU’s eLearning Strategy.•   Quantitative reporting.•   What is quality in learning?•   What is quality in eLe...
eLearning Strategy Level One• Teaching and learning is assisted by technology for  efficiency and cost-effectiveness. Stud...
eLearning Strategy Level One
eLearning Strategy Level Two• Teaching and learning is enriched by technology  through enhancing opportunities for active ...
eLearning Strategy Level 2                     6
eLearning Strategy Level 3• “Innovative pedagogy, curriculum design and  assessment are brought to new heights by  technol...
eLearning Strategy Level 3                     8
eLearning Strategy Level 3                     9
Faculty Moodle Statistics• Right now the reality is Moodle!                                                            No....
Moodle Activity• Moodle Activity Levels  – Level 1 - one-way transmission of information from    teacher to students – the...
Faculty Moodle Statistics• The news forum is created automatically which may  account for the level 1 and level 3 activiti...
Quantitative Reporting• “Quantitative Performance Indicator (PI) data has a  limited role to play in determining impact of...
Quantitative Reporting• These statistics tell us about activity levels in Moodle.• The statistics do not tell us anything ...
Quality Learning• When we say that a course is a quality course “we”  are broadly making a judgment that the course is fit...
Quality Learning QAC• “The main objective of QAC quality audits is to  assure the quality of student learning in UGC-funde...
Quality Learning QAC• QAC is concerned with learning outcomes:• “the audit process is particularly concerned with the  way...
Quality Learning• Quality also has something to do with the student  learning experience.• In other words being fit for pu...
Quality Learning QAC• The QAC, through its audit process, aims to give  confidence, in general, to students and their pare...
Quality in Learning• At HKU the criteria for a quality course are given in  the Student Experience of Teaching and Learnin...
Quality in eLearning• If we know what constitutes quality in teaching and  learning we can ask about whether things are  d...
Quality in eLearning• “I believe the distinction between face-to-face and  online will soon merge in both quality standard...
Quality in eLearning• Things are different at the micro level because there  is a question of the use of technologies in t...
Quality in eLearning• The following three questions have been put forward  for use in HKUs Student Learning Experience  Qu...
Quality in eLearning• What would a more robust quality assurance  program look like?• There are issues here:  – A robust q...
Concluding Remarks• There has to be a program of work if we want actual  evidence that the eLearning strategy has resulted...
References• European Association of Distance Teaching  Universities (EADTU). (2007). Quality Manual for E-  learning in Hi...
References• Frydenberg, J. (2002). Quality Standards in  eLearning: A Matrix of Analysis. The International  Review of Res...
References• Herrington, A., Herrington, J., Oliver, R., Stoney, S.,  & Willis, J. (2001). Quality Guidelines for Online  C...
References• TESEP (Transforming and Enhancing the Student  Experience through Pedagogy). (2007). TESEP: Re-  thinking Qual...
Upcoming SlideShare
Loading in...5
×

Evaluating the Impact of an eLearning Strategy on the Quality of Teaching and Learning

566

Published on

This is a presentation that I gave at the CITE 2012 Research Symposium at The University of Hong Kong. Essentially the presentation makes a case for the need to look at learning experiences and learning outcomes in order to truly determine whether or not an eLearning strategy has been effective. In other words, Moodle generated data will not do the trick.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
566
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Evaluating the Impact of an eLearning Strategy on the Quality of Teaching and Learning"

  1. 1. Evaluating the Impact of an eLearningStrategy on the Quality of Teaching and Learning. Dr. Iain Doherty Associate Professor Director eLearning Pedagogical Support Unit Centre for the Enhancement of Teaching and Learning 16th May 2012
  2. 2. Overview• HKU’s eLearning Strategy.• Quantitative reporting.• What is quality in learning?• What is quality in eLearning.• What would a program of evaluation look like?• Concluding remarks. 2
  3. 3. eLearning Strategy Level One• Teaching and learning is assisted by technology for efficiency and cost-effectiveness. Students are provided with on-line access to: – Information regarding their programmes of study, including administrative procedures relating to course enrolment, assessments, degree audit, student evaluation of teaching and learning (SETL); and – Resources for learning, including all course outlines and digital materials, course learning outcomes and assessments.
  4. 4. eLearning Strategy Level One
  5. 5. eLearning Strategy Level Two• Teaching and learning is enriched by technology through enhancing opportunities for active learning within and beyond the classroom, provision of links to digital library resources, provision of just-in-time formative and summative feedback and assessment for learning, fostering teaching-research nexus, establishing a closer link with schools, community partners and employers.
  6. 6. eLearning Strategy Level 2 6
  7. 7. eLearning Strategy Level 3• “Innovative pedagogy, curriculum design and assessment are brought to new heights by technology, including, but not restricted to, internationalization of the curriculum, collaborative teaching and learning within HKU courses and with overseas universities, integration of campus-based and experiential learning, involvement of employers and community partners in the learning processes”. 7
  8. 8. eLearning Strategy Level 3 8
  9. 9. eLearning Strategy Level 3 9
  10. 10. Faculty Moodle Statistics• Right now the reality is Moodle! No. of Moodle courses Faculty/Others Total Available Created Enabled Active 821 265 87 821. “Total available” - the course templates created based on SIS data or the courses created explicitly as per requests from teachers2. “Created” - courses that their teachers have created and can start editing its eLearning materials3. “Enabled” - courses that their teachers have granted students’ access4. “Active” – courses with both teacher logins > 5 and student logins > 5 10
  11. 11. Moodle Activity• Moodle Activity Levels – Level 1 - one-way transmission of information from teacher to students – the data is gathered as "the total count of Moodle files/ folders in the Moodle courses" – Level 2 - two-way interaction between teachers & students – total count of Moodle assignment, quizzes and choices in the Moodle courses – Level 3 - three way interaction between teacher & students and among students – total count of Moodle forum, chat and wikis in the Moodle courses 11
  12. 12. Faculty Moodle Statistics• The news forum is created automatically which may account for the level 1 and level 3 activities. So the only “real figure” may be levels 1, 2 and 3. Number of active Moodle courses having only Level 1 only Level 2 only Level 3 Level 1 and Level 1 and Level 2 and Level 1, Level activities activities activities Level 2 Level 3 Level 3 2 and Level 3 Faculty activities activities activities activities /Others 0 0 0 0 48 0 34 12
  13. 13. Quantitative Reporting• “Quantitative Performance Indicator (PI) data has a limited role to play in determining impact of new learning and teaching practices on the learner experience. PIs are ‘input’ and ‘output’ measures and TESEP [Transforming and Enhancing the Student Experience through Pedagogy] found that they were best used as ‘diagnostic tools’ which could be used on a number of levels.” 13
  14. 14. Quantitative Reporting• These statistics tell us about activity levels in Moodle.• The statistics do not tell us anything about the quality of eLearning e.g. – A lecturer on any particular course might be delivering all the course content (level 1) via Moodle. – The same lecturer might be engaging students with multiple choice assessments (level 2) to test understanding.• What does this tell us about quality? – Nothing! 14
  15. 15. Quality Learning• When we say that a course is a quality course “we” are broadly making a judgment that the course is fit for purpose where the purpose is institutional purpose.• For example, university which efficiently helps students to achieve the educational aims and learning outcomes can be considered as a quality institution.• The same is true at a course level. 15
  16. 16. Quality Learning QAC• “The main objective of QAC quality audits is to assure the quality of student learning in UGC-funded institutions. The audits are intended to assure the UGC and the public that institutions deliver on the promises they make in their role and mission statements. A QAC audit is therefore an audit of an institution’s Fitness for Purpose in teaching and learning” (Quality Assurance Council, 2008, p.10). 16
  17. 17. Quality Learning QAC• QAC is concerned with learning outcomes:• “the audit process is particularly concerned with the ways institutions articulate and measure the student learning outcomes they expect or aspire to” (Quality Assurance Council, 2008, p.11) 17
  18. 18. Quality Learning• Quality also has something to do with the student learning experience.• In other words being fit for purpose is a necessary but not sufficient condition of a quality course.• The reason for this is that students may achieve educational aims but have a very negative experience on the course.• So, we need to include some criteria referenced to experience. 18
  19. 19. Quality Learning QAC• The QAC, through its audit process, aims to give confidence, in general, to students and their parents, employers and sponsors, that our institutions provide a quality and internationally recognised student learning experience (Quality Assurance Council, 2008, p.3.). 19
  20. 20. Quality in Learning• At HKU the criteria for a quality course are given in the Student Experience of Teaching and Learning (SETL) questionnaire and the Student Learning Experience Questionnaire (SLEQ).• Both questionnaires consist of items that relate to achieving course aims, developing generic skills / attributes and having a positive experience on a course.• So a quality course is a course that is rated positively with respect to these aspects. 20
  21. 21. Quality in eLearning• If we know what constitutes quality in teaching and learning we can ask about whether things are different with eLearning.• The short answer is “no” because at a macro level the teaching and learning conditions are the same: – The course must enable students to achieve ILOs; – The course must develop generic attributes / skills; and – Students must have a positive experience on the course. 21
  22. 22. Quality in eLearning• “I believe the distinction between face-to-face and online will soon merge in both quality standard setting and practice. As pedagogy and learning needs drive educational design, every possible mutation of physical and virtual meetings of minds will be created and be grounded in pedagogical purpose. It is a grand time to be an educator” (Frydenberg, 2002).• There is an important point here – pedagogy and learning drive technology use – but this is not the whole picture for quality in eLearning. 22
  23. 23. Quality in eLearning• Things are different at the micro level because there is a question of the use of technologies in teaching and learning: – Technical infrastructure; – Learning management system; – Technical support for students; – Use of additional technologies; – Use of a variety of media;• The big question, “Did eLearning help?” 23
  24. 24. Quality in eLearning• The following three questions have been put forward for use in HKUs Student Learning Experience Questionnaire: – The Learning Management System Moodle was easy to access and use; – eLearning has been effective in aiding learning in courses in my discipline; and – eLearning has been effective in aiding communication in courses in my discipline. 24
  25. 25. Quality in eLearning• What would a more robust quality assurance program look like?• There are issues here: – A robust quality assurance program would be much more comprehensive in terms of the range of questions asked about eLearning; – A robust quality assurance program would move beyond questions that ask about “perceptions”; – The program would look for evidence of e.g. improved communication, improved understanding. 25
  26. 26. Concluding Remarks• There has to be a program of work if we want actual evidence that the eLearning strategy has resulted in improved quality in teaching and learning;• Funding would be nice because in the end showing actual gains will require a lot of work.• Don’t really see this program of work as optional because without it what do we have in terms of evidence for effectiveness of the eLearning strategy? 26
  27. 27. References• European Association of Distance Teaching Universities (EADTU). (2007). Quality Manual for E- learning in Higher Education,. European Association of Distance Teaching Universities (EADTU). Retrieved from http://www.eadtu.nl/e-xcellenceQS/files/members/E-xcelle 27
  28. 28. References• Frydenberg, J. (2002). Quality Standards in eLearning: A Matrix of Analysis. The International Review of Research in Open and Distance Learning, 3(2), 1-15. Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticl e/109 28
  29. 29. References• Herrington, A., Herrington, J., Oliver, R., Stoney, S., & Willis, J. (2001). Quality Guidelines for Online Courses: The Development of an Instrument to Audit Online Units. In G. Kennedy & M. Keppell (Eds.), Meeting at the Crossroads.18th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education. (pp. 263-270). Melbourne, Australia. Retrieved from http://www.ascilite.org.au/conferences/melbourne01/ pdf/papers/herringtona.pdf 29
  30. 30. References• TESEP (Transforming and Enhancing the Student Experience through Pedagogy). (2007). TESEP: Re- thinking Quality Enhancement (pp. 1-4). Edinburgh, UK. Retrieved from http://www2.napier.ac.uk/transform/rethinking_qe.htm 30
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×