Evaluating the Impact of an eLearning Strategy on the Quality of Teaching and Learning.

Evaluating the Impact of an eLearningStrategy on the Quality of Teaching and Learning. Dr. Iain Doherty Associate Professor Director eLearning Pedagogical Support Unit Centre for the Enhancement of Teaching and Learning 13th May 2012

Overview• HKU’s eLearning Strategy.• Quantitative reporting.• What is quality in learning?• What is quality in eLearning.• What would a program of evaluation look like?• Concluding remarks. 2

eLearning Strategy Level 2 3

eLearning Strategy Level 3• “Innovative pedagogy, curriculum design and assessment are brought to new heights by technology, including, but not restricted to, internationalization of the curriculum, collaborative teaching and learning within HKU courses and with overseas universities, integration of campus-based and experiential learning, involvement of employers and community partners in the learning processes”. 4

Faculty Moodle Statistics• Right now the reality is Moodle! No. of Moodle courses Faculty/Others Total Available Created Enabled Active Business and Economics 821 265 87 82 1. “Total available” - the course templates created based on SIS data or the courses created explicitly as per requests from teachers 2. “Created” - courses that their teachers have created and can start editing its eLearning materials 3. “Enabled” - courses that their teachers have granted students’ access 4. “Active” – courses with both teacher logins > 5 and student logins > 5 7

Moodle Activity• Moodle Activity Levels – Level 1 - one-way transmission of information from teacher to students – the data is gathered as "the total count of Moodle files/ folders in the Moodle courses" – Level 2 - two-way interaction between teachers & students – total count of Moodle assignment, quizzes and choices in the Moodle courses – Level 3 - three way interaction between teacher & students and among students – total count of Moodle forum, chat and wikis in the Moodle courses 8

Faculty Moodle Statistics• The news forum is created automatically which may account for the level 1 and level 3 activities. So the only “real figure” may be levels 1, 2 and 3. Number of active Moodle courses having only only only Level 1 Level 1 Level 2 Level 1, Level 1 Level 2 Level 3 and and and Level 2 activities activities activities Level 2 Level 3 Level 3 and Faculty activities activities activitiesLevel 3 /Others activities B&E 0 0 0 0 48 0 34 9

Quantitative Reporting• “Quantitative Performance Indicator (PI) data has a limited role to play in determining impact of new learning and teaching practices on the learner experience. PIs are ‘input’ and ‘output’ measures and TESEP [Transforming and Enhancing the Student Experience through Pedagogy] found that they were best used as ‘diagnostic tools’ which could be used on a number of levels.” 10

Quantitative Reporting• These statistics tell us about activity levels in Moodle.• The statistics do not tell us anything about the quality of eLearning e.g. – A lecturer on any particular course might be delivering all the course content (level 1) via Moodle. – The same lecturer might be engaging students with multiple choice assessments (level 2) to test understanding.• What does this tell us about quality? – Nothing! 11

Quality Learning• When we say that a course is a quality course “we” are broadly making a judgment that the course is fit for purpose where the purpose is institutional purpose.• For example, university which efficiently helps students to achieve the educational aims and learning outcomes can be considered as a quality institution.• The same is true at a course level. 12

Quality Learning• Quality also has something to do with the student learning experience.• In other words being fit for purpose is a necessary but not sufficient condition of a quality course.• The reason for this is that students may achieve educational aims but have a very negative experience on the course.• So, we need to include some criteria referenced to experience. 13

Quality in Learning• At HKU the criteria for a quality course are given in the Student Experience of Teaching and Learning (SETL) questionnaire and the Student Learning Experience Questionnaire (SLEQ).• Both questionnaires consist of items that relate to achieving course aims, developing generic skills / attributes and having a positive experience on a course.• So a quality course is a course that is rated positively with respect to these aspects. 14

Quality in eLearning• If we know what constitutes quality in teaching and learning we can ask about whether things are different with eLearning.• The short answer is “no” because at a macro level the teaching and learning conditions are the same: – The course must enable students to achieve ILOs; – The course must develop generic attributes / skills; and – Students must have a positive experience on the course. 15

Quality in eLearning• “I believe the distinction between face-to-face and online will soon merge in both quality standard setting and practice. As pedagogy and learning needs drive educational design, every possible mutation of physical and virtual meetings of minds will be created and be grounded in pedagogical purpose. It is a grand time to be an educator” (Frydenberg, 2002).• There is an important point here – pedagogy and learning drive technology use – but this is not the whole picture for quality in eLearning. 16

Quality in eLearning• Things are different at the micro level because there is a question of the use of technologies in teaching and learning: – Technical infrastructure; – Learning management system; – Technical support for students; – Use of additional technologies; – Use of a variety of media;• The big question, “Did eLearning help?” 17

Quality in eLearning• The following three questions have been put forward for use in SLEQ: – The Learning Management System Moodle was easy to access and use; – eLearning has been effective in aiding learning in courses in my discipline; and – eLearning has been effective in aiding communication in courses in my discipline.• No more than 3 questions because SLEQ is already very large. 18

Quality in eLearning• What would a more robust quality assurance program look like?• There are issues here: – A robust quality assurance program would be much more comprehensive in terms of the range of questions asked about eLearning; – A robust quality assurance program would move beyond questions that ask about “perceptions”; – The program would look for evidence of e.g. improved communication, improved understanding. 19

Concluding Remarks• There has to be a program of work if we want actual evidence that the eLearning strategy has resulted in improved quality in teaching and learning;• Funding would be nice because in the end showing actual gains will require a lot of work.• Don’t really see this program of work as optional because without it what do we have in terms of evidence for effectiveness of the eLearning strategy? 20

References• European Association of Distance Teaching Universities (EADTU). (2007). Quality Manual for E- learning in Higher Education,. European Association of Distance Teaching Universities (EADTU). Retrieved from http://www.eadtu.nl/e-xcellenceQS/files/members/E-xcelle 21

References• Frydenberg, J. (2002). Quality Standards in eLearning: A Matrix of Analysis. The International Review of Research in Open and Distance Learning, 3(2), 1-15. Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticl e/109 22

References• Herrington, A., Herrington, J., Oliver, R., Stoney, S., & Willis, J. (2001). Quality Guidelines for Online Courses: The Development of an Instrument to Audit Online Units. In G. Kennedy & M. Keppell (Eds.), Meeting at the Crossroads.18th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education. (pp. 263-270). Melbourne, Australia. Retrieved from http://www.ascilite.org.au/conferences/melbourne01/ pdf/papers/herringtona.pdf 23

References• TESEP (Transforming and Enhancing the Student Experience through Pedagogy). (2007). TESEP: Re- thinking Quality Enhancement (pp. 1-4). Edinburgh, UK. Retrieved from http:// www2.napier.ac.uk/transform/rethinking_qe.htm 24

http://www.scoop.it	486
http://citers2012.cite.hku.hk	30
http://teldevelopers.wordpress.com	14
http://webcache.googleusercontent.com	1

Evaluating the Impact of an eLearning Strategy on the Quality of Teaching and Learning.

by CITE on Jun 17, 2012

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 531

Statistics

Evaluating the Impact of an eLearning Strategy on the Quality of Teaching and Learning. Presentation Transcript