Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons from 10 years of E-xcellence quality reviews for e-learning

728 views

Published on

Lessons from 10 years of E-xcellence quality reviews for e-learning by Karen Kear & Jon Rosewell, OUUK, 20 September 2018

Published in: Education
  • Be the first to comment

Lessons from 10 years of E-xcellence quality reviews for e-learning

  1. 1. Lessons from 10 years of E-xcellence quality reviews for e-learning Karen Kear & Jon Rosewell Open University, UK Quality in Higher Education EMPOWER / EADTU Webinar Week 18-20 Sept 2018
  2. 2. Poll: Where is the greatest challenge facing universities when introducing e-learning? • Strategic Management – a high level view of how the institution plans its e-learning • Curriculum Design – how e-learning is used across a whole programme of study • Course Design – how e-learning is used in the design of individual courses • Course Delivery – the technical and practical aspects of e-learning delivery • Staff Support – the support and training provided to staff • Student Support – the support, information and guidance provided to students
  3. 3. Poll: …more specific e-learning challenges? • Agreeing an e-learning strategy • Developing academic communities • Using OER • Greater interactivity • Technical infrastructure (VLE, LMS…) • Providing wide range of e-learning tools • Staff workload management • Encouraging educational innovation • Administrative support for students
  4. 4. What do we mean by ‘quality’ in HE? • Reputation – Recruit good students, produce good graduates • Compliance & consumer protection – Accreditation – Guarantee of uniform standards • Process improvement – To meet institutional mission (so internal focus) – Stakeholder engagement – Measures of added value (for external use?)
  5. 5. Approaches to QA in e-learning • Compliance or enhancement? • Norms? • Process or product? • Pedagogical models? • Input elements? • Outcome measures? • Self-assessment or external review? Holistic: emphasis on process & context as well as product
  6. 6. The future will be blended campus — distance full time — part time face-to-face — online synchronous — asynchronous paper — digital physical — virtual Challenge: finding a ‘principled’ mix to meet needs ‘Principled’: justified on basis of research / evaluation / experience / …
  7. 7. Enablers of blended provision Underpinned by ubiquitous access to internet through personal devices • Digital media / content • VLE / LMS / registration / support • Social media use • Online library provision • Open educational resources • (MOOCs?)
  8. 8. Challenges for QA of e/b-learning • Tension between: – High level standards that are independent of mode – Detailed indicators for e-learning, campus, blend… • Tension between: – External, national/international – Internal • Tension between: – Compliance, checking, retrospective – Enhancement, prospective • Tension between: – Content: components and product – Process and context
  9. 9. E-xcellence label http://e-xcellencelabel.eadtu.eu/ Manual 1. Strategic management 2. Curriculum design 3. Course design 4. Course delivery 5. Staff support 6. Student support
  10. 10. Organisation of resources Strategic Management – a high level view of how the institution plans its e-learning Curriculum Design – how e-learning is used across a whole programme of study Course Design – how e-learning is used in the design of individual courses Course Delivery – the technical and practical aspects of e- learning delivery Staff Support – the support and training provided to staff Student Support – the support, information and guidance provided to students
  11. 11. Organisation of resources Strategic Management 5 benchmark statements Curriculum Design 4 benchmark statements Course Design 9 benchmark statements Course Delivery 6 benchmark statements Staff Support 6 benchmark statements Student Support 5 benchmark statements
  12. 12. Topic area Sample Benchmark Strategic Management 1. The institution has an e-learning strategy that is widely understood and integrated into the overall strategies for institutional development and quality improvement. E-learning policies conform to legal and ethical frameworks. Curriculum Design 9. Curricula are designed to enable participation in academic communities via social media tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities. Course Design 14.E-learning materials have sufficient interactivity (student-to-content, student- to-student and student-to-teacher) to encourage active engagement and enable students to test their knowledge, understanding and skills. Course Delivery 20.The systems for communication and storage of data are secure, reliable and assure appropriate levels of privacy. Measures are in place for system recovery in the event of failure or breakdown. Staff Support 29.The institution ensures that issues of staff workload, and any other implications of staff participation in e-learning activities, are taken into account when managing courses or programmes. Student Support 32.Students are provided with guidelines stating their rights, roles and responsibilities and those of their institution. Guidelines of specific relevance to e-learning include provision of hardware, information on accessibility and expected participation in collaborative activities.
  13. 13. Sample benchmark Curriculum design 8. … 9. Curricula are designed to enable participation in academic communities via social media tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities. 10. …
  14. 14. Sample indicators • There are institutional policies relating to the provision of online community spaces for student-student and student-teacher interactions. • Curriculum designers specify clearly the educational role that student-student interaction plays in their programmes. • Criteria for the assessment of student online collaboration exist and are applied consistently across programmes and courses. At excellence level • Teaching staff are supported by formal and informal staff development activity in the use of online tools for community building. • The institution works closely with professional bodies in the development of online professional communities. • Innovative assessment approaches, such as online collaborative work, peer assessment and self-assessment, form a part of the institution’s practice in this area.
  15. 15. E-xcellence benchmarks • Benchmark = statement of best practice • Benchmarking as quality enhancement tool – Identification of weaknesses & strengths by collecting evidence – Roadmap of actions for improvement • Generic in form – Resilient to developments in technology and pedagogy • Reflect current experience and practice – Informed by identifiable trends, evolve over time • Complementary with OpenupEd MOOC benchmarks • Published under Creative Commons licensing
  16. 16. Modes of use • Informal self-assessment using QuickScan – Identify ‘hot’ and ‘cold’ spots • Full internal self-assessment – Stakeholders collect evidence and prepare road map of actions • Integrate with institutional process – Embed selected benchmarks in internal process • EADTU E-xcellence Associates Label – Self-assessment, roadmap, external review → recognition by EADTU • Integrate into QA agencies formal process – External agencies embed benchmarks in formal process NB: Resources such as manual and benchmarks are freely available! Seminar / workshop
  17. 17. Review of reviews • Nineteen reviews 2009-2016 • For each benchmark, counted no. of reviews with: – ‘Problematic’ in self-assessments – Actions planned in road maps • Grouped by major area (chapter of manual)
  18. 18. Emerging issues • Workload management (of staff) • E-learning strategy • Academic communities / social media • Interactivity / e-learning tools Non-issues: • Student support generally • Reliability / performance (of VLE)
  19. 19. Local seminar feedback • Framework – Quickscan is valuable to structure discussion – Completeness of the framework is appreciated • Team working – People exchange perspectives with other departments • External perspective – Exchange of experience between the evaluators and staff was valuable – New ideas surfaced for course design • Reflection – A valued ‘moment of reflection’ on quality – People become aware of choices and implementations – Gives insight into strengths and weaknesses • Analysis – Opportunity to formulate e-learning policy – Provides foundations for decision making E-xcellence NEXT
  20. 20. What aspects did you like the most about the E-xcellence seminar? • Open discussion, sharing views and experiences • The process of completing the Quick Scan • Opportunity to exchange experiences between local team and experts from other countries • Sharing experience, revealing strengths and weaknesses • Improvement plan • E-xcellence tool is extremely useful for improving quality • Commitment of participants: managers, teachers, students, incl. representative of National accreditation agency • Discussions that are relevant for future development • It was an opportunity to revise our master degree program SEQUENT
  21. 21. Has the use of the instrument had any impact in your organisation so far? • Discussion on some important issues was initiated, involving stakeholders not involved before • We have identified issues for further e-learning development • The participation in the project has already led to adjustments of the QA system of the university • It has opened a dialogue between different stakeholders • We are working on improvement on e-learning performance along the road map • Definitely, QA in e-learning became more important • Since the project teachers see how to improve their courses • Too early for an official impact, but it enhances fruitful discussions • E-xcellence framework is very useful in the process of designing an internal QA system on e-learning • Not yet, some changes require decision making at the central level
  22. 22. What role could QA agencies play in relation to QA in e-learning? • They should regard e-learning as an indispensable element of modern education • Create an independent assessment of e-learning/on-line programmes • They should prepare recommendations in QA and e-learning • Provide a more detailed implementation plan • They should both play a constructive and an accreditive role in the development of a methodology to assess quality in e- learning • Organise workshops on QA benchmarks SEQUENT
  23. 23. ICDE Global Overview of Quality Models 2015 • Norm based approaches are subject to rapid obsolescence • Staff development in institutions and agencies is a challenge • Focus on principles and flexibility • As systems converge, face-to-face sector can learn from distance education sector Ebba Ossiannilsson, Keith Williams, Anthony F. Camilleri, and Mark Brown (2015) Quality models in online and open education around the globe: State of the art and recommendations, ICDE Report http://www.icde.org/quality
  24. 24. EADTU-ENQA: QA in blended and online Main challenges for quality blended and online education – Governmental and European strategies and frameworks for innovation – Institutional strategy & cultural changes / mind-set – Pedagogical models and design for blended courses – New expertise needed and staff support – Student support – Including blended education in internal quality frameworks – Funding of innovation – Increased workload – Institutional evaluation, research and innovation See EADTU EMPOWER webinar 19-Sept-2018 by George Ubachs
  25. 25. ENQA: Considerations for QA of e-learning • Just published! • Supplement to ‘European Standards and Guidelines’ 2015 • Additional guidance and indicators • See EMPOWER webinar 19-Sept-2018 by Esther Huertas Hidalgo Huertas et al (2018) ENQA Occasional Papers, No. 26 https://enqa.eu/index.php/publications/papers-reports/occasional-papers/
  26. 26. ESG and e-learning 1.1 Policy for QA 1.2 Design and approval of programme 1.3 Student-centred learning, teaching & assessment 1.4 Student admission, progression, recognition & certification 1.5 Teaching staff 1.6 Learning resources and student support 1.6 Information management 1.8 Public information 1.9 Ongoing monitoring and periodic review 1.10 Cyclical external quality assurance
  27. 27. ESG and E-xcellence 1.1 Policy for QA 1.2 Design and approval of programme 1.3 Student-centred learning, teaching & assessment 1.4 Student admission, progression, recognition & certification 1.5 Teaching staff 1.6 Learning resources and student support 1.6 Information management 1.8 Public information 1.9 Ongoing monitoring and periodic review 1.10 Cyclical external quality assurance  Strategic management  Curriculum design  Course design  Course delivery  Staff support  Student support
  28. 28. Questions? Thanks for your attention Jon.Rosewell@open.ac.uk George.Ubachs@eadtu.eu
  29. 29. Discussion • Are the challenges the same in different types of university? • Are the challenges the same in different countries? • What will the challenges be over the next 5 years?

×