Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[EADTU-ENQA PLA] The E-xcellence QA methodology: lessons learned over ten years of development and implementation


Published on

Presentation by Jon Rosewell (OUUK) during the EADTU-ENQA Peer Learning Activity in Brussels on 21-22 September 2017

Published in: Education
  • Be the first to comment

  • Be the first to like this

[EADTU-ENQA PLA] The E-xcellence QA methodology: lessons learned over ten years of development and implementation

  1. 1. The E-xcellence QA methodology: Lessons learned over ten years of development and implementation Jon Rosewell (Open University, UK), Karen Kear(OU), Keith Williams (OU), Covadonga Rodrigo (UNED), Angeles Sanchez Elvira Paniagua (UNED) QA of Blended and Online Programmes EADTU-ENQA Peer Learning Activity Brussels, 21-22 Sept 2017
  2. 2. What do we mean by ‘quality’ in HE? • Reputation – Recruit good students, produce good graduates • Compliance & consumer protection – Accreditation – Guarantee of uniform standards • Process improvement – To meet institutional mission (so internal focus) – Stakeholder engagement – Measures of added value (for external use?)
  3. 3. Approaches to QA in e-learning • Compliance or enhancement? • Norms? • Process or product? • Pedagogical models? • Input elements? • Outcome measures? • Self-assessment or external review? Holistic: emphasis on process & context as well as product
  4. 4. The future will be blended campus — distance full time — part time face-to-face — online synchronous — asynchronous paper — digital physical — virtual Challenge: finding a ‘principled’ mix to meet needs ‘Principled’: justified on basis of research / evaluation / experience / …
  5. 5. Enablers of blended provision Underpinned by ubiquitous access to internet through personal devices • Digital media / content • VLE / LMS / registration / support • Social media use • Online library provision • Open educational resources • (MOOCs?)
  6. 6. Challenges for QA of e/b-learning • Tension between: – High level standards that are independent of mode – Detailed indicators for e-learning, campus, blend… • Tension between: – External, national/international – Internal • Tension between: – Compliance, checking, retrospective – Enhancement, prospective • Tension between: – Content: components and product – Process and context
  7. 7. E-xcellence label Manual 1. Strategic management 2. Curriculum design 3. Course design 4. Course delivery 5. Staff support 6. Student support
  8. 8. Organisation of resources Strategic Management 5 benchmark statements Curriculum Design 4 benchmark statements Course Design 9 benchmark statements Course Delivery 6 benchmark statements Staff Support 6 benchmark statements Student Support 5 benchmark statements
  9. 9. Sample benchmark Curriculum design 8. … 9. Curricula are designed to enable participation in academic communities via social media tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities. 10. …
  10. 10. Sample indicators • There are institutional policies relating to the provision of online community spaces for student-student and student-teacher interactions. • Curriculum designers specify clearly the educational role that student-student interaction plays in their programmes. • Criteria for the assessment of student online collaboration exist and are applied consistently across programmes and courses. At excellence level • Teaching staff are supported by formal and informal staff development activity in the use of online tools for community building. • The institution works closely with professional bodies in the development of online professional communities. • Innovative assessment approaches, such as online collaborative work, peer assessment and self-assessment, form a part of the institution’s practice in this area.
  11. 11. E-xcellence benchmarks • Benchmark = statement of best practice • Benchmarking as quality enhancement tool – Identification of weaknesses & strengths by collecting evidence – Action plan for improvement • Generic in form – Resilient to developments in technology and pedagogy • Reflect current experience and practice – Informed by identifiable trends, evolve over time • Complementary with OpenupEd MOOC benchmarks • Published under Creative Commons licensing
  12. 12. Modes of use • Informal self-assessment using QuickScan – Identify ‘hot’ and ‘cold’ spots • Full internal self-assessment – Stakeholders collect evidence and prepare road map of actions • Integrate with institutional process – Embed selected benchmarks in internal process • EADTU E-xcellence Associates Label – Self-assessment and external review  recognition by EADTU • Integrate into QA agencies formal process – External agencies embed benchmarks in formal process NB: Resources such as manual and benchmarks are freely available! Seminar / workshop
  13. 13. Review of reviews • Nineteen reviews 2009-2016 • For each benchmark, counted no. of reviews with: – ‘Problematic’ in self-assessments – Actions planned in road maps • Grouped by major area (chapter of manual)
  14. 14. Emerging issues • Workload management (of staff) • E-learning strategy • Academic communities / social media • Interactivity / e-learning tools Non-issues: • Student support generally • Reliability / performance (of VLE)
  15. 15. Local seminar feedback • Framework – Quickscan is valuable to structure discussion – Completeness of the framework is appreciated • Team working – People exchange perspectives with other departments • External perspective – Exchange of experience between the evaluators and staff was valuable – New ideas surfaced for course design • Reflection – A valued ‘moment of reflection’ on quality – People become aware of choices and implementations – Gives insight into strengths and weaknesses • Analysis – Opportunity to formulate e-learning policy – Provides foundations for decision making E-xcellence NEXT
  16. 16. What aspects did you like the most about the E-xcellence seminar? • Open discussion, sharing views and experiences • The process of completing the Quick Scan • Opportunity to exchange experiences between local team and experts from other countries • Sharing experience, revealing strengths and weaknesses • Improvement plan • E-xcellence tool is extremely useful for improving quality • Commitment of participants: managers, teachers, students, incl. representative of National accreditation agency • Discussions that are relevant for future development • It was an opportunity to revise our master degree program SEQUENT
  17. 17. Has the use of the instrument had any impact in your organisation so far? • Discussion on some important issues was initiated, involving stakeholders not involved before • We have identified issues for further e-learning development • The participation in the project has already led to adjustments of the QA system of the university • It has opened a dialogue between different stakeholders • We are working on improvement on e-learning performance along the road map • Definitely, QA in e-learning became more important • Since the project teachers see how to improve their courses • Too early for an official impact, but it enhances fruitful discussions • E-xcellence framework is very useful in the process of designing an internal QA system on e-learning • Not yet, some changes require decision making at the central level
  18. 18. What role could QA agencies play in relation to QA in e-learning? • They should regard e-learning as an indispensable element of modern education • Create an independent assessment of e-learning/on-line programmes • They should prepare recommendations in QA and e-learning • Provide a more detailed implementation plan • They should both play a constructive and an accreditive role in the development of a methodology to assess quality in e- learning • Organise workshops on QA benchmarks SEQUENT
  19. 19. ICDE Global Overview of Quality Models 2015 • Norm based approaches are subject to rapid obsolescence • Staff development in institutions and agencies is a challenge • Focus on principles and flexibility • As systems converge, face-to-face sector can learn from distance education sector Ebba Ossiannilsson, Keith Williams, Anthony F. Camilleri, and Mark Brown (2015) Quality models in online and open education around the globe: State of the art and recommendations, ICDE Report
  20. 20. Questions? Thanks for your attention