E-xcellence (http://e-xcellencelabel.eadtu.eu/) is a methodology published by EADTU designed to assess the quality of e-learning. It takes a benchmarking approach, providing a set of 35 benchmarks against which e-learning may be judged. The benchmarks are supported by a manual which provides more detailed indicators and background guidance, organised into six main areas of concern: Strategic management, Curriculum design, Course design, Course delivery, Staff support and Student support. E-xcellence is intended to be used as part of a quality enhancement process, including internal self-assessment, external review and discussion, and action planning for improvement. Engagement with the full process is recognised by the E-xcellence Associates Label. In addition, the E-xcellence manual (http://e-xcellencelabel.eadtu.eu/tools/manual) should prove useful to a wide range of stakeholders involved in the development of e-learning. E-learning and the contexts in which it occurs have continued to evolve since the E-xcellence methodology was first developed. The 2016 update of the benchmarks and manual reflects recent trends: the rapid rise of Massive Open Online Courses (MOOCs), a surge of interest in learning analytics, and an increasing use of learning design in a more systematic approach to the development of e-learning courses. A number of other topics have also been included, such as an increased focus on personalisation, flipped approaches to teaching, virtual and remote laboratories, digital badges and e-portfolios. In this workshop, participants will explore the new benchmarks and detailed indicators. What indicators are appropriate when judging quality in the rapidly changing e-learning landscape?

1. E-xcellence:
quality in the changing e-learning
landscape
Karen Kear, Jon Rosewell, Angeles Sánchez-Elvira
Paniagua, Miguel Santamaria Lancho, Covadonga
Rodrigo, Keith Williams, Ebba Ossiannilsson,
Harvey Mellar
The Online, Open and Flexible Higher Education Conference
19-21 October 2016, Rome

2. THE ONLINE, OPEN AND FLEXIBLE
HIGHER EDUCATION CONFERENCE
ENHANCING EUROPEANHIGHER EDUCATION:
“OPPORTUNITIESAND IMPACT OFNEWMODESOFTEACHING”
19-21 OCTOBER 2016

3. Quality Assurance for e-learning
• Context:
–Increasing use of e-learning in Higher Education
–QA procedures were designed for face-to-face
education
–They don’t necessarily fit e-learning and blended
learning
• E-xcellence approach:
–Provide resources and processes for QA of e-learning

4. Quality Assurance spectrum
Check compliance
with quality standards
Support institutions in
quality enhancement
E-xcellence
Flexibility to:
 set relevant goals
 demonstrate quality of performance
 plan for improvement

5. E-xcellence resources
• E-xcellence manual (in PDF format)
– 35 benchmark statements
– Performance indicators
– Supporting text
• Quickscan tool
• Assessors’ notes
• Framework for action roadmap
See: e-xcellencelabel.eadtu.eu/

6. E-xcellence manual and framework
1. Strategic Management
2. Curriculum Design
3. Course Design
4. Course Delivery
5. Student Support
6. Staff Support
See: e-xcellencelabel.eadtu.eu/

7. Quickscan
• Quick self-assessment of the HE institution’s e-learning
• Uses the 35 E-xcellence benchmarks
• Helps institution to identifies strengths and weaknesses
• Online version provides guidance and feedback
• Best used collaboratively by an interdisciplinary team

8. The Quickscan tool

9. E-xcellence review
• Onsite visit (1-2 days) or online meetings
• Engage with the HE institution in constructive
discussions about the quality of their e-learning, and
how it might be improved
• Provide an independent external review, with advice for
improvement
• Collect feedback on the E-xcellence resources, in order
to improve them

10. Review format
• Preparation
– Decide programme(s) to be reviewed
– Form interdisciplinary team of the HE institution’s managers, course
designers, educators, students, technical staff
– Team meets to complete QuickScan self-evaluation
• Visit or online meetings
– HE team meets with E-xcellence reviewers (2 - 4 e-learning experts)
– Discuss institution’s e-learning offerings and the QuickScan self-evaluation
– Reviewers give initial feedback and suggestions
• Reports
– Summary report from E-xcellence reviewers
– Reviewers consider and agree Roadmap for improvement from the
institution

11. E-xcellence manual update
• Third edition 2016
• Reflects recent trends in e-learning
– rapid rise of MOOCs
– surge of interest in learning analytics
– increasing use of learning design in course development
• Includes new topics
– increased focus on personalization
– flipped approaches to teaching
– virtual and remote laboratories
– digital badges and e-portfolios.

12. In this workshop we …
• Explore the new E-xcellence benchmarks and indicators
• Consider which indicators are helpful for assessing
quality in the rapidly changing e-learning landscape
e-xcellencelabel.eadtu.eu

13. Strategic management
• Example benchmark
– The institution has an e-learning strategy that is widely
understood and integrated into the overall strategies for
institutional development and quality improvement. E-learning
policies conform to legal and ethical frameworks.

14. Strategic management
…significant changes
• Changes to benchmarks
– Benchmark 2 … there is an organisational framework that can foster
innovation and development, and evaluation by scholarship and research.
– Benchmark 5 … development of Open Educational Resources and Massive
Open Online Courses, or use of social media), the roles and responsibilities
are clearly defined, communicated to those concerned, and controlled by
operational agreements where appropriate.
• Text updates for additional issues
– New section on Learning analytics
– Paragraphs on MOOCs
– New indicators for policies on: MOOCs, learning analytics, monitoring
changing technology and educational practice

15. Curriculum Design
• Example benchmark
– Curricula that use e-learning offer personalisation and a flexible
path for the learner, while ensuring the achievement of learning
outcomes.

16. Curriculum Design
...significant changes
• Flexibility
– Pre-requisites for registration and course stay to foster inclusion of life-long learners
– Possibility of conforming personalized curricula
– Existence of virtual mobility
• Academic community development: Evidence of online collaborative learning and team work
• Professional Networks: Existence of virtual mobility for professional practices
• Knowledge and Skills
– Existence of mechanisms for recognizing prior learning
– Results of student’s survey regarding fulfilment of user’s expectations
• Assessment:
– Clarification of Formative and Summative assessment definitions
– Evidence of online collaborative learning and team work by existence of:
• Student’s peer-to-peer review
• Project-based work assessment
– Information on student identification in non-proctored examining procedures: use of identity
cards, fingerprints, …

17. Curriculum Design
• No significant changes in benchmarks
• Additional indicators at excellence level:
– The curriculum includes authentic practical work which may be achieved
online by virtual or remote laboratory facilities. (2.2.3)
– The institution provides digital badges that can be used to recognise generic
skills and other achievements not covered by formal assessment. (2.3.1)
– There are opportunities for virtual internships or other ways of developing
employability skills. (2.3.2)

18. Course Design
• Example benchmark
– E-learning materials have sufficient interactivity (student-to-
content, student-to-student and student-to-teacher) to encourage
active engagement and enable students to test their knowledge,
understanding and skills.

19. Course Design
… significant changes
• Minor changes in benchmark wording
• Additional indicator at excellence level
• Learning analytic data is used to gain insight into the success of aspects of
e-learning design. This is used to improve the design of future courses.
(3.5)
• Text changes
– New section on MOOCs (no indicators – see OpenupEd!)
– Terminology: ‘components’  ‘elements and activities’
– Other issues:
• Monitoring of student feedback
• Vocational / professional competencies in outcomes
• Student workload specified as part of design
• Focus on user friendly & motivating user interface

20. Course Delivery
• Example benchmark
– The systems for communication and storage of data are secure,
reliable and assure appropriate levels of privacy. Measures are
in place for system recovery in the event of failure or breakdown.

21. Course Delivery
• No significant changes in benchmarks
• Changes to indicators:
– There are secure systems for storing and analysing data on students’ learning activity
and interactions with university online systems (learning analytics data). (4.1.1)
– Operating standards are implemented effectively (deleted) (4.1.2)
– The VLE captures data that can be used for learning analytics approaches and there
are appropriate reporting tools such as dashboards for users of this data. (4.2.1)
– The system allows for personalisation. (4.2.1)
– Plagiarism detection systems are available and staff conform to policies on the use of
these systems. (4.2.5)
– Learning analytics approaches use data from online assessments. (4.2.5)
• Changes to text:
– Notes on mobile
– Notes on learning analytics

22. Staff support
• Example benchmark
– The institution ensures that appropriate training and support is
provided for staff and that this training is enhanced in the light of
technological and educational developments.
Technical aspects
Technical support
Technical training
Educational aspects
Educational support
Educational innovation
Resources
Information and media support
Administrative support
Career development, incentives and recognition
6 benchmarks
• 30 indicators
• 17 indicators of excellence

23. Staff support
• Significant changes: New indicators to address e-learning evolution
– Technical Support for …
• Use of new types of software
– Tools for multimedia self-production
• Learning analytics
– Educational Support for …
• Mobile and ubiquitous learning
• Merging of formal and informal learning
• Pedagogical use of learning analytics
• Indexing and archiving of e-learning materials to promote institutional reuse and OER
– Administrative Support
• Improvement of the administrative Support provided by Study Centres
– Career Development
• Formal organization of new roles and functions associated with e-learning within the
institution
• Clear recognition of career progression and incentives in terms of e-learning contributions

24. Student support
• Example benchmark
– Students have access to learning resources, including online library
access, study skills development and a study advisor, and they
receive guidelines and training in using these resources.
5 benchmarks
• 67 indicators
• 22 indicators of excellence
3rd Edition
The chapter has been restructured
and reorganized
Specific areas have been renamed
New areas and indicators have
been added

25. Student support
Student support organization
Institutional student support planning
Support needs for different learner groups
Support staff
Human resource planning
Support role definitions
Administrative support
Technical support
Online services availability
Professional management of technical support
Online technical support system availability
Pedagogical support
Advice and guidance on study skills development
Support for e-learning skills development
Support resources
Course choice advice and guidance
Library resources
Learner communities
Role of study centres in student support delivery
 Significant changes
A systemic institutional
perspective

26. Over to you…
• Work in groups of three
• Choose one chapter that is of interest to you
• Think of a specific new e-learning development
–Can you find an appropriate indicator / benchmark?
• this development is evidence of achievement
–No appropriate indicator / benchmark?
• suggest brief paragraph for ‘Additional notes’
e-xcellencelabel.eadtu.eu/tools/manual
1. Strategic Management
2. Curriculum Design
3. Course Design
4. Course Delivery
5. Student Support
6. Staff Support

27. Plenary discussion

28. Thank you!
• Please contact us to continue discussions
• For more info on E-xcellence:
w: e-xcellencelabel.eadtu.eu
e: secretariat@eadtu.eu