Next Steps for Excellence in the
Quality of e-Learning
Jon Rosewell, Karen Kear, Keith Williams
Dept of Communications and Systems,
Faculty of Maths Computing and Technology,
The Open University, UK

EADTU Masterclass, 23rd Oct 2013, Paris
E-xcellence project 2005–present
Funded by EU Lifelong Learning programme
Managed by EADTU
• E-xcellence 2005-06
– Development and trialling of criteria, handbooks and
methodology
• E-xcellence plus 2008-09
– Dissemination to institutions and to QA agencies in 9 European
countries
• E-xcellence NEXT 2011-12
– Continuing dissemination and updating of criteria and
resources
http://e-xcellencelabel.eadtu.eu
E-xcellence: modes of use
• Informal self-evaluation
– Use Quickscan
• Local seminar
– Local use of Quickscan with justification for rating
– Meeting: institution, project team, national QA agency
– Improvement roadmap
• Full assessment
– As above but part of formal accreditation
– Evidence provided for benchmarks
E-xcellence NEXT: updating
• General updating of manual
– Clarifying language / terminology
• Deal with emerging trends
– Convergence between distance and F2F  blended modes
– Social networking in HE
– Use of Open Educational Resources
• Process:
– Quickscan comments from partners
– Participatory workshops
– Feedback from local seminars
What do we mean by ‘social
networking’?
• ‘Social networking’ can be interpreted broadly to cover a
range of online communication processes
– e.g. via forums, blogs, wikis
• It can also be interpreted more narrowly to focus on
social network sites that provide accessible tools
– e.g. Facebook, Twitter, LinkedIn
Why use social networking?
• Social networking has two primary purposes in
education:
– facilitating learning
• social learning theories
• focused pedagogic function such as group work,
peer assessment
– building communities
• motivation and progress
• informal and social
Social networking tools
•
•
•
•

Forums
Wikis
Blogs
Social network sites

discussion and debate
co-creation of resources
reflection, sharing and feedback
sense of community

• Public (Facebook etc) or walled-garden (VLE)?
– Boundaries and invasion of student space?
Social network sites
Benefits
• many students already
use them regularly
• seen as more social,
informal and flexible

Challenges
• privacy issues
• lack of control
• blurring of boundaries
between social and
academic life.
E-xcellence NEXT: social networking
• How might social networking contribute to high quality in
e-learning?
• What risks to quality might arise?
• Which of the existing E-xcellence quality benchmarks
might apply in this context?
• Are any new benchmarks needed to cover this
scenario?
Social networking
• Where:
– Online communication: forums, blogs, wikis, …
– Sites: Facebook, Twitter, LinkedIn, …
• Why:
– Learning: social learning, collaborative work
– Building communities: motivation, progress, social
• Issues:
– Public (Facebook etc) or walled-garden (VLE)?
– Boundaries and invasion of student space?
Revised benchmarks
– social networking
Curriculum design
10. Curricula are designed to enable participation in
academic communities via online social networking
tools. These online communities provide opportunities
for collaborative learning, contact with external
professionals and involvement in research and
professional activities.
Indicators – social networking
• There are institutional policies relating to the provision of online community
spaces for student-student and student-teacher interactions.
• Curriculum designers specify clearly the educational role that student-student
interaction plays in their programmes.
• Criteria for the assessment of student online collaboration exist and are applied
consistently across programmes and courses.
At excellence level:
• Teaching staff are supported by formal and informal staff development activity
in the use of online tools for community building.
• The institution works closely with professional bodies in the development of
online professional communities.
• Innovative assessment approaches, such as online collaborative work, peer
assessment and self-assessment, form a part of the institution’s practice in this
area.
OER use-cases
•
•
•
•
•
•

Life-long learner finds material for independent study
Individual teacher uses assets in own material
Course uses podcasts from iTunes U
Course uses a 10-hour unit
Entire 100-hour module reused, with new assessment
Course and assignments in OER; tutorial / marking /
accreditation offered for fee
• Consortium develops material for own use and ‘frees’ it
• MOOC
Quality points
Provenance
Provenance
Reputation
Reputation
Brand
Brand

checking
OER
OER
repository
repository
creation
peer review

use

user recommendation
Quality Dimensions
Content
Accuracy
Currency
Relevance

Ease of use
Clarity
Visual attractiveness,
engaging
Clear navigation
Functional!

Pedagogic Effectiveness
Learning objectives
Prerequisites
Learning design
Learning styles
Assessment

Reusability & openness
Format & interoperability
Localisation
Discoverability: metadata
Digital preservation
Accessibility
Trends toward Open Educational Practice?
use
teacher centred
transmission
(sage on stage)
focus on outcome
standardised
individual

 create
 learner centred
 constructivism
 (guide on side)
 focus on process
 personalised learning
 social/ peer learning

Capability maturity model:
Use OERs  Adapt OER material  Create OER material
See, for example, OPAL OEP Guide
E-xcellence NEXT: OERs
• How might OERs contribute to high quality in e-learning?
• What risks to quality might arise?
• Which of the existing E-xcellence quality benchmarks
might apply in this context?
• Are any new benchmarks needed to cover this
scenario?
Revised benchmarks – OERs
Course design
14. OER material is selected with regard to learning
outcome, tailored if necessary for fit to the learning
context, and integrated with other learning materials.
OER materials are subject to the same review
processes as other course materials.
Indicators – OERs
•

The institution has a policy for use of independent learning
materials from a number of quality assured sources, including
OER.
• Course materials obtained from OER are judged fit for purpose
by students and external assessors.
• There is a principled approach to judging the quality of material
obtained from an OER repository.
• There is a process for tracking intellectual property rights
associated with e-learning components.
At excellence level
• E-learning components are contributed to repositories as OER.
Comments and feedback?
Web:

http://www.eadtu.nl/e-xcellencelabel/

Email:

J.P.Rosewell@open.ac.uk
K.L.Kear@open.ac.uk
K.Williams@open.ac.uk

Thank you for your attention

Next steps for excellence in the quality of e-learning (EADTU Paris masterclass)

  • 1.
    Next Steps forExcellence in the Quality of e-Learning Jon Rosewell, Karen Kear, Keith Williams Dept of Communications and Systems, Faculty of Maths Computing and Technology, The Open University, UK EADTU Masterclass, 23rd Oct 2013, Paris
  • 2.
    E-xcellence project 2005–present Fundedby EU Lifelong Learning programme Managed by EADTU • E-xcellence 2005-06 – Development and trialling of criteria, handbooks and methodology • E-xcellence plus 2008-09 – Dissemination to institutions and to QA agencies in 9 European countries • E-xcellence NEXT 2011-12 – Continuing dissemination and updating of criteria and resources
  • 5.
  • 7.
    E-xcellence: modes ofuse • Informal self-evaluation – Use Quickscan • Local seminar – Local use of Quickscan with justification for rating – Meeting: institution, project team, national QA agency – Improvement roadmap • Full assessment – As above but part of formal accreditation – Evidence provided for benchmarks
  • 8.
    E-xcellence NEXT: updating •General updating of manual – Clarifying language / terminology • Deal with emerging trends – Convergence between distance and F2F  blended modes – Social networking in HE – Use of Open Educational Resources • Process: – Quickscan comments from partners – Participatory workshops – Feedback from local seminars
  • 9.
    What do wemean by ‘social networking’? • ‘Social networking’ can be interpreted broadly to cover a range of online communication processes – e.g. via forums, blogs, wikis • It can also be interpreted more narrowly to focus on social network sites that provide accessible tools – e.g. Facebook, Twitter, LinkedIn
  • 10.
    Why use socialnetworking? • Social networking has two primary purposes in education: – facilitating learning • social learning theories • focused pedagogic function such as group work, peer assessment – building communities • motivation and progress • informal and social
  • 11.
    Social networking tools • • • • Forums Wikis Blogs Socialnetwork sites discussion and debate co-creation of resources reflection, sharing and feedback sense of community • Public (Facebook etc) or walled-garden (VLE)? – Boundaries and invasion of student space?
  • 12.
    Social network sites Benefits •many students already use them regularly • seen as more social, informal and flexible Challenges • privacy issues • lack of control • blurring of boundaries between social and academic life.
  • 13.
    E-xcellence NEXT: socialnetworking • How might social networking contribute to high quality in e-learning? • What risks to quality might arise? • Which of the existing E-xcellence quality benchmarks might apply in this context? • Are any new benchmarks needed to cover this scenario?
  • 14.
    Social networking • Where: –Online communication: forums, blogs, wikis, … – Sites: Facebook, Twitter, LinkedIn, … • Why: – Learning: social learning, collaborative work – Building communities: motivation, progress, social • Issues: – Public (Facebook etc) or walled-garden (VLE)? – Boundaries and invasion of student space?
  • 15.
    Revised benchmarks – socialnetworking Curriculum design 10. Curricula are designed to enable participation in academic communities via online social networking tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities.
  • 16.
    Indicators – socialnetworking • There are institutional policies relating to the provision of online community spaces for student-student and student-teacher interactions. • Curriculum designers specify clearly the educational role that student-student interaction plays in their programmes. • Criteria for the assessment of student online collaboration exist and are applied consistently across programmes and courses. At excellence level: • Teaching staff are supported by formal and informal staff development activity in the use of online tools for community building. • The institution works closely with professional bodies in the development of online professional communities. • Innovative assessment approaches, such as online collaborative work, peer assessment and self-assessment, form a part of the institution’s practice in this area.
  • 17.
    OER use-cases • • • • • • Life-long learnerfinds material for independent study Individual teacher uses assets in own material Course uses podcasts from iTunes U Course uses a 10-hour unit Entire 100-hour module reused, with new assessment Course and assignments in OER; tutorial / marking / accreditation offered for fee • Consortium develops material for own use and ‘frees’ it • MOOC
  • 18.
  • 19.
    Quality Dimensions Content Accuracy Currency Relevance Ease ofuse Clarity Visual attractiveness, engaging Clear navigation Functional! Pedagogic Effectiveness Learning objectives Prerequisites Learning design Learning styles Assessment Reusability & openness Format & interoperability Localisation Discoverability: metadata Digital preservation Accessibility
  • 20.
    Trends toward OpenEducational Practice? use teacher centred transmission (sage on stage) focus on outcome standardised individual  create  learner centred  constructivism  (guide on side)  focus on process  personalised learning  social/ peer learning Capability maturity model: Use OERs  Adapt OER material  Create OER material See, for example, OPAL OEP Guide
  • 21.
    E-xcellence NEXT: OERs •How might OERs contribute to high quality in e-learning? • What risks to quality might arise? • Which of the existing E-xcellence quality benchmarks might apply in this context? • Are any new benchmarks needed to cover this scenario?
  • 22.
    Revised benchmarks –OERs Course design 14. OER material is selected with regard to learning outcome, tailored if necessary for fit to the learning context, and integrated with other learning materials. OER materials are subject to the same review processes as other course materials.
  • 23.
    Indicators – OERs • Theinstitution has a policy for use of independent learning materials from a number of quality assured sources, including OER. • Course materials obtained from OER are judged fit for purpose by students and external assessors. • There is a principled approach to judging the quality of material obtained from an OER repository. • There is a process for tracking intellectual property rights associated with e-learning components. At excellence level • E-learning components are contributed to repositories as OER.
  • 24.

Editor's Notes

  • #2 {"5":"Resources: manual including benchmarks and indicators\nStructure\nStrategic Management\nCurriculum Design\nCourse Design\nCourse Delivery\nStudent Support\nStaff Support\n","6":"Online Quickscan\n35 Benchmark statements\nQuick self-assessment of e-learning performance\nRate programme/course on the most relevant aspects \nIdentifies hot / cold spots of e-learning programme/course\nOnline version provides feedback:\nTo identify elements to be improved\nTo guide the internal discussion \nTo learn if a full quality assessment procedure is useful\n","1":"Need to do:\nGet final version of manual – benchmarks changed\nMaybe cut background slides on social networking and OER\nMaybe give indicators as well as benchmarks\nMaybe add slide re process\nDelete slide re OE practices\n","18":"Issue: OER use is very varied in scale – from single assets to whole courses – so QA procedures could be very different in different contexts\n","7":"Online Quickscan\n35 Benchmark statements\nQuick self-assessment of e-learning performance\nRate programme/course on the most relevant aspects \nIdentifies hot / cold spots of e-learning programme/course\nOnline version provides feedback:\nTo identify elements to be improved\nTo guide the internal discussion \nTo learn if a full quality assessment procedure is useful\n","2":"What is the problem?\nThere are established HE QA procedures in Europe\nThese were designed for conventional universities\nThey don’t necessarily fit e-learning\nSolution:\nProvide resources and processes for QA of e-learning\nThese can be adapted for local/national purposes\n","19":"Is it possible to evaluate quality of components in isolation, or only in the context of their use?\nQuality process\nChecking\nPeer review\nFeedback\nRating / voting / recommendation\nBranding / provenance / reputation\n","3":"Resources: manual including benchmarks and indicators\nStructure\nStrategic Management\nCurriculum Design\nCourse Design\nCourse Delivery\nStudent Support\nStaff Support\n","4":"Resources: manual including benchmarks and indicators\nStructure\nStrategic Management\nCurriculum Design\nCourse Design\nCourse Delivery\nStudent Support\nStaff Support\n","10":"Note categories of users\n"}