Quality assurance of MOOCs from
an institutional perspective:
The OpenupEd label
Jon Rosewell & Karen Kear, OUUK
Quality in Higher Education
EMPOWER / EADTU Webinar Week
18-20 Sept 2018

Why worry about MOOC quality?
Students – know what they are committing to
Employers – recognition of content and skills
Authors – personal reputation, 'glow' of success
Universities / Institutions – brand reputation
Funders – philanthropists, government, investors
Quality agencies – on behalf of all above
Participants don’t judge correctly – for the same MOOC:
‘The worse course I’ve taken’ (158 posts)
‘Thanks Dr. ____ for a great course’ (177 posts)

Are MOOCs different from e-learning?
• MOOC vs Higher Education e-learning
– Short, free, no entry requirements
• MOOC participants
– Motivations differ from degree students
– Completion may not be not their goal
• But a MOOC is a Course so maybe it should be
judged like any other HE course?

OpenupEd Quality Label
• Framework for quality
• Formal label
– External review
• Self-assessment
• Include in internal
process
www.openuped.eu/quality-label

OpenupEd MOOC features
• Openness to learners
• Digital openness
• Learner-centred approach
• Independent learning
• Media-supported interaction
• Recognition options
• Quality focus
• Spectrum of diversity

OpenupEd MOOC benchmarks
• Derived from E-xcellence benchmarks
• For the institution:
– To be checked every 3-5 years
– 21 benchmark statements, in six groups:
Strategic management, Curriculum design, Course design, Course
delivery, Staff support, Student support
• For the course:
– To be checked for each MOOC
– 11 benchmark statements

OpenupEd MOOC benchmarks
• Benchmark = statement of best practice
• Benchmarking as quality enhancement tool
– Identification of weaknesses and strengths
– Action plan for improvement
• It is not expected that every institution will
achieve every benchmark or feature
– Improvement is the goal

Course design
9 The institution provides templates or guidelines for layout and
presentation of MOOCS to support consistency across the portfolio. These
templates have the flexibility to accommodate a range of teaching and
learning methods.
10 Course materials, including the intended learning outcomes, are regularly
reviewed, up-dated and improved using feedback from stakeholders.
11 The institution specifies an open licence for MOOC components, and has a
mechanism to track intellectual property rights.
Course delivery
12 The MOOC platform is reliable, secure and assures appropriate levels of
privacy. Provision is made for system maintenance, monitoring and review
of performance.
13 The MOOC platform provides a range of online tools which are appropriate
for the educational models adopted.
Benchmarks – institutional level

22 A clear statement of learning outcomes for both knowledge and skills is
provided.
23 There is reasoned coherence between learning outcomes, course content,
teaching and learning strategy (including use of media), and assessment
methods.
24 Course activities aid participants to construct their own learning and to
communicate it to others.
25 The course content is relevant, accurate, and current.
26 Staff who write and deliver the course have the skills and experience to do
so successfully.
27 Course components have an open licence and are correctly attributed.
Reuse of material is supported by the appropriate choice of formats and
standards.
28 The course conforms to guidelines for layout, presentation and
accessibility.
Benchmarks – course level

Quick scan

Quick scan
NA: Not achieved
PA: Partially achieved
LA: Largely achieved
FA: Fully achieved

OL: Openness to learners
DO: Digital openness
LC: Learner-centred approach
IL: Independent learning
MI: Media-supported interaction
RO: Recognition options
QF: Quality focus
SD: Spectrum of diversity
Quick scan

Case study
• A representative Open University MOOC…
• …published on FutureLearn
• Evidence for OpenupEd features and
benchmarks
• Captured from report/description of processes
Jansen, D., Rosewell, J., & Kear, K. (2017). ‘Quality Frameworks for
MOOCs.’ In: M. Jemni, Kinshuk, & M. K. Khribi (Eds.), Open Education:
from OERs to MOOCs, 261–281. Springer http://oro.open.ac.uk/47595/

Both the UKOU and FutureLearn have clear strategies and processes for MOOC production which are seen as essential to ensuring
quality [#3, #5, QF]. These include commissioning processes on both sides so that course proposals are scrutinised at an early stage,
one output of which is a course description [#18, #22, OL, IL]. This ensures that the course will meet the needs of learners [LC], as well
as contribute to a MOOC portfolio that meets the strategic goals of both the HEI and platform [#1, #8, OL]. The UKOU delivers MOOCs
on FutureLearn (with certificates) and on OpenLearn (with badges) [RO], which also includes access material [#8, OL, SD] and tasters for
core non-MOOC curriculum [#7].
Both the UKOU and FutureLearn take very clear positions on aspects such as openness [#11, #27, DO], accessibility and inclusion [#4,
#28, OL], and these values therefore permeate normal work, helping to ensure that material is produced that conforms to accepted
standards without needing rework at a late stage.
Course design is mainly the responsibility of the HEI, but is supported by guidance documents from FutureLearn [#9]. A strong steer is
provided by the affordances of the platform, which is directed to a particular pedagogical model [#13, #23, LC, IL, MI]. This model
appears to be successful, although it may limits the freedom of course authors to take alternative approaches. At a practical level, this
can be seen in the way that FutureLearn currently only hosts a restricted set of resource types and activities [#13, #23], requiring the
author or HEI to make alternative arrangements for some resources; the result is that not all FutureLearn courses are entirely self-
contained [#5]. Course design is mainly the responsibility of the HEI, but is supported by guidance documents from FutureLearn [#9]. A
strong steer is provided by the affordances of the platform, which is directed to a particular pedagogical model [#13, #23, LC, IL, MI].
This model appears to be successful, although it may limits the freedom of course authors to take alternative approaches. At a practical
level, this can be seen in the way that FutureLearn currently only hosts a restricted set of resource types and activities [#13, #23],
requiring the author or HEI to make alternative arrangements for some resources; the result is that not all FutureLearn courses are
entirely self-contained [#5].
The UKOU process for course design follows the model used in development of their standard non-MOOC provision [#6, QF], although
with fewer staff and at an accelerated pace. The early learning design workshop ensures that there is coherence between content,
teaching and learning strategy and assessment [#23, LC, IL]. This workshop, together with guidelines from FutureLearn and the
affordances of the platform itself (with its clear design in ‘steps’ and the emphasis on social learning [#20, #24, LC, SD]), also ensures
that there is interactivity (student-to-student and student-to-content) to encourage active engagement [#29, LC, IL, MI]. Team writing
and critical reading of drafts help to assure that content is relevant, accurate and current [#25, QF]. The process of course approval,
which includes choice of authors, helps to ensure that staff have the required skills to develop material suitable
Case study: OU + FutureLearn
Tagged for OpenupEd features
Tagged for OpenupEd benchmarks

Both the UKOU and FutureLearn have clear strategies and processes for
MOOC production which are seen as essential to ensuring quality [#3, #5,
QF]. These include commissioning processes on both sides so that course
proposals are scrutinised at an early stage, one output of which is a
course description [#18, #22, OL, IL]. This ensures that the course will
meet the needs of learners [LC], as well as contribute to a MOOC portfolio
that meets the strategic goals of both the HEI and platform [#1, #8, OL].
The UKOU delivers MOOCs on FutureLearn (with certificates) and on
OpenLearn (with badges) [RO], which also includes access material [#8,
OL, SD] and tasters for core non-MOOC curriculum [#7].
Both the UKOU and FutureLearn take very clear positions on aspects such
as openness [#11, #27, DO], accessibility and inclusion [#4, #28, OL], and
these values therefore permeate normal work, helping to ensure that
material is produced that conforms to accepted standards without
needing rework at a late stage.
Case study: OU + FutureLearnInstitution level benchmark
#11: The institution specifies an open licence for MOOC components,
and has a mechanism to track intellectual property rights.
Course level benchmark
#27: Course components have an open licence and are correctly
attributed. Reuse of material is supported by the appropriate choice
of formats and standards.
Features
DO: Digital openness

Both the UKOU and FutureLearn have clear strategies and processes for
MOOC production which are seen as essential to ensuring quality [#3, #5,
QF]. These include commissioning processes on both sides so that course
proposals are scrutinised at an early stage, one output of which is a
course description [#18, #22, OL, IL]. This ensures that the course will
meet the needs of learners [LC], as well as contribute to a MOOC portfolio
that meets the strategic goals of both the HEI and platform [#1, #8, OL].
The UKOU delivers MOOCs on FutureLearn (with certificates) and on
OpenLearn (with badges) [RO], which also includes access material [#8,
OL, SD] and tasters for core non-MOOC curriculum [#7].
Both the UKOU and FutureLearn take very clear positions on aspects such
as openness [#11, #27, DO], accessibility and inclusion [#4, #28, OL], and
these values therefore permeate normal work, helping to ensure that
material is produced that conforms to accepted standards without
needing rework at a late stage.
Case study: OU + FutureLearnInstitution level benchmark
#4: The institution has a service relationship to MOOC participants
that addresses ethical and legal dimensions including accessibility
and data protection.
Course level benchmark
#28: The course conforms to guidelines for layout, presentation and
accessibility.
Features
OL: Openness to learners

Case study: OU + FutureLearn
• Quality emerges from joint efforts of OU (HEI)
& FutureLearn (platform provider)
• Holistic approach:
– Institutional and course level
– Process as well as product
• Structures and processes embed a concern for
quality throughout development, delivery and
evaluation
• There is no ‘Quality Assurance’ stage!

Questions?
www.openuped.eu/quality-label
Email: Jon.Rosewell@open.ac.uk
Karen.Kear@open.ac.uk
Thank you!