SlideShare a Scribd company logo
MOOQ and the
Quality of MOOCs:
Findings & Tools
Christian M. Stracke & Esther Tan
Open University of the Netherlands
Global cooperation: ECNU & KNOU
Global initiative ICORE for OR & OE
International WLS / LINQ Conference
eLC European Institute
ICDE Chair in OER
Dr. Christian M. Stracke:
Open Learning & Education, Innovations,
Policies, Quality & Competences, Impact
Open University of the Netherlands
The Quality of MOOCs
Let’s Learn to Learn
Seamless Learning
Dr. Esther Tan
Technology-Enhanced Learning,
Innovations in & out Classroom
Presentation Outline
1. The Quality of MOOCs: What is the problem?
2. A Conceptual Framework towards MOOC QRF:
Theoretical & Methodological Approach
3. Findings from MOOC surveys:
Learners, Designers & Facilitators
4. Findings from MOOC OEQ:
Learners, Designers & Facilitators
5. Findings from MOOC semi-structured interviews:
Providers
Any discussion on the quality of MOOCs should consider the goals
of both, the MOOC learner and the provider. (Hayes, 2015)
Critical questions to be addressed:
 Who are the MOOC users, and why?
 What makes a good MOOC from the learners’ experiences
with MOOCs?
 What are the best design principles and best practices
as indicators of quality?
 Are these quality indicators also MOOC domain specific and/or
MOOC type specific?
Problem Statement
A Research Framework for MOOQ
“When one designs any course, one has to have some learner
cohort in mind.” (Macleod et al., 2015, p. 9)
Main research goals:
1. Establish a research framework for the subsequent analysis of
MOOC design patterns and best practices
2. Develop a Quality Reference Framework (QRF) for MOOCs
Theoretical Framework
Global MOOC
Quality Survey
Quality Reference Framework with
criteria & checklist for MOOC design
Our main goal is the collaboration with all
to improve Open Education & MOOCs
www.MOOC-quality.eu
 Subjects of Investigation:
Learners, Designers, Facilitators & Providers
 Mixed Methods Approach:
Quantitative & qualitative data
1. Global MOOC Quality Survey,
2. Open-Ended questions (OEQ) &
3. Semi-structured interviews
Methodological Framework
Instruments of Measure
MOOC
Learners
MOOC
Designers
MOOC
Facilitators
MOOC
Providers
TOTAL
Global
MOOC
Quality
Survey
166
(69 qns.)
68
(89 qns.)
33
(58 qns.)
- 267
Open-
ended
Questions
118
(4 qns.)
42
(4 qns.)
27
(4 qns.)
- 185
Semi-
structured
interviews
-
12
(15 qns.)
12
(10 qns.)
12
(13 qns.)
36
MOOC Survey Constructs
Constructs Learners Designers Facilitators
Experience with MOOC X X X
Learning Objectives X X X
Duration and Structure X X
Learning Resources X X X
Accessibility and Inclusion X X
Learning Progress X
Learning Environment X X
Learning Assessment X X X
Learning Certification X X
Design Process X
Pedagogical Decisions X
Learning Support
 Feedback & Facilitation
 Interaction & Collaboration
X X X
Global MOOC
Quality Survey
(GMQS)
Global MOOC Quality Survey
Demographic Profile
Age range of all survey participants (learners, designers & facilitators) by gender
Demographic Profile
Educational level of all survey participants (learners, designers & facilitators) by gender
n VB B N G VG
Learning
experience
166 4 4 13 75 70
Learning Experience (Learners)
n VB B N G VG
Design
experience
68 1 2 13 33 19
Design Experience (Designers)
Interaction from Learners‘ Perspective
n N/A SD D N A SA
LF 146 20 5 13 48 37 23
LL 146 15 3 17 34 51 26
LR 146 9 2 8 25 61 41
GG 146 37 4 15 50 24 16
Note:
LF: Interaction between learners and facilitators
LL: Interaction among learners
LR: Interaction between learners and learning resources
GG: Interaction among teams and groups
n R2 M2 p
LF by
learners
125 .094 9.382 .000***
LL by
learners
130 .101 10.818 .000***
LR by
learners
136 .112 12.286 .000***
GG by
learners
108 .045 4.131 .026*
Bivariate Correlations between LLR4 and LLE4
Interaction from Designers‘ Perspective
n N/A SD D N A SA
LF 52 2 1 5 11 24 9
LL 52 1 1 3 11 19 17
LR 52 3 1 0 4 22 22
GG 52 8 2 10 14 13 5
Note:
LF: Interaction between learners and facilitators
LL: Interaction among learners
LR: Interaction between learners and learning resources
GG: Interaction among teams and groups
n R2 M2 p
LF by
designers
49 .003 0.109 .703
LL by
designers
50 .043 1.595 .143
LR by
designers
48 .046 1.537 .138
GG by
designers
43 .001 0.038 .821
Bivariate Correlations between DLR4 and DDE4
Interaction from Facilitators‘ Perspective
n N/A SD D N A SA
LF 32 0 1 3 3 15 10
LL 32 1 0 3 2 11 15
LR 32 0 0 0 5 12 15
GG 32 5 4 3 3 9 8
Note:
LF: Interaction between learners and facilitators
LL: Interaction among learners
LR: Interaction between learners and learning resources
GG: Interaction among teams and groups
n R2 M2 p
LF by fa-
cilitators
30 .181 0.242 .015*
LL by fa-
cilitators
29 .021 0.296 .435
LR by fa-
cilitators
30 .030 0.287 .342
GG by fa-
cilitators
25 .000 0.305 .971
Bivariate Correlations between FLR4 and FDE4
Open-ended
Questions
(Learners)
No of respondents across the six domains
Domain Learners Designers Facilitators
Social Sciences, Humanities &
Law
24 9 2
Education & Lifelong Learning 19 12 18
Computing & Informatics 18 5 2
Science, Math & Engineering 16 5 1
Nature, Environment & Health 21 6 1
Business, Management &
Economics
20 5 3
Total 118 42 27
MOOC Open-ended Questions (OEQ)
MOOC Learners
Q1. What were the three main strengths of the MOOC?
Q2. What were the three main weaknesses of the MOOC?
Q3. What was missing in the MOOC?
Q4. Looking ahead into the development of this type of
learning experiences, what could be improved in
future MOOCs?
Age Range of MOOC Learners
18-24 25-34 35-44 45-54 55-64 65-74 >75 Total
Business,
Management &
Economics
0 3 6 9 1 1 0 20
Nature, Environment
& Health 1 5 4 1 6 4 0 21
Science, Math &
Engineering 0 3 4 6 0 3 0 16
Computing &
Informatics 1 4 5 4 4 0 0 18
Education & Lifelong
Learning 1 1 7 3 5 2 0 19
Social Sciences,
Humanities &Law 1 4 6 6 4 2 1 24
Total 4 20 32 29 20 12 1 118
Demographics of MOOC Learners
Educational Level of Learners
No
School-
ing
High
school
Bachelor’s
degree
Masters
degree
Doctorate
degree Total
Business,
Management &
Economics
0 1 4 10 5 20
Nature, Environment
& Health 0 2 6 12 1 21
Science, Math &
Engineering 0 1 4 7 4 16
Computing &
Informatics 0 0 3 9 6 18
Education & Lifelong
Learning 1 0 4 5 9 19
Social Sciences,
Humanities &Law 0 0 4 12 8 24
Total 1 4 25 55 33 118
Demographics of MOOC Learners
Gender of MOOC Learners
Male Female Other Total
Business,
Management &
Economics
12 8 0 20
Nature,
Environment &
Health
14 7 0 21
Science, Math &
Engineering
7 9 0 16
Computing &
Informatics
8 10 0 18
Education &
Lifelong Learning
11 8 0 19
Social Sciences,
Humanities &Law
10 13 1 24
Total 62 55 1 118
Demographics of MOOC Learners
Learning Experience
Very
Bad Bad Neutral Good Very
Good Total
Business,
Management &
Economics
1 0 2 9 8 20
Nature,
Environment &
Health
0 0 0 9 12 21
Science, Math &
Engineering
0 0 1 2 13 16
Computing &
Informatics
1 1 1 8 7 18
Education &
Lifelong Learning
1 1 0 9 8 19
Social Sciences,
Humanities &Law
0 0 1 15 8 24
Total 3 2 5 52 56 118
Open-ended Questions (MOOC Learners)
Open-ended Questions (MOOC Learners)
% of Activities Completed
None <25% =50% >75% >100% Total
Business, Management &
Economics
1 2 2 3 12 20
Nature, Environment &
Health
0 5 2 4 10 21
Science, Math &
Engineering
0 1 3 3 9 16
Computing & Informatics 2 3 1 4 8 18
Education & Lifelong
Learning
1 3 2 5 8 19
Social Sciences, Humanities
&Law
1 2 3 9 9 24
Total 5 16 13 28 56 118
Open-ended Questions (MOOC Learners)
Q1. What were the three main strengths of the MOOC?
0
10
20
30
40
50
60
70
80
90
Curriculum
Design &
Delivery
Instructional
Design &
Technology
Assessment &
Evaluation
Facilitation &
Feedback
Interaction &
Collaboration
NumberofComments
Social Sciences, Humanities
and Law
Education and Lifelong
Learning
Computing and Informatics
Science, Maths and
Engineering
Nature, Environment and
Health
Business, Management and
Economics
Open-ended Questions (MOOC Learners)
Q2. What were the three main weaknesses of the MOOC?
0
5
10
15
20
25
30
35
Curriculum Design Instructional
Design &
Technology
Assessment &
Evaluation
Faciliation &
Feedback
Interaction &
Collaboration
NumberofComments
Social Sciences, Humanities and Law
Education and Lifelong Learning
Computing and Informatics
Science, Maths and Engineering
Nature, Environment and Health
Business, Management and Economics
Open-ended Questions (MOOC Learners)
Three main strengths and weaknesses of the MOOC
Strengths Weaknesses
Curriculum Design & Delivery
• Good choice & quality of content
• Good teachers, presenters & tutors
• LO aligns with content
• Weak choice & quality of content
• Short duration
Instructional Design & Technology
• Good integration of IT & media
• Support self-regulation & individual
learning paths
• Poor use of IT technological tools
• Resources lack variety & quality
Interaction & Collaboration
• Encourage local group discussion &
activities
• Foster interaction with field experts
• Lack interaction: learner-tutor
• No support for community building
Open-ended
Questions
(Designers)
MOOC Open-ended Questions (OEQ)
MOOC Designers
Q1. Which were the main design decisions that you made
during the development of the MOOC that later proved to
be successful?
Q2. Which were the three biggest difficulties that you faced
when designing the MOOC?
Q3. Which design decisions did not pay off as you expected?
Q4. Looking ahead into the development of this type of learning
design experiences, what methods and tools could be
helpful to improve the design of future MOOCs?
Age Range of MOOC Designers
25-34 35-44 45-54 55-64 65-74 >75 Total
Business, Management
& Economics
1 1 3 0 0 0 5
Nature, Environment &
Health
1 3 0 1 0 1 6
Science, Math &
Engineering
1 1 1 1 1 0 5
Computing &
Informatics
0 1 4 0 0 0 5
Education & Lifelong
Learning
1 2 5 3 1 0 12
Social Sciences,
Humanities &Law
1 1 1 4 1 1 9
Total 5 9 14 9 3 2 42
Demographics of MOOC Designers
Educational Level of Designers
High
school
Bachelor’s
degree
Masters
degree
Doctorate
degree Total
Business, Management
& Economics
0 0 4 1 5
Nature, Environment &
Health
0 0 3 3 6
Science, Math &
Engineering
0 0 1 4 5
Computing &
Informatics
1 0 1 3 5
Education & Lifelong
Learning
0 0 3 9 12
Social Sciences,
Humanities &Law
0 2 3 4 9
Total 1 2 15 24 42
Demographics of MOOC Designers
Gender of MOOC Designers
Male Female Total
Business, Management
& Economics
3 2 5
Nature, Environment &
Health
2 4 6
Science, Math &
Engineering
2 3 5
Computing &
Informatics
2 3 5
Education & Lifelong
Learning
5 7 12
Social Sciences,
Humanities &Law
6 3 9
Total 20 22 42
Demographics of MOOC Designers
Open-ended Questions (MOOC Designers)
No. of MOOCs Designed
1 2 - 4 5 - 9 >10 Total
Business, Management &
Economics
3 2 0 0 5
Nature, Environment &
Health
2 1 2 1 6
Science, Math &
Engineering
1 2 2 0 5
Computing & Informatics 3 2 0 0 5
Education & Lifelong
Learning
4 5 1 2 12
Social Sciences,
Humanities &Law
3 4 2 0 9
Total 16 16 7 3 42
Design Experience
Very
Bad Bad Neutral Good Very
Good Total
Business,
Management &
Economics
0 0 0 5 0 5
Nature,
Environment &
Health
1 0 1 3 1 6
Science, Math &
Engineering
0 0 1 2 2 5
Computing &
Informatics
0 0 1 3 1 5
Education &
Lifelong Learning
0 0 4 5 3 12
Social Sciences,
Humanities &Law
0 1 0 3 5 9
Total 1 1 7 21 12 42
Open-ended Questions (MOOC Designers)
Open-ended Questions (MOOC Designers)
Q1. Which were the main design decisions that you made during the
development of the MOOC that later proved to be successful?
1
3
3
42
1
1
3
3
6
5
4
0
4
8
12
16
20
24
Assessibility&
Inclusion
Assessment &
Evaluation
Interaction &
Collaboration
Certification&
Accreditation
Curriculum
Design& Delivery
Expertise &
Manpower
Feedback &
Facilitation
Instructional
Design&
Technology
No.ofComments
Social Sciences, Humanities and Law
Education and Lifelong Learning
Computer and Informatics
Science, Maths and Engineering
Nature, Environment and Health
Business, Management and Economics
Open-ended Questions (MOOC Designers)
Q2. Which were the three biggest difficulties that you faced when designing the
MOOC?
3 3
1
3
3
2
8
1
2
2
1
1
1
2
3
5
4
4
2
5
3
0
4
8
12
16
20
24
Assessment &
Evaluation
Curriculum
Design&
Delivery
Expertise &
Manpower
Feedback &
Facilitation
Institutional
Support &
Funding
Instructional
Design&
Technology
Interaction &
Collaboration
Open Access,
Copyrights &
Licensing
No.ofComments
Social Sciences, Humanities and Law
Education and Lifelong Learning
Computer and Informatics
Science, Maths and Engineering
Nature, Environment and Health
Business, Management and
Economics
Open-ended Questions (MOOC Designers)
Successful Decisions Biggest Challenges
Curriculum Design & Delivery
• Content delivery format
• Content structure & LOs
• Weak choice & quality of content
• Short duration
Instructional Design & Technology
• Choice of learning activities
• Integration of IT & media
• Platform, software & production
decisions
Interaction & Collaboration Expertise & Manpower
• Creating community of learners
• Foster interaction between learner
& tutor/ facilitator
• Gap in content & instructional
design knowledge
• Coordination & collaboration, e.g.,
different experts & teaching staff
Three main successful decisions and three biggest challenges
Open-ended
Questions
(Facilitators)
MOOC Open-ended Questions (OEQ)
MOOC Facilitators
Q1. Which were the main decisions that you made during the
facilitation of the MOOC that later proved to be successful?
Q2.Which were the three biggest difficulties that you faced
when facilitating the MOOC?
Q3 Which facilitation decisions did not pay off as you expected?
Q4 Looking ahead into the development of this type of learning
experiences, what methods and tools could be helpful to
improve the facilitation of future MOOCs?
Age Range of MOOC Facilitators
25-34 35-44 45-54 55-64 65-74 >75 Total
Business, Management
& Economics
0 1 0 0 0 2 3
Nature, Environment &
Health
0 0 0 1 0 0 1
Science, Math &
Engineering
0 1 0 0 0 0 1
Computing &
Informatics
1 0 0 1 0 0 2
Education & Lifelong
Learning
0 6 6 3 3 0 18
Social Sciences,
Humanities &Law
0 0 1 1 0 0 2
Total 1 8 7 6 3 2 27
Demographics of MOOC Facilitators
Educational Level of MOOC Facilitators
High
school
Bachelor’s
degree
Masters
degree
Doctorate
degree Total
Business, Management
& Economics
0 0 1 2 3
Nature, Environment &
Health
0 0 0 1 1
Science, Math &
Engineering
0 0 0 1 1
Computing &
Informatics
1 0 1 0 2
Education & Lifelong
Learning
0 4 8 6 18
Social Sciences,
Humanities &Law
0 0 2 0 2
Total 1 4 12 10 27
Demographics of MOOC Facilitators
Gender of MOOC Facilitators
Male Female Total
Business, Management
& Economics
0 3 3
Nature, Environment &
Health
1 0 1
Science, Math &
Engineering
1 0 1
Computing &
Informatics
0 2 2
Education & Lifelong
Learning
10 8 18
Social Sciences,
Humanities &Law
0 2 2
Total 12 15 27
Demographics of MOOC Facilitators
Open-ended Questions (MOOC Facilitators)
No. of MOOCs Facilitated (OEQ)
1 2 - 4 5 - 9 >10 Total
Business, Management &
Economics
2 0 0 1 3
Nature, Environment &
Health
0 0 0 1 1
Science, Math &
Engineering
1 0 0 0 1
Computing & Informatics 1 1 0 0 2
Education & Lifelong
Learning
6 7 2 3 18
Social Sciences,
Humanities &Law
0 2 0 0 2
Total 10 10 2 5 27
Facilitation Experience
Good Very Good Total
Business, Management
& Economics
3 0 3
Nature, Environment &
Health
0 1 1
Science, Math &
Engineering
1 0 1
Computing &
Informatics
1 1 2
Education & Lifelong
Learning
12 6 18
Social Sciences,
Humanities &Law
1 1 2
Total 18 9 27
Open-ended Questions (MOOC Facilitators)
0
1
2
3
4
5
6
7
8
9
Provide
guidelines on
attendance, quiz
& test
Monitor peer
review process
& forum
discussion
Show presence
& provide
feedback
Leverage
technological
tools to support
e-learning
Promote
interaction &
group dynamic
Provide quality
content &
appropriate
pedagogical
approach
No.ofComments
Successful Facilitation Decisions
Social Sciences, Humanities &
Law
Education & Lifelong
Learning
Computing & Informatics
Science, Math & Engineering
Nature, Environment & Health
Business, Management &
Economics
0
1
2
3
4
5
Difficult to
engage &
retain learners
Integrate IT
& Media to
support
learning
Lack IT
competencies
(learners)
Mismatch of
content and
target learners
Overload
learners with
quizzes &
assignments
Provide free
certfication
and course
Unable to
foster
interaction &
collaboration
No.ofComments
Unsucessful Facilitation Decisions
Social Sciences,
Humanities & Law
Education & Lifelong
Learning
Computing & Informatics
Science, Math &
Engineering
Nature, Environment &
Health
Business, Management &
Economics
Semi-Structured
Interviews Questions
(Providers)
Categories Descriptor Indicators
Role of MOOCs in HE Providers perspective
on the role of MOOCs
in the current HE
national and
international scenario
 Pedagogical innovation
 Institutional culture: Toward
hybrid & blended courses;
teaching internally & reaching
out externally
Reasons to provide
MOOCs
Identification of causes
for offering MOOCs
• Institutional mission: to “place
people” as the largest course
provider in FutureLearn; foster
openlearn in the eco system;
educational research on
MOOCs; public engagement to
showcase Uni; capacity
development within the Uni
• Institutional goal: Foster
innovation in digital learning;
stay relevant in the changing
educational ecosystems
Summative Report of Coded Interviews
Categories Descriptor Indicators
Partnerships Establishment, organization
and management of
partnerships
 Partnership structure: Shared
network with a core group of
universities to share research
project
 Great collaboration between
researchers from various
disciplines (excellent model);
good to have external driver to
promote internal change; helped
in best practices.
Institutional
Implementation
Policies
Theoretical design of the
MOOC business model to
implement
 More strategic approach
developed: lifelong learning,
apprenticeship; MOOCs as
experimentation spaces -
separate from other learning &
teaching platforms
 Business model for delivering
MOOCs: undergraduate and
masters degrees) & chargeable
certification (for some types)
Summative Report of Coded Interviews
Categories Descriptor Indicators
Institutional
organizational
strategies
Description of the MOOC
organization and
development process in
the institution
 Support structures: 1. Set up a
content team – work with partners
to design well-designed courses
with quality; 2. Uni put in place a
number pedagogical coordinators
to help teaching staff to prepare
the course and provide ideas and
experiences, technical staff for
recording and post production,
staff for subtitles. 3. Central unit
works with individual academics
to find the best plan/ fit
 Re-train/ Assimilate teachers into
the MOOC culture: provide
teachers a checklist on dos &
donts for content in an online
learning environment.
Summative Report of Coded Interviews
Categories Descriptor Indicators
Sustainability Characterization of the
return on investment
 Funding source(s):
1. Business model: a number of bis
model. Some funded by external org.
most funded internally thru OU open
learning budget (by media budget fr
media unit)
2. Budget from uni for central unit
plus some money from Edx (but not
sufficient) received from Edx every 3
months some money; the objective is
to improve education; not profit-
making.
3. Four mio raised funding externally
for the fully online masters study
Summative Report of Coded Interviews
Categories Descriptor Indicators
Evaluation Design of the evaluation
process for the MOOC
policy established
 Level of success for the
institution:
1. Grant awareness: revenue gains,
paid courses
2. Prestige & research gains.
3. Educational success: moving from
standard to hybrid & blended
learning in accredited courses
4. Community of learners: with other
likeminded people/ other Uni
5. Met objectives, research, Uni
promotion & capacity building
 Specialist evaluation apart from
usual questionnaire on
institutional e-learning and
pedagogical innovation
pedagogical model
Summative Report of Coded Interviews
 Every institution should always have a roadmap to
improve and extend the use of current courses, such
as MOOCs:
1. facilitates implicit social learning, and may also help
to attract new students to programmes.
2, the sociocultural nature of students is changing in
Europe and beyond so it is important for to undertake
multicultural adaptation of courses for the different
student types
3. ECTS for MOOCs
 Globalization strategies of the training spaces:
MOOCs raise new research questions/problems that
deserve further study (e.g., what are the
consequences of these new models to knowledge
sharing/building on a global level).
Summative Report of Coded Interviews
Final words from MOOC Provider
christian.stracke@ou.nl
esther.tan@ou.nl
@ChrMStracke
@Taneste
www.opening-up.education
Let’s collaborate!
Thank you!
Your questions?
To be continued…
Open CC License for
sharing & re-using slides
This work is free to share under the creative commons licence:
"Attribution – Noncommercial – Share Alike 3.0"
You can copy, distribute and transmit the work under the following conditions:
1. Attribution –
2. Noncommercial –
3. Share Alike
Licence: Attribution-Noncommercial-Share Alike
Some rights reserved, see: http://creativecommons.org/licenses/by-nc-sa/3.0/
First References for GMQS
Stracke, C. M., et al. (2018). Gap between MOOC designers' and MOOC
learners' perspectives on interaction and experiences in MOOCs: Findings
from the Global MOOC Quality Survey. In M. Chang, N.-S. Chen, R. Huang,
Kinshuk, K. Moudgalya, S. Murthy, & D. G. Sampson (Eds.), Proceedings
18th IEEE International Conference on Advanced Learning Technologies
(ICALT) (pp. 1-5). IEEE: Computer Society. DOI 10.1109/ICALT.2018.0000
Stracke, C. M., & Tan, E. (2018). The Quality of Open Online Learning and
Education: Towards a Quality Reference Framework for MOOCs. In J. Kay, &
R. Luckin (Eds.), Rethinking learning in the digital age. Making the Learning
Sciences Count: The International Conference of the Learning Sciences
(ICLS) 2018 (pp. 1029-1032). London: ISLS.
Stracke, C. M. et al. (2017). The Quality of Open Online Education: Towards
a Reference Framework for MOOCs. In Proceedings of 2017 IEEE Global
Engineering Education Conference (EDUCON) (pp. 1712-1715).
IEEE Xplore. DOI: 10.1109/EDUCON.2017.7943080
To be continued …
The Quality Reference Framework
Dimension 1: Phases Analysis, Design, Implementation, Realization, Evaluation
Dimension 2: Perspectives Pedagogical, Technological, and Strategic
Dimension 3: Roles Designer, Facilitator, and Provider
MOOQ project:
www.MOOC-quality.eu
Online community for the
Quality Reference Framework:
www.MOOC-quality.net
Session Title:
Minds-on session
on Design
Processes for
MOOCs
Facilitators:
Christian M. Stracke
Cleo Sgouropoulou
Nikos Palavitsinis
Session Title:
Dealing with
practical quality
problems when
running a MOOC
Facilitators:
Bill Vassiliadis
Antonia Stefani
Esther Tan
Achilles Kameas
Parallel Sessions
Session 1:
Minds-on session on Design
Processes for MOOCs
Insights on:
• The Quality Reference
Framework in practice,
• Design decisions,
• Roles of designers,
facilitators and providers
Room: 3rd floor
Session 2:
Dealing with practical
quality problems when
running a MOOC
Insights on:
• Drop-out rates,
• Failure to collaborate,
• Fairness in automatic
assessment
Room: here (1st floor)
Parallel Sessions

More Related Content

What's hot

Redesigning assessment and feedback - landscape review and areas for development
Redesigning assessment and feedback - landscape review and areas for developmentRedesigning assessment and feedback - landscape review and areas for development
Redesigning assessment and feedback - landscape review and areas for development
Jisc
 
2017-05-19 ZUID Conference Future Open Education Libraries Stracke
2017-05-19 ZUID Conference Future Open Education Libraries Stracke2017-05-19 ZUID Conference Future Open Education Libraries Stracke
2017-05-19 ZUID Conference Future Open Education Libraries Stracke
Christian M. Stracke
 
SEMAT & SE Education - LACREST 2013 Keynote
SEMAT & SE Education - LACREST 2013 KeynoteSEMAT & SE Education - LACREST 2013 Keynote
SEMAT & SE Education - LACREST 2013 Keynote
peraire
 
Learning Design
Learning DesignLearning Design
Learning Design
Scot Aldred
 
Electronic management of assessment: challenges to solutions
Electronic management of assessment: challenges to solutionsElectronic management of assessment: challenges to solutions
Electronic management of assessment: challenges to solutions
Jisc
 
Apps for teaching and learning: An institutional approach
Apps for teaching and learning: An institutional approachApps for teaching and learning: An institutional approach
Apps for teaching and learning: An institutional approach
Jisc
 
Professional Development Programme on OER-based e-learning
Professional Development Programme on OER-based e-learningProfessional Development Programme on OER-based e-learning
Professional Development Programme on OER-based e-learning
Pat Toh
 
Understanding student needs and potential for standardisation in module maps
Understanding student needs and potential for standardisation in module mapsUnderstanding student needs and potential for standardisation in module maps
Understanding student needs and potential for standardisation in module maps
Jisc
 
Denver Chadron
Denver ChadronDenver Chadron
Denver Chadron
Ronda Dorsey
 
Motivate-ing
Motivate-ingMotivate-ing
Motivate-ing
Jisc
 
Case study: MOOCs for professional development in global eye health
Case study: MOOCs for professional development in global eye healthCase study: MOOCs for professional development in global eye health
Case study: MOOCs for professional development in global eye health
Sally Parsley
 
The power of inner feedback
The power of inner feedbackThe power of inner feedback
The power of inner feedback
Jisc
 
Champ Oer and MOOC Presentation December 2014
Champ Oer and MOOC Presentation December 2014Champ Oer and MOOC Presentation December 2014
Champ Oer and MOOC Presentation December 2014
cccschamp
 
Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...
Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...
Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...
Michael Clark
 
Online Learning ROI: Building the case for your online learning program
Online Learning ROI: Building the case for your online learning program Online Learning ROI: Building the case for your online learning program
Online Learning ROI: Building the case for your online learning program
Julie Evans
 
The New Math for Justifying Online Learning
The New Math for Justifying Online LearningThe New Math for Justifying Online Learning
The New Math for Justifying Online Learning
Julie Evans
 
Australian Academic Leadership Survey &;Interviews – An interim report - Alla...
Australian Academic Leadership Survey &;Interviews – An interim report - Alla...Australian Academic Leadership Survey &;Interviews – An interim report - Alla...
Australian Academic Leadership Survey &;Interviews – An interim report - Alla...
Blackboard APAC
 
Lgug01 mar11 presentation
Lgug01 mar11 presentationLgug01 mar11 presentation
Lgug01 mar11 presentation
miravogel
 
Blended Welding
Blended WeldingBlended Welding
Blended Welding
cccschamp
 
The assessment diaries
The assessment diariesThe assessment diaries
The assessment diaries
Jisc
 

What's hot (20)

Redesigning assessment and feedback - landscape review and areas for development
Redesigning assessment and feedback - landscape review and areas for developmentRedesigning assessment and feedback - landscape review and areas for development
Redesigning assessment and feedback - landscape review and areas for development
 
2017-05-19 ZUID Conference Future Open Education Libraries Stracke
2017-05-19 ZUID Conference Future Open Education Libraries Stracke2017-05-19 ZUID Conference Future Open Education Libraries Stracke
2017-05-19 ZUID Conference Future Open Education Libraries Stracke
 
SEMAT & SE Education - LACREST 2013 Keynote
SEMAT & SE Education - LACREST 2013 KeynoteSEMAT & SE Education - LACREST 2013 Keynote
SEMAT & SE Education - LACREST 2013 Keynote
 
Learning Design
Learning DesignLearning Design
Learning Design
 
Electronic management of assessment: challenges to solutions
Electronic management of assessment: challenges to solutionsElectronic management of assessment: challenges to solutions
Electronic management of assessment: challenges to solutions
 
Apps for teaching and learning: An institutional approach
Apps for teaching and learning: An institutional approachApps for teaching and learning: An institutional approach
Apps for teaching and learning: An institutional approach
 
Professional Development Programme on OER-based e-learning
Professional Development Programme on OER-based e-learningProfessional Development Programme on OER-based e-learning
Professional Development Programme on OER-based e-learning
 
Understanding student needs and potential for standardisation in module maps
Understanding student needs and potential for standardisation in module mapsUnderstanding student needs and potential for standardisation in module maps
Understanding student needs and potential for standardisation in module maps
 
Denver Chadron
Denver ChadronDenver Chadron
Denver Chadron
 
Motivate-ing
Motivate-ingMotivate-ing
Motivate-ing
 
Case study: MOOCs for professional development in global eye health
Case study: MOOCs for professional development in global eye healthCase study: MOOCs for professional development in global eye health
Case study: MOOCs for professional development in global eye health
 
The power of inner feedback
The power of inner feedbackThe power of inner feedback
The power of inner feedback
 
Champ Oer and MOOC Presentation December 2014
Champ Oer and MOOC Presentation December 2014Champ Oer and MOOC Presentation December 2014
Champ Oer and MOOC Presentation December 2014
 
Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...
Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...
Creating a Leading Applied Computing Department (Tony Clark, Head of Departme...
 
Online Learning ROI: Building the case for your online learning program
Online Learning ROI: Building the case for your online learning program Online Learning ROI: Building the case for your online learning program
Online Learning ROI: Building the case for your online learning program
 
The New Math for Justifying Online Learning
The New Math for Justifying Online LearningThe New Math for Justifying Online Learning
The New Math for Justifying Online Learning
 
Australian Academic Leadership Survey &;Interviews – An interim report - Alla...
Australian Academic Leadership Survey &;Interviews – An interim report - Alla...Australian Academic Leadership Survey &;Interviews – An interim report - Alla...
Australian Academic Leadership Survey &;Interviews – An interim report - Alla...
 
Lgug01 mar11 presentation
Lgug01 mar11 presentationLgug01 mar11 presentation
Lgug01 mar11 presentation
 
Blended Welding
Blended WeldingBlended Welding
Blended Welding
 
The assessment diaries
The assessment diariesThe assessment diaries
The assessment diaries
 

Similar to 2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - Findings and Tools Stracke Tan

2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...
2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...
2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...
Christian M. Stracke
 
2017-03-09 OE Global MOOQ Interactive Workshop Results
2017-03-09 OE Global MOOQ Interactive Workshop Results2017-03-09 OE Global MOOQ Interactive Workshop Results
2017-03-09 OE Global MOOQ Interactive Workshop Results
Christian M. Stracke
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EUmoocs
 
TESTA Interactive Masterclass
TESTA Interactive MasterclassTESTA Interactive Masterclass
TESTA Interactive Masterclass
Tansy Jessop
 
Learning design and learning analytics
Learning design and learning analyticsLearning design and learning analytics
Learning design and learning analytics
Rebecca Ferguson
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EUmoocs
 
Logic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptx
Logic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptxLogic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptx
Logic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptx
abhishekdeo19
 
2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke
2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke
2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke
Christian M. Stracke
 
Best Practices in Designing MOOC
Best Practices in Designing MOOCBest Practices in Designing MOOC
Best Practices in Designing MOOC
David Asirvatham
 
Quarterly Membership Meeting Sept 23 2015
Quarterly Membership Meeting Sept 23 2015Quarterly Membership Meeting Sept 23 2015
Quarterly Membership Meeting Sept 23 2015
Open Education Global (OEGlobal)
 
Lecture 1-Introduction to labour market skills (1).pptx
Lecture 1-Introduction to labour market skills (1).pptxLecture 1-Introduction to labour market skills (1).pptx
Lecture 1-Introduction to labour market skills (1).pptx
ShorooqSuleiman1
 
Learning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCsLearning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCs
Rebecca Ferguson
 
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
Christian M. Stracke
 
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results
Christian M. Stracke
 
Mnet moo cs_ict-bett2015
Mnet moo cs_ict-bett2015Mnet moo cs_ict-bett2015
Mnet moo cs_ict-bett2015
Creative Digital Solutions Limited
 
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
Christian M. Stracke
 
How to Launch an OER Degree | 2017 NE OER Summit
How to Launch an OER Degree | 2017 NE OER SummitHow to Launch an OER Degree | 2017 NE OER Summit
How to Launch an OER Degree | 2017 NE OER Summit
Achieving the Dream
 
Zero Textbook Cost Degrees
Zero Textbook Cost DegreesZero Textbook Cost Degrees
Zero Textbook Cost Degrees
Achieving the Dream
 
Electronic management of assessment webinar slides
Electronic management of assessment webinar slidesElectronic management of assessment webinar slides
Electronic management of assessment webinar slides
Jisc
 
Measuring Our Impact: The Open.Michigan Initiative
Measuring Our Impact: The Open.Michigan InitiativeMeasuring Our Impact: The Open.Michigan Initiative
Measuring Our Impact: The Open.Michigan Initiative
Emily Puckett Rodgers
 

Similar to 2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - Findings and Tools Stracke Tan (20)

2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...
2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...
2018-08-23 EARLI Conference in Bonn Quality Reference Framework for MOOCs Str...
 
2017-03-09 OE Global MOOQ Interactive Workshop Results
2017-03-09 OE Global MOOQ Interactive Workshop Results2017-03-09 OE Global MOOQ Interactive Workshop Results
2017-03-09 OE Global MOOQ Interactive Workshop Results
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
 
TESTA Interactive Masterclass
TESTA Interactive MasterclassTESTA Interactive Masterclass
TESTA Interactive Masterclass
 
Learning design and learning analytics
Learning design and learning analyticsLearning design and learning analytics
Learning design and learning analytics
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
 
Logic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptx
Logic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptxLogic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptx
Logic_Models_for_Program_Design_Implementation_and_Evaluation_Slide_Deck.pptx
 
2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke
2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke
2017-09-02 EARLI 2017 We Need High Drop-out Rates in MOOCs Stracke
 
Best Practices in Designing MOOC
Best Practices in Designing MOOCBest Practices in Designing MOOC
Best Practices in Designing MOOC
 
Quarterly Membership Meeting Sept 23 2015
Quarterly Membership Meeting Sept 23 2015Quarterly Membership Meeting Sept 23 2015
Quarterly Membership Meeting Sept 23 2015
 
Lecture 1-Introduction to labour market skills (1).pptx
Lecture 1-Introduction to labour market skills (1).pptxLecture 1-Introduction to labour market skills (1).pptx
Lecture 1-Introduction to labour market skills (1).pptx
 
Learning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCsLearning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCs
 
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
 
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Results
 
Mnet moo cs_ict-bett2015
Mnet moo cs_ict-bett2015Mnet moo cs_ict-bett2015
Mnet moo cs_ict-bett2015
 
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
 
How to Launch an OER Degree | 2017 NE OER Summit
How to Launch an OER Degree | 2017 NE OER SummitHow to Launch an OER Degree | 2017 NE OER Summit
How to Launch an OER Degree | 2017 NE OER Summit
 
Zero Textbook Cost Degrees
Zero Textbook Cost DegreesZero Textbook Cost Degrees
Zero Textbook Cost Degrees
 
Electronic management of assessment webinar slides
Electronic management of assessment webinar slidesElectronic management of assessment webinar slides
Electronic management of assessment webinar slides
 
Measuring Our Impact: The Open.Michigan Initiative
Measuring Our Impact: The Open.Michigan InitiativeMeasuring Our Impact: The Open.Michigan Initiative
Measuring Our Impact: The Open.Michigan Initiative
 

More from Christian M. Stracke

20221205_Online_Education_ChrMStracke_EP.pdf
20221205_Online_Education_ChrMStracke_EP.pdf20221205_Online_Education_ChrMStracke_EP.pdf
20221205_Online_Education_ChrMStracke_EP.pdf
Christian M. Stracke
 
GSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdf
GSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdfGSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdf
GSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdf
Christian M. Stracke
 
2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...
2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...
2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...
Christian M. Stracke
 
2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al
2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al
2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al
Christian M. Stracke
 
2018-04-24 OE Global Better MOOCs Stracke et al
2018-04-24 OE Global Better MOOCs Stracke et al2018-04-24 OE Global Better MOOCs Stracke et al
2018-04-24 OE Global Better MOOCs Stracke et al
Christian M. Stracke
 
2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...
2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...
2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...
Christian M. Stracke
 
2017-10-27 OEE Online Discussion Quality in Open Education Stracke
2017-10-27 OEE Online Discussion Quality in Open Education Stracke2017-10-27 OEE Online Discussion Quality in Open Education Stracke
2017-10-27 OEE Online Discussion Quality in Open Education Stracke
Christian M. Stracke
 
2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke
2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke
2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke
Christian M. Stracke
 
2017-09-13 EC-TEL MOOQ Interactive Workshop Results
2017-09-13 EC-TEL MOOQ Interactive Workshop Results2017-09-13 EC-TEL MOOQ Interactive Workshop Results
2017-09-13 EC-TEL MOOQ Interactive Workshop Results
Christian M. Stracke
 
2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan
2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan
2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan
Christian M. Stracke
 
2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results
Christian M. Stracke
 
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
Christian M. Stracke
 
2016-03-11 Open Education Week 2016 Open Learning Stracke
2016-03-11 Open Education Week 2016 Open Learning Stracke2016-03-11 Open Education Week 2016 Open Learning Stracke
2016-03-11 Open Education Week 2016 Open Learning Stracke
Christian M. Stracke
 
2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke
2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke
2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke
Christian M. Stracke
 
2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke
2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke
2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke
Christian M. Stracke
 
2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...
2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...
2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...
Christian M. Stracke
 
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke
Christian M. Stracke
 
2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke
2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke
2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke
Christian M. Stracke
 
2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE
2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE
2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE
Christian M. Stracke
 
2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke
2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke
2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke
Christian M. Stracke
 

More from Christian M. Stracke (20)

20221205_Online_Education_ChrMStracke_EP.pdf
20221205_Online_Education_ChrMStracke_EP.pdf20221205_Online_Education_ChrMStracke_EP.pdf
20221205_Online_Education_ChrMStracke_EP.pdf
 
GSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdf
GSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdfGSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdf
GSE_2022_Keynote_Learning_Quality_AI_Ethics_Stracke.pdf
 
2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...
2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...
2019-07-09 Quality and Future of Open Education and MOOCs - Keynote at IEC201...
 
2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al
2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al
2018-04-24 OE Global OER Community for UNESCO OER Action Plan Stracke et al
 
2018-04-24 OE Global Better MOOCs Stracke et al
2018-04-24 OE Global Better MOOCs Stracke et al2018-04-24 OE Global Better MOOCs Stracke et al
2018-04-24 OE Global Better MOOCs Stracke et al
 
2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...
2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...
2018-03-05 Keynote Quality Design Online Courses OpenEd Framework Mooc Survey...
 
2017-10-27 OEE Online Discussion Quality in Open Education Stracke
2017-10-27 OEE Online Discussion Quality in Open Education Stracke2017-10-27 OEE Online Discussion Quality in Open Education Stracke
2017-10-27 OEE Online Discussion Quality in Open Education Stracke
 
2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke
2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke
2017-06-09 WLS LINQ Evidence-based Inclusive School Education Stracke
 
2017-09-13 EC-TEL MOOQ Interactive Workshop Results
2017-09-13 EC-TEL MOOQ Interactive Workshop Results2017-09-13 EC-TEL MOOQ Interactive Workshop Results
2017-09-13 EC-TEL MOOQ Interactive Workshop Results
 
2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan
2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan
2017-09-13 EC-TEL MOOQ Interactive Workshop Presentation Stracke Tan
 
2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results
 
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
 
2016-03-11 Open Education Week 2016 Open Learning Stracke
2016-03-11 Open Education Week 2016 Open Learning Stracke2016-03-11 Open Education Week 2016 Open Learning Stracke
2016-03-11 Open Education Week 2016 Open Learning Stracke
 
2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke
2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke
2017-04-27 IEEE EDUCON Open Education Learning Quality Stracke
 
2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke
2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke
2017-05-14 KNOU Seminar Open Education OER MOOCs Learning Analytics Stracke
 
2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...
2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...
2017-05-12 KNOU Symposium Education Changes need Open Education and Quality S...
 
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke
2017-04-25 IEEE EDUCON MOOQ Interactive Workshop Presentation Stracke
 
2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke
2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke
2017-03-09 OE Global MOOQ Interactive Workshop Presentation Stracke
 
2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE
2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE
2017-03-08 OE Global Panel ICORE CC OEC UNESCO IITE
 
2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke
2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke
2016-12-02 ICDE Digital Transformation Quality HE Christian M. Stracke
 

Recently uploaded

How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
HajraNaeem15
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
iammrhaywood
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
PsychoTech Services
 
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptxSWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
zuzanka
 
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
EduSkills OECD
 
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
imrankhan141184
 
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
Nguyen Thanh Tu Collection
 
Skimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S EliotSkimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S Eliot
nitinpv4ai
 
Pharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brubPharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brub
danielkiash986
 
Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10
nitinpv4ai
 
Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"
National Information Standards Organization (NISO)
 
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumPhilippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
MJDuyan
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
Krassimira Luka
 
Data Structure using C by Dr. K Adisesha .ppsx
Data Structure using C by Dr. K Adisesha .ppsxData Structure using C by Dr. K Adisesha .ppsx
Data Structure using C by Dr. K Adisesha .ppsx
Prof. Dr. K. Adisesha
 
Nutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour TrainingNutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour Training
melliereed
 
Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...
Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...
Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...
ImMuslim
 
How to Predict Vendor Bill Product in Odoo 17
How to Predict Vendor Bill Product in Odoo 17How to Predict Vendor Bill Product in Odoo 17
How to Predict Vendor Bill Product in Odoo 17
Celine George
 
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
TechSoup
 
Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)
nitinpv4ai
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
deepaannamalai16
 

Recently uploaded (20)

How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
 
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptxSWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
 
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
 
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
 
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
 
Skimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S EliotSkimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S Eliot
 
Pharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brubPharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brub
 
Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10
 
Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"
 
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumPhilippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
 
Data Structure using C by Dr. K Adisesha .ppsx
Data Structure using C by Dr. K Adisesha .ppsxData Structure using C by Dr. K Adisesha .ppsx
Data Structure using C by Dr. K Adisesha .ppsx
 
Nutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour TrainingNutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour Training
 
Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...
Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...
Geography as a Discipline Chapter 1 __ Class 11 Geography NCERT _ Class Notes...
 
How to Predict Vendor Bill Product in Odoo 17
How to Predict Vendor Bill Product in Odoo 17How to Predict Vendor Bill Product in Odoo 17
How to Predict Vendor Bill Product in Odoo 17
 
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
 
Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
 

2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - Findings and Tools Stracke Tan

  • 1. MOOQ and the Quality of MOOCs: Findings & Tools Christian M. Stracke & Esther Tan
  • 2. Open University of the Netherlands Global cooperation: ECNU & KNOU Global initiative ICORE for OR & OE International WLS / LINQ Conference eLC European Institute ICDE Chair in OER Dr. Christian M. Stracke: Open Learning & Education, Innovations, Policies, Quality & Competences, Impact
  • 3. Open University of the Netherlands The Quality of MOOCs Let’s Learn to Learn Seamless Learning Dr. Esther Tan Technology-Enhanced Learning, Innovations in & out Classroom
  • 4. Presentation Outline 1. The Quality of MOOCs: What is the problem? 2. A Conceptual Framework towards MOOC QRF: Theoretical & Methodological Approach 3. Findings from MOOC surveys: Learners, Designers & Facilitators 4. Findings from MOOC OEQ: Learners, Designers & Facilitators 5. Findings from MOOC semi-structured interviews: Providers
  • 5. Any discussion on the quality of MOOCs should consider the goals of both, the MOOC learner and the provider. (Hayes, 2015) Critical questions to be addressed:  Who are the MOOC users, and why?  What makes a good MOOC from the learners’ experiences with MOOCs?  What are the best design principles and best practices as indicators of quality?  Are these quality indicators also MOOC domain specific and/or MOOC type specific? Problem Statement
  • 6. A Research Framework for MOOQ “When one designs any course, one has to have some learner cohort in mind.” (Macleod et al., 2015, p. 9) Main research goals: 1. Establish a research framework for the subsequent analysis of MOOC design patterns and best practices 2. Develop a Quality Reference Framework (QRF) for MOOCs Theoretical Framework
  • 8. Quality Reference Framework with criteria & checklist for MOOC design Our main goal is the collaboration with all to improve Open Education & MOOCs www.MOOC-quality.eu
  • 9.  Subjects of Investigation: Learners, Designers, Facilitators & Providers  Mixed Methods Approach: Quantitative & qualitative data 1. Global MOOC Quality Survey, 2. Open-Ended questions (OEQ) & 3. Semi-structured interviews Methodological Framework
  • 10. Instruments of Measure MOOC Learners MOOC Designers MOOC Facilitators MOOC Providers TOTAL Global MOOC Quality Survey 166 (69 qns.) 68 (89 qns.) 33 (58 qns.) - 267 Open- ended Questions 118 (4 qns.) 42 (4 qns.) 27 (4 qns.) - 185 Semi- structured interviews - 12 (15 qns.) 12 (10 qns.) 12 (13 qns.) 36
  • 11. MOOC Survey Constructs Constructs Learners Designers Facilitators Experience with MOOC X X X Learning Objectives X X X Duration and Structure X X Learning Resources X X X Accessibility and Inclusion X X Learning Progress X Learning Environment X X Learning Assessment X X X Learning Certification X X Design Process X Pedagogical Decisions X Learning Support  Feedback & Facilitation  Interaction & Collaboration X X X
  • 14. Demographic Profile Age range of all survey participants (learners, designers & facilitators) by gender
  • 15. Demographic Profile Educational level of all survey participants (learners, designers & facilitators) by gender
  • 16. n VB B N G VG Learning experience 166 4 4 13 75 70 Learning Experience (Learners)
  • 17. n VB B N G VG Design experience 68 1 2 13 33 19 Design Experience (Designers)
  • 18. Interaction from Learners‘ Perspective n N/A SD D N A SA LF 146 20 5 13 48 37 23 LL 146 15 3 17 34 51 26 LR 146 9 2 8 25 61 41 GG 146 37 4 15 50 24 16 Note: LF: Interaction between learners and facilitators LL: Interaction among learners LR: Interaction between learners and learning resources GG: Interaction among teams and groups
  • 19. n R2 M2 p LF by learners 125 .094 9.382 .000*** LL by learners 130 .101 10.818 .000*** LR by learners 136 .112 12.286 .000*** GG by learners 108 .045 4.131 .026* Bivariate Correlations between LLR4 and LLE4
  • 20. Interaction from Designers‘ Perspective n N/A SD D N A SA LF 52 2 1 5 11 24 9 LL 52 1 1 3 11 19 17 LR 52 3 1 0 4 22 22 GG 52 8 2 10 14 13 5 Note: LF: Interaction between learners and facilitators LL: Interaction among learners LR: Interaction between learners and learning resources GG: Interaction among teams and groups
  • 21. n R2 M2 p LF by designers 49 .003 0.109 .703 LL by designers 50 .043 1.595 .143 LR by designers 48 .046 1.537 .138 GG by designers 43 .001 0.038 .821 Bivariate Correlations between DLR4 and DDE4
  • 22. Interaction from Facilitators‘ Perspective n N/A SD D N A SA LF 32 0 1 3 3 15 10 LL 32 1 0 3 2 11 15 LR 32 0 0 0 5 12 15 GG 32 5 4 3 3 9 8 Note: LF: Interaction between learners and facilitators LL: Interaction among learners LR: Interaction between learners and learning resources GG: Interaction among teams and groups
  • 23. n R2 M2 p LF by fa- cilitators 30 .181 0.242 .015* LL by fa- cilitators 29 .021 0.296 .435 LR by fa- cilitators 30 .030 0.287 .342 GG by fa- cilitators 25 .000 0.305 .971 Bivariate Correlations between FLR4 and FDE4
  • 25. No of respondents across the six domains Domain Learners Designers Facilitators Social Sciences, Humanities & Law 24 9 2 Education & Lifelong Learning 19 12 18 Computing & Informatics 18 5 2 Science, Math & Engineering 16 5 1 Nature, Environment & Health 21 6 1 Business, Management & Economics 20 5 3 Total 118 42 27
  • 26. MOOC Open-ended Questions (OEQ) MOOC Learners Q1. What were the three main strengths of the MOOC? Q2. What were the three main weaknesses of the MOOC? Q3. What was missing in the MOOC? Q4. Looking ahead into the development of this type of learning experiences, what could be improved in future MOOCs?
  • 27. Age Range of MOOC Learners 18-24 25-34 35-44 45-54 55-64 65-74 >75 Total Business, Management & Economics 0 3 6 9 1 1 0 20 Nature, Environment & Health 1 5 4 1 6 4 0 21 Science, Math & Engineering 0 3 4 6 0 3 0 16 Computing & Informatics 1 4 5 4 4 0 0 18 Education & Lifelong Learning 1 1 7 3 5 2 0 19 Social Sciences, Humanities &Law 1 4 6 6 4 2 1 24 Total 4 20 32 29 20 12 1 118 Demographics of MOOC Learners
  • 28. Educational Level of Learners No School- ing High school Bachelor’s degree Masters degree Doctorate degree Total Business, Management & Economics 0 1 4 10 5 20 Nature, Environment & Health 0 2 6 12 1 21 Science, Math & Engineering 0 1 4 7 4 16 Computing & Informatics 0 0 3 9 6 18 Education & Lifelong Learning 1 0 4 5 9 19 Social Sciences, Humanities &Law 0 0 4 12 8 24 Total 1 4 25 55 33 118 Demographics of MOOC Learners
  • 29. Gender of MOOC Learners Male Female Other Total Business, Management & Economics 12 8 0 20 Nature, Environment & Health 14 7 0 21 Science, Math & Engineering 7 9 0 16 Computing & Informatics 8 10 0 18 Education & Lifelong Learning 11 8 0 19 Social Sciences, Humanities &Law 10 13 1 24 Total 62 55 1 118 Demographics of MOOC Learners
  • 30. Learning Experience Very Bad Bad Neutral Good Very Good Total Business, Management & Economics 1 0 2 9 8 20 Nature, Environment & Health 0 0 0 9 12 21 Science, Math & Engineering 0 0 1 2 13 16 Computing & Informatics 1 1 1 8 7 18 Education & Lifelong Learning 1 1 0 9 8 19 Social Sciences, Humanities &Law 0 0 1 15 8 24 Total 3 2 5 52 56 118 Open-ended Questions (MOOC Learners)
  • 31. Open-ended Questions (MOOC Learners) % of Activities Completed None <25% =50% >75% >100% Total Business, Management & Economics 1 2 2 3 12 20 Nature, Environment & Health 0 5 2 4 10 21 Science, Math & Engineering 0 1 3 3 9 16 Computing & Informatics 2 3 1 4 8 18 Education & Lifelong Learning 1 3 2 5 8 19 Social Sciences, Humanities &Law 1 2 3 9 9 24 Total 5 16 13 28 56 118
  • 32. Open-ended Questions (MOOC Learners) Q1. What were the three main strengths of the MOOC? 0 10 20 30 40 50 60 70 80 90 Curriculum Design & Delivery Instructional Design & Technology Assessment & Evaluation Facilitation & Feedback Interaction & Collaboration NumberofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computing and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  • 33. Open-ended Questions (MOOC Learners) Q2. What were the three main weaknesses of the MOOC? 0 5 10 15 20 25 30 35 Curriculum Design Instructional Design & Technology Assessment & Evaluation Faciliation & Feedback Interaction & Collaboration NumberofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computing and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  • 34. Open-ended Questions (MOOC Learners) Three main strengths and weaknesses of the MOOC Strengths Weaknesses Curriculum Design & Delivery • Good choice & quality of content • Good teachers, presenters & tutors • LO aligns with content • Weak choice & quality of content • Short duration Instructional Design & Technology • Good integration of IT & media • Support self-regulation & individual learning paths • Poor use of IT technological tools • Resources lack variety & quality Interaction & Collaboration • Encourage local group discussion & activities • Foster interaction with field experts • Lack interaction: learner-tutor • No support for community building
  • 36. MOOC Open-ended Questions (OEQ) MOOC Designers Q1. Which were the main design decisions that you made during the development of the MOOC that later proved to be successful? Q2. Which were the three biggest difficulties that you faced when designing the MOOC? Q3. Which design decisions did not pay off as you expected? Q4. Looking ahead into the development of this type of learning design experiences, what methods and tools could be helpful to improve the design of future MOOCs?
  • 37. Age Range of MOOC Designers 25-34 35-44 45-54 55-64 65-74 >75 Total Business, Management & Economics 1 1 3 0 0 0 5 Nature, Environment & Health 1 3 0 1 0 1 6 Science, Math & Engineering 1 1 1 1 1 0 5 Computing & Informatics 0 1 4 0 0 0 5 Education & Lifelong Learning 1 2 5 3 1 0 12 Social Sciences, Humanities &Law 1 1 1 4 1 1 9 Total 5 9 14 9 3 2 42 Demographics of MOOC Designers
  • 38. Educational Level of Designers High school Bachelor’s degree Masters degree Doctorate degree Total Business, Management & Economics 0 0 4 1 5 Nature, Environment & Health 0 0 3 3 6 Science, Math & Engineering 0 0 1 4 5 Computing & Informatics 1 0 1 3 5 Education & Lifelong Learning 0 0 3 9 12 Social Sciences, Humanities &Law 0 2 3 4 9 Total 1 2 15 24 42 Demographics of MOOC Designers
  • 39. Gender of MOOC Designers Male Female Total Business, Management & Economics 3 2 5 Nature, Environment & Health 2 4 6 Science, Math & Engineering 2 3 5 Computing & Informatics 2 3 5 Education & Lifelong Learning 5 7 12 Social Sciences, Humanities &Law 6 3 9 Total 20 22 42 Demographics of MOOC Designers
  • 40. Open-ended Questions (MOOC Designers) No. of MOOCs Designed 1 2 - 4 5 - 9 >10 Total Business, Management & Economics 3 2 0 0 5 Nature, Environment & Health 2 1 2 1 6 Science, Math & Engineering 1 2 2 0 5 Computing & Informatics 3 2 0 0 5 Education & Lifelong Learning 4 5 1 2 12 Social Sciences, Humanities &Law 3 4 2 0 9 Total 16 16 7 3 42
  • 41. Design Experience Very Bad Bad Neutral Good Very Good Total Business, Management & Economics 0 0 0 5 0 5 Nature, Environment & Health 1 0 1 3 1 6 Science, Math & Engineering 0 0 1 2 2 5 Computing & Informatics 0 0 1 3 1 5 Education & Lifelong Learning 0 0 4 5 3 12 Social Sciences, Humanities &Law 0 1 0 3 5 9 Total 1 1 7 21 12 42 Open-ended Questions (MOOC Designers)
  • 42. Open-ended Questions (MOOC Designers) Q1. Which were the main design decisions that you made during the development of the MOOC that later proved to be successful? 1 3 3 42 1 1 3 3 6 5 4 0 4 8 12 16 20 24 Assessibility& Inclusion Assessment & Evaluation Interaction & Collaboration Certification& Accreditation Curriculum Design& Delivery Expertise & Manpower Feedback & Facilitation Instructional Design& Technology No.ofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computer and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  • 43. Open-ended Questions (MOOC Designers) Q2. Which were the three biggest difficulties that you faced when designing the MOOC? 3 3 1 3 3 2 8 1 2 2 1 1 1 2 3 5 4 4 2 5 3 0 4 8 12 16 20 24 Assessment & Evaluation Curriculum Design& Delivery Expertise & Manpower Feedback & Facilitation Institutional Support & Funding Instructional Design& Technology Interaction & Collaboration Open Access, Copyrights & Licensing No.ofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computer and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  • 44. Open-ended Questions (MOOC Designers) Successful Decisions Biggest Challenges Curriculum Design & Delivery • Content delivery format • Content structure & LOs • Weak choice & quality of content • Short duration Instructional Design & Technology • Choice of learning activities • Integration of IT & media • Platform, software & production decisions Interaction & Collaboration Expertise & Manpower • Creating community of learners • Foster interaction between learner & tutor/ facilitator • Gap in content & instructional design knowledge • Coordination & collaboration, e.g., different experts & teaching staff Three main successful decisions and three biggest challenges
  • 46. MOOC Open-ended Questions (OEQ) MOOC Facilitators Q1. Which were the main decisions that you made during the facilitation of the MOOC that later proved to be successful? Q2.Which were the three biggest difficulties that you faced when facilitating the MOOC? Q3 Which facilitation decisions did not pay off as you expected? Q4 Looking ahead into the development of this type of learning experiences, what methods and tools could be helpful to improve the facilitation of future MOOCs?
  • 47. Age Range of MOOC Facilitators 25-34 35-44 45-54 55-64 65-74 >75 Total Business, Management & Economics 0 1 0 0 0 2 3 Nature, Environment & Health 0 0 0 1 0 0 1 Science, Math & Engineering 0 1 0 0 0 0 1 Computing & Informatics 1 0 0 1 0 0 2 Education & Lifelong Learning 0 6 6 3 3 0 18 Social Sciences, Humanities &Law 0 0 1 1 0 0 2 Total 1 8 7 6 3 2 27 Demographics of MOOC Facilitators
  • 48. Educational Level of MOOC Facilitators High school Bachelor’s degree Masters degree Doctorate degree Total Business, Management & Economics 0 0 1 2 3 Nature, Environment & Health 0 0 0 1 1 Science, Math & Engineering 0 0 0 1 1 Computing & Informatics 1 0 1 0 2 Education & Lifelong Learning 0 4 8 6 18 Social Sciences, Humanities &Law 0 0 2 0 2 Total 1 4 12 10 27 Demographics of MOOC Facilitators
  • 49. Gender of MOOC Facilitators Male Female Total Business, Management & Economics 0 3 3 Nature, Environment & Health 1 0 1 Science, Math & Engineering 1 0 1 Computing & Informatics 0 2 2 Education & Lifelong Learning 10 8 18 Social Sciences, Humanities &Law 0 2 2 Total 12 15 27 Demographics of MOOC Facilitators
  • 50. Open-ended Questions (MOOC Facilitators) No. of MOOCs Facilitated (OEQ) 1 2 - 4 5 - 9 >10 Total Business, Management & Economics 2 0 0 1 3 Nature, Environment & Health 0 0 0 1 1 Science, Math & Engineering 1 0 0 0 1 Computing & Informatics 1 1 0 0 2 Education & Lifelong Learning 6 7 2 3 18 Social Sciences, Humanities &Law 0 2 0 0 2 Total 10 10 2 5 27
  • 51. Facilitation Experience Good Very Good Total Business, Management & Economics 3 0 3 Nature, Environment & Health 0 1 1 Science, Math & Engineering 1 0 1 Computing & Informatics 1 1 2 Education & Lifelong Learning 12 6 18 Social Sciences, Humanities &Law 1 1 2 Total 18 9 27 Open-ended Questions (MOOC Facilitators)
  • 52. 0 1 2 3 4 5 6 7 8 9 Provide guidelines on attendance, quiz & test Monitor peer review process & forum discussion Show presence & provide feedback Leverage technological tools to support e-learning Promote interaction & group dynamic Provide quality content & appropriate pedagogical approach No.ofComments Successful Facilitation Decisions Social Sciences, Humanities & Law Education & Lifelong Learning Computing & Informatics Science, Math & Engineering Nature, Environment & Health Business, Management & Economics
  • 53. 0 1 2 3 4 5 Difficult to engage & retain learners Integrate IT & Media to support learning Lack IT competencies (learners) Mismatch of content and target learners Overload learners with quizzes & assignments Provide free certfication and course Unable to foster interaction & collaboration No.ofComments Unsucessful Facilitation Decisions Social Sciences, Humanities & Law Education & Lifelong Learning Computing & Informatics Science, Math & Engineering Nature, Environment & Health Business, Management & Economics
  • 55. Categories Descriptor Indicators Role of MOOCs in HE Providers perspective on the role of MOOCs in the current HE national and international scenario  Pedagogical innovation  Institutional culture: Toward hybrid & blended courses; teaching internally & reaching out externally Reasons to provide MOOCs Identification of causes for offering MOOCs • Institutional mission: to “place people” as the largest course provider in FutureLearn; foster openlearn in the eco system; educational research on MOOCs; public engagement to showcase Uni; capacity development within the Uni • Institutional goal: Foster innovation in digital learning; stay relevant in the changing educational ecosystems Summative Report of Coded Interviews
  • 56. Categories Descriptor Indicators Partnerships Establishment, organization and management of partnerships  Partnership structure: Shared network with a core group of universities to share research project  Great collaboration between researchers from various disciplines (excellent model); good to have external driver to promote internal change; helped in best practices. Institutional Implementation Policies Theoretical design of the MOOC business model to implement  More strategic approach developed: lifelong learning, apprenticeship; MOOCs as experimentation spaces - separate from other learning & teaching platforms  Business model for delivering MOOCs: undergraduate and masters degrees) & chargeable certification (for some types) Summative Report of Coded Interviews
  • 57. Categories Descriptor Indicators Institutional organizational strategies Description of the MOOC organization and development process in the institution  Support structures: 1. Set up a content team – work with partners to design well-designed courses with quality; 2. Uni put in place a number pedagogical coordinators to help teaching staff to prepare the course and provide ideas and experiences, technical staff for recording and post production, staff for subtitles. 3. Central unit works with individual academics to find the best plan/ fit  Re-train/ Assimilate teachers into the MOOC culture: provide teachers a checklist on dos & donts for content in an online learning environment. Summative Report of Coded Interviews
  • 58. Categories Descriptor Indicators Sustainability Characterization of the return on investment  Funding source(s): 1. Business model: a number of bis model. Some funded by external org. most funded internally thru OU open learning budget (by media budget fr media unit) 2. Budget from uni for central unit plus some money from Edx (but not sufficient) received from Edx every 3 months some money; the objective is to improve education; not profit- making. 3. Four mio raised funding externally for the fully online masters study Summative Report of Coded Interviews
  • 59. Categories Descriptor Indicators Evaluation Design of the evaluation process for the MOOC policy established  Level of success for the institution: 1. Grant awareness: revenue gains, paid courses 2. Prestige & research gains. 3. Educational success: moving from standard to hybrid & blended learning in accredited courses 4. Community of learners: with other likeminded people/ other Uni 5. Met objectives, research, Uni promotion & capacity building  Specialist evaluation apart from usual questionnaire on institutional e-learning and pedagogical innovation pedagogical model Summative Report of Coded Interviews
  • 60.  Every institution should always have a roadmap to improve and extend the use of current courses, such as MOOCs: 1. facilitates implicit social learning, and may also help to attract new students to programmes. 2, the sociocultural nature of students is changing in Europe and beyond so it is important for to undertake multicultural adaptation of courses for the different student types 3. ECTS for MOOCs  Globalization strategies of the training spaces: MOOCs raise new research questions/problems that deserve further study (e.g., what are the consequences of these new models to knowledge sharing/building on a global level). Summative Report of Coded Interviews Final words from MOOC Provider
  • 62. Thank you! Your questions? To be continued…
  • 63. Open CC License for sharing & re-using slides This work is free to share under the creative commons licence: "Attribution – Noncommercial – Share Alike 3.0" You can copy, distribute and transmit the work under the following conditions: 1. Attribution – 2. Noncommercial – 3. Share Alike Licence: Attribution-Noncommercial-Share Alike Some rights reserved, see: http://creativecommons.org/licenses/by-nc-sa/3.0/
  • 64. First References for GMQS Stracke, C. M., et al. (2018). Gap between MOOC designers' and MOOC learners' perspectives on interaction and experiences in MOOCs: Findings from the Global MOOC Quality Survey. In M. Chang, N.-S. Chen, R. Huang, Kinshuk, K. Moudgalya, S. Murthy, & D. G. Sampson (Eds.), Proceedings 18th IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 1-5). IEEE: Computer Society. DOI 10.1109/ICALT.2018.0000 Stracke, C. M., & Tan, E. (2018). The Quality of Open Online Learning and Education: Towards a Quality Reference Framework for MOOCs. In J. Kay, & R. Luckin (Eds.), Rethinking learning in the digital age. Making the Learning Sciences Count: The International Conference of the Learning Sciences (ICLS) 2018 (pp. 1029-1032). London: ISLS. Stracke, C. M. et al. (2017). The Quality of Open Online Education: Towards a Reference Framework for MOOCs. In Proceedings of 2017 IEEE Global Engineering Education Conference (EDUCON) (pp. 1712-1715). IEEE Xplore. DOI: 10.1109/EDUCON.2017.7943080 To be continued …
  • 65. The Quality Reference Framework Dimension 1: Phases Analysis, Design, Implementation, Realization, Evaluation Dimension 2: Perspectives Pedagogical, Technological, and Strategic Dimension 3: Roles Designer, Facilitator, and Provider
  • 66. MOOQ project: www.MOOC-quality.eu Online community for the Quality Reference Framework: www.MOOC-quality.net
  • 67. Session Title: Minds-on session on Design Processes for MOOCs Facilitators: Christian M. Stracke Cleo Sgouropoulou Nikos Palavitsinis Session Title: Dealing with practical quality problems when running a MOOC Facilitators: Bill Vassiliadis Antonia Stefani Esther Tan Achilles Kameas Parallel Sessions
  • 68. Session 1: Minds-on session on Design Processes for MOOCs Insights on: • The Quality Reference Framework in practice, • Design decisions, • Roles of designers, facilitators and providers Room: 3rd floor Session 2: Dealing with practical quality problems when running a MOOC Insights on: • Drop-out rates, • Failure to collaborate, • Fairness in automatic assessment Room: here (1st floor) Parallel Sessions