Evaluating MOOCs – what is really happening?
Gráinne Conole,
University of Leicester
11th June 2014
EDEN conference, Zagreb
National
Teaching
Fellow 2012 Ascilite fellow 2012EDEN fellow 2013
Outline
• Emergence of MOOCs
• FutureLearn
• UoL MOOC evaluation
• Methodology
• A new MOOC classification
• Survey findings
• Interview themes
• Recommendations
• Conclusion
The emergence of MOOCs
• CCK08
– Connectivist MOOC (cMOOC)
– Siemens, Downes and Cormier
– Evaluation (Fini, 2009)
– http://www.irrodl.org/index.php/irrodl/article/view/643/1402
• What are MOOCs?
– http://www.youtube.com/watch?v=eW3gMGqcZQc
• Emergence of large-scale xMOOCs
• Recent developments
– UK-based FutureLearn
– Launch of Massey on Open2Study
• List of MOOCs
– http://www.mooc-list.com/
• EFQUEL series of blogs
– http://mooc.efquel.org/
• ICDE list of MOOC reports
– http://tinyurl.com/gconole-MOOC
Massive Open Online Courses (MOOCs)
Free
Distributed global community
Social inclusion
High dropout rates
Learning income not learning outcome
Marketing exercise
http://alternative-educate.blogspot.co.uk/2012/12/audio-ascilite-2012-great-debate-moocs.html
JOLT, Vol. 9, No. 2, http://jolt.merlot.org
cMOOCs or xMOOCs?
• Weekly centred
• Participant reflective
spaces
• Social and networked
participation
• Hashtag: #etmooc
• Use of a range of social
media
• Linear learning pathway
• Mainly text and video
• Formative feedback
through MCQs
• Individually focused
Dimension Characteristics
Context
Open Degree to which the MOOC is open
Massive How large the MOOC is
Diversity The diversity of the learners
Learning
Use of multimedia Extent of use of rich multimedia
Degree of communication Amount of communication incorporated
Degree of collaboration Amount of collaboration incorporated
Amount of reflection Ways in which reflection is encouraged
Learning pathway Degree to which the learning pathway is supported
Quality assurance Degree of quality assurance
Certification Mechanisms for accreditation
Formal learning Feed into formal learning offerings
Autonomy Degree of learner autonomy
A taxonomy of MOOCs
http://e4innovation.com/?p=727
A new MOOC classification
Dimension Connectivist Siemens MOOC
Context
Open 3
Massive 2
Diversity 3
Learning
Use of multimedia 2
Degree of communication 3
Degree of collaboration 2
Amount of reflection 3
Learning pathway 1
Quality assurance 1
Certification 1
Formal learning 1
Autonomy 3
For each
dimension, give
the MOOC a
score:
Low=1,
Medium=2
High=3
A new MOOC classification
Dimension Connectivist Siemens MOOC
Context
Open 3
Massive 2
Diversity 3
Learning
Use of multimedia 2
Degree of communication 3
Degree of collaboration 2
Amount of reflection 3
Learning pathway 1
Quality assurance 1
Certification 1
Formal learning 1
Autonomy 3
How to rate Open?
It’s free = 1
At least some CC
materials = 2
All materials CC,
and non-registered
students can view
materials=3
How to rate
Massive?
Under 500=1
500-10,000=2
Over 10,000=3
http://tinyurl.com/OEWBirdConole
FutureLearn
• Launched Dec 2012
• 37 partners
• Inspire learning for life
• Diverse range of courses
• Globally connecting
learners with experts
• Bite-size steps of learning
• Learning through
storytelling
• Mix of text, video, audio
and activities
https://www.futurelearn.com/
Principles
• Open
• Listen to learners
• Tell stories
• Provoke conversation
• Create connections
• Keep it simple
• Learn from others
• Celebrate progress
• Embrace FutureLearners
UoL MOOCs
• Courses
– England in the time of King
Richard III
– Forensic Science and
Criminal Justice
• Key features
– 6 weeks, 3 hours a week
– Bite-size chunks of learning
– Text, audio, videos, forums
and activities
– Certificate of participation
Methodology
• Focus:
– Interviews
• developers, tutors,
learners, UoL lead
– Survey (participants)
– Analysis of courses
– Learning analytics
– No. registered
– Drop out rates
Aims
• Rationale for joining
• Patterns of interactions
• Perceptions
• Development time
• Time on different
components
• Reasons for participating
• Reasons for drop out
• Recommendations
Activities
Survey findings
• Cost £28, 500
– 130 days of staff time
• 52% had prior MOOC
experience
• Richard III (10, 066)
• Most visited a few times a
week
• Average No. posts 8
• 87% no contact with tutors
• 47% still active in final week
Survey findings
• 97% structure clear
• 91.5% Engaging
• About right
– 67% level
– 85% time
– 69% length
• Participating
– Learn new things (85%)
– Try a MOOC (53%)
– Online learning (46%)
• 91% a positive experience
• Interesting, enjoyable,
informative
Interview themes
• Marketing
• Use of video and chunks
• Learner analytics
• Need for Learning Technologists
• Unknown audience
• Platform
– Good for learners
– Not good for developers
• Expectation management
• Not much discussion
• Good profile
• Learner discussions and content generation
• Public good
• Pedagogical innovation
• Expert learners
http://wp.europeanmoocs.eu/
Related projects
http://wikieducator.org/Emundus
http://vmpass.eu/
Formal
Informal
Individual Social
Blended courses
DL+ social media
Trad. campus courses
DL courses
OER
xMOOCs
OER + Social media
cMOOCs
APEL
ePortforlios
OERu
Badges
Formal/informal landscape
Recommendations
• More strategic choice of
courses
• Support from LTs
• Effective use of learner
analytics
• Explore new business
models
• Feed into campus courses
• More pedagogical
innovation
MOOCs as a disruptive technology
• Disruptive technologies are
challenging traditional
institutions
• New business models emerging
• New approaches needed for
designing and delivering MOOCs
• Blurring of boundaries:
– formal/informal, real/virtual,
teacher/learner, cross cultural
• Need for new pedagogies
• Disaggregation of education
– High quality resources
– Learning pathways
– Support
– Accreditation
http://www.le.ac.uk/ili
http://www.slideshare.net/GrainneConole
grainne.conole@le.ac.uk
http://e4innovation.com
@gconole

Conole eden _mooc_evaln

  • 1.
    Evaluating MOOCs –what is really happening? Gráinne Conole, University of Leicester 11th June 2014 EDEN conference, Zagreb National Teaching Fellow 2012 Ascilite fellow 2012EDEN fellow 2013
  • 2.
    Outline • Emergence ofMOOCs • FutureLearn • UoL MOOC evaluation • Methodology • A new MOOC classification • Survey findings • Interview themes • Recommendations • Conclusion
  • 3.
    The emergence ofMOOCs • CCK08 – Connectivist MOOC (cMOOC) – Siemens, Downes and Cormier – Evaluation (Fini, 2009) – http://www.irrodl.org/index.php/irrodl/article/view/643/1402 • What are MOOCs? – http://www.youtube.com/watch?v=eW3gMGqcZQc • Emergence of large-scale xMOOCs • Recent developments – UK-based FutureLearn – Launch of Massey on Open2Study • List of MOOCs – http://www.mooc-list.com/ • EFQUEL series of blogs – http://mooc.efquel.org/ • ICDE list of MOOC reports – http://tinyurl.com/gconole-MOOC
  • 4.
    Massive Open OnlineCourses (MOOCs) Free Distributed global community Social inclusion High dropout rates Learning income not learning outcome Marketing exercise http://alternative-educate.blogspot.co.uk/2012/12/audio-ascilite-2012-great-debate-moocs.html JOLT, Vol. 9, No. 2, http://jolt.merlot.org
  • 5.
    cMOOCs or xMOOCs? •Weekly centred • Participant reflective spaces • Social and networked participation • Hashtag: #etmooc • Use of a range of social media • Linear learning pathway • Mainly text and video • Formative feedback through MCQs • Individually focused
  • 6.
    Dimension Characteristics Context Open Degreeto which the MOOC is open Massive How large the MOOC is Diversity The diversity of the learners Learning Use of multimedia Extent of use of rich multimedia Degree of communication Amount of communication incorporated Degree of collaboration Amount of collaboration incorporated Amount of reflection Ways in which reflection is encouraged Learning pathway Degree to which the learning pathway is supported Quality assurance Degree of quality assurance Certification Mechanisms for accreditation Formal learning Feed into formal learning offerings Autonomy Degree of learner autonomy A taxonomy of MOOCs http://e4innovation.com/?p=727
  • 7.
    A new MOOCclassification Dimension Connectivist Siemens MOOC Context Open 3 Massive 2 Diversity 3 Learning Use of multimedia 2 Degree of communication 3 Degree of collaboration 2 Amount of reflection 3 Learning pathway 1 Quality assurance 1 Certification 1 Formal learning 1 Autonomy 3 For each dimension, give the MOOC a score: Low=1, Medium=2 High=3
  • 8.
    A new MOOCclassification Dimension Connectivist Siemens MOOC Context Open 3 Massive 2 Diversity 3 Learning Use of multimedia 2 Degree of communication 3 Degree of collaboration 2 Amount of reflection 3 Learning pathway 1 Quality assurance 1 Certification 1 Formal learning 1 Autonomy 3 How to rate Open? It’s free = 1 At least some CC materials = 2 All materials CC, and non-registered students can view materials=3 How to rate Massive? Under 500=1 500-10,000=2 Over 10,000=3 http://tinyurl.com/OEWBirdConole
  • 9.
    FutureLearn • Launched Dec2012 • 37 partners • Inspire learning for life • Diverse range of courses • Globally connecting learners with experts • Bite-size steps of learning • Learning through storytelling • Mix of text, video, audio and activities https://www.futurelearn.com/
  • 10.
    Principles • Open • Listento learners • Tell stories • Provoke conversation • Create connections • Keep it simple • Learn from others • Celebrate progress • Embrace FutureLearners
  • 11.
    UoL MOOCs • Courses –England in the time of King Richard III – Forensic Science and Criminal Justice • Key features – 6 weeks, 3 hours a week – Bite-size chunks of learning – Text, audio, videos, forums and activities – Certificate of participation
  • 12.
    Methodology • Focus: – Interviews •developers, tutors, learners, UoL lead – Survey (participants) – Analysis of courses – Learning analytics – No. registered – Drop out rates
  • 13.
    Aims • Rationale forjoining • Patterns of interactions • Perceptions • Development time • Time on different components • Reasons for participating • Reasons for drop out • Recommendations
  • 14.
  • 15.
    Survey findings • Cost£28, 500 – 130 days of staff time • 52% had prior MOOC experience • Richard III (10, 066) • Most visited a few times a week • Average No. posts 8 • 87% no contact with tutors • 47% still active in final week
  • 16.
    Survey findings • 97%structure clear • 91.5% Engaging • About right – 67% level – 85% time – 69% length • Participating – Learn new things (85%) – Try a MOOC (53%) – Online learning (46%) • 91% a positive experience • Interesting, enjoyable, informative
  • 17.
    Interview themes • Marketing •Use of video and chunks • Learner analytics • Need for Learning Technologists • Unknown audience • Platform – Good for learners – Not good for developers • Expectation management • Not much discussion • Good profile • Learner discussions and content generation • Public good • Pedagogical innovation • Expert learners
  • 18.
  • 19.
    Formal Informal Individual Social Blended courses DL+social media Trad. campus courses DL courses OER xMOOCs OER + Social media cMOOCs APEL ePortforlios OERu Badges Formal/informal landscape
  • 20.
    Recommendations • More strategicchoice of courses • Support from LTs • Effective use of learner analytics • Explore new business models • Feed into campus courses • More pedagogical innovation
  • 21.
    MOOCs as adisruptive technology • Disruptive technologies are challenging traditional institutions • New business models emerging • New approaches needed for designing and delivering MOOCs • Blurring of boundaries: – formal/informal, real/virtual, teacher/learner, cross cultural • Need for new pedagogies • Disaggregation of education – High quality resources – Learning pathways – Support – Accreditation
  • 22.

Editor's Notes