Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MOOCs and Quality Issues - QAQE
Steering Group
Quality Assurance and Quality Enhancement in E-learning
Special Interest Gr...
Session Overview
• Definition of MOOCs
• Online learning
• The Quality Code
3
MOOCs and Quality Issues

So what is a MOOC?
•
•
•
•

Massive
Open
Online
Course
MOOC background information
• Term coined in 2008
• Connectivism and Connective Knowledge
▫ 25 tuition paying students
▫ 2...
Providers
Context
What are the key challenges associated with assuring the quality
of MOOCs?
8
MOOCs and Quality Issues

Why bother with quality?
•
•
•
•
•

Students – know what they are committing to
Employers – re...
9
MOOCs and Quality Issues

Tensions
Delivery:
Accreditation:
Price:
Entry:
Scale:
Support:
Pedagogy:
Teacher:

face-to-fa...
Massive
• Issue is not large size, but scale independence
• Aspects:
▫ Financial
▫ Technical
▫ Pedagogical
Learning design at scale
• Individual learning
• Small group collaboration
• Large communities
Open
• Different meanings:
▫
▫
▫
▫

Open = free = gratis
Open = free = libre
Open entry
Open = transparency
Course
• What distinguishes MOOC from OER or learning
object?
▫
▫
▫
▫

Size?
Goal, learning outcomes?
Measure of completio...
Quality and learners
“What are MOOCs actually aiming at?
“Can the quality of MOOCs be assessed in the same way as
any defi...
Quality Assurance Agency
“Our job is to safeguard quality and
standards in UK universities and colleges,
so that students ...
QAA event
• Quality Code
“Factors which apply to all learning
opportunities regardless of location,
mode of delivery, acad...
The Quality Code
The Quality Code
Part A: Setting and
maintaining threshold
academic standards
Part B: Assuring and
enhanc...
The Quality Code
Part A: Setting and maintaining
threshold academic standards
• A1: The national level
• A2: The subject l...
Considering MOOCs through the lens of
the Quality Code
B1: Programme design and approval
• Procedures for design and appro...
Course approval
Considering MOOCs through the lens of
the Quality Code
B3; Learning and Teaching
• Constructive alignment
• Levelness
• Te...
Considering MOOCs through the lens of
the Quality Code
• B5; Student engagement
• “improving the motivation of students to...
“the participation of students in quality enhancement and
quality assurance processes, resulting in the improvement of
the...
Considering MOOCs through the lens of
the Quality Code
• B6; Assessment of students and the recognition of prior learning
Mozilla badge infrastructure
http://openbadges.org

http://scouts.org.uk/

A badge is a validated indicator of
accomplishm...
Back to basics…
• Are students learning?
▫
▫
▫
▫
▫

Self assessment
Quizzes
Peer assessment
Analytics
Survey evaluation
Monitoring and evaluation
Completion rates
• Reported completion may be very low (1-10%)
• Does that matter?
▫ With very large starting numbers, the...
Quality points
Provenance
Reputation
Brand

checking
MOOC
creation

peer review

use

user recommendation
Quality Dimensions
Content
Accuracy
Currency
Relevance

Ease of use
Clarity
Visual attractiveness,
engaging
Clear navigati...
Benchmarks / indicators
(NB: an unashamed plug!)
• Conventional HE benchmarks not be appropriate
• E-learning benchmarks a...
OpenupEd www.openuped.eu
• ‘Reflecting European values such as equity, equality
and diversity’
• Courses should show featu...
UK’s first credit rated MOOC
• Level 4
• 20 credit points
• Taught as MOOC for
undergraduate students and as
open access
•...
And the latest…
• University of California, Irvine,
• “Tackle serious scientific issues,
related to events in the show”
▫ ...
MOOCs and Quality Issues
MOOCs and Quality Issues
Upcoming SlideShare
Loading in …5
×

MOOCs and Quality Issues

1,464 views

Published on

A presentation on 'MOOCs and Quality Issues' given at a workshop organised by the QA-QE special interest group of the UK Higher Education Academy (HEA) [http://qaqe-sig.net/?page_id=8]

Published in: Education, Technology
  • Be the first to comment

MOOCs and Quality Issues

  1. 1. MOOCs and Quality Issues - QAQE Steering Group Quality Assurance and Quality Enhancement in E-learning Special Interest Group (QE-SIG) Jon Rosewell The Open University Helen Barefoot University of Hertfordshire
  2. 2. Session Overview • Definition of MOOCs • Online learning • The Quality Code
  3. 3. 3 MOOCs and Quality Issues So what is a MOOC? • • • • Massive Open Online Course
  4. 4. MOOC background information • Term coined in 2008 • Connectivism and Connective Knowledge ▫ 25 tuition paying students ▫ 2300 other students from the general public taking course online and free of charge
  5. 5. Providers
  6. 6. Context
  7. 7. What are the key challenges associated with assuring the quality of MOOCs?
  8. 8. 8 MOOCs and Quality Issues Why bother with quality? • • • • • Students – know what they are committing to Employers – recognition of content and skills Authors – personal reputation, 'glow' of success Institutions – brand reputation Funders – philanthropic, venture caps, governments • Quality agencies – on behalf of above
  9. 9. 9 MOOCs and Quality Issues Tensions Delivery: Accreditation: Price: Entry: Scale: Support: Pedagogy: Teacher: face-to-face . distance credit . none cost . free selective . open personal . massive intensive . not supported constructivism . transmission star . anonymous
  10. 10. Massive • Issue is not large size, but scale independence • Aspects: ▫ Financial ▫ Technical ▫ Pedagogical
  11. 11. Learning design at scale • Individual learning • Small group collaboration • Large communities
  12. 12. Open • Different meanings: ▫ ▫ ▫ ▫ Open = free = gratis Open = free = libre Open entry Open = transparency
  13. 13. Course • What distinguishes MOOC from OER or learning object? ▫ ▫ ▫ ▫ Size? Goal, learning outcomes? Measure of completion / achievement / attainment? Structure  As narrative (xMOOC)?  Negotiated experience (cMOOC)? ▫ Time?
  14. 14. Quality and learners “What are MOOCs actually aiming at? “Can the quality of MOOCs be assessed in the same way as any defined university course with traditional degree awarding processes? “Or do we have to take into account a different type of objective with MOOC learners? Are the learners mostly interested in only small sequences of learning, tailored to their own individual purpose, and then sign off and move to other MOOCs because their own learning objective was fulfilled?” Ulf-Daniel Ehlers, Ebba Ossiannilsson, Alastair Creelman http://mooc.efquel.org/
  15. 15. Quality Assurance Agency “Our job is to safeguard quality and standards in UK universities and colleges, so that students have the best possible learning experience” Quality Code for Higher Education sets out the expectations all providers of UK higher education are required to meet. It gives all higher education providers a shared starting point for setting, describing and assuring the academic standards of their higher education awards and programmes and the quality of the learning opportunities they provide.
  16. 16. QAA event • Quality Code “Factors which apply to all learning opportunities regardless of location, mode of delivery, academic subject; MOOCs are no exception to that” Anthony McClaran, QAA event (July, 2013)
  17. 17. The Quality Code The Quality Code Part A: Setting and maintaining threshold academic standards Part B: Assuring and enhancing academic quality Part C: Information about higher education provision
  18. 18. The Quality Code Part A: Setting and maintaining threshold academic standards • A1: The national level • A2: The subject level and qualification level • A3: The programme level • A4: Approval and review • A5: Externality • A6: Assessment of achievement of learning outcomes Part C: Information about higher education provision Part B: Assuring and enhancing academic quality • • • • • • • • • • • B1: Programme design and approval B2: Admissions B3: Learning and Teaching B4: Enabling student development and achievement B5: Student engagement B6: Assessment of students and the recognition of prior learning B7: External examining B8: Programme monitoring and review B9: Academic appeals and student complaints B10: Managing HE provision with others B11: Research degrees
  19. 19. Considering MOOCs through the lens of the Quality Code B1: Programme design and approval • Procedures for design and approval for MOOC courses? • Are external partners involved in the design and approval? • Assessing the effectiveness of the course and processes for monitoring and reviewing the course?
  20. 20. Course approval
  21. 21. Considering MOOCs through the lens of the Quality Code B3; Learning and Teaching • Constructive alignment • Levelness • Teaching methods ▫ ▫ ▫ ▫ ▫ ▫ Videos Embedded questions Tutorials Discussion forums Wikis Assessments
  22. 22. Considering MOOCs through the lens of the Quality Code • B5; Student engagement • “improving the motivation of students to engage in learning and to learn independently ▫ data • the participation of students in quality enhancement and quality assurance processes, resulting in the improvement of their educational experience”. Source: edX – Presented by Lori Breslow – Enhancement Themes conference, June 2013
  23. 23. “the participation of students in quality enhancement and quality assurance processes, resulting in the improvement of their educational experience” Traditional courses MOOCs
  24. 24. Considering MOOCs through the lens of the Quality Code • B6; Assessment of students and the recognition of prior learning
  25. 25. Mozilla badge infrastructure http://openbadges.org http://scouts.org.uk/ A badge is a validated indicator of accomplishment, skill, quality or interest that can be earned in any environment http://www.dmlcompetition.net/
  26. 26. Back to basics… • Are students learning? ▫ ▫ ▫ ▫ ▫ Self assessment Quizzes Peer assessment Analytics Survey evaluation
  27. 27. Monitoring and evaluation
  28. 28. Completion rates • Reported completion may be very low (1-10%) • Does that matter? ▫ With very large starting numbers, there are still many learners completing ▫ Maybe learners achieve personal goals even if they don’t complete ▫ Can MOOCs encourage access to HE if >90% have an experience which is a ‘failure’?
  29. 29. Quality points Provenance Reputation Brand checking MOOC creation peer review use user recommendation
  30. 30. Quality Dimensions Content Accuracy Currency Relevance Ease of use Clarity Visual attractiveness, engaging Clear navigation Functional! Pedagogic Effectiveness Learning objectives Prerequisites Learning design Learning styles Assessment Reusability & openness Format & interoperability Localisation Discoverability: metadata Digital preservation Accessibility
  31. 31. Benchmarks / indicators (NB: an unashamed plug!) • Conventional HE benchmarks not be appropriate • E-learning benchmarks a better starting point ▫ E-xcellence NEXT  http://e-xcellencelabel.eadtu.eu/ • MOOC benchmarks better! ▫ OpenupEd / E-xcellence MOOC (to come…)  http://openuped.eu/
  32. 32. OpenupEd www.openuped.eu • ‘Reflecting European values such as equity, equality and diversity’ • Courses should show features: ▫ ▫ ▫ ▫ ▫ ▫ ▫ Openness to learners Digital openness Learner-centred approach Media supported interaction Recognition options Quality focus Spectrum of diversity
  33. 33. UK’s first credit rated MOOC • Level 4 • 20 credit points • Taught as MOOC for undergraduate students and as open access • 1000 students • 200 students going for credit • Assessed by a • 10-minute podcast presentation (40%) • 1500-word critical blog (60%)
  34. 34. And the latest… • University of California, Irvine, • “Tackle serious scientific issues, related to events in the show” ▫ Physics- "science behind decay“ ▫ Public health department- "What can we learn from past epidemics?“ ▫ Maths - "post-calculus maths can be used to model population and epidemic dynamics" • TV series - audience of 10 million • Student numbers for online course expected to be in hundreds of thousands

×