Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - Findings and Tools Stracke Tan

88 views

Published on

2018-07-13 Presentation at European MOOQ Conference in Athens on MOOQ and the Quality of MOOCs - Findings and Tools by Christian M. Stracke and Esther Tan from OUNL

Published in: Education
  • Be the first to comment

  • Be the first to like this

2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - Findings and Tools Stracke Tan

  1. 1. MOOQ and the Quality of MOOCs: Findings & Tools Christian M. Stracke & Esther Tan
  2. 2. Open University of the Netherlands Global cooperation: ECNU & KNOU Global initiative ICORE for OR & OE International WLS / LINQ Conference eLC European Institute ICDE Chair in OER Dr. Christian M. Stracke: Open Learning & Education, Innovations, Policies, Quality & Competences, Impact
  3. 3. Open University of the Netherlands The Quality of MOOCs Let’s Learn to Learn Seamless Learning Dr. Esther Tan Technology-Enhanced Learning, Innovations in & out Classroom
  4. 4. Presentation Outline 1. The Quality of MOOCs: What is the problem? 2. A Conceptual Framework towards MOOC QRF: Theoretical & Methodological Approach 3. Findings from MOOC surveys: Learners, Designers & Facilitators 4. Findings from MOOC OEQ: Learners, Designers & Facilitators 5. Findings from MOOC semi-structured interviews: Providers
  5. 5. Any discussion on the quality of MOOCs should consider the goals of both, the MOOC learner and the provider. (Hayes, 2015) Critical questions to be addressed:  Who are the MOOC users, and why?  What makes a good MOOC from the learners’ experiences with MOOCs?  What are the best design principles and best practices as indicators of quality?  Are these quality indicators also MOOC domain specific and/or MOOC type specific? Problem Statement
  6. 6. A Research Framework for MOOQ “When one designs any course, one has to have some learner cohort in mind.” (Macleod et al., 2015, p. 9) Main research goals: 1. Establish a research framework for the subsequent analysis of MOOC design patterns and best practices 2. Develop a Quality Reference Framework (QRF) for MOOCs Theoretical Framework
  7. 7. Global MOOC Quality Survey
  8. 8. Quality Reference Framework with criteria & checklist for MOOC design Our main goal is the collaboration with all to improve Open Education & MOOCs www.MOOC-quality.eu
  9. 9.  Subjects of Investigation: Learners, Designers, Facilitators & Providers  Mixed Methods Approach: Quantitative & qualitative data 1. Global MOOC Quality Survey, 2. Open-Ended questions (OEQ) & 3. Semi-structured interviews Methodological Framework
  10. 10. Instruments of Measure MOOC Learners MOOC Designers MOOC Facilitators MOOC Providers TOTAL Global MOOC Quality Survey 166 (69 qns.) 68 (89 qns.) 33 (58 qns.) - 267 Open- ended Questions 118 (4 qns.) 42 (4 qns.) 27 (4 qns.) - 185 Semi- structured interviews - 12 (15 qns.) 12 (10 qns.) 12 (13 qns.) 36
  11. 11. MOOC Survey Constructs Constructs Learners Designers Facilitators Experience with MOOC X X X Learning Objectives X X X Duration and Structure X X Learning Resources X X X Accessibility and Inclusion X X Learning Progress X Learning Environment X X Learning Assessment X X X Learning Certification X X Design Process X Pedagogical Decisions X Learning Support  Feedback & Facilitation  Interaction & Collaboration X X X
  12. 12. Global MOOC Quality Survey (GMQS)
  13. 13. Global MOOC Quality Survey
  14. 14. Demographic Profile Age range of all survey participants (learners, designers & facilitators) by gender
  15. 15. Demographic Profile Educational level of all survey participants (learners, designers & facilitators) by gender
  16. 16. n VB B N G VG Learning experience 166 4 4 13 75 70 Learning Experience (Learners)
  17. 17. n VB B N G VG Design experience 68 1 2 13 33 19 Design Experience (Designers)
  18. 18. Interaction from Learners‘ Perspective n N/A SD D N A SA LF 146 20 5 13 48 37 23 LL 146 15 3 17 34 51 26 LR 146 9 2 8 25 61 41 GG 146 37 4 15 50 24 16 Note: LF: Interaction between learners and facilitators LL: Interaction among learners LR: Interaction between learners and learning resources GG: Interaction among teams and groups
  19. 19. n R2 M2 p LF by learners 125 .094 9.382 .000*** LL by learners 130 .101 10.818 .000*** LR by learners 136 .112 12.286 .000*** GG by learners 108 .045 4.131 .026* Bivariate Correlations between LLR4 and LLE4
  20. 20. Interaction from Designers‘ Perspective n N/A SD D N A SA LF 52 2 1 5 11 24 9 LL 52 1 1 3 11 19 17 LR 52 3 1 0 4 22 22 GG 52 8 2 10 14 13 5 Note: LF: Interaction between learners and facilitators LL: Interaction among learners LR: Interaction between learners and learning resources GG: Interaction among teams and groups
  21. 21. n R2 M2 p LF by designers 49 .003 0.109 .703 LL by designers 50 .043 1.595 .143 LR by designers 48 .046 1.537 .138 GG by designers 43 .001 0.038 .821 Bivariate Correlations between DLR4 and DDE4
  22. 22. Interaction from Facilitators‘ Perspective n N/A SD D N A SA LF 32 0 1 3 3 15 10 LL 32 1 0 3 2 11 15 LR 32 0 0 0 5 12 15 GG 32 5 4 3 3 9 8 Note: LF: Interaction between learners and facilitators LL: Interaction among learners LR: Interaction between learners and learning resources GG: Interaction among teams and groups
  23. 23. n R2 M2 p LF by fa- cilitators 30 .181 0.242 .015* LL by fa- cilitators 29 .021 0.296 .435 LR by fa- cilitators 30 .030 0.287 .342 GG by fa- cilitators 25 .000 0.305 .971 Bivariate Correlations between FLR4 and FDE4
  24. 24. Open-ended Questions (Learners)
  25. 25. No of respondents across the six domains Domain Learners Designers Facilitators Social Sciences, Humanities & Law 24 9 2 Education & Lifelong Learning 19 12 18 Computing & Informatics 18 5 2 Science, Math & Engineering 16 5 1 Nature, Environment & Health 21 6 1 Business, Management & Economics 20 5 3 Total 118 42 27
  26. 26. MOOC Open-ended Questions (OEQ) MOOC Learners Q1. What were the three main strengths of the MOOC? Q2. What were the three main weaknesses of the MOOC? Q3. What was missing in the MOOC? Q4. Looking ahead into the development of this type of learning experiences, what could be improved in future MOOCs?
  27. 27. Age Range of MOOC Learners 18-24 25-34 35-44 45-54 55-64 65-74 >75 Total Business, Management & Economics 0 3 6 9 1 1 0 20 Nature, Environment & Health 1 5 4 1 6 4 0 21 Science, Math & Engineering 0 3 4 6 0 3 0 16 Computing & Informatics 1 4 5 4 4 0 0 18 Education & Lifelong Learning 1 1 7 3 5 2 0 19 Social Sciences, Humanities &Law 1 4 6 6 4 2 1 24 Total 4 20 32 29 20 12 1 118 Demographics of MOOC Learners
  28. 28. Educational Level of Learners No School- ing High school Bachelor’s degree Masters degree Doctorate degree Total Business, Management & Economics 0 1 4 10 5 20 Nature, Environment & Health 0 2 6 12 1 21 Science, Math & Engineering 0 1 4 7 4 16 Computing & Informatics 0 0 3 9 6 18 Education & Lifelong Learning 1 0 4 5 9 19 Social Sciences, Humanities &Law 0 0 4 12 8 24 Total 1 4 25 55 33 118 Demographics of MOOC Learners
  29. 29. Gender of MOOC Learners Male Female Other Total Business, Management & Economics 12 8 0 20 Nature, Environment & Health 14 7 0 21 Science, Math & Engineering 7 9 0 16 Computing & Informatics 8 10 0 18 Education & Lifelong Learning 11 8 0 19 Social Sciences, Humanities &Law 10 13 1 24 Total 62 55 1 118 Demographics of MOOC Learners
  30. 30. Learning Experience Very Bad Bad Neutral Good Very Good Total Business, Management & Economics 1 0 2 9 8 20 Nature, Environment & Health 0 0 0 9 12 21 Science, Math & Engineering 0 0 1 2 13 16 Computing & Informatics 1 1 1 8 7 18 Education & Lifelong Learning 1 1 0 9 8 19 Social Sciences, Humanities &Law 0 0 1 15 8 24 Total 3 2 5 52 56 118 Open-ended Questions (MOOC Learners)
  31. 31. Open-ended Questions (MOOC Learners) % of Activities Completed None <25% =50% >75% >100% Total Business, Management & Economics 1 2 2 3 12 20 Nature, Environment & Health 0 5 2 4 10 21 Science, Math & Engineering 0 1 3 3 9 16 Computing & Informatics 2 3 1 4 8 18 Education & Lifelong Learning 1 3 2 5 8 19 Social Sciences, Humanities &Law 1 2 3 9 9 24 Total 5 16 13 28 56 118
  32. 32. Open-ended Questions (MOOC Learners) Q1. What were the three main strengths of the MOOC? 0 10 20 30 40 50 60 70 80 90 Curriculum Design & Delivery Instructional Design & Technology Assessment & Evaluation Facilitation & Feedback Interaction & Collaboration NumberofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computing and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  33. 33. Open-ended Questions (MOOC Learners) Q2. What were the three main weaknesses of the MOOC? 0 5 10 15 20 25 30 35 Curriculum Design Instructional Design & Technology Assessment & Evaluation Faciliation & Feedback Interaction & Collaboration NumberofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computing and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  34. 34. Open-ended Questions (MOOC Learners) Three main strengths and weaknesses of the MOOC Strengths Weaknesses Curriculum Design & Delivery • Good choice & quality of content • Good teachers, presenters & tutors • LO aligns with content • Weak choice & quality of content • Short duration Instructional Design & Technology • Good integration of IT & media • Support self-regulation & individual learning paths • Poor use of IT technological tools • Resources lack variety & quality Interaction & Collaboration • Encourage local group discussion & activities • Foster interaction with field experts • Lack interaction: learner-tutor • No support for community building
  35. 35. Open-ended Questions (Designers)
  36. 36. MOOC Open-ended Questions (OEQ) MOOC Designers Q1. Which were the main design decisions that you made during the development of the MOOC that later proved to be successful? Q2. Which were the three biggest difficulties that you faced when designing the MOOC? Q3. Which design decisions did not pay off as you expected? Q4. Looking ahead into the development of this type of learning design experiences, what methods and tools could be helpful to improve the design of future MOOCs?
  37. 37. Age Range of MOOC Designers 25-34 35-44 45-54 55-64 65-74 >75 Total Business, Management & Economics 1 1 3 0 0 0 5 Nature, Environment & Health 1 3 0 1 0 1 6 Science, Math & Engineering 1 1 1 1 1 0 5 Computing & Informatics 0 1 4 0 0 0 5 Education & Lifelong Learning 1 2 5 3 1 0 12 Social Sciences, Humanities &Law 1 1 1 4 1 1 9 Total 5 9 14 9 3 2 42 Demographics of MOOC Designers
  38. 38. Educational Level of Designers High school Bachelor’s degree Masters degree Doctorate degree Total Business, Management & Economics 0 0 4 1 5 Nature, Environment & Health 0 0 3 3 6 Science, Math & Engineering 0 0 1 4 5 Computing & Informatics 1 0 1 3 5 Education & Lifelong Learning 0 0 3 9 12 Social Sciences, Humanities &Law 0 2 3 4 9 Total 1 2 15 24 42 Demographics of MOOC Designers
  39. 39. Gender of MOOC Designers Male Female Total Business, Management & Economics 3 2 5 Nature, Environment & Health 2 4 6 Science, Math & Engineering 2 3 5 Computing & Informatics 2 3 5 Education & Lifelong Learning 5 7 12 Social Sciences, Humanities &Law 6 3 9 Total 20 22 42 Demographics of MOOC Designers
  40. 40. Open-ended Questions (MOOC Designers) No. of MOOCs Designed 1 2 - 4 5 - 9 >10 Total Business, Management & Economics 3 2 0 0 5 Nature, Environment & Health 2 1 2 1 6 Science, Math & Engineering 1 2 2 0 5 Computing & Informatics 3 2 0 0 5 Education & Lifelong Learning 4 5 1 2 12 Social Sciences, Humanities &Law 3 4 2 0 9 Total 16 16 7 3 42
  41. 41. Design Experience Very Bad Bad Neutral Good Very Good Total Business, Management & Economics 0 0 0 5 0 5 Nature, Environment & Health 1 0 1 3 1 6 Science, Math & Engineering 0 0 1 2 2 5 Computing & Informatics 0 0 1 3 1 5 Education & Lifelong Learning 0 0 4 5 3 12 Social Sciences, Humanities &Law 0 1 0 3 5 9 Total 1 1 7 21 12 42 Open-ended Questions (MOOC Designers)
  42. 42. Open-ended Questions (MOOC Designers) Q1. Which were the main design decisions that you made during the development of the MOOC that later proved to be successful? 1 3 3 42 1 1 3 3 6 5 4 0 4 8 12 16 20 24 Assessibility& Inclusion Assessment & Evaluation Interaction & Collaboration Certification& Accreditation Curriculum Design& Delivery Expertise & Manpower Feedback & Facilitation Instructional Design& Technology No.ofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computer and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  43. 43. Open-ended Questions (MOOC Designers) Q2. Which were the three biggest difficulties that you faced when designing the MOOC? 3 3 1 3 3 2 8 1 2 2 1 1 1 2 3 5 4 4 2 5 3 0 4 8 12 16 20 24 Assessment & Evaluation Curriculum Design& Delivery Expertise & Manpower Feedback & Facilitation Institutional Support & Funding Instructional Design& Technology Interaction & Collaboration Open Access, Copyrights & Licensing No.ofComments Social Sciences, Humanities and Law Education and Lifelong Learning Computer and Informatics Science, Maths and Engineering Nature, Environment and Health Business, Management and Economics
  44. 44. Open-ended Questions (MOOC Designers) Successful Decisions Biggest Challenges Curriculum Design & Delivery • Content delivery format • Content structure & LOs • Weak choice & quality of content • Short duration Instructional Design & Technology • Choice of learning activities • Integration of IT & media • Platform, software & production decisions Interaction & Collaboration Expertise & Manpower • Creating community of learners • Foster interaction between learner & tutor/ facilitator • Gap in content & instructional design knowledge • Coordination & collaboration, e.g., different experts & teaching staff Three main successful decisions and three biggest challenges
  45. 45. Open-ended Questions (Facilitators)
  46. 46. MOOC Open-ended Questions (OEQ) MOOC Facilitators Q1. Which were the main decisions that you made during the facilitation of the MOOC that later proved to be successful? Q2.Which were the three biggest difficulties that you faced when facilitating the MOOC? Q3 Which facilitation decisions did not pay off as you expected? Q4 Looking ahead into the development of this type of learning experiences, what methods and tools could be helpful to improve the facilitation of future MOOCs?
  47. 47. Age Range of MOOC Facilitators 25-34 35-44 45-54 55-64 65-74 >75 Total Business, Management & Economics 0 1 0 0 0 2 3 Nature, Environment & Health 0 0 0 1 0 0 1 Science, Math & Engineering 0 1 0 0 0 0 1 Computing & Informatics 1 0 0 1 0 0 2 Education & Lifelong Learning 0 6 6 3 3 0 18 Social Sciences, Humanities &Law 0 0 1 1 0 0 2 Total 1 8 7 6 3 2 27 Demographics of MOOC Facilitators
  48. 48. Educational Level of MOOC Facilitators High school Bachelor’s degree Masters degree Doctorate degree Total Business, Management & Economics 0 0 1 2 3 Nature, Environment & Health 0 0 0 1 1 Science, Math & Engineering 0 0 0 1 1 Computing & Informatics 1 0 1 0 2 Education & Lifelong Learning 0 4 8 6 18 Social Sciences, Humanities &Law 0 0 2 0 2 Total 1 4 12 10 27 Demographics of MOOC Facilitators
  49. 49. Gender of MOOC Facilitators Male Female Total Business, Management & Economics 0 3 3 Nature, Environment & Health 1 0 1 Science, Math & Engineering 1 0 1 Computing & Informatics 0 2 2 Education & Lifelong Learning 10 8 18 Social Sciences, Humanities &Law 0 2 2 Total 12 15 27 Demographics of MOOC Facilitators
  50. 50. Open-ended Questions (MOOC Facilitators) No. of MOOCs Facilitated (OEQ) 1 2 - 4 5 - 9 >10 Total Business, Management & Economics 2 0 0 1 3 Nature, Environment & Health 0 0 0 1 1 Science, Math & Engineering 1 0 0 0 1 Computing & Informatics 1 1 0 0 2 Education & Lifelong Learning 6 7 2 3 18 Social Sciences, Humanities &Law 0 2 0 0 2 Total 10 10 2 5 27
  51. 51. Facilitation Experience Good Very Good Total Business, Management & Economics 3 0 3 Nature, Environment & Health 0 1 1 Science, Math & Engineering 1 0 1 Computing & Informatics 1 1 2 Education & Lifelong Learning 12 6 18 Social Sciences, Humanities &Law 1 1 2 Total 18 9 27 Open-ended Questions (MOOC Facilitators)
  52. 52. 0 1 2 3 4 5 6 7 8 9 Provide guidelines on attendance, quiz & test Monitor peer review process & forum discussion Show presence & provide feedback Leverage technological tools to support e-learning Promote interaction & group dynamic Provide quality content & appropriate pedagogical approach No.ofComments Successful Facilitation Decisions Social Sciences, Humanities & Law Education & Lifelong Learning Computing & Informatics Science, Math & Engineering Nature, Environment & Health Business, Management & Economics
  53. 53. 0 1 2 3 4 5 Difficult to engage & retain learners Integrate IT & Media to support learning Lack IT competencies (learners) Mismatch of content and target learners Overload learners with quizzes & assignments Provide free certfication and course Unable to foster interaction & collaboration No.ofComments Unsucessful Facilitation Decisions Social Sciences, Humanities & Law Education & Lifelong Learning Computing & Informatics Science, Math & Engineering Nature, Environment & Health Business, Management & Economics
  54. 54. Semi-Structured Interviews Questions (Providers)
  55. 55. Categories Descriptor Indicators Role of MOOCs in HE Providers perspective on the role of MOOCs in the current HE national and international scenario  Pedagogical innovation  Institutional culture: Toward hybrid & blended courses; teaching internally & reaching out externally Reasons to provide MOOCs Identification of causes for offering MOOCs • Institutional mission: to “place people” as the largest course provider in FutureLearn; foster openlearn in the eco system; educational research on MOOCs; public engagement to showcase Uni; capacity development within the Uni • Institutional goal: Foster innovation in digital learning; stay relevant in the changing educational ecosystems Summative Report of Coded Interviews
  56. 56. Categories Descriptor Indicators Partnerships Establishment, organization and management of partnerships  Partnership structure: Shared network with a core group of universities to share research project  Great collaboration between researchers from various disciplines (excellent model); good to have external driver to promote internal change; helped in best practices. Institutional Implementation Policies Theoretical design of the MOOC business model to implement  More strategic approach developed: lifelong learning, apprenticeship; MOOCs as experimentation spaces - separate from other learning & teaching platforms  Business model for delivering MOOCs: undergraduate and masters degrees) & chargeable certification (for some types) Summative Report of Coded Interviews
  57. 57. Categories Descriptor Indicators Institutional organizational strategies Description of the MOOC organization and development process in the institution  Support structures: 1. Set up a content team – work with partners to design well-designed courses with quality; 2. Uni put in place a number pedagogical coordinators to help teaching staff to prepare the course and provide ideas and experiences, technical staff for recording and post production, staff for subtitles. 3. Central unit works with individual academics to find the best plan/ fit  Re-train/ Assimilate teachers into the MOOC culture: provide teachers a checklist on dos & donts for content in an online learning environment. Summative Report of Coded Interviews
  58. 58. Categories Descriptor Indicators Sustainability Characterization of the return on investment  Funding source(s): 1. Business model: a number of bis model. Some funded by external org. most funded internally thru OU open learning budget (by media budget fr media unit) 2. Budget from uni for central unit plus some money from Edx (but not sufficient) received from Edx every 3 months some money; the objective is to improve education; not profit- making. 3. Four mio raised funding externally for the fully online masters study Summative Report of Coded Interviews
  59. 59. Categories Descriptor Indicators Evaluation Design of the evaluation process for the MOOC policy established  Level of success for the institution: 1. Grant awareness: revenue gains, paid courses 2. Prestige & research gains. 3. Educational success: moving from standard to hybrid & blended learning in accredited courses 4. Community of learners: with other likeminded people/ other Uni 5. Met objectives, research, Uni promotion & capacity building  Specialist evaluation apart from usual questionnaire on institutional e-learning and pedagogical innovation pedagogical model Summative Report of Coded Interviews
  60. 60.  Every institution should always have a roadmap to improve and extend the use of current courses, such as MOOCs: 1. facilitates implicit social learning, and may also help to attract new students to programmes. 2, the sociocultural nature of students is changing in Europe and beyond so it is important for to undertake multicultural adaptation of courses for the different student types 3. ECTS for MOOCs  Globalization strategies of the training spaces: MOOCs raise new research questions/problems that deserve further study (e.g., what are the consequences of these new models to knowledge sharing/building on a global level). Summative Report of Coded Interviews Final words from MOOC Provider
  61. 61. christian.stracke@ou.nl esther.tan@ou.nl @ChrMStracke @Taneste www.opening-up.education Let’s collaborate!
  62. 62. Thank you! Your questions? To be continued…
  63. 63. Open CC License for sharing & re-using slides This work is free to share under the creative commons licence: "Attribution – Noncommercial – Share Alike 3.0" You can copy, distribute and transmit the work under the following conditions: 1. Attribution – 2. Noncommercial – 3. Share Alike Licence: Attribution-Noncommercial-Share Alike Some rights reserved, see: http://creativecommons.org/licenses/by-nc-sa/3.0/
  64. 64. First References for GMQS Stracke, C. M., et al. (2018). Gap between MOOC designers' and MOOC learners' perspectives on interaction and experiences in MOOCs: Findings from the Global MOOC Quality Survey. In M. Chang, N.-S. Chen, R. Huang, Kinshuk, K. Moudgalya, S. Murthy, & D. G. Sampson (Eds.), Proceedings 18th IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 1-5). IEEE: Computer Society. DOI 10.1109/ICALT.2018.0000 Stracke, C. M., & Tan, E. (2018). The Quality of Open Online Learning and Education: Towards a Quality Reference Framework for MOOCs. In J. Kay, & R. Luckin (Eds.), Rethinking learning in the digital age. Making the Learning Sciences Count: The International Conference of the Learning Sciences (ICLS) 2018 (pp. 1029-1032). London: ISLS. Stracke, C. M. et al. (2017). The Quality of Open Online Education: Towards a Reference Framework for MOOCs. In Proceedings of 2017 IEEE Global Engineering Education Conference (EDUCON) (pp. 1712-1715). IEEE Xplore. DOI: 10.1109/EDUCON.2017.7943080 To be continued …
  65. 65. The Quality Reference Framework Dimension 1: Phases Analysis, Design, Implementation, Realization, Evaluation Dimension 2: Perspectives Pedagogical, Technological, and Strategic Dimension 3: Roles Designer, Facilitator, and Provider
  66. 66. MOOQ project: www.MOOC-quality.eu Online community for the Quality Reference Framework: www.MOOC-quality.net
  67. 67. Session Title: Minds-on session on Design Processes for MOOCs Facilitators: Christian M. Stracke Cleo Sgouropoulou Nikos Palavitsinis Session Title: Dealing with practical quality problems when running a MOOC Facilitators: Bill Vassiliadis Antonia Stefani Esther Tan Achilles Kameas Parallel Sessions
  68. 68. Session 1: Minds-on session on Design Processes for MOOCs Insights on: • The Quality Reference Framework in practice, • Design decisions, • Roles of designers, facilitators and providers Room: 3rd floor Session 2: Dealing with practical quality problems when running a MOOC Insights on: • Drop-out rates, • Failure to collaborate, • Fairness in automatic assessment Room: here (1st floor) Parallel Sessions

×