Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Land of The
Learning Giants:
The Rise of MOOCs
ESAI Conference
April 2nd 2016
Galway
Eamon Costello, Mark Brown,
Enda Donl...
Image by @dkernohan 26 Sep 2012 https://twitter.com/dkernohan/status/250889990828089344/photo/1
http://www.onlinecoursereport.com/state-of-the-mooc-2016-a-year-of-massive-landscape-change-for-massive-open-online-course...
Good MOOCs/Bad MOOCs
photo by zoomar https://flickr.com/photos/zoomar/145322737 (BY-NC-SA)
Jones A, Leta R &Regner, L, Use...
Why would
Institutions bet on
MOOCs?
DCU and Ireland
Representations of
MOOCs in the Media:
Ireland
MCQs in MOOCs
A questi...
Completionrates
MOOCLab (2016) UK MOOC Report
http://www.mooclab.club/Reports/UK%20MOOC%20Report%202016.pdf
Course length
To MOOC or not to MOOC?
Allen, I. E., & Seaman, J. (2015). Grade level: Tracking online education in the United States. Ba...
Ireland (n = 9)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 1
New method in big business
Reduce the costs of HE
New form to edu...
MOOCs in the Media
• Methodology
• Media Sources
• Discourse Analysis Protocol
- 20 Analysis Items
- Descriptive | Interpr...
0 5 10 15 20 25 30
Fear of Missing Out
Institutional Branding
Increase Student Recruitment
Extend Access
Reduce Costs
Gene...
Elite
73%
Non Elite
27%
Institutional Status
MOOCs Drivers at DCU
• Assisting Students in Course Selection
• Promoting an Open Philosophy
• Shaping Direction of Future...
DCU Open Academy
Irish Language MOOC
Getting Ready to Study MOOC
21st Century Learning Design Microsoft in Educa21CLD Course Introduction
43 Office Mixes
Classroom in action videos
Classr...
Good MOOCs Bad MOOCs
Dillenbourg, P. (2015) Proposal for a Digital Education Strategy for Flanders Universities.
“Thinkers...
Research Question:
What is the Quality of Multiple Choice Questions in
MOOCs?
Best Practice
Case, S. M., & Swanson, D. B. (2003). Constructing written test questions for the basic
and clinical science...
Methodology
• Qualitative Study
• Use Tarrant et. al.’s (2006) diagnostic tool to
analyse MCQs in MOOC systematically
• Lo...
1. Ambiguous or unclear information
2. Negative worded stem (not, incorrect, except)
3. Implausible distracters
4. Gratuit...
Data and Analysis
• 18 Courses
– from five MOOC platforms
edX, Coursera, Futurelearn, Iversity and Eliademy
• Total MCQs: ...
Findings (n = 204)
• 202 item writing flaws (errors)
• At least 1 error in 112 questions (54.9%)
• More than 2 errors in 5...
Potential Impact?
• Downing’s (2005) study
– found 33–46% of MCQs flawed
– 10–15% of students who failed should have passe...
To conclude
over to you…
EASI SIG on Digital Learning
Upcoming SlideShare
Loading in …5
×

Land of The Learning Giants: The Rise of MOOCs

822 views

Published on

Massive Open Online Courses (MOOCs) have been heralded and decried in something of equal measure over the last four years. Their ultimate purpose and the effect they are having are still uncertain but given the level of maturity that has now reached we ought now to be able to attempt to answer some questions of this phenomenon. Following an overview of key issues for educational research on the topic of MOOCs this paper presents findings from studies we have conducted into

* Representations of MOOCs in the Irish Print Media: What are the narratives, who is telling it and why?
* Quality of education in MOOCs in particular regarding online testing
* The strategic drivers for higher education institutions in Ireland to develop MOOCs

Published in: Education
  • Be the first to comment

  • Be the first to like this

Land of The Learning Giants: The Rise of MOOCs

  1. 1. Land of The Learning Giants: The Rise of MOOCs ESAI Conference April 2nd 2016 Galway Eamon Costello, Mark Brown, Enda Donlon, Mairead Ni Ghiolla Mhichil
  2. 2. Image by @dkernohan 26 Sep 2012 https://twitter.com/dkernohan/status/250889990828089344/photo/1
  3. 3. http://www.onlinecoursereport.com/state-of-the-mooc-2016-a-year-of-massive-landscape-change-for-massive-open-online-courses/
  4. 4. Good MOOCs/Bad MOOCs photo by zoomar https://flickr.com/photos/zoomar/145322737 (BY-NC-SA) Jones A, Leta R &Regner, L, Users or Students? Privacy in University MOOCs (August 10, 2015). Science & Engineering Ethics, Forthcoming.
  5. 5. Why would Institutions bet on MOOCs? DCU and Ireland Representations of MOOCs in the Media: Ireland MCQs in MOOCs A question of Quality ? • a type of marketing • an academic labor policy • speculative financial instruments • an expression of Silicon Valley values • a kind of entertainment media MOOCs are… Peters, M. (2013). Massive Open Online Courses and Beyond: the Revolution to Come. Truthout, August 17
  6. 6. Completionrates MOOCLab (2016) UK MOOC Report http://www.mooclab.club/Reports/UK%20MOOC%20Report%202016.pdf Course length
  7. 7. To MOOC or not to MOOC? Allen, I. E., & Seaman, J. (2015). Grade level: Tracking online education in the United States. Babson Park, MA: Babson Survey Research Group and Quahog Research Group
  8. 8. Ireland (n = 9) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 1 New method in big business Reduce the costs of HE New form to educate the many Need for (e-)skills and jobs. Technical innovation push Improving the quality of learning Business models based on ‘free’ Openness as business driver Globalization and internationalization ncreasing shared services and unbundling Macro drivers for institutions 2015 (all) Not at all relevant for my institution Somewhat relevant for my institution Neither irrelevant or relevant Relevant for my institution Highly relevant for my institution
  9. 9. MOOCs in the Media • Methodology • Media Sources • Discourse Analysis Protocol - 20 Analysis Items - Descriptive | Interpretative - Revolutionary | Evolutionary - Major Drivers
  10. 10. 0 5 10 15 20 25 30 Fear of Missing Out Institutional Branding Increase Student Recruitment Extend Access Reduce Costs Generate Revenue Unbundling of Services Innovation Curriculum Renewal Industry Training Supporting Student Readiness Promote Research Not Stated Major Drivers
  11. 11. Elite 73% Non Elite 27% Institutional Status
  12. 12. MOOCs Drivers at DCU • Assisting Students in Course Selection • Promoting an Open Philosophy • Shaping Direction of Future Developments in HE • Supporting Research on New Models of Teaching and Learning
  13. 13. DCU Open Academy
  14. 14. Irish Language MOOC
  15. 15. Getting Ready to Study MOOC
  16. 16. 21st Century Learning Design Microsoft in Educa21CLD Course Introduction 43 Office Mixes Classroom in action videos Classroom practice analysis videos 21CLD Course OneNote
  17. 17. Good MOOCs Bad MOOCs Dillenbourg, P. (2015) Proposal for a Digital Education Strategy for Flanders Universities. “Thinkers in Residence” Programme from KVABKoninklijke Vlaamse Academie van België voor Wetenschappen en Kunsten. “Sooner or later, online tests will be as reliable or even more reliable than on campus exams”
  18. 18. Research Question: What is the Quality of Multiple Choice Questions in MOOCs?
  19. 19. Best Practice Case, S. M., & Swanson, D. B. (2003). Constructing written test questions for the basic and clinical sciences (3rd ed.). Philadelphia, PA: National Board of Medical Examiners.
  20. 20. Methodology • Qualitative Study • Use Tarrant et. al.’s (2006) diagnostic tool to analyse MCQs in MOOC systematically • Look for item writing flaws Tarrant, M., Knierim, A., Hayes, S. K., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education Today, 26(8), 662-671
  21. 21. 1. Ambiguous or unclear information 2. Negative worded stem (not, incorrect, except) 3. Implausible distracters 4. Gratuitous information in stem 5. More than one or no correct answer 6. Longest option is correct 7. Logical cues in stem 8. Word repeats in stem and correct answer 9. Unfocused stem 10. True/false question 11. Use of all of the above 12. Vague terms (sometimes, frequently) 13. Absolute terms (never, always) 14. Use of none of the above 15. Fill-in-blank 16. Complex or K-type 17. Grammatical cues in sentence completion 18. Convergence cues Tool Tarrant, M., Knierim, A., Hayes, S. K., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education Today, 26(8), 662-671
  22. 22. Data and Analysis • 18 Courses – from five MOOC platforms edX, Coursera, Futurelearn, Iversity and Eliademy • Total MCQs: 204 – 934 data points • Two Evaluators
  23. 23. Findings (n = 204) • 202 item writing flaws (errors) • At least 1 error in 112 questions (54.9%) • More than 2 errors in 57 questions (27.9%) COSTELLO, E., BROWN, M., & HOLLAND, J. (2016). What Questions are MOOCs asking? An Evidence-Based Investigation. Proceedings of the European Stakeholder Summit on experiences and best practices in and around MOOCs (EMOOCS 2016), 211.
  24. 24. Potential Impact? • Downing’s (2005) study – found 33–46% of MCQs flawed – 10–15% of students who failed should have passed Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education, 10(2), 133-143.
  25. 25. To conclude over to you… EASI SIG on Digital Learning

×