Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Systematic Analysis And Synthesis of the Empirical MOOC Literature Published in 2013-2015

4,878 views

Published on

A deluge of empirical research became available on MOOCs in 2013-2015 and this research is available in disparate sources. This paper addresses a number of gaps in the scholarly understanding of MOOCs and presents a comprehensive picture of the literature by examining the geographic distribution, publication outlets, citations, data collection and analysis methods, and research strands of empirical research focusing on MOOCs during this time period. Results demonstrate that: more than 80% of this literature is published by individuals whose home institutions are in North America and Europe; a select few papers are widely cited while nearly half of the papers are cited zero times; and researchers have favored a quantitative if not positivist approach to the conduct of MOOC research, preferring the collection of data via surveys and automated methods. While some interpretive research was conducted on MOOCs in this time period, it was often basic and only a handful of studies were informed by methods traditionally associated with qualitative research (e.g., interviews, observations, focus groups). Analysis shows that there is limited research reported on instructor-related topics, and that even though researchers have attempted to identify and classify learners into various groupings, very little research examines the experiences of learner subpopulations.

Published in: Education
  • Be the first to comment

A Systematic Analysis And Synthesis of the Empirical MOOC Literature Published in 2013-2015

  1. 1. A Systematic Analysis And Synthesis of the Empirical MOOC Literature Published in 2013-2015 George Veletsianos - Canada Research Chair in Innovative Learning & Technology, Royal Roads University, Peter Shepherdson - University of Zurich, Laura Pasquini - University of North Texas, & Rich McCue - UVic The 9th Collaboration for Online Higher Education & Research (COHERE) Conference October 2015, Victoria & Halifax, Canada
  2. 2. Rich McCue University of Victoria George Veletsianos, Canada Research Chair in Innovative Learning & Technology, Royal Roads University Peter Shepherdson University of Zurich Laura Pasquini University of North Texas
  3. 3. • What is the state of MOOC research (very briefly)? • Where are there gaps & what gaps can we fill? • Research Methods. • Did we find anything interesting? Overview
  4. 4. Introduction We wanted to address a number of gaps in the scholarly understanding of MOOCs and present a comprehensive picture of the MOOC literature by examining: geographic distribution, publication outlets, citations, data collection and analysis methods, and research strands of empirical research focusing on MOOCs.
  5. 5. Review of the Literature
  6. 6. cMOOCs and xMOOCs? ● cMOOCs are described as being “based on principles of connectivism, openness, and participatory teaching” (Jacoby, 2014, p. 76), “through a dynamic network of connections afforded by online technology” (Ebben & Murphy, 2014, p. 333). ● xMOOCs resemble “traditional teacher-directed course[s], yet automated, massive, and online” (Kennedy, 2014, p. 8). ● Early MOOCs tended to follow the cMOOC model, whereas more recently the number of xMOOCs delivered has been growing rapidly.
  7. 7. Expected impacts of MOOCs on education The size and openness of MOOCs have the potential to disrupt conventional thinking about the role, value, and cost of higher education. The discourse around MOOCs could diminish of the authority and importance of the educational leader. MOOCs may threaten the business models of universities.
  8. 8. Demographics of MOOC users • Vast majority male, 20-40 yrs old. • College degree or higher. • More than 1/2 of learners from outside the United States. • Majority of participants are from North America and Europe, with a small minority being from Asia, Southeast Asia, or Africa.
  9. 9. Challenges for MOOC’s Completion rates in MOOCs are less than 10%. Why? A lack of incentive, many courses free. Insufficient prior knowledge (e.g., lack of math or language skills). Failure to understand the content and having no one to turn to for help, Lack of time due to having other priorities and commitments. 90%
  10. 10. Research Questions: RQ1: How is MOOC research geographically distributed? RQ2: Is MOOC research usually published in journal or conference proceedings? In which journals and conference proceedings is MOOC research currently being published? RQ3: Which empirical MOOC studies are cited the most? RQ4: What data collection methods and data analysis methods are used in empirical studies of MOOCs? RQ5: What are the research strands of empirical MOOC research?
  11. 11. Research Methods
  12. 12. Literature discovery searches were conducted using the [NEXT] keywords “MOOC” or “Massive Open Online Course.” To be included articles had to be: (1) empirical, (2) published in a peer-reviewed journal, in conference proceedings, or in Educause Review, (3) published or was available online as in press between January 2013 and January 2015, and (4) written in English. All identified papers were examined by [NEXT] two researchers to ensure consensus that they fit the inclusion criteria. Forwarding references: In [NEXT] Google scholar looked up the papers and the looked at papers that cited those papers to discover new papers. Found 60 new papers. Completeness Search: Examined references of 17 papers published in 2015 in our corpus to identify any papers that we missed. They were more likely to reference literature published in 2013-2014 than the papers published in 2013 or 2014. 5 new papers found. [NEXT] The final number of published papers that constituted the corpus of this study was 183. Data Collection Search Keywords: “MOOC” or “Massive Open Online Course” Published Papers: n=183 Authors: n=460
  13. 13. Limitations 1. Limited duration - 3 years. 2. Only English language data. 3. Quality of research not evaluated. 4. Some content analyzed, but not all.
  14. 14. Results
  15. 15. RQ1: How is MOOC research geographically distributed? Country Corpus USA 50.2% UK Kingdom 10% Australia 7.7% China 5.4% Spain 4.8% Canada 4.5% Germany 2.2% Switzerland 1.3% Netherlands 1.1% Other (29 countries) 12.8% Author Affiliations
  16. 16. RQ2a: Is MOOC research usually published in journals or conference proceedings?
  17. 17. RQ2b: In which journals and conference proceedings is MOOC research currently being published? Outlet name # of papers Type International Review of Research in Open and Distributed Learning (IRRODL) 18 Journal Proceedings of the first ACM conference on Learning @ scale (L@S '14) 11 Conference Journal of Online Learning and Teaching (JOLT) 7 Journal Conference on Empirical Methods in Natural Language Processing (EMNLP) 5 Conference Distance Education 5 Journal NIPS Workshop on Data Driven Education 5 Conference Proceedings of the 2014 ASCILITE Conference 4 Conference EDUCAUSE Review 4 Journal
  18. 18. RQ3: Which empirical MOOC studies are cited the most? Paper # of citations - Studying learning in the worldwide classroom research into edX’s first MOOC. 124 - Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. 117 - MOOCs and the funnel of participation. 83 - Tuned Models of Peer Assessment in MOOCs. 73 - Automated Feedback Generation for Introductory Programming Assignments. 56 - Retention and Intention in Massive Open Online Courses: In Depth. 41 - “Turn on, Tune in, Drop out”: Anticipating student dropouts in Massive Open Online Courses. 35
  19. 19. RQ4: What data collection methods & analysis are used in empirical studies of MOOCs? Analytic method Frequency (%) of total papers Descriptive statistics 93.4 Correlational 52.5 Basic qualitative study 38.8 Experimental and quasi-experimental 25.7 Grounded Theory 7.6 Natural Language processing 7.6 Social Network Analysis 6.6 Ethnography 4.4 Phenomenology 2.2 Discourse analysis 1.0
  20. 20. RQ5: What are the research strands of empirical MOOC research? ● [NEXT] 83.6% of the papers focused on student-related areas. ● [NEXT] Nearly half (46.4%) of the papers identified in the literature search had some focus on topics relating to the design, creation, and implementation of MOOCs themselves. This included research pertaining to methods of assessment, the description of unique learning environments, the creation of MOOCs on specific topics, and the evaluation of course success. ● [NEXT] Only 8.2% of papers focused on topics related to instructors and teaching. Papers within this theme largely focused on academics’ awareness, perspectives of, and experiences with MOOCs. ● [NEXT] 10.9% of the corpus contained content pertaining to the context and impact of MOOCs. This included research into perceptions of MOOCs, their usefulness as an educational medium, and their economic impact . ● [NEXT] Other. 84% 46% 8% 11%
  21. 21. Discussion & Conclusions
  22. 22. Dependence on particular research methods may restrict our understanding of MOOCs Analysis suggests that researchers have favored a quantitative, if not positivist approach to the conduct of MOOC research.
  23. 23. Survey data and secondary data collected via automated methods were also favored. While some interpretive research was conducted in MOOCs in this time period, it was often basic and only a handful of studies were informed by methods traditionally associated with qualitative research (e.g., interviews, observations, focus groups).
  24. 24. Thus, even though results suggest that research on MOOCs focuses on student-related topics, learners’ voices are largely absent in the literature. Based on these results, we suggest that an expansion of the methodological approaches used in MOOC research is urgently needed.
  25. 25. Paucity of research examining instructor-related topics Analysis shows that there is limited research reported on instructor-related topics. This is a rich area for future research. Topics of interest in this area may include instructor motivations, experiences, and perceptions.
  26. 26. Understanding Learner Subpopulations Completing Auditing Disengaging Sampling No-shows Observers Drop-ins Latecomers Drop-in latecomers Non-engaged Engaged
  27. 27. Understanding Learner Subpopulations Little research examines the experiences of different populations and how and why learning experiences differ between groups. Future research into this area for instance could examine why some learners disengage, how the learning experience of drop-ins differs from the learning experience of those who are non- engaged, and what interventions may scaffold different types of learners.
  28. 28. The geography of MOOC research Our geographical analysis of author affiliations showed that over half of the authors conducted their research in the USA. Over 80% of authors were affiliated with institutions in North America or Europe.
  29. 29. Percentage of Research Published in USA In contrast, according to a Scimago search of all citable documents published in 2013 across all disciplines only 19% came from the USA, and it takes the top 20 countries (including at least seven countries from outside of North America and Europe) to account for 80% of academic output. As such, current MOOC research is simply not a reflection of geographical contributions to academic output in general. 0% 15% 30% 45% 60% MOOC Resarch All Other Research
  30. 30. Future Directions We hope that this systematic analysis enables researchers to make better sense of the empirical literature on MOOCs and its direction and limitations. There are many possibilities for future research in this area. Future systematic reviews of the literature may focus on: Synthesizing the knowledge on particular areas of interest, (completion and retention in MOOCs; learner motivations in MOOCs). Examining whether research methods used to understand MOOCs follow standard methods of inquiry. Follow methods that take into advantage the digital nature of learning and teaching in this context.
  31. 31. Final Thoughts Finally, we hope that our results highlight the need for a critical reflection on the part of researchers as to the methods they have used to understand MOOCs to date and encourage a greater diversity in research methods.
  32. 32. Thank you! Research available at: http://veletsianos/publications This presentation: http://slideshare.net/richmccue Contact: veletsianos@gmail.com @veletsianos on twitter @richmccue on twitter Photo credit: Got Credit
  33. 33. Special Thanks to the Creators of the Creative Commons Images used in this presentation: • https://www.flickr.com/photos/47572798@N00/8397808475 • https://commons.wikimedia.org/wiki/File:Wet_winding_road.jpg • https://pixabay.com/en/world-map-map-world-black-earth-297446/ • http://creativecommons.org/weblog/entry/26283 • https://en.wikipedia.org/wiki/Printing_press#/media/File:PrintMus_038.jpg • https://commons.wikimedia.org/wiki/File:2011_Library_of_Congress_USA_5466788868_card_catalog.jpg • https://upload.wikimedia.org/wikipedia/commons/7/72/Rhll_wire_rope.jpg • https://upload.wikimedia.org/wikipedia/commons/0/0a/Urval_av_de_bocker_som_har_vunnit_Nordiska_radets_litteraturpris_unde r_de_50_ar_som_priset_funnits_(3).jpg • https://www.flickr.com/photos/mathplourde/8620174342/sizes/l/in/photostream/ • http://orig00.deviantart.net/34c5/f/2011/263/d/6/destruction_by_tbh_1138-d4af5vp.png • https://www.flickr.com/photos/68751915@N05/6355351769 • https://upload.wikimedia.org/wikipedia/commons/1/10/ECLA_of_Bard_Seminar,_2012.jpg • https://www.flickr.com/photos/carbonnyc/143186839 • https://pixabay.com/en/beer-quit-quitting-can-vintage-398742/ • https://pixabay.com/en/blueprint-ruler-architecture-964629/ • https://www.flickr.com/photos/keithroper/8139626676 • http://www.picserver.org/images/highway/phrases/results.jpg • https://upload.wikimedia.org/wikipedia/commons/4/4b/Colossus.jpg • http://www.apa.org/images/apastyle_logo.gif • https://upload.wikimedia.org/wikipedia/commons/3/39/Student_in_Class_(3618969705).jpg • https://upload.wikimedia.org/wikipedia/commons/e/e6/Avrocar_schematic_(high_resolution).jpg • http://teachers.net/teachers.jpg • https://pixabay.com/p-90781/?no_redirect • https://pixabay.com/en/people-sitting-chairs-in-front-690810/ • https://upload.wikimedia.org/wikipedia/commons/1/17/1895-Dictionary-Phrenolog.png • https://www.flickr.com/photos/safari_vacation/10362491406 • https://www.flickr.com/photos/16210667@N02/15617432129 • http://www.veletsianos.com/wp-content/uploads/2013/05/edtechmooc.jpg • https://www.flickr.com/photos/intersectionconsulting/7537238368/ • https://pixabay.com/en/compass-antique-map-of-the-world-429772/ • http://asiimdesgraphic.deviantart.com/art/Mirror-Ball-Reflection-214407739

×