Expert Perceptions of the
Feasibility of MOOCs
Reporting on a Modified E-Delphi Study
Shalin Hai-Jew
Paper Forum
2014 Big ...
Presentation description
 For any innovation to take, experts and other decision-makers in a field have to
buy into the e...
Welcome!
 What are your interests re: this presentation?
 Any experiences with modified e-Delphi studies?
 What are you...
A modified e-Delphi study
 A classic Delphi study (a structured group procedure):
 Delphi study as a research method ori...
A modified e-Delphi study (cont.)
 A classic Delphi study (a structured group procedure) (cont.):
 Sharing the expert in...
A modified e-Delphi study(cont.)
 A modified e-Delphi study:
 May be one-stage or multi-stage; may manifest differently ...
An overview of the research workflow
Expert Perceptions of the Feasibility of MOOCs
7
The extant literature on MOOCs
(in 2013)
 Impetuses for MOOCs
 Types of MOOCs
 Teaching and learning platforms with dig...
The extant literature on MOOCs (cont.)
 Challenges with learner participation, retention, and completion
 Big data resea...
The survey instrument
For the modified e-Delphi study
Expert Perceptions of the Feasibility of MOOCs
10
The open-ended survey instrument
 About You
 Part 1: Impressions of MOOCs (open-ended questions with text-based response...
The open-ended survey instrument (cont.)
 Part 3: Ensuring MOOC Quality
 Strategies for managing learner “massiveness”
...
Expert Perceptions of the Feasibility of MOOCs
13 Sliders
Indicating
Future
Likelihood for
the “Potential
Scenarios”
(last...
The open-ended survey instrument (cont.)
 Potential Scenarios
 Near-term and midterm scenario for MOOCs
 Popularity
 U...
Seating the e-Delphi expert panel
 Announcements through professional electronic mailing lists; announcements in
the micr...
Panel: Years spent in online learning in
any capacity
Expert Perceptions of the Feasibility of MOOCs
16
Panel: Professional roles17
Panel: Direct experiences with MOOCs
18
Panel: Types of workplaces19
Panel: Years in the current professional
position
 9.5 years average
 Median: 10 years
 Range from 1 – 30 years in the ...
Panel: Direct experiences w/ MOOC
courses
21
Panel: Ways respondents learned about
MOOCs (check all that apply)
22
Panel: Familiarity measures with MOOCs23
Panel: Familiarity measures with MOOCs
(on a bell curve)
24
Some findings…
Expert Perceptions of the Feasibility of MOOCs
25
Potential positive social implications of
MOOCs in a globalist context
26
Potential negative social implications of
MOOCs in a globalist context
27
Possible benefits of MOOCs to the local
university
28
Anticipated pre-requisites for launching
MOOCs
29
Understood strategies for managing
massive numbers of students
30
A Wordle™ of text responses for the
definition of quality in MOOCs
Expert Perceptions of the Feasibility of MOOCs
31
Identity verification strategies for MOOC
learners
32
Strategies for the selection of MOOC
course topics
 Current areas of research to spark innovations
 General areas
 Univ...
Criteria for the selection of faculty to lead
MOOCs
34
MOOC platforms disruptive or
evolutionary?
35
Intellectual property (IP) and MOOCs36
MOOC suitability for various types of
learning
37
Long-term potentials for MOOCs38
MOOC scenarios
Near- and mid- term
Expert Perceptions of the Feasibility of MOOCs
39
Summary of MOOC predictions
 Little chance MOOCs will disappear in the near-term because of a lack of clear
funding model...
Summary of MOOC predictions (cont.)
 A better than chance likelihood that non-star professors will lead MOOCs
 A 50% cha...
Summary of MOOC predictions (cont.)
 A high likelihood that MOOC learners will be separated into an elite tuition-paying
...
Iff and other conditionals
 Actual and documented learning
 Effective design
 Respect for the individual learner and hi...
Real-world test of the findings?
 In the year since this study was conducted (June 2013), what predictions do you
see as ...
Implications? Questions? Comments?
 Are some of these findings intuitive? Others counter-intuitive? Elaborate.
 How woul...
References
 Hai-Jew, S. (2014). Iff and other conditionals: Expert perceptions of the feasibility
of massive open online ...
An addendum re: the research method
Expert Perceptions of the Feasibility of MOOCs
47
Some thoughts on modified e-Delphi
studies
Overall
 Serves as an effective research methodology that uses survey system t...
Some thoughts on modified e-Delphi
studies (cont.)
Creating Related Survey Instruments
 Requires in-depth and thorough se...
Some thoughts on modified e-Delphi
studies (cont.)
Light Use
 May work as a near-term and mid-term forecasting method (a ...
The benefits of maintaining a research
journal
 Structured reflective practice around
the research
 A documented venue f...
Conclusion and contact
 Dr. Shalin Hai-Jew
 Instructional Designer
 Information Technology Assistance Center (iTAC)
 K...
Upcoming SlideShare
Loading in...5
×

Expert Perceptions of the Feasibility of MOOCs

373

Published on

This slideshow highlights some findings from a modified e-Delphi studied about the feasibility of MOOCs.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
373
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Expert Perceptions of the Feasibility of MOOCs

  1. 1. Expert Perceptions of the Feasibility of MOOCs Reporting on a Modified E-Delphi Study Shalin Hai-Jew Paper Forum 2014 Big XII Teaching and Learning Conference Oklahoma State University Stillwater, Oklahoma Aug. 4 – 5, 2014
  2. 2. Presentation description  For any innovation to take, experts and other decision-makers in a field have to buy into the endeavor. This is likely so for the adoption of MOOCs. To determine the feasibility of MOOC adoption, a modified e-Delphi study was conducted recently to capture the thinking of an expert panel of individuals involved in e- learning. This presentation describes the processes of setting up the modified e- Delphi study, beginning with the extensive literature review undertaken for the development of the survey instrument (and the use of NVivo 10 to code the sources). The major findings from this qualitative and mixed methods research suggests critical design, policy, and other structural issues that need to be addressed individually and collectively in order to roll out successful MOOCs.  Caveats: This is a quick view, with various types of data reduction from the content analysis of the responses in the modified e-Delphi study. Expert Perceptions of the Feasibility of MOOCs 2
  3. 3. Welcome!  What are your interests re: this presentation?  Any experiences with modified e-Delphi studies?  What are your experiences with MOOCs? (Designing? Creating? Teaching? Administering? Supporting? Taking?) Expert Perceptions of the Feasibility of MOOCs 3
  4. 4. A modified e-Delphi study  A classic Delphi study (a structured group procedure):  Delphi study as a research method originating in the 1950s by the RAND Corporation (“Research ANd Development”); first described in research in Dalkey & Helmer, 1965  Seating an expert panel from the same field or widely disparate ones  Working with experts in a field (or in unrelated fields) on a difficult problem  Eliciting insights individually (and anonymously) in an unstructured way in an early exploration phase and then moving to structured elicitation of the experts Expert Perceptions of the Feasibility of MOOCs 4
  5. 5. A modified e-Delphi study (cont.)  A classic Delphi study (a structured group procedure) (cont.):  Sharing the expert insights and tacit knowledge; examining whether there is a move toward consensus or convergence on a difficult or complex problem  Evolving the process based on new information  Forecasting (~ Delphic oracle) and enhancing long-range human policy-making and decision-making Expert Perceptions of the Feasibility of MOOCs 5
  6. 6. A modified e-Delphi study(cont.)  A modified e-Delphi study:  May be one-stage or multi-stage; may manifest differently over time  Brings in various sampling technologies including through information and communication technology (ICT)  Enables geographically distributed expert panels  Uses a range of survey techniques and technologies (with a focus on experts and on forecasting and on policymaking)  Does not strive for consensus per se (researchers expect a wide range of responses, including dissensus) Expert Perceptions of the Feasibility of MOOCs 6
  7. 7. An overview of the research workflow Expert Perceptions of the Feasibility of MOOCs 7
  8. 8. The extant literature on MOOCs (in 2013)  Impetuses for MOOCs  Types of MOOCs  Teaching and learning platforms with digital content delivery, comment ecosystems, mass assessments, back-end tracking, and augmentation by social media  Teaching strategies  Application of automation  Research on MOOC learning experiences and learner “types” (such as completing, auditing, disengaging, and sampling)  Funding Expert Perceptions of the Feasibility of MOOCs 8
  9. 9. The extant literature on MOOCs (cont.)  Challenges with learner participation, retention, and completion  Big data research potentials  Types of courses and course contents  Credentialing and badging Expert Perceptions of the Feasibility of MOOCs 9
  10. 10. The survey instrument For the modified e-Delphi study Expert Perceptions of the Feasibility of MOOCs 10
  11. 11. The open-ended survey instrument  About You  Part 1: Impressions of MOOCs (open-ended questions with text-based responses for all sections with a few exceptions)  Experiences  Impressions  Sentiments  Part 2: Local Adoptions of MOOCs as Course Offerings  Benefits of MOOCs (for stakeholder groups, for reputation, for budgets, for resources)  Pre-requisites for adoption Expert Perceptions of the Feasibility of MOOCs 11
  12. 12. The open-ended survey instrument (cont.)  Part 3: Ensuring MOOC Quality  Strategies for managing learner “massiveness”  Definition of “quality learning” in MOOCs  Student identity in MOOCs  Criteria for course topics selection for MOOCs  Preferred instructor characteristics  Part 4: MOOCs in the Future  MOOC platforms as disruptive technologies or not  Open-courseware movement? Open-source movement? As critical parts of MOOCs?  Informal learning and certification or badging? Formal learning?  Long-term potentials? Expert Perceptions of the Feasibility of MOOCs 12
  13. 13. Expert Perceptions of the Feasibility of MOOCs 13 Sliders Indicating Future Likelihood for the “Potential Scenarios” (last) Question
  14. 14. The open-ended survey instrument (cont.)  Potential Scenarios  Near-term and midterm scenario for MOOCs  Popularity  Use by universities for formal learning and college credit? Use by corporations for training?  Star professors or common professors  Learner-taught MOOCs  Used for development in developing countries  Cobbled MOOC platforms using Web 2.0 technologies  Copyright or open-source  Big data advancements and insights Expert Perceptions of the Feasibility of MOOCs 14
  15. 15. Seating the e-Delphi expert panel  Announcements through professional electronic mailing lists; announcements in the microblogging site Twitter (social network connectivity), and others  Self-identified and self-selected experts  Anonymous respondents  A fairly small dataset (may range from a few to a thousand or so traditionally)  Some 103 total respondents but not responses on all questions Expert Perceptions of the Feasibility of MOOCs 15
  16. 16. Panel: Years spent in online learning in any capacity Expert Perceptions of the Feasibility of MOOCs 16
  17. 17. Panel: Professional roles17
  18. 18. Panel: Direct experiences with MOOCs 18
  19. 19. Panel: Types of workplaces19
  20. 20. Panel: Years in the current professional position  9.5 years average  Median: 10 years  Range from 1 – 30 years in the current professional position Expert Perceptions of the Feasibility of MOOCs 20
  21. 21. Panel: Direct experiences w/ MOOC courses 21
  22. 22. Panel: Ways respondents learned about MOOCs (check all that apply) 22
  23. 23. Panel: Familiarity measures with MOOCs23
  24. 24. Panel: Familiarity measures with MOOCs (on a bell curve) 24
  25. 25. Some findings… Expert Perceptions of the Feasibility of MOOCs 25
  26. 26. Potential positive social implications of MOOCs in a globalist context 26
  27. 27. Potential negative social implications of MOOCs in a globalist context 27
  28. 28. Possible benefits of MOOCs to the local university 28
  29. 29. Anticipated pre-requisites for launching MOOCs 29
  30. 30. Understood strategies for managing massive numbers of students 30
  31. 31. A Wordle™ of text responses for the definition of quality in MOOCs Expert Perceptions of the Feasibility of MOOCs 31
  32. 32. Identity verification strategies for MOOC learners 32
  33. 33. Strategies for the selection of MOOC course topics  Current areas of research to spark innovations  General areas  University specialists  Learning groups of non-traditional and mid-career professionals to meet employer needs  Long-tail niche courses  Topics that other MOOCs haven’t covered  Future focused topics  “Why not anything?” Expert Perceptions of the Feasibility of MOOCs 33
  34. 34. Criteria for the selection of faculty to lead MOOCs 34
  35. 35. MOOC platforms disruptive or evolutionary? 35
  36. 36. Intellectual property (IP) and MOOCs36
  37. 37. MOOC suitability for various types of learning 37
  38. 38. Long-term potentials for MOOCs38
  39. 39. MOOC scenarios Near- and mid- term Expert Perceptions of the Feasibility of MOOCs 39
  40. 40. Summary of MOOC predictions  Little chance MOOCs will disappear in the near-term because of a lack of clear funding models  A fairly high probability MOOCs will be popularized beyond well-funded universities  A fairly high likelihood that MOOCs would be used for training by corporations  A low likelihood that MOOCs would be used as formal college courses  A high likelihood that MOOCs would be used as short courses, tutorials, and automated trainings  A low probability that star professors will be the main ones teaching MOOCs Expert Perceptions of the Feasibility of MOOCs 40
  41. 41. Summary of MOOC predictions (cont.)  A better than chance likelihood that non-star professors will lead MOOCs  A 50% chance likelihood that MOOCs will be taught by the learners in that MOOC  A low probability that MOOCs will only be developed in advanced or well developed countries (a high probability that MOOCs will be adopted and developed in developing countries)  A high likelihood that MOOC platforms will be developed and maintained commercially  A high likelihood that MOOC platforms will be created by cobbling Web 2.0 technologies Expert Perceptions of the Feasibility of MOOCs 41
  42. 42. Summary of MOOC predictions (cont.)  A high likelihood that MOOC learners will be separated into an elite tuition-paying core and a non-paying general public periphery  An equal chance that learners will be treated equally (without a paying elite vs. a general non-paying public)  A fairly high probability that MOOC courses would fall under copyright protection (and other intellectual property laws); also a fairly high probability that MOOCs will be released in an open-source way  A high probability of new insights of human learners and their needs from “big data” resulting from MOOCs; respondents also suggested that there is a low probability of not finding out new insights from the big data related to MOOCs (in a conversely-phrased “checking” question) Expert Perceptions of the Feasibility of MOOCs 42
  43. 43. Iff and other conditionals  Actual and documented learning  Effective design  Respect for the individual learner and his /her personhood  Faculty voice and choices  Egalitarianism in terms of faculty leading  A clear (and fair) funding model  Useful research from the big data  Proper identification of learners  Proper crediting of learning (in the various contexts)… Expert Perceptions of the Feasibility of MOOCs 43
  44. 44. Real-world test of the findings?  In the year since this study was conducted (June 2013), what predictions do you see as having been borne out? (Is this a valid test of the “predictions” of the modified e-Delphi study results? Are there extant trends that were not addressed here?  Any core changes to the status quo in regards to MOOCs? Expert Perceptions of the Feasibility of MOOCs 44
  45. 45. Implications? Questions? Comments?  Are some of these findings intuitive? Others counter-intuitive? Elaborate.  How would you answer some of these survey questions? Why?  What other questions would you have asked in this instrument? Why?  In terms of variations on MOOCs (MCs, MOCs, MOCs, OOCs, OCs, OCs), which do you see as having the most potential? Why? (A recent suggestion involved the creation of LOOCs or “local open online courses”.)  What do you think is next? Expert Perceptions of the Feasibility of MOOCs 45
  46. 46. References  Hai-Jew, S. (2014). Iff and other conditionals: Expert perceptions of the feasibility of massive open online courses (MOOCs)—A modified e-Delphi study. Ch. 13. In Hai-Jew’s Remote Workforce Training: Effective Technologies and Strategies. IGI- Global. Expert Perceptions of the Feasibility of MOOCs 46
  47. 47. An addendum re: the research method Expert Perceptions of the Feasibility of MOOCs 47
  48. 48. Some thoughts on modified e-Delphi studies Overall  Serves as an effective research methodology that uses survey system technologies well (particularly those that incorporate multimedia elements like videos, audio files, slideshows, and images)  Works well as an open-ended method with plenty of designed open-ended questions (without any forced consensus or forced dissensus) Seating Expert Panels  May be benefitted by the far-distance and near-in access to various experts  May be weakened by self-assertions of expertise by anonymous participants Expert Perceptions of the Feasibility of MOOCs 48
  49. 49. Some thoughts on modified e-Delphi studies (cont.) Creating Related Survey Instruments  Requires in-depth and thorough secondary research about the topic to create depth and breadth  Requires plenty of vetting of test instruments Single or Multi-Staging?  May be more effective as a multi-stage process particularly for more complex issues Expert Perceptions of the Feasibility of MOOCs 49
  50. 50. Some thoughts on modified e-Delphi studies (cont.) Light Use  May work as a near-term and mid-term forecasting method (a kind of predictive analytics), if held lightly  If something has to be put at risk based on the results of the modified e-Delphi study, it would be good to conduct much more research Additional Research Necessary about Effective Modified e-Delphi Study Methods  Would benefit from more research in terms of efficacy Expert Perceptions of the Feasibility of MOOCs 50
  51. 51. The benefits of maintaining a research journal  Structured reflective practice around the research  A documented venue for questioning at every stage of the research work  Not released with the work at publication Expert Perceptions of the Feasibility of MOOCs 51
  52. 52. Conclusion and contact  Dr. Shalin Hai-Jew  Instructional Designer  Information Technology Assistance Center (iTAC)  Kansas State University  212 Hale Library  Manhattan, KS 66506-1200  785-532-5262 (work phone)  shalin@k-state.edu  A special thanks to the attendees at the SIDLIT Special Interest Group (SIG) lunch session on MOOCs who helped catch a few problematic charts, which have since been re-drawn and labeled. :P It’s always good to have many eyes on a work, especially before going public! 52
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×