3 adopted any ethical review or ethical standards.
3 adopted any ethical review or ethical standards.Only 1 case “controlled study” of any methodological quality.Only 1 study (Sri Lanka) investigated social inclusion and digital access.
OpenEd13 UTAH MOOCs and social responsibility to learners V Rolfe Nov2013
Free online courses and social
responsibility toward learners.
A systematic literature and blogature review.
Dr Vivien Rolfe
University of the West of England, Bristol UK
National Teacher Fellow
Science Open Educator
CC BY Vivien Rolfe
Open Education Conference
November 6-8, Utah Park City
Why this question?
• I would argue that campus and distance learning
programmes have the following ethical stance:
– Students have to meet entry criteria for enrolment
– Curricula place great emphasis on student experience and
– Strategies address academic quality, accessibility,
inclusivity and diversity
– Academics care for student welfare
• What is the academic and ethical stance surrounding
newer free online courses? xMoocs, cMoocs?
• The aim was to systematically review the research literature and
“blogature” to identify evidence (and the gaps) relating to open
online courses and the student experience. Areas of interest:
Student experience and success
Social and digital inclusion
Quality and governance
• Did not achieve a second aim to survey academic and student
opinion – moved house, jobs and cities over the summer!
CC BY Vivien Rolfe
#Searching “NOT Minnie the Moocher”
Six research databases – keywords & MeSH
Google Blogs (first 200 blogs of 2 searches)
Peers, Twitter and Google alerts for six months
• cMOOX*, xMOOC*, massive online open
• Student experience, social responsibility, ethics,
• NOT molybdenum (MoOCl)
• NOT “Minnie the Moocher”
Wikimedia Commons, Public Domain
Lit and Blog Analysis
Read titles and abstracts, and retrieved full document.
Searched blogs / web documents / PDFs for keywords using the “search option”.
EXCLUSION LEG 1
Duplicates were excluded. Those with no keyword matches within the article were excluded. No
research was excluded on the basis of quality (which would be the norm for a systematic review).
(Rolfe and Gray 2011)
EXCLUSION LEG 2
Articles, reviews, blogs were then excluded on fuller examination of the text (e.g. if a blog about a
MOOC, personal experience of a MOOC, keyword matched but out of context with my themes).
Search results were recorded on a spreadsheet.
Separate analyses for 1) peer-reviewer research studies; 2) peer-reviewed journal review articles; 3)
reports, eBooks and blog articles.
Review and blog themes were clustered, and new relevant themes identified.
Lit and Blog Identification
Scholar did not
retrieve a single
Lit and Blog Identification
The MOOCs of Hazard
I’m in the mood for MOOCs
Some MOOC points
MOOC or not to MOOC
Lit Analysis – Leg 2; 129 Articles down to 38.
Only 1 “controlled” academic study.
3 gained ethical approval to use student data/blog narratives for research.
1 study from Sri Lanka looked at digital and social inclusion.
Learner data not the learner!
• Liyanagunawardena et al (2013) – systematic account of
MOOC literature (research + other documents); 45 items
and 9 matched for student experience. Not evaluated for
• Means et al (2010); search of 1132 and 45 included for
“The most unexpected finding was that an extensive initial search
of the published literature from 1996 through 2006 found no
experimental or controlled quasi-experimental studies”
“Elements such as video or online quizzes do not appear to
influence the amount that students learn in online classes. The
research does not support the use of some frequently
recommended online learning practices”.
Blog Analysis – Leg 2; 75 Articles down to 39.
“And what does “primarily free” mean?
MOOCs are only available to people with
excellent broadband internet access and fast
OUR ACADEMIC SOCIAL RESPONSIBILITY
“Drop out rates. But what about the drop outs?”
“I would hate to see people get a bad taste of
university because there's just too many
students in there to get personal attention.”
“The question is if students without the
literacies required to take a Higher Education
degree are able to succeed”
IP AND PRIVACY
“Then there are the sticky matters of
plagiarism, identity fraud, peer grading, and a
lack of creditable assessment of learning
“There will be no private, “safe” spaces for
“Assuming you have the right tech, you’ll also
have to be comfortable with being tracked and
ACADEMIC QUALITY ASSURANCE / PEDAGOGY
“Who will validate student learning?”
“Fairly unaltered in relation to the important
stuff like instructional design, instructional
delivery, and authentic assessment.”
Summary – learner experience?
• Concerns for learner experience and our social
responsibility was reflected in blog
opinions, but was not the subject of good
• A lack of concern with who drops out and
why, and the impact on their future learning.
Summary – pedagogy?
• Insights into online learning design from Means et al’s review (2010) are
not reflected in xMOOC design. Students online performed slightly better
than face-to-face (although there was no mention of time invested).
Learning outcomes were enhanced by:
– Collaborative or instructor-directed learning rather than independent.
– Giving learners control of their interactions and time for reflection.
• Learning was not enhanced by:
– Use of video or online quizzes.
– Providing guidance to groups as opposed to individuals.
• Discussions about pedagogy were reflected in blog opinions but not
Summary – unethical?
• What are the user rights - for privacy? A safe
learning space? Is there any data protection?
Who owns the user data, and how will it be
used next? Affiliate marketing?
• Where are the ethical standards? Blog and
personal learning narratives readily used in
research. Where is the informed consent?
Summary – scope of research?
• The current work concurs with past systematic
reviews on the lack of good quality
educational research in online learning (Dean
2010, Rolfe 2011).
• Research emerging over the last 5 years is
interested in learner DATA not the learner.
Let’s not do education by numbers!
CC BY Vivien Rolfe
For further information go to:
• Slides on Slideshare
• Reference Lists on Google Docs
• Methods blogs – searching literature and “blogature”
• Learning by numbers. Pitfalls of pursuing targets.
• Full paper – results of systematic review, research findings
• Liyanagunawardena TR, Adams AAA and Williams SA (2013) MOOCs: A
systematic study of the published literature 2008-2012. The International
Review of Research in Open and Distance Learning, Volume 14(3), 202-227.
• Means B, Toyama Y, Murphy R, Bakia M and Jones K (revised 2010)
Evaluation of evidence-based practices in online learning. A meta-analysis
and review of online learning studies. US Department of Education.
• Rolfe VE and Gray D (2011) Are Multimedia Resources Effective in Life
Science Education? A Meta-Analysis. Bioscience Education, Volume 18.