Ancient idea, older than even the University of London
Is there anyone here who doesn’t know what a MOOC is? Genuine question.
But first, a commercial break. FutureLearn announced this time last year. Now has real students on it. No real data – yet.
Technical The diversity of online tools often employed can make it hard to collect and process data; the disconnection from formal educational systems means that much useful data - such as demographics, previous experience, other courses being studied, etc - is unavailable. Theoretical To date, MOOCs have not led directly to the award of credit, so analytics in these terms is problematic: endpoints, progress and feedback are different. Philosophical For connectivist MOOCs, the idea of defining learning endpoints ahead of time, let alone tracking and measuring progress towards them, is anathema. Resource An important factor in the rapid spread of MOOCs is their low cost base: this means there is far less resource available per student for analytics activities – and, more importantly, far less for human mediation of analytics feedback
There was a point where you could be fully briefed on the research literature on MOOCs in an afternoon. It was about this time last year. Interesting models – Stanford U Lytics Lab behaviour; Coursera/Ng/Koller two populations high/low retention MOOC Research Initiative, Gates Foundation grants
This *was* the state of the art.
One of our graduate students, Katy Jordan, has done the job properly. The scale only went up to 20%.
Even more data! A few high ones – one over 50% now. But most have rates <15%. Possible exception of Open2Study (Australia); small, interesting.
Enroll = prospectus, or UCAS application
Background in online activity. Three examples of informal learning communities, dropout/participation.
Wired, 2004. Steeply unequal distribution of popularity – purchases, downloads, clicks, likes. Learns? Contributions online, to forums, Flickr.
Sometimes it is, but you don’t have the data. Need lots. Sometimes log normal. As silly as ‘chaos theory’ for ‘it’s messy’
Three examples. See the drop at each stage.
Compare those to 15% dropout … but this is the people who started at all, not completions. Maybe == submitted first assignment
Buy a degree gag: we may be private sector, but we don’t sell degrees, we sell learning opportunities Big drop each time
Drop off each time
Highly unequal, steeply unequal. A few people do lots, lots of people do a little. It can be exponential fall off – or steeper than that. But IT’S NOT A POWER LAW. You need a lot of data.
Stanford/Oxford/MIT. Post-92. OU, U Phoenix. The university figures here are for whole degrees, so 15% MOOC is understatement here: if 12 courses for a degree, independent probability, <1 in 7bn complete! World pop. (Of course, the probabilities aren’t independent.) The traditional universities – mass, distance/open/online - are doing something right that MOOCs are not doing. All that work on student support is not wasted. Commitment too. We can’t not care about drop out. Climb out, drop out – study aim.
Half Dome, Yosemite
- QAA, Institutional Review Further Guidance, 2011-12
About 75% completion rate. One of our level 1 modules. Dropout isn’t a sudden fall-off near the start.
2 component: high-retention (45%), and low-retention populations.
But not a power law!
Proto-MOOC. Wasn’t at start, was at the end.
The funnel of participation: beyond dropout in MOOCs, informal learning and universities
The funnel of participation:
Moving beyond dropout
Doug Clow, Institute of Educational Technology,
The Open University, UK
Centre for Distance Education, University of London, 10 Dec 2013.
• To change the way you think about dropout.
With a funnel.
Photo (cc) Nina Hale http://www.flickr.com/photos/94693506@N00/614751836/
Funnel of participation
cc licensed ( BY SA ) flickr photo by Rob Sinclair: http://flickr.com/photos/rob-sinclair/6025086793/
• Learning analytics on MOOCs is hard:
– Technical challenges
– Theoretical challenges
– Philosophical challenges
– Resource challenges
Honeybees Apis mellifera
Photo (cc) David Goehring http://www.flickr.com/photos/carbonnyc/5041997785/
Main findings of MOOC research
•People drop out a lot
•More research required
•They really do drop out a lot
•More research arriving
Photo (cc) jeroen bennink http://www.flickr.com/photos/jeroenbennink/2355768494/
State of the art in MOOC Completion Rates Nov 2012
At least 16
Photo (cc) Seth Tisue http://www.flickr.com/photos/tisue/254308538/
MOOC Completion Rates April 2013
MOOC Completion Rates December 2013
Who counts to
• Anyone who clicked ‘enroll’?
• Anyone who logged in once?
• Anyone who watched ≥ one video?
– Could yield >100%!
Photo (cc) wales_gibbons
= 1 in 7bn for 12
Photo (cc) Walt Hubis http://www.flickr.com/photos/walthubis/4346378552/
Put your thinking funnel on
• Think less about total dropout
• Think more about why participation
reduces at each stage
• Think about patterns of participation
• Benchmark whether it’s a problem
Photo (cc) Tom Bayly http://www.flickr.com/photos/tombayly13/6006166585/
What is to be done
• Pour more in the top
• Widen the funnel at each point
– Pinch points
• Put better in the top
– Right person, right course
cc licensed ( BY ) flickr photo by Steve Dunleavy: http://flickr.com/photos/stevedunleavy/5142841381/
Look at the data ...
… as a whole.
“retention and progression data, viewed in isolation, have limited
value as measures of student success in the first year” – QAA.
cc licensed ( BY ) flickr photo by Klearchos Kapoutsis: http://flickr.com/photos/klearchos/3601801484/
This work is licensed under a Creative Commons Attribution 3.0 Unported License
cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/
Ng, Koller, Ko and Chen (2013.
funnel of participation =
steep drop off from
one stage to the next
“We have a lousy product”
– Sebastian Thrun, Udacity CEO, Nov 14, 2013
Photo (c) Fast Company & Inc 2013 Mansueto Ventures, LLC.
Free, open, online, OER, CC BY-SA
Seven EU partners
Three presentations, 2009-2012
Joomla, email, IRC, Elluminate
Explored business models, found none