The funnel of participation:
Moving beyond dropout

Doug Clow, Institute of Educational Technology,
The Open University, U...
Aim

• To change the way you think about dropout.
With a funnel.
Photo (cc) Nina Hale http://www.flickr.com/photos/9469350...
Overview
•
•
•
•
•

MOOC dropout
Informal learning
Funnel of participation
Universities
Improving learning

cc licensed ( ...
Nuremberg funnel,
1902
Just say no!
to antediluvian,
ineffective,
unsophisticaed
epistemologies of
pedagogical
ontology
MOOC dropout
futurelearn.com
• Learning analytics on MOOCs is hard:
– Technical challenges
– Theoretical challenges
– Philosophical challenges
– Resour...
Main findings of MOOC research
2012:
•People drop out a lot
•More research required
2013:
•They really do drop out a lot
•...
State of the art in MOOC Completion Rates Nov 2012

CCK08

PLENK2010 MITx
Circuits &
Electronics

Coursera
Software
Engine...
MOOC Completion Rates April 2013

http://www.katyjordan.com/MOOCproject.html
MOOC Completion Rates December 2013

http://www.katyjordan.com/MOOCproject.html
Who counts to
start with?

• Anyone who clicked ‘enroll’?
• Anyone who logged in once?
• Anyone who watched ≥ one video?
–...
informal learning
Long tail graph (cc) Chris Anderson http://www.longtail.com/about.html
it’s not a power law
 

iSpot

Visits

1,100,000

275,000

30,000

390,000
(35%)

165,000
(60%)

15,500
(52%)

Registrations

21,000
(5%)

5,23...
 

iSpot

Visits

1,100,000

275,000

30,000

390,000
(35%)

165,000
(60%)

15,500
(52%)

Registrations

21,000
(5%)

5,23...
the funnel of participation
Awareness
Interest
Desire
the marketing
funnel
Purchase
Awareness
Registration
Activity
the funnel of
participation
Progress
funnel of participation =

steep drop off from
one stage to the next
+
highly unequal
participation pattern
universities
Distance /
Elite
Mass
open /
university university
online
university
Representative
completion rate

~ 90%

~ 60%

~ 35%

...
making things better
Put your thinking funnel on
• Think less about total dropout
• Think more about why participation
reduces at each stage
• ...
What is to be done
• Pour more in the top
• Widen the funnel at each point
– Expectations
– Motivation
– Pinch points
– Ro...
Look at the data ...

… as a whole.

“retention and progression data, viewed in isolation, have limited

value as measures...
Doug Clow
@dougclow
doug.clow@open.ac.uk
dougclow.org/mooc-funnel
This work is licensed under a Creative Commons Attributi...
cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/
Ng, Koller, Ko and Chen (2013.
http://www.educause.edu/ero/article/retention-and-intention-massive-open-online-courses-dep...
funnel of participation =

steep drop off from
one stage to the next
+
highly unequal
participation pattern
“We have a lousy product”
– Sebastian Thrun, Udacity CEO, Nov 14, 2013
http://www.fastcompany.com/3021473/udacity-sebastia...
•
•
•
•
•
•

http://www.open-ed.eu
Free, open, online, OER, CC BY-SA
Seven EU partners
Three presentations, 2009-2012
Joom...
The funnel of participation: beyond dropout in MOOCs, informal learning and universities
The funnel of participation: beyond dropout in MOOCs, informal learning and universities
Upcoming SlideShare
Loading in...5
×

The funnel of participation: beyond dropout in MOOCs, informal learning and universities

1,753

Published on

Slides for a talk at the Centre for Distance Education event, "InFocus: Learner analytics and big data", #CDEInFocus, University of London, Senate House, 10th December 2013.

Published in: Education, Sports
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,753
On Slideshare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
4
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • Dangerous aim to make people think.
  • Ancient idea, older than even the University of London
  • Is there anyone here who doesn’t know what a MOOC is? Genuine question.
  • But first, a commercial break.
    FutureLearn announced this time last year.
    Now has real students on it. No real data – yet.
  • Technical The diversity of online tools often employed can make it hard to collect and process data; the disconnection from formal educational systems means that much useful data - such as demographics, previous experience, other courses being studied, etc - is unavailable.
    Theoretical To date, MOOCs have not led directly to the award of credit, so analytics in these terms is problematic: endpoints, progress and feedback are different.
    Philosophical For connectivist MOOCs, the idea of defining learning endpoints ahead of time, let alone tracking and measuring progress towards them, is anathema.
    Resource An important factor in the rapid spread of MOOCs is their low cost base: this means there is far less resource available per student for analytics activities – and, more importantly, far less for human mediation of analytics feedback
  • There was a point where you could be fully briefed on the research literature on MOOCs in an afternoon.
    It was about this time last year.
    Interesting models – Stanford U Lytics Lab behaviour; Coursera/Ng/Koller two populations high/low retention
    MOOC Research Initiative, Gates Foundation grants
  • This *was* the state of the art.
  • One of our graduate students, Katy Jordan, has done the job properly.
    The scale only went up to 20%.
  • Even more data!
    A few high ones – one over 50% now. But most have rates <15%.
    Possible exception of Open2Study (Australia); small, interesting.
  • Enroll = prospectus, or UCAS application
  • Background in online activity.
    Three examples of informal learning communities, dropout/participation.
  • Wired, 2004.
    Steeply unequal distribution of popularity – purchases, downloads, clicks, likes. Learns?
    Contributions online, to forums, Flickr.
  • Sometimes it is, but you don’t have the data. Need lots.
    Sometimes log normal.
    As silly as ‘chaos theory’ for ‘it’s messy’
  • Three examples. See the drop at each stage.
  • Compare those to 15% dropout … but this is the people who started at all, not completions.
    Maybe == submitted first assignment
  • Buy a degree gag: we may be private sector, but we don’t sell degrees, we sell learning opportunities
    Big drop each time
  • Drop off each time
  • Highly unequal, steeply unequal. A few people do lots, lots of people do a little. It can be exponential fall off – or steeper than that. But IT’S NOT A POWER LAW. You need a lot of data.
  • Stanford/Oxford/MIT. Post-92. OU, U Phoenix.
    The university figures here are for whole degrees, so 15% MOOC is understatement here: if 12 courses for a degree, independent probability, <1 in 7bn complete! World pop.
    (Of course, the probabilities aren’t independent.)
    The traditional universities – mass, distance/open/online - are doing something right that MOOCs are not doing. All that work on student support is not wasted.
    Commitment too.
    We can’t not care about drop out.
    Climb out, drop out – study aim.
  • Half Dome, Yosemite
  • - QAA, Institutional Review Further Guidance, 2011-12
  • About 75% completion rate. One of our level 1 modules.
    Dropout isn’t a sudden fall-off near the start.
  • 2 component: high-retention (45%), and low-retention populations.
  • But not a power law!
  • Proto-MOOC. Wasn’t at start, was at the end.
  • The funnel of participation: beyond dropout in MOOCs, informal learning and universities

    1. 1. The funnel of participation: Moving beyond dropout Doug Clow, Institute of Educational Technology, The Open University, UK Centre for Distance Education, University of London, 10 Dec 2013.
    2. 2. Aim • To change the way you think about dropout. With a funnel. Photo (cc) Nina Hale http://www.flickr.com/photos/94693506@N00/614751836/
    3. 3. Overview • • • • • MOOC dropout Informal learning Funnel of participation Universities Improving learning cc licensed ( BY SA ) flickr photo by Rob Sinclair: http://flickr.com/photos/rob-sinclair/6025086793/
    4. 4. Nuremberg funnel, 1902
    5. 5. Just say no! to antediluvian, ineffective, unsophisticaed epistemologies of pedagogical ontology
    6. 6. MOOC dropout
    7. 7. futurelearn.com
    8. 8. • Learning analytics on MOOCs is hard: – Technical challenges – Theoretical challenges – Philosophical challenges – Resource challenges Honeybees Apis mellifera Photo (cc) David Goehring http://www.flickr.com/photos/carbonnyc/5041997785/
    9. 9. Main findings of MOOC research 2012: •People drop out a lot •More research required 2013: •They really do drop out a lot •It’s complicated •More research arriving Photo (cc) jeroen bennink http://www.flickr.com/photos/jeroenbennink/2355768494/
    10. 10. State of the art in MOOC Completion Rates Nov 2012 CCK08 PLENK2010 MITx Circuits & Electronics Coursera Software Engineerin g Registered 2,200 1,641 150,000 50,000 Completed At least 16 40-60 ‘active’ 7,157 3,500 Not Not 7% definable definable Photo (cc) Seth Tisue http://www.flickr.com/photos/tisue/254308538/ 7% Completion rate
    11. 11. MOOC Completion Rates April 2013 http://www.katyjordan.com/MOOCproject.html
    12. 12. MOOC Completion Rates December 2013 http://www.katyjordan.com/MOOCproject.html
    13. 13. Who counts to start with? • Anyone who clicked ‘enroll’? • Anyone who logged in once? • Anyone who watched ≥ one video? – Could yield >100%! Photo (cc) wales_gibbons Survival analysis
    14. 14. informal learning
    15. 15. Long tail graph (cc) Chris Anderson http://www.longtail.com/about.html
    16. 16. it’s not a power law
    17. 17.   iSpot Visits 1,100,000 275,000 30,000 390,000 (35%) 165,000 (60%) 15,500 (52%) Registrations 21,000 (5%) 5,239 (3%) 1,429 (9%) Contributors 9,000 (43%) 1,750 (33%) 198 (14%) Unique visitors Cloudworks Data from respective site launch to 7 November 2012 openED
    18. 18.   iSpot Visits 1,100,000 275,000 30,000 390,000 (35%) 165,000 (60%) 15,500 (52%) Registrations 21,000 (5%) 5,239 (3%) 1,429 (9%) Contributors 9,000 (43%) 1,750 (33%) 198 (14%) Unique visitors Cloudworks Data from respective site launch to 7 November 2012 openED
    19. 19. the funnel of participation
    20. 20. Awareness Interest Desire the marketing funnel Purchase
    21. 21. Awareness Registration Activity the funnel of participation Progress
    22. 22. funnel of participation = steep drop off from one stage to the next + highly unequal participation pattern
    23. 23. universities
    24. 24. Distance / Elite Mass open / university university online university Representative completion rate ~ 90% ~ 60% ~ 35% MOOCs < 15% = 1 in 7bn for 12 Photo (cc) Walt Hubis http://www.flickr.com/photos/walthubis/4346378552/
    25. 25. making things better
    26. 26. Put your thinking funnel on • Think less about total dropout • Think more about why participation reduces at each stage • Think about patterns of participation • Benchmark whether it’s a problem Photo (cc) Tom Bayly http://www.flickr.com/photos/tombayly13/6006166585/
    27. 27. What is to be done • Pour more in the top • Widen the funnel at each point – Expectations – Motivation – Pinch points – Roadblocks – Interventions • Put better in the top – Right person, right course cc licensed ( BY ) flickr photo by Steve Dunleavy: http://flickr.com/photos/stevedunleavy/5142841381/
    28. 28. Look at the data ... … as a whole. “retention and progression data, viewed in isolation, have limited value as measures of student success in the first year” – QAA. cc licensed ( BY ) flickr photo by Klearchos Kapoutsis: http://flickr.com/photos/klearchos/3601801484/
    29. 29. Doug Clow @dougclow doug.clow@open.ac.uk dougclow.org/mooc-funnel This work is licensed under a Creative Commons Attribution 3.0 Unported License
    30. 30. cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/
    31. 31. Ng, Koller, Ko and Chen (2013. http://www.educause.edu/ero/article/retention-and-intention-massive-open-online-courses-depth-0
    32. 32. funnel of participation = steep drop off from one stage to the next + highly unequal participation pattern
    33. 33. “We have a lousy product” – Sebastian Thrun, Udacity CEO, Nov 14, 2013 http://www.fastcompany.com/3021473/udacity-sebastian-thrun-uphill-climb Photo (c) Fast Company & Inc 2013 Mansueto Ventures, LLC.
    34. 34. • • • • • • http://www.open-ed.eu Free, open, online, OER, CC BY-SA Seven EU partners Three presentations, 2009-2012 Joomla, email, IRC, Elluminate Explored business models, found none
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×