DECIPHERING THE MASSIVE OPEN ONLINE
COURSES (MOOCS) ECOSYSTEM: WHAT
MAKES SUCCESSFUL MOOCS?
Ido Roll, Will Engle, Simon Ba...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
Please raise your hand if you have pre-teen
child(ren) at home.!
Please ...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
LEARNING FROM VIDEO !
IS POSSIBLE
3
If someone wants to learn, you canno...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
WE KNOW A LOT ABOUT
UNIVERSITY TEACHING
What works?!
Small classrooms!
I...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
THREE KEY QUESTIONS
5
Who are !
our learners?
What defines !
success?
Wha...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
CONTEXT
Four UBC courses (Spring/Summer 2013)!
Climate Literacy; Useful ...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
VIDEO PRODUCTION
7
Averagelengthpervideo(min)
0
6
12
18
24
Average # of ...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
THREE KEY QUESTIONS
8
Who are
our learners?
What defines !
success?
What ...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
WHO ARE OUR LEARNERS?
General demographics consistent across courses
and...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
WHO ARE OUR LEARNERS?
General demographics consistent across courses
and...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
ROLE OF GEOGRAPHY
Does location matter? Are MOOCs “local”?
11
Country of...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
ROLE OF GEOGRAPHY
Does location matter? Are MOOCs “local”?
12
Country of...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
ROLE OF GEOGRAPHY
Break down by course.
weighting by overall
registratio...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
THREE KEY QUESTIONS
14
Who are !
our learners?
What defines
success?
What...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
WHAT DEFINES SUCCESS?
Definitions:!
Visitors: log in at least once. Our d...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
METRICS OF SUCCESS
Definitions:!
Visitors: log in at least once. Our deno...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
METRICS OF SUCCESS
Definitions:!
Visitors: log in at least once. Our deno...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
METRICS OF SUCCESS
Definitions:!
Visitors: log in at least once. Our deno...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
METRICS OF SUCCESS
Definitions:!
Visitors: log in at least once. Our deno...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
METRICS OF SUCCESS
Metrics:!
Attractiveness: convertVisitors to Tasters....
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
METRICS OF SUCCESS
21
0
0.15
0.3
0.45
0.6
Attractiveness Retention Inter...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
THREE KEY QUESTIONS
22
Who are !
our learners?
What defines !
success?
Wh...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
WHAT AFFECTS CONVERSION
RATES?
Correlate with:!
Average length per video...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
ATTRACTIVENESS
Visitors -> Tasters: How many of those who
visited the we...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
ATTRACTIVENESS
Visitors -> Tasters: How many of those who
visited the we...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
RETENTION
Tasters -> Learners: How many of those who
watched @ Week 2 al...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
RETENTION
Tasters -> Learners: How many of those who
watched @ Week 2 al...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
INTERACTIVITY
How many of the students who watch videos also
submit prob...
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
SUMMARY
Local matters!
Courses have unique “signatures”!
Structure, e.g....
@hummus_monster @simonpbates @UBC_CTLT #STLHE2014
DESIGN RECOMMENDATIONS
Long videos may be okay!
Especially later in the ...
Upcoming SlideShare
Loading in …5
×

Deciphering the MOOC ecosystem: what makes successful MOOCs

465 views

Published on

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
465
On SlideShare
0
From Embeds
0
Number of Embeds
18
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Deciphering the MOOC ecosystem: what makes successful MOOCs

  1. 1. DECIPHERING THE MASSIVE OPEN ONLINE COURSES (MOOCS) ECOSYSTEM: WHAT MAKES SUCCESSFUL MOOCS? Ido Roll, Will Engle, Simon Bates! @hummus_monster @infology @simonpbates @CTLT_UBC! Centre for Teaching, Learning, and Technology! University of British Columbia 1
  2. 2. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 Please raise your hand if you have pre-teen child(ren) at home.! Please keep it up if you know where the following sentence is taken from:! “The snow glows white on the mountain tonight” 2
  3. 3. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 LEARNING FROM VIDEO ! IS POSSIBLE 3 If someone wants to learn, you cannot stop them.! @eric_mazur #STLHE2014
  4. 4. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WE KNOW A LOT ABOUT UNIVERSITY TEACHING What works?! Small classrooms! Interactivity! With material! With peers! With instructors! Feedback! … 4 MOOCs
  5. 5. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 5 Who are ! our learners? What defines ! success? What course design supports that?
  6. 6. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 CONTEXT Four UBC courses (Spring/Summer 2013)! Climate Literacy; Useful Genetics; Program Design; Game Theory! 330,000 learners registered. ! Five Edinburgh courses (Winter/Spring 2013)! Introduction to Philosophy; Artificial Intelligence; Astrobiology; Nutrition; Critical Thinking.! 309,000 learners registered.! https://www.era.lib.ed.ac.uk/handle/1842/6683! 6
  7. 7. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 VIDEO PRODUCTION 7 Averagelengthpervideo(min) 0 6 12 18 24 Average # of videos per week 0 4 8 12 16 UBC 1 UBC 2 UBC 3 UBC 4
  8. 8. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 8 Who are our learners? What defines ! success? What course design supports that?
  9. 9. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHO ARE OUR LEARNERS? General demographics consistent across courses and with Coursera. 9 Education level %responses 0 25 50 75 100 UBC 1 UBC 2 UBC 3 Coursera High School Diploma Professional School Some College Bachelor’s Degree Master’s Degree Doctorate Degree
  10. 10. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHO ARE OUR LEARNERS? General demographics consistent across courses and with Coursera. 10 Ethnicity %responses 0 25 50 75 100 UBC 1 UBC 2 UBC 3 Coursera White or Caucasian South Asian East Asian Other Asian Black or African American Decline to State
  11. 11. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Does location matter? Are MOOCs “local”? 11 Country of Origin %responses 0 15 30 45 60 UBC overall Coursera Edinburgh 11 3.6 3.9 United States India Canada United Kingdom Spain Brazil Germany
  12. 12. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Does location matter? Are MOOCs “local”? 12 Country of Origin %responses 0 15 30 45 60 UBC overall Coursera Edinburgh 43.9 5.2 United States India Canada United Kingdom Spain Brazil Germany
  13. 13. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Break down by course. weighting by overall registration, population of country:! How many more students are local, per 10,000 registrants, per capita?! In Canada, ! ~150 additional local students! ~500 Canadians overall 13 Extralocallearnersper10,000 registrants,1,000,000residents 0 2.5 5 7.5 10 UBC Edinburgh 2.0 1.0 55.7 3.3 5.5 4.8
  14. 14. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 14 Who are ! our learners? What defines success? What course design support that?
  15. 15. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHAT DEFINES SUCCESS? Definitions:! Visitors: log in at least once. Our denominator. 15 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  16. 16. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2. 16 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  17. 17. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. 17 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  18. 18. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. ! Active learners: problem sets, discussion forums. 18 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  19. 19. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. ! Active learners: problem sets, discussion forums. 19 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  20. 20. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Metrics:! Attractiveness: convertVisitors to Tasters.! Retention: convert Tasters to Learners! Interactivity: convert Learners to Active Learners. 20 Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners
  21. 21. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS 21 0 0.15 0.3 0.45 0.6 Attractiveness Retention Interactivity 48% 54% 35% 50% 31%29% 52% 58% 36% 35% 57% 24% Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners 17k 13k 49k 85k 2k 3k 4k 16k
  22. 22. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 22 Who are ! our learners? What defines ! success? What course design support that?
  23. 23. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHAT AFFECTS CONVERSION RATES? Correlate with:! Average length per video! Average length per week 23 Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners
  24. 24. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ATTRACTIVENESS Visitors -> Tasters: How many of those who visited the website watched Week 2 videos?! By length per week: r = -0.72*, r² = 0.52! 24 %ofVisitorswho arealsoTasters 0% 20% 40% 60% Overall legth of videos per week (min) 0 45 90 135 180 UBC Edinburgh
  25. 25. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ATTRACTIVENESS Visitors -> Tasters: How many of those who visited the website watched Week 2 videos?! By length per video: r = -0.33! 25 %ofVisitorswho arealsoTasters 0% 20% 40% 60% Average length per video 5 8 11 14 17 UBC Edinburgh
  26. 26. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 RETENTION Tasters -> Learners: How many of those who watched @ Week 2 also watched @ Week 5?! By length per week: r = -0.29! 26 %ofTasterswho arealsoLearners 0% 26.667% 53.333% 80% Overall legth of videos per week (min) 0 45 90 135 180 UBC Edinburgh
  27. 27. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 RETENTION Tasters -> Learners: How many of those who watched @ Week 2 also watched @ Week 5?! By length per video: r = 0.55, r² = 0.30 27 %ofTasterswho arealsoLearners 0% 20% 40% 60% 80% Average length per video 5 8 11 14 17 UBC Edinburgh
  28. 28. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 INTERACTIVITY How many of the students who watch videos also submit problem sets? 28 %ofvideowatchers whosubmitted problemsets 20% 30% 40% 50% 60% 70% Week 0 2 4 6 8 UBC 1 UBC 2 UBC 3 UBC 4
  29. 29. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 SUMMARY Local matters! Courses have unique “signatures”! Structure, e.g., # and length of videos! Engagement, e.g., % active learning.! Overall length of videos is negatively correlated with retention! But length of individual videos does not show such relationship. ! True for both universities 29
  30. 30. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 DESIGN RECOMMENDATIONS Long videos may be okay! Especially later in the course! Keep overall load reasonable.! Active learning is not affected by week-by-week variations! Two-stage courses: attractiveness and retention. 30

×