Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Learning at Scale
Using Research To Improve Learning Practices
and Technology for Teaching Math
Maria H. Andersen, Ph.D.
C...
What have we
learned?
10 Lessons
from
Learning at
Scale
Stand up if you know what a
MOOC is.
Stay standing if you have ever
signed up for a MOOC.
Stay standing if you have ever
f...
HarvardX and MITx: Four Years of Open
Online Courses
Authors: I. Chuang, A. Ho
Published: 2016
Study size: 4.4 million stu...
If a MOOC held 100 students …
(Chuang & Ho, 2016)
(Chuang & Ho, 2016)
How many times a year do
you get a chance to start
over?
1. We need to improve
findability.
Learners are in the maze, and
they can’t see the big picture.
Many of you have
been in a maze like
what our students
experience.
Wouldn’t it be nice to
have a transporter to
get right to where
you want to go?
Why don’t students
seem to be able to
find information that
is clearly printed in
the syllabus?
The impact of findability on student motivation, self-
efficacy, and perceptions of online course quality
Authors: B. Simu...
What else can we do to help
learners navigate this maze?
Help students to build a physical
map to anchor memories as they
learn.
7.00x Introduction to Biology: The Secret of
Life MITx on EdX Course Report - 2013 Spring
Authors: D. Seaton, J. Reich, S....
6.00x Introduction to Computer Science and
Programming MITx on edX – 2012 Fall
Authors: S. Features, P. Grimson, J. Guttag...
14.73x The Challenges of Global Poverty
MITx on edX – 2013 Spring
Authors: P. Black, P. White
Published: 2014
Study size: ...
(Seaton et al, 2014)
Introduction to Biology
(Features et al, 2014)
Introduction to Computer Science and Programming
(Black & White, 2014)
The Challenges of Global Poverty
Examining Engagement: Analysing Learner
Subpopulations in MOOCs
Authors: R. Ferguson, D. Clow
Published: 2015
Study size: ...
(Ferguson & Clow, 2015)
HarvardX and MITx: Four Years of Open
Online Courses
Authors: I. Chuang, A. Ho
Published: 2016
Study size: 4.4 million stu...
Find
Calculus.
(Chuang & Ho, 2016)
2. Students need to be coerced
into more frequent engagement.
Are you ever surprised by things
you can find on the Internet?
Stand up if you know who the
Borg are.
Stay standing if you...
3. Use an
online math
learning
system.
Traditional
Course
Online homework
(quantity and
difficulty adapts
to mastery levels
of student)
Online homework
(all stud...
Supports each
students’ pace
Interaction around content
Frequent accountability
Immediate feedback
Learning is not a spectator sport: doing is better
than watching for learning from a MOOC
Authors: K. Koedinger, J. Kim, J...
Our digital lives are with us everywhere,
our learning should be too.
Stand up if you teach online or
rely heavily on digital learning
materials.
Stay standing if you think it’s easy to
get st...
4. Get students to
participate in their learning
community.
1-9-90 Rule
1% creators
9% commenters/sharers
90% lurkers
Comments in MOOCs: who is doing the
talking and does it help?
Authors: B. Swinnerton, S. Hotchkiss & N.P. Morris
Published...
Predicting Student Retention in Massive Open
Online Courses using Hidden Markov Models
Authors: G. Balakrishnan
Published:...
(Balakrishnan, 2013)
Commenting in discussion boards
(Balakrishnan, 2013)
Commenting in discussion boards
(Balakrishnan, 2013)
Lurking in discussion boards
Use
announcements
to showcase great
discussion posts.
5. Use personas to
strategize about
interventions.
Using Data Mining to Differentiate Instruction
in College Algebra
Authors: R. Manspeaker
Published: 2011
Study size: 524 s...
(Manspeaker, 2011)
Overachievers (OA)
Characteristics What do they struggle with?
• see mathematics as useful
• do not see...
Underachievers (UA)
Characteristics What do they struggle with?
• Intelligent and well-
prepared
• Bored and frustrated wi...
“Employees” (E)
Characteristics What do they struggle with?
• Feel the class is a menial job
for which they get “paid”
wit...
(Manspeaker, 2011)
Rote Memorizers (RM)
Characteristics What do they struggle with?
• Rely on memorizing to get
through ma...
Sisyphean Strivers (SS)
Characteristics What do they struggle with?
• Excellent Attendance
• Do well on written
homework
•...
Stand up if students drop your
classes.
Stay standing if the digital systems
you use make it easy to see which
students yo...
6. Anticipate and catch
students as they fall off the
path to success.
Community Insights: Emerging Benchmarks &
Student Success Trends from across the Civitas
Authors: Civitas Learning
Publish...
LMS activity in the first 14 days
LMS activity in the first 14 days
Persistence: 92%
(or more)
Persistence: 76%
LMS activity in the first 14 days
Persistence: 47%
LMS activity in the first 14 days
Using Bayesian Learning to Classify College
Algebra Students by Understanding in Real-Time
Authors: A. Cousino
Published: ...
(Cousino, 2013)
Examining Engagement: Analysing Learner
Subpopulations in MOOCs
Authors: R. Ferguson, D. Clow
Published: 2015
Study size: ...
(Ferguson & Clow, 2015)
Samplers Visit, but only briefly (commented in some courses)
Strong Starters Complete the first week, but then dropout
Ret...
(Graph generated from data in Ferguson & Clow, 2015)
Deconstructing Disengagement: Analyzing Learner
Subpopulations in Massive Open Online Courses
Authors: R. Kizilcec, C. Pie...
7. Courses need a
catch-up option.
Why don’t we
offer 1-2 week
”slipped
schedule”
courses?
This might
mean the
courses have
to be offered
on 12-week
schedules.
8. Tweak instructional videos.
How Video Production Affects Student
Engagement: An Empirical Study of MOOC Videos
Authors: P. Guo, J. Kim, R. Rubin
Publi...
46% 33%
(Guo et al, 2014)
(Guo et al, 2014)
(Guo et al, 2014)
Planned vs. Chopped
9. Don’t aim for perfect.
Let students get stuck a little
more.
Your click decides your fate : Inferring Information
Processing and Attrition Behavior from MOOC
Video Clickstream Interac...
10 GB of JSON video click data
Play (Pl)
Pause (Pa)
SeekFw (Sf)
SeekBw (Sb)
ScrollFw (SSf)
ScrollBw (SSb)
RatechangeFast (...
(Sinha et al., 2014)
Information Processing Index (IPI)
(Sinha et al., 2014)
If students’ rewatching behavior
changes from low to high IPI, they
are 33% less likely to drop out.
Why do we lose students before
graduation?
Community Insights: Emerging Benchmarks &
Student Success Trends from across the Civitas
Authors: Civitas Learning
Publish...
(Civitas, 2016)
Academic Performance is not the Primary Risk to Departure
10. Make sure your courses
are have opportunities for
curiosity, challenge, and
creativity.
Let’s see what you remember…
1. Improve findability.
2. Coerce students into
more frequent
engagement.
3. Use an
online math
system.
4. Get students to participate
in their learning community.
5. Use personas
to strategize
about
interventions.
6. Anticipate
and catch
students as
they fall off the
path to
success.
7. Courses need a
catch-up option.
8. Tweak
Instructional
Videos.
9. Let students get stuck a little
more.
10. Make sure your courses
are have opportunities for
curiosity, challenge, and
creativity.
Cool. You
remember
everything. My
work here is done.
busynessgirl@gmail.c
om
busynessgirl.com
@busynessgirl
http://tinyletter.com/teachingchallenge
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math
Upcoming SlideShare
Loading in …5
×

Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math

2,385 views

Published on

In the last 5 years, there has been a rise in what we might call "large-scale digital learning experiments." These take the form of centralized courses, vendor-created courseware, online homework systems, MOOCs, and free-range learning platforms. If we mine the research, successes, and failures coming out of these experiments, what can we discover about designing better digital learning experiences and technology for the learning of mathematics?

Published in: Education
  • Be the first to comment

Learning at Scale: Using Research To Improve Learning Practices and Technology for Teaching Math

  1. 1. Learning at Scale Using Research To Improve Learning Practices and Technology for Teaching Math Maria H. Andersen, Ph.D. Chief Troublemaker, Edge of Learning Math Faculty, Westminster College
  2. 2. What have we learned?
  3. 3. 10 Lessons from Learning at Scale
  4. 4. Stand up if you know what a MOOC is. Stay standing if you have ever signed up for a MOOC. Stay standing if you have ever finished a MOOC.
  5. 5. HarvardX and MITx: Four Years of Open Online Courses Authors: I. Chuang, A. Ho Published: 2016 Study size: 4.4 million students across 290 MOOCs
  6. 6. If a MOOC held 100 students … (Chuang & Ho, 2016)
  7. 7. (Chuang & Ho, 2016)
  8. 8. How many times a year do you get a chance to start over?
  9. 9. 1. We need to improve findability.
  10. 10. Learners are in the maze, and they can’t see the big picture.
  11. 11. Many of you have been in a maze like what our students experience.
  12. 12. Wouldn’t it be nice to have a transporter to get right to where you want to go?
  13. 13. Why don’t students seem to be able to find information that is clearly printed in the syllabus?
  14. 14. The impact of findability on student motivation, self- efficacy, and perceptions of online course quality Authors: B. Simunich, D. Robins & V. Kelly Published: 2015 Study size: n=81, students had to complete 7 tasks, navigating either a well-constructed course (according to QM) or a “broken” course
  15. 15. What else can we do to help learners navigate this maze?
  16. 16. Help students to build a physical map to anchor memories as they learn.
  17. 17. 7.00x Introduction to Biology: The Secret of Life MITx on EdX Course Report - 2013 Spring Authors: D. Seaton, J. Reich, S. Nesterko, et al. Published: 2014 Study size: 38k signups, 3K certified (Seaton et al, 2014)
  18. 18. 6.00x Introduction to Computer Science and Programming MITx on edX – 2012 Fall Authors: S. Features, P. Grimson, J. Guttag Published: 2014 Study size: 84.5k signups, 5.7K certified
  19. 19. 14.73x The Challenges of Global Poverty MITx on edX – 2013 Spring Authors: P. Black, P. White Published: 2014 Study size: 40k signups, 4.6K certified
  20. 20. (Seaton et al, 2014) Introduction to Biology
  21. 21. (Features et al, 2014) Introduction to Computer Science and Programming
  22. 22. (Black & White, 2014) The Challenges of Global Poverty
  23. 23. Examining Engagement: Analysing Learner Subpopulations in MOOCs Authors: R. Ferguson, D. Clow Published: 2015 Study size: 34k signups, 7k full participants, 4 MOOCs
  24. 24. (Ferguson & Clow, 2015)
  25. 25. HarvardX and MITx: Four Years of Open Online Courses Authors: I. Chuang, A. Ho Published: 2016 Study size: 4.4 million students across 290 MOOCs
  26. 26. Find Calculus. (Chuang & Ho, 2016)
  27. 27. 2. Students need to be coerced into more frequent engagement.
  28. 28. Are you ever surprised by things you can find on the Internet? Stand up if you know who the Borg are. Stay standing if you’ve ever done an internet search for a cat in a Borg costume.
  29. 29. 3. Use an online math learning system.
  30. 30. Traditional Course Online homework (quantity and difficulty adapts to mastery levels of student) Online homework (all students do same problems) < < A hierarchy of online math learning based on principles from learning research Online homework (adapts to mastery levels of student and learner profile or preferences) <
  31. 31. Supports each students’ pace
  32. 32. Interaction around content
  33. 33. Frequent accountability
  34. 34. Immediate feedback
  35. 35. Learning is not a spectator sport: doing is better than watching for learning from a MOOC Authors: K. Koedinger, J. Kim, J. Jia, et al Published: 2015 Study size: 27k registrants, 1k completers, “Introduction to Psychology as a Science” delivered through Coursera and OLI platform
  36. 36. Our digital lives are with us everywhere, our learning should be too.
  37. 37. Stand up if you teach online or rely heavily on digital learning materials. Stay standing if you think it’s easy to get students to participate in these. Stay standing if you use discussion boards with your students.
  38. 38. 4. Get students to participate in their learning community.
  39. 39. 1-9-90 Rule 1% creators 9% commenters/sharers 90% lurkers
  40. 40. Comments in MOOCs: who is doing the talking and does it help? Authors: B. Swinnerton, S. Hotchkiss & N.P. Morris Published: 2017 Study size: 25k active learners, 8k commenters, 5k students who completed a pre-course survey (roughly half commenters and half non-commenters)
  41. 41. Predicting Student Retention in Massive Open Online Courses using Hidden Markov Models Authors: G. Balakrishnan Published: 2013 Study size: 30k enrolled students in “Software as a Service” MOOC
  42. 42. (Balakrishnan, 2013) Commenting in discussion boards
  43. 43. (Balakrishnan, 2013) Commenting in discussion boards
  44. 44. (Balakrishnan, 2013) Lurking in discussion boards
  45. 45. Use announcements to showcase great discussion posts.
  46. 46. 5. Use personas to strategize about interventions.
  47. 47. Using Data Mining to Differentiate Instruction in College Algebra Authors: R. Manspeaker Published: 2011 Study size: 524 students in College Algebra using digital homework, over 2 semesters
  48. 48. (Manspeaker, 2011) Overachievers (OA) Characteristics What do they struggle with? • see mathematics as useful • do not see how the class specifically applies to their lives • high grades in math • non-standard problems • problems not found on homework or worked in class
  49. 49. Underachievers (UA) Characteristics What do they struggle with? • Intelligent and well- prepared • Bored and frustrated with the course • Do reasonably well on the first exam • Procedural and non-standard problems • Get low scores in most aspects of the course • High risk for dropping out (Manspeaker, 2011)
  50. 50. “Employees” (E) Characteristics What do they struggle with? • Feel the class is a menial job for which they get “paid” with a passing grade • Dislike math • Learn through memorizing • Excel at procedural problems • Problems involving creative thinking or independent thought • Attendance (starts strong, then drops) (Manspeaker, 2011)
  51. 51. (Manspeaker, 2011) Rote Memorizers (RM) Characteristics What do they struggle with? • Rely on memorizing to get through math • Negative views of math which will get worse over time • Poorest scores • Emporium-style courses • Everything
  52. 52. Sisyphean Strivers (SS) Characteristics What do they struggle with? • Excellent Attendance • Do well on written homework • Have high understanding of math if interviewed • Do very poorly on exams • Non-standard questions • Application problems • Exams • Online homework (Manspeaker, 2011)
  53. 53. Stand up if students drop your classes. Stay standing if the digital systems you use make it easy to see which students you should intervene with. Stay standing if you have strategies in place to try to intervene before disengagement.
  54. 54. 6. Anticipate and catch students as they fall off the path to success.
  55. 55. Community Insights: Emerging Benchmarks & Student Success Trends from across the Civitas Authors: Civitas Learning Published: 2016 Study size: 2 million learners, 55 colleges
  56. 56. LMS activity in the first 14 days
  57. 57. LMS activity in the first 14 days Persistence: 92% (or more)
  58. 58. Persistence: 76% LMS activity in the first 14 days
  59. 59. Persistence: 47% LMS activity in the first 14 days
  60. 60. Using Bayesian Learning to Classify College Algebra Students by Understanding in Real-Time Authors: A. Cousino Published: 2013 Study size: 505 students over 2 semesters
  61. 61. (Cousino, 2013)
  62. 62. Examining Engagement: Analysing Learner Subpopulations in MOOCs Authors: R. Ferguson, D. Clow Published: 2015 Study size: 34k signups, 7k full participants, 4 MOOCs
  63. 63. (Ferguson & Clow, 2015)
  64. 64. Samplers Visit, but only briefly (commented in some courses) Strong Starters Complete the first week, but then dropout Returners Made it through 2 weeks, but then dropout Midway Dropouts Dropped out around assessment 3 or 4 Nearly There’s Made it through ¾ of the course, then dropped Late Completers Completed course, but submitted things late Keen Completers Completed course, mostly on time (>80%) (Ferguson & Clow, 2015)
  65. 65. (Graph generated from data in Ferguson & Clow, 2015)
  66. 66. Deconstructing Disengagement: Analyzing Learner Subpopulations in Massive Open Online Courses Authors: R. Kizilcec, C. Piech, E. Schneider Published: 2013 Study size: 94k active learners in 3 computer science MOOCs
  67. 67. 7. Courses need a catch-up option.
  68. 68. Why don’t we offer 1-2 week ”slipped schedule” courses?
  69. 69. This might mean the courses have to be offered on 12-week schedules.
  70. 70. 8. Tweak instructional videos.
  71. 71. How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos Authors: P. Guo, J. Kim, R. Rubin Published: 2014 Study size: 6.9 million video watching sessions across four courses on the edX MOOC platform
  72. 72. 46% 33% (Guo et al, 2014)
  73. 73. (Guo et al, 2014)
  74. 74. (Guo et al, 2014)
  75. 75. Planned vs. Chopped
  76. 76. 9. Don’t aim for perfect. Let students get stuck a little more.
  77. 77. Your click decides your fate : Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions Authors: T. Sinha, P. Jermann, N. Li et al. Published: 2014 Study size: 66k enrolled, 36.5k interacted with videos, there were 48 video lectures (producing 10 Gb of JSON data) from “Functional Programming in Scala” run on Coursera
  78. 78. 10 GB of JSON video click data Play (Pl) Pause (Pa) SeekFw (Sf) SeekBw (Sb) ScrollFw (SSf) ScrollBw (SSb) RatechangeFast (Rf) RatechangeSlow (Rs) PlPaSbPl RfPaPlPa (rewatch) (fast watching) SSbSbPaPl (clear concept) (Sinha et al., 2014)
  79. 79. (Sinha et al., 2014) Information Processing Index (IPI)
  80. 80. (Sinha et al., 2014) If students’ rewatching behavior changes from low to high IPI, they are 33% less likely to drop out.
  81. 81. Why do we lose students before graduation?
  82. 82. Community Insights: Emerging Benchmarks & Student Success Trends from across the Civitas Authors: Civitas Learning Published: 2016 Study size: 4 million learners (this part was conducted looking at 2 years of data), 55 colleges
  83. 83. (Civitas, 2016) Academic Performance is not the Primary Risk to Departure
  84. 84. 10. Make sure your courses are have opportunities for curiosity, challenge, and creativity.
  85. 85. Let’s see what you remember…
  86. 86. 1. Improve findability.
  87. 87. 2. Coerce students into more frequent engagement.
  88. 88. 3. Use an online math system.
  89. 89. 4. Get students to participate in their learning community.
  90. 90. 5. Use personas to strategize about interventions.
  91. 91. 6. Anticipate and catch students as they fall off the path to success.
  92. 92. 7. Courses need a catch-up option.
  93. 93. 8. Tweak Instructional Videos.
  94. 94. 9. Let students get stuck a little more.
  95. 95. 10. Make sure your courses are have opportunities for curiosity, challenge, and creativity.
  96. 96. Cool. You remember everything. My work here is done.
  97. 97. busynessgirl@gmail.c om busynessgirl.com @busynessgirl http://tinyletter.com/teachingchallenge

×