Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Getting Students To Do the Work 
Using Learner Analytics Data to Inform Instructional Strategies 
New Directions in Online...
Study Questions vs. Assignments
Study Questions vs. Assignments 
WEBTEXT
Study Questions vs. Assignments 
WEBTEXT
Study Questions vs. Assignments 
WEBTEXT 
380 Study Questions
Study Questions vs. Assignments 
WEBTEXT 
• 33 Audio / Video Selections 
• 34 Articles and Studies 
• 8 Infographics, Webs...
Study Questions vs. Assignments 
WEBTEXT 
Study questions 
on each page 
Assignments 
in each chapter
Data Collected From Student Activities 
Log-In 
- first enrollment 
- each log-in time 
- most recent date 
Active Time 
T...
About Our Data 
In two studies, assignment scores (non-resettable, application-type 
questions) are significant predictors...
A Tale of Two Courses
Percentage 
Complete 
100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 
0% 
Webtext Completion* 
PSY Pilot Instructor 1 
...
! 
Just 20-30% 
of students 
do assigned 
reading. 
(Hobson, 2004)
100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 
0% 
Webtext Completion* 
*N=46 students; excludes 1 student with 0 time...
100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 
0% 
Webtext Completion* 
*N=46 students; excludes 1 student with 0 time...
Percentage 
Complete 
100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 
0% 
Webtext Completion* 
PSY Pilot Instructor 1 
...
What We Learned 
Students don’t do optional 
Assign credit for formative activities that will prepare students to do well ...
What We Learned 
Students don’t do optional 
Assign credit for formative activities that will prepare students to do well ...
Percentage 
Complete 
100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 
0% 
Webtext Completion* 
PSY Pilot Instructor 1 
...
Successful Student Behaviors
1. Successful students 
do optional activities 
like study questions.
Median Webtext Completion Percentages 
Assignments 
Study Questions 
Polls 
Your Turns 
By Assignment Scores 
Fall 2012 
0...
“[L]earning techniques such as taking practice tests and spreading 
study sessions out over time — known as distributed pr...
Credit buys time. 
! 
(and time is fixed.)
2. Successful students 
complete more work.
Assignments 
Study Questions 
Average Webtext Completion 
By Final Grade POL 
0 10 20 30 40 50 60 70 80 90 100 
A’s B’s C’...
Median Webtext Scores (Lines) 
100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 
As of October 28, 2013 PSY 
As 
Bs 
Cs 
Fs 
Ch ...
Median Webtext Scores (Lines) and Completion (Bars) 
100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 
As of October 28, 2013 PS...
3. Successful students 
invest more time.
Webtext Hours 
Median Active Time in Webtext - Hours 
By Webtext Scores 
As of October 28, 2013 
0 4.5 9 13.5 18 
A’s B’s ...
By Webtext Scores HIS 
4 
3.2 
2.4 
1.6 
0.8 
0 
100% 
80% 
60% 
40% 
20% 
0% 
Median Assignment TIme and Scores 
As of Oc...
500 
450 
400 
350 
300 
250 
200 
150 
100 
50 
0 
Median Time 
Median Correct 
0 500 1000 1500 2000 2500 3000 3500 4000 ...
4. Successful students 
start working before the 
first day of class.
With Median Hours Per Week PSY 
5 
3.75 
2.5 
1.25 
0 
100% 
80% 
60% 
40% 
20% 
0% 
Median Webtext Scores By Week 
As of ...
Successful Instructor Behaviors
Follow up. 
(Which is why instructors need actionable analytics.)
WHAT IF WE KNEW 
STUDENTS WERE 
DOING THE WORK?
Questions?
david.lindrum@soomolearning.com
Using Analytics to Increase Student Success
Using Analytics to Increase Student Success
Using Analytics to Increase Student Success
Using Analytics to Increase Student Success
Upcoming SlideShare
Loading in …5
×

Using Analytics to Increase Student Success

324 views

Published on

Presentation from the 2014 New Directions in Online Learning Conference in San Diego. The gist is that faculty can see which students need help early on, intervene with those students, and thereby increase the number of students who successfully complete the course. All data has been anonymized.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Using Analytics to Increase Student Success

  1. 1. Getting Students To Do the Work Using Learner Analytics Data to Inform Instructional Strategies New Directions in Online Learning San Diego, 2014 David Lindrum
  2. 2. Study Questions vs. Assignments
  3. 3. Study Questions vs. Assignments WEBTEXT
  4. 4. Study Questions vs. Assignments WEBTEXT
  5. 5. Study Questions vs. Assignments WEBTEXT 380 Study Questions
  6. 6. Study Questions vs. Assignments WEBTEXT • 33 Audio / Video Selections • 34 Articles and Studies • 8 Infographics, Websites, etc.
  7. 7. Study Questions vs. Assignments WEBTEXT Study questions on each page Assignments in each chapter
  8. 8. Data Collected From Student Activities Log-In - first enrollment - each log-in time - most recent date Active Time Time spent on each page typing, clicking, or scrolling. Participation - pages viewed - questions answered - number of resets Scoring - on multiple choice - first attempt - most recent attempt Higher Order - assignments - short answer - response boards
  9. 9. About Our Data In two studies, assignment scores (non-resettable, application-type questions) are significant predictors of midterm and final scores. Other webtext activities are also correlated with course success, including minutes per question and question completion. In multiple online courses (over 8,000 students), webtext success pass closely matches course pass rate.
  10. 10. A Tale of Two Courses
  11. 11. Percentage Complete 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Webtext Completion* PSY Pilot Instructor 1 *N=35 students; excludes 5 students with 0 time. PSY Summer 2012 median ✦ 21% webtext completion ✦ 102 questions answered ✦ 8.8 hours active time ✦ 6.2 minutes per correct question median
  12. 12. ! Just 20-30% of students do assigned reading. (Hobson, 2004)
  13. 13. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Webtext Completion* *N=46 students; excludes 1 student with 0 time. PSY Fall 2012 median ✦ 99% webtext completion ✦ 475 questions answered ✦ 16.4 hours active time ✦ 2.9 minutes per correct question PSY Pilot Instructor 2 median
  14. 14. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Webtext Completion* *N=46 students; excludes 1 student with 0 time. PSY Fall 2012 median ✦ 99% webtext completion ✦ 475 questions answered ✦ 16.4 hours active time ✦ 2.9 minutes per correct question PSY Pilot Instructor 2 median 30% Course Credit for Webtext Assignments and Study Questions
  15. 15. Percentage Complete 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Webtext Completion* PSY Pilot Instructor 1 *N=35 students; excludes 5 students with 0 time. PSY Summer 2012 median ✦ 21% webtext completion ✦ 102 questions answered ✦ 8.8 hours active time ✦ 6.2 minutes per correct question median 0% Course Credit for Webtext Assignments and Study Questions
  16. 16. What We Learned Students don’t do optional Assign credit for formative activities that will prepare students to do well on final. Credit should generally be between 20-30% of course credit.
  17. 17. What We Learned Students don’t do optional Assign credit for formative activities that will prepare students to do well on final. Credit should generally be between 20-30% of course credit. Students respond to personal outreach (nagging, parenting) Instructor 2 emailed students at the end of every week Go get your learning resource. You can’t pass this class without it. I see that you’re falling behind. If you don’t catch up, you won’t pass this class.
  18. 18. Percentage Complete 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Webtext Completion* PSY Pilot Instructor 1 What exactly are successful-student behaviors? *N=35 students; excludes 5 students with 0 time. PSY Summer 2012 median ✦ 21% webtext completion ✦ 102 questions answered ✦ 8.8 hours active time ✦ 6.2 minutes per correct question median
  19. 19. Successful Student Behaviors
  20. 20. 1. Successful students do optional activities like study questions.
  21. 21. Median Webtext Completion Percentages Assignments Study Questions Polls Your Turns By Assignment Scores Fall 2012 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% A’s B’s C’s D’s F’s POL No credit n=48 at R1, f2f
  22. 22. “[L]earning techniques such as taking practice tests and spreading study sessions out over time — known as distributed practice — were found to be of high utility because they benefited students of many different ages and ability levels and enhanced performance in many different areas.” Improving Students’ Learning with Effective Learning Techniques (Dunlosky, et al. 2013)
  23. 23. Credit buys time. ! (and time is fixed.)
  24. 24. 2. Successful students complete more work.
  25. 25. Assignments Study Questions Average Webtext Completion By Final Grade POL 0 10 20 30 40 50 60 70 80 90 100 A’s B’s C’s D’s F’s n=57 at R1, f2f
  26. 26. Median Webtext Scores (Lines) 100 90 80 70 60 50 40 30 20 10 0 As of October 28, 2013 PSY As Bs Cs Fs Ch 1 Ch 2 Ch 3 Ch 4 Ch 5 Ch 6 Bars represent completion. ! Lines represent percentage of points earned per module. ! Ds n=539
  27. 27. Median Webtext Scores (Lines) and Completion (Bars) 100 90 80 70 60 50 40 30 20 10 0 As of October 28, 2013 PSY Ch 1 Ch 2 Ch 3 Ch 4 Ch 5 Ch 6 As Bs Cs Fs Bars represent completion. ! Lines represent percentage of points earned per module. ! Ds n=539
  28. 28. 3. Successful students invest more time.
  29. 29. Webtext Hours Median Active Time in Webtext - Hours By Webtext Scores As of October 28, 2013 0 4.5 9 13.5 18 A’s B’s C’s D’s F’s ORG n=1,040 at online
  30. 30. By Webtext Scores HIS 4 3.2 2.4 1.6 0.8 0 100% 80% 60% 40% 20% 0% Median Assignment TIme and Scores As of October 28, 2013 As Bs Cs Ds Fs Assignment Score Assignment Completion Hours n=560 at online
  31. 31. 500 450 400 350 300 250 200 150 100 50 0 Median Time Median Correct 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Active Time in Minutes Correct Responses Active Time and Correct Responses All Students, All Questions (large, 2-year)
  32. 32. 4. Successful students start working before the first day of class.
  33. 33. With Median Hours Per Week PSY 5 3.75 2.5 1.25 0 100% 80% 60% 40% 20% 0% Median Webtext Scores By Week As of October 28, 2013 Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Pre-Term Score Pre-Term Hours Week 1 Score Week 1 Hours Week 2+ Score Week 2+ Hours n=539 at online
  34. 34. Successful Instructor Behaviors
  35. 35. Follow up. (Which is why instructors need actionable analytics.)
  36. 36. WHAT IF WE KNEW STUDENTS WERE DOING THE WORK?
  37. 37. Questions?
  38. 38. david.lindrum@soomolearning.com

×