Bridging the Gap fromKnowledge to Action:Putting Analytics in theHands of AcademicAdvisors                                ...
Research Setting:M-STEM Academy• Undergraduate engineering mentoring program• Historically underrepresented students• 200 ...
Goals of the project• Utilize data stored in campus learning  management system to:  • Provide timely and targeted data on...
Supporting M-STEM mentors• Iteratively develop      • Metrics for comparing        students using LMS data   • Classificat...
How does mentor’s use of EWS affectstudent outcomes? USE Lab Digital Media Commons                5
Improved                             Performance                                            Action                        ...
Improved                             Performance                                            Action                        ...
Assignments               GradebookWeekly%query%of%LMS%for%courses%that%include%an%MASTEM%student%and%use%the%Gradebook%or...
Measures from LMS data• Gradebook and Assignments tools allow up-to-date  tracking of student performances• Report student...
Measures from LMS data• “Presence” events serve as a proxy for effort and are  events common to all courses   • Cumulative...
75%                        mean                        25%USE LabDigital Media Commons          11
Cumulative “Presence”                  events can be highly               predictive for students’ final   75%            ...
USE LabDigital Media Commons   12
Classification scheme• Absolute grade thresholds• Difference from course average• Presence cutoff    USE Lab    Digital Me...
Classification scheme• Absolute grade thresholds• Difference from course average• Presence cutoffComparisons• Grades (cour...
Classification scheme   Student %                    Relative Distance       Presence Percentile Rank       E3>=0.85      ...
Mentor summary  USE Lab  Digital Media Commons   14
Student Detail Report   USE Lab   Digital Media Commons   15
Benefits of EWS use• Contacting students• Shortening time to intervention• Viewing longitudinal trends   • By individual c...
How mentors use the EWS USE Lab Digital Media Commons   17
Next Steps• New infrastructure• New versions   • Instructor   • Students• Messaging system   • Recommendations (from perso...
Improved                                       Performance                                                      Action    ...
Conclusion• Closing the gap between problem identification and  intervention• Organizational capacity and the success of l...
CollaboratorsM-STEM                      ITS• Cinda-Sue Davis           • Bryan Hartman• Guy Meadows               • Jeff ...
QuestionsSteve                      slonn@umich.edu      @stevelonnStephanie                  steasley@umich.edu   @stepht...
Upcoming SlideShare
Loading in …5
×

Bridging the Gap from Knowledge to Action: Putting Analytics in the Hands of Academic Advisors

1,010 views

Published on

Short Paper Presentation at Learning Analytics and Knowledge Conference 2012, May 1. #LAK12

This paper presents current findings from an ongoing design- based research project aimed at developing an early warning system (EWS) for academic mentors in an undergraduate engineering mentoring program. This paper details our progress in mining Learning Management System data and translating these data into an EWS for academic mentors. We focus on the role of mentors and advisors, and elaborate on their importance in learning analytics-based interventions developed for higher education.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,010
On SlideShare
0
From Embeds
0
Number of Embeds
93
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Bridging the Gap from Knowledge to Action: Putting Analytics in the Hands of Academic Advisors

  1. 1. Bridging the Gap fromKnowledge to Action:Putting Analytics in theHands of AcademicAdvisors Steven Lonn Andrew Krumm R. Joseph Waddington Stephanie Teasley USE Lab University of Michigan Digital Media Commons www.umich.edu/~uselab
  2. 2. Research Setting:M-STEM Academy• Undergraduate engineering mentoring program• Historically underrepresented students• 200 Engineering students in 4 cohorts USE Lab Digital Media Commons 2
  3. 3. Goals of the project• Utilize data stored in campus learning management system to: • Provide timely and targeted data on student performance to M-STEM mentors • Shorten the timespan from problem identification to intervention USE Lab Digital Media Commons 3
  4. 4. Supporting M-STEM mentors• Iteratively develop • Metrics for comparing students using LMS data • Classification schemes • Visualizations of student performances• Send mentors weekly updates Photo%Credit:%h,p://teacherrogers.wordpress.com USE Lab Digital Media Commons 4
  5. 5. How does mentor’s use of EWS affectstudent outcomes? USE Lab Digital Media Commons 5
  6. 6. Improved Performance Action academic resources,Face-to-Face / Email Communication study strategiesMentor Audience StudentEWS ProductClassification Analysis Data 6
  7. 7. Improved Performance Action academic resources,Face-to-Face / Email Communication study strategiesMentor Audience StudentEWS ProductClassification Analysis Data 6
  8. 8. Assignments GradebookWeekly%query%of%LMS%for%courses%that%include%an%MASTEM%student%and%use%the%Gradebook%or%Assignments% “Presence”tool 8
  9. 9. Measures from LMS data• Gradebook and Assignments tools allow up-to-date tracking of student performances• Report student-level information for M-STEM students • Percent of available points earned • Course averages (all students) USE Lab Digital Media Commons 9
  10. 10. Measures from LMS data• “Presence” events serve as a proxy for effort and are events common to all courses • Cumulative and week-to-week “Presence” USE Lab Digital Media Commons 10
  11. 11. 75% mean 25%USE LabDigital Media Commons 11
  12. 12. Cumulative “Presence” events can be highly predictive for students’ final 75% course grade performance mean 25%USE LabDigital Media Commons 11
  13. 13. USE LabDigital Media Commons 12
  14. 14. Classification scheme• Absolute grade thresholds• Difference from course average• Presence cutoff USE Lab Digital Media Commons 12
  15. 15. Classification scheme• Absolute grade thresholds• Difference from course average• Presence cutoffComparisons• Grades (course average)• Percentile Ranks (presence) USE Lab Digital Media Commons 12
  16. 16. Classification scheme Student % Relative Distance Presence Percentile Rank E3>=0.85 . . Encourage0.75<=X<0.85 <-0.15 . Explore0.75<=X<0.85 >=-0.15 <0.25 Explore0.75<=X<0.85 >=-0.15 >=0.25 Encourage0.65<=X<0.75 <-0.15 <0.25 Engage0.65<=X<0.75 <-0.15 >=0.25 Explore0.65<=X<0.75 >=-0.15 . Explore0.55<=X<0.65 >=-0.10 . Explore0.55<=X<0.65 <-0.10 . Engage<0.55 . . Engage USE Lab Digital Media Commons 13
  17. 17. Mentor summary USE Lab Digital Media Commons 14
  18. 18. Student Detail Report USE Lab Digital Media Commons 15
  19. 19. Benefits of EWS use• Contacting students• Shortening time to intervention• Viewing longitudinal trends • By individual course • Across all courses• Contextualizing M-STEM student performance USE Lab Digital Media Commons 16
  20. 20. How mentors use the EWS USE Lab Digital Media Commons 17
  21. 21. Next Steps• New infrastructure• New versions • Instructor • Students• Messaging system • Recommendations (from person, from system) USE Lab Digital Media Commons 18
  22. 22. Improved Performance Action academic resources,Face-to-Face / Email Communication study strategiesMentor Audience StudentEWS ProductClassification Analysis USE Lab Digital Media Commons Data 19
  23. 23. Conclusion• Closing the gap between problem identification and intervention• Organizational capacity and the success of learning analytics • “Analytics” is but a small part• Information is always subject to interpretation • How can we scaffold interpretation and effective action-taking? USE Lab Digital Media Commons 20
  24. 24. CollaboratorsM-STEM ITS• Cinda-Sue Davis • Bryan Hartman• Guy Meadows • Jeff Jenkins• James Holloway • Dan Kiskis• Daryl Koch• Mark Jones USE Lab• Debbie Taylor • Gierad Laput USE Lab Digital Media Commons 21
  25. 25. QuestionsSteve slonn@umich.edu @stevelonnStephanie steasley@umich.edu @stephteasleywww.umich.edu/~uselabslides: www.slideshare.net/stevelonn USE Lab Digital Media Commons 22

×