Your SlideShare is downloading. ×
Learner Analytics Presentation to ATSC Committee
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Learner Analytics Presentation to ATSC Committee

236
views

Published on

Presentation by John Whitmer and Kathy Fernandes to the California State University Academic Technology Steering Committee on December 12, 2012.

Presentation by John Whitmer and Kathy Fernandes to the California State University Academic Technology Steering Committee on December 12, 2012.

Published in: Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
236
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Kathy
  • John
  • John
  • John
  • Kathy
  • Kathy
  • Kathy
  • John
  • John
  • John
  • John
  • Kathy
  • Kathy
  • Kathy
  • Transcript

    • 1. System-wide LMS Learner Analytics ProjectsPresenters: Kathy Fernandes and John Whitmer ATSC Virtual Meeting December 13, 2012 Slides @ http://goo.gl/DYqJU
    • 2. Agenda1. Chico State Learner Analytics Research Study • EDUCAUSE Article (http://goo.gl/tESoi)2. Current Projects • Moodle • Blackboard 2
    • 3. 1. CHICO STATE LEARNER ANALYTICSRESEARCH STUDY“Logging on to Improve Achievement” by John WhitmerEdD. Dissertation (UC Davis & Sonoma State) 3
    • 4. Case Study: Intro to Religious Studies• Redesigned to hybrid delivery through Academy eLearning 54 F’s• Enrollment: 373 students (54% increase on largest section)• Highest LMS (Vista) usage entire campus Fall 2010 (>250k hits)• Bimodal outcomes: • 10% increased SLO mastery • 7% & 11% increase in DWF• Why? Can’t tell with aggregated reporting data 4
    • 5. Driving Conceptual Questions1. How is student LMS use related to academic achievement in a single course section?2. How does that finding compare to the relationship of achievement with traditional student characteristic variables?3. How are these relationships different for “at-risk” students (URM & Pell-eligible)?4. What data sources, variables and methods are most useful to answer these questions? 5
    • 6. VariablesStudent Characteristic Independent VariablesGenderUnder Represented Minority (URM)Pell-EligibleHigh School GPAFirst in Family to Attend CollegeStudent Major (Discipline)Enrollment StatusInteraction URM & GenderInteraction URM & Pell-EligibilityLearning Management System Usage VariablesTotal LMS course website hitsTotal LMS course dwell timeAdministrative tool website hitsAssessment tool website hitsContent tool website hitsEngagement tool website hitsDependent Variable: Final Course Grade 6
    • 7. Clear Trend: Grade w/Mean LMS Hits 7
    • 8. Separate Variables: Correlation LMS Use & Student Characteristic with Final Grade LMS Student > Use Characteristic Variables Variables 18% Average 4% Average (r = 0.35–0.48) (r = -0.11–0.31)Explanation of change Explanation of change in final grade in final grade 8
    • 9. Combined Variables: Regression Final Grade by LMS Use & Student Characteristic Variables LMS Student > Use Characteristic Variables Variables 25% +10% (r2=0.25) (r2=0.35)Explanation of change Explanation of change in final grade in final grade 9
    • 10. At-Risk Students: “Over-Working Gap” 10
    • 11. Filtering Data – Lots of “Noise”; Low “Signal” 500 450 400 350 300 382 250 200 150 Raw Average 151 Hits/Student 100 49 Filtered Average 58 50 26 Hits/Student 54 51 23 36 16 0 Discussion Activity Content Activity Hits Assessment Activity Mail Activity Hits Administrative Hits Hits Activity Hits Final data set: 72,000 records (-73%) 11
    • 12. 2. CURRENT PROJECTS 12
    • 13. Moodle and Bb Learner AnalyticsWhat do these have in common?• Multi-campus CSU groups discussing common analytics questions & query definitions 13
    • 14. Moodle vs. Bb Learner AnalyticsMoodle CIG (18 months old) Blackboard Learn GroupChair: Andrew Roderick, SFSU (just starting) CIG Chair: Terry Smith, CSUEB DIY, adopt and evaluate  Bb Learn Analytics product available “off the shelf”; solutions from other defined and integrated with Moodlers Peoplesoft Starting with technical  Pre-built Reports and reporting to build accurate Dashboards to ANYONE on indicators of use campus (admin. or faculty if 2 rounds of data collection authenticated) already completed and  Charts available inside LMS discussed for Faculty and Student Views 14
    • 15. Moodle Reporting & Analytics, Round 1 Prioritized Moodle Queries from S&PG governance group Focused on measures of adoption (% faculty, % students, % course sections) For expediency, campuses reported using current queries used for reporting 15
    • 16. “How many sections are using the LMS (out of all sections offered that term)?”CSU_09 671 2,191CSU_08 1,098 1,162CSU_06 2,997 7,064 Active Sections Inactive SectionsCSU_05 2,492 3,687CSU_04 553 614CSU_02 2,270 3,911 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 16
    • 17. “How many sections are using the LMS (out of all sections offered that term)?”CSU_09 671 2,191CSU_08 1,098 1,162CSU_06 2,997 Use = “visible”+”student activity” 7,064 Active Sections Inactive SectionsCSU_05 2,492 3,687CSU_04 553 Use = “visible” 614CSU_02 2,270 3,911 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 17
    • 18. Round 2: mCURL(Moodle Common Usage and Learning Analytics) 8 active CSU & 2 UC campuses – Co-chaired: John Whitmer, CO ATS and Mike Haskell, Cal Poly SLO Starting with same measures of adoption, prioritizing “wish list” of more advanced analytics Local database conventions and campus practices make accurate comps. challenging 18
    • 19. Faculty LMS AdoptionHow many faculty are using the LMS in one or more course sections? 19
    • 20. mCURL Next Steps Refine queries for accurate comparative course and student adoption measures Select additional queries: depth and breadth of use – # tools used – # students in each section – frequency of use Create repositories for campuses to share unique local queries 20
    • 21. Blackboard Analytics for Learn (A4L) CSU ATS Co-Lab Agreement – working together – Functionality: from early alerts/course reporting to institutional-level analytics – Up to 4 campuses participating (3 confirmed) – Period: December 2012-December 2013 – Individual campus Scope of Work for setup of infrastructure and services Kick-off meeting next week 21
    • 22. Co-Lab Goals1. Develop methodologies and processes to identify, aggregate, and transform LMS usage data into information for analytics.2. Improve campus usage of learning analytics for decision- making for student success, curriculum improvement, and technical services.3. Create shared measures, database reports, and algorithms, drawing on campus best practices and research innovations.4. Increase campus awareness of applications and technical tools.5. Document campus efforts and disseminate to other campuses.6. Provide professional development in learning analytics. 22
    • 23. Student at a Glance 23
    • 24. Instructor at a Glance 24
    • 25. Dean Dashboard 25
    • 26. Feedback? Questions?Kathy Fernandes(kfernandes@csuchico.edu)John Whitmer(jwhitmer@calstate.edu) 26