Your SlideShare is downloading.
×

×
# Saving this for later?

### Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

#### Text the download link to your phone

Standard text messaging rates apply

Like this presentation? Why not share!

- CSU System-wide Learning Analytics ... by John Whitmer, Ed.D. 1626 views
- Logging on to Improve Achievement by John Whitmer, Ed.D. 220 views
- Learner Analytics Presentation to A... by John Whitmer, Ed.D. 233 views
- CSU System-wide Learning Analytics ... by John Whitmer, Ed.D. 1626 views
- Learner Analytics Panel Session: D... by John Whitmer, Ed.D. 167 views
- this document.doc by Saina33 1631 views
- 2008 â€" 2009 Course Guide.doc.doc.doc by Saina33 1553 views
- Whitmer, Fernandes, Kodai CSU Chico... by DET/CHE Directors... 350 views
- Learner Analytics: from Buzz to St... by John Whitmer, Ed.D. 647 views
- Learning Analytics: Realizing the ... by John Whitmer, Ed.D. 538 views
- UNDERGRADUATE and GRADUATE by Terry34 3727 views
- Stevens Henager College Online Cata... by ScottC4 3663 views

Like this? Share it with your network
Share

468

views

views

Published on

Case study presentation for the Learning and Knowledge Analytics 2013 MOOC.

Case study presentation for the Learning and Knowledge Analytics 2013 MOOC.

Published in:
Education

No Downloads

Total Views

468

On Slideshare

0

From Embeds

0

Number of Embeds

1

Shares

0

Downloads

10

Comments

0

Likes

2

No embeds

No notes for slide

- 1. Using Learner Analytics toUnderstand Student Achievement in a Large Enrollment Hybrid Course slides posted: John Whitmer, Ed.D. Associate Director, Academic Technology Services California State University, Office of the Chancellor Society for Learning Analytics Research | LAK 2013 Case Study February 19, 2013
- 2. Outline1. Context2. Methods & Tools3. Findings4. Conclusions & Next Steps
- 3. 1. CONTEXT
- 4. Founded in 1887 15,257 FTES, 95% from California, serves 12 counties Primarily residential, undergraduate teaching college Campus in California State University system (23 colleges, 44,000 faculty and staff, 437,000 students)
- 5. CSU Budget Proposed Increase! Source: CSU Chancellor’s Office http://bit.ly/X7LYeK
- 6. Case Study: Intro to Religious Studies• Undergraduate, introductory, high demand• Redesigned to hybrid delivery format 54 F’s through “academy eLearning program”• Enrollment: 373 students (54% increase on largest section)• Highest LMS (Vista) usage entire campus Fall 2010 (>250k hits)• Bimodal outcomes: • 10% increase on final exam • 7% & 11% increase in DWF• Why? Can’t tell with aggregated data
- 7. Driving Conceptual Questions1. How is student LMS use related to academic achievement in a single course section?2. How does that finding compare to the relationship of achievement with traditional student characteristic variables?3. How are these relationships different for “at-risk” students (URM & Pell-eligible)?4. What data sources, variables and methods are most useful to answer these questions?
- 8. UniversityGender Freq. Percent Average Difference Female 231 62% 51% 11% Male 142 38% 48% -10%Age 0% 17 22 6% 18-21 302 81% 22-30 22 6% 31+ 1 0%Under-representedMinority No 264 71% 73% -2% Yes 109 29% 27% 2%Pell-eligible Freq. Percent No 210 56% Yes 163 44%First Attend College Freq. No 268 72% Yes 105 28%Enrollment Status Freq. Continuing Student 217 58% Transfer 17 5% First-Time Student 139 37%
- 9. 2. METHODS & TOOLS
- 10. Methods at a Glance Data Sources: 1) LMS logfiles, 2) SIS data, 3) Course data Process 1. Clean/filter/transform/reduce data (70% effort) 2. Descriptive / exploratory analysis (20% effort) 3. Statistical analysis (10% effort) Factor analysis Correlation single variables Regression multiple variables; partial & complete
- 11. Tools UsedApp FunctionExcel Early data exploration; simple sorting; tables for print/publicationTableau Complex data summaries and explorations; complex charts; presentation charts Final/formal descriptive data; statistical analysis; some charts (scatterplots) Statistical analysis (factor analysis)
- 12. VariablesStudent Characteristic Independent VariablesGenderUnder Represented Minority (URM)Pell-EligibleHigh School GPAFirst in Family to Attend CollegeStudent Major (Discipline)Enrollment StatusInteraction URM & GenderInteraction URM & Pell-EligibilityLearning Management System Usage VariablesTotal LMS course website hitsTotal LMS course dwell timeAdministrative tool website hitsAssessment tool website hitsContent tool website hitsEngagement tool website hitsDependent Variable: Final Course Grade
- 13. Missing Data On Critical Indicators
- 14. Final data set: 72,000 records (-73%)
- 15. LMS Use Consistent across CategoriesFactor Analysis of LMS Use Categories
- 16. 3. FINDINGS
- 17. Clear Trend: Grade w/Mean LMS Hits
- 18. Question 1 Results: Correlation LMS Use w/Final Grade Scatterplot ofAssessment Activity Hits vs. Course Grade
- 19. Question 2 Results:Correlation: Student Char. w/Final Grade Scatterplot of HS GPA vs. Course Grade
- 20. Conclusion: LMS Use Variables better Predictors than Student Characteristics LMS Student Use Characteristic Variables Variables 18% Average (r = 0.35–0.48) > 4% Average (r = -0.11–0.31)Explanation of change Explanation of change in final grade in final grade
- 21. Smallest LargestLMS Use Variable Student > Characteristic (Administrative Activities) (HS GPA) r = 0.35 r = 0.31
- 22. Combined Variables Regression Final Grade by LMS Use & Student Characteristic Variables LMS Student Use Characteristic Variables Variables 25% (r2=0.25) > +10% (r2=0.35)Explanation of change Explanation of change in final grade in final grade
- 23. Question 3 Results:Regression by “At Risk” Population Subsamples
- 24. At-Risk Students: “Over-Working Gap” 24
- 25. Activities by Pell and GradeExtra effortin content-relatedactivities
- 26. 4. CONCLUSIONS & NEXT STEPS
- 27. Conclusions1. At the course level, LMS use better predictor of academic achievement than student demographics (what do, not who are).2. Small strength magnitude of complete model demonstrates relevance of data, but suggests that better methods could produce stronger results.3. LMS data requires extensive filtering to be useful; student variables need pre-screening for missing data.
- 28. More Conclusions4. LMS use frequency is a proxy for effort. Not a very complex indicator.5. Student demographic measures need revision for utility in Postmodern era (importance to student, more frequent sampling, etc.).6. LMS effectiveness for at-risk students may be caused by non-technical barriers. Need additional research!
- 29. Ideas & FeedbackPotential for improved LMS analysis methods: social learning activity patterns discourse content analysis time series analysisGroup students by broader identity, with uniquevariables: Continuing student (Current college GPA, URM, etc. First-time freshman (HS GPA, SAT/Act, etc)
- 30. Feedback? Questions?John Whitmerjwhitmer@calstate.eduSlidesComplete monographhttp://bit.ly/15ijySPTwitter: johncwhitmer

Be the first to comment