Logging On To Improve AchievementEvaluating the relationship between use of the LearningManagement System, student charact...
Introduction• Educational Doctorate Degree (EdD) candidate (University of California Davis & Sonoma State University)• Adv...
Presentation Outline1. Study Case & Context2. Results for Instructional Practices3. Results for LMS Data Analysis4. Conclu...
STUDY CASE &CONTEXT
Problem: Student Graduation• Less than 50% of college/university students graduate within 6 years  • California State Univ...
Case: Introduction to Religious Studies•   Redesigned to hybrid            54 F’s    delivery through Academy    eLearning...
Research Questions1)   Is there a relationship between student LMS usage and academic     performance? Does this relations...
Independent Variables: Student Characteristics 1. Gender 2. Under-Represented Minority 3. Pell-Eligible 4. High School GPA...
Independent Variables: LMS Usage#    LMS Usage Category             LMS Tools within                                    Ca...
Research Methods (Cliff’s notes version)1.   Extract data, validate with appropriate “owner”2.   Transform variables     •...
Results for InstructionalPractices
Correlation: LMS Usage w/Final Grade        Scatterplot of   Assessment Activity Hits v.        Course Grade
Correlation: Student Char. w/Final Grade
Most interesting finding (so far):       Smallest                           Largest                              >   LMS U...
Regression R2 Results Comparison
RESULTS FOR LMSDATA ANALYSIS
Lms Logfiles: “Data Exhaust”1. Logfile tracks server actions   (not educationally relevant activity)2. Duplicate logfile h...
Logfile Data Filtering Results                                             450     382                                    ...
LMS Use Consistent across CategoriesFactor Analysis of LMS Use Categories
Missing Data On Critical Indicators
Conclusions1. At the course level, LMS use better predictor of   academic achievement than any student characteristic   va...
Ideas & FeedbackPotential for improved LMS analysis methods:• social learning• activity patterns• discourse content analys...
Contact InfoJohn Whitmerjwhitmer@calstate.eduSkype: john.whitmerUSA Phone: 530.554.1528                          By Winged...
Upcoming SlideShare
Loading in …5
×

Logging on to Improve Achievement

460 views

Published on

Improving student persistence, especially among under-represented minority students, is a driving goal at many colleges and universities. Academic technologies, such as the Learning Management System (LMS), are frequently used to deliver innovative pedagogical strategies to increase engagement and improve persistence. This study presents research on a redesigned hybrid high-enrollment undergraduate course exploring the relationship between LMS activity, student background characteristics, current enrollment information, and student achievement.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
460
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Kathy
  • John
  • John
  • John
  • John
  • John
  • John
  • Kathy
  • Logging on to Improve Achievement

    1. 1. Logging On To Improve AchievementEvaluating the relationship between use of the LearningManagement System, student characteristics, and academicachievement in a hybrid large enrollment undergraduate course Research Highlights: Presentation to SoLAR Storm November 15, 2012 John C Whitmer (jwhitmer@calstate.edu)Committee Chair: Dr. Paul Porter, Sonoma State University Slides: http://slidesha.re/sFKjcm
    2. 2. Introduction• Educational Doctorate Degree (EdD) candidate (University of California Davis & Sonoma State University)• Advanced to candidacy, defending ~ January 14• Associate Director, California State University LMSS Project, Chancellor’s Office
    3. 3. Presentation Outline1. Study Case & Context2. Results for Instructional Practices3. Results for LMS Data Analysis4. Conclusions
    4. 4. STUDY CASE &CONTEXT
    5. 5. Problem: Student Graduation• Less than 50% of college/university students graduate within 6 years • California State University: 52.4% (first-time freshman, 2000 cohort) (CSU Analytic Studies, 2011)• Students from under-represented minority racial/ethnic groups graduate at much lower rates • California State University: 38.3% (African American students, first-time freshman, 2000 cohort) (CSU Analytic Studies, 2011)• Contributing factor: mega-enrollment intro courses • Infrequent interaction, prevent faculty/student relationships
    6. 6. Case: Introduction to Religious Studies• Redesigned to hybrid 54 F’s delivery through Academy eLearning• Highest LMS usage entire campus Fall 2010 (>250k hits)• 373 students (54% increase)• Bimodal results • 10% increased SLO mastery • 7-11% increase in DWF
    7. 7. Research Questions1) Is there a relationship between student LMS usage and academic performance? Does this relationship vary by the pedagogical purpose underlying LMS usage? (correlation)2) Is there a relationship between student background characteristics or current enrollment information and academic performance? (correlation)3) Does analyzing combined student characteristics and current enrollment information increase the predictive relationship between combined LMS usage data and student success? (multivariate regression)4) Does a student’s economic status and student of color status vary the predictive relationship between combined LMS usage, combined background characteristics and current enrollment information? (multivariate regression, restricted model)
    8. 8. Independent Variables: Student Characteristics 1. Gender 2. Under-Represented Minority 3. Pell-Eligible 4. High School GPA 5. First in Family to Attend College 6. Major-College 7. Enrollment Status Under-Represented Minority and Pell-Eligible 8. (interaction) Under-Represented Minority and Gender 9. (interaction)
    9. 9. Independent Variables: LMS Usage# LMS Usage Category LMS Tools within Category1. Administration Activity Hits Announcement Calendar2. Assessment Activity Hits Assessment Assignments My-grades3. Content Activity Hits Content-page Web-links4. Engagement Activity Hits Discussion Mail
    10. 10. Research Methods (Cliff’s notes version)1. Extract data, validate with appropriate “owner”2. Transform variables • measures of interest (e.g. “URM”, not race/ethnicity) • analysis methods (categorical into numeric)3. Examine data for • outliers, missing data, data distributions, etc. • colinearity between variables (e.g. independence)4. Join data into single data file, collapse to one record/student5. Run analysis
    11. 11. Results for InstructionalPractices
    12. 12. Correlation: LMS Usage w/Final Grade Scatterplot of Assessment Activity Hits v. Course Grade
    13. 13. Correlation: Student Char. w/Final Grade
    14. 14. Most interesting finding (so far): Smallest Largest > LMS Use Variable Student Characteristic(Administrative Activities) (HS GPA) r=0.3459 r=0.3055
    15. 15. Regression R2 Results Comparison
    16. 16. RESULTS FOR LMSDATA ANALYSIS
    17. 17. Lms Logfiles: “Data Exhaust”1. Logfile tracks server actions (not educationally relevant activity)2. Duplicate logfile hits for single student action3. To remedy, filtered logfiles by: • Time (> 5sec, <3600 sec) • Actions (no “index views”, more)
    18. 18. Logfile Data Filtering Results 450 382 400 350 300 250 200 151 150 58 100 54 51 49 36 23 26 16 50 0 Final data set: 72,000 records (from 250K+) Filtered Measure Raw Avg Avg Reduction Discussion Activity Hits 382 54 706% Content Activity Hits 151 51 296% Assessment Activity Hits 58 23 249% Mail Activity Hits 49 36 136% Administrative Activity Hits 26 16 159%
    19. 19. LMS Use Consistent across CategoriesFactor Analysis of LMS Use Categories
    20. 20. Missing Data On Critical Indicators
    21. 21. Conclusions1. At the course level, LMS use better predictor of academic achievement than any student characteristic variable. Behavioral data appears to supercede demographic information (what do, not who are).2. Moderate strength magnitude of complete model demonstrates relevance of data, but suggests that refinement of methods could produce stronger results.3. LMS data requires extensive filtering to be useful; student variables need pre-screening for missing data.
    22. 22. Ideas & FeedbackPotential for improved LMS analysis methods:• social learning• activity patterns• discourse content analysis• time series analysisGroup students by broader identity, with unique variables:• Continuing student (Current college GPA, URM, etc.• First-time freshman (HS GPA, SAT/Act, etc)
    23. 23. Contact InfoJohn Whitmerjwhitmer@calstate.eduSkype: john.whitmerUSA Phone: 530.554.1528 By WingedWolf Damián Navas

    ×