• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
LRT Talks 2013-03-12 CETIS
 

LRT Talks 2013-03-12 CETIS

on

  • 739 views

 

Statistics

Views

Total Views
739
Views on SlideShare
704
Embed Views
35

Actions

Likes
0
Downloads
0
Comments
0

3 Embeds 35

https://twitter.com 30
https://t.co 4
http://tweetedtimes.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    LRT Talks 2013-03-12 CETIS LRT Talks 2013-03-12 CETIS Presentation Transcript

    • First steps with learning analytics CETIS 2013 Birmingham | Mar 2013 Professor Mark Stubbs Head of Learning & Research Technologies m.stubbs@mmu.ac.uk http://twitter.com/thestubbs http://slideshare.net/markstubbsTuesday, March 12, 2013 1
    • Analytics to develop understanding 2. Identify 1.Appreciate relevant data the issue sources Refining 3.Summarise 6.Analyse & visualise understanding individual data sources of a problem space 5 Prep the 4 Join on data for common analysis identifiers
    • Analytics case study oneUnderstanding impacts of a new VLE on student success• PhD Student (Julie Hardman) led evaluation strand for MMU’s (WebCT Vista) VLE project from 2006-2009• Stakeholders wanted to understand impacts of new VLE• Suspicion that some patterns of VLE use were more or less likely to produce successful outcomes for students• Lots of small-scale studies, but lack of institutional-scale research
    • Analytics case study oneAppreciation of 2. Identify Student 1.Appreciate relevant datamore and less the issue demographics sourcessuccessful VLE logspatterns of VLE Exam boarduse outcomes Understanding 3.Summarise 6.Analyse & visualise impacts of new individual data sources VLE on student Categorise VLE usage success & count hits by category 5 Prep the 4 Join on data for common analysis identifiers
    • 3.Summarise individual data sourcesAnalytics case study oneWebCT Vista kept a detailed click-track log (Oracle DB)• Custom SQL generated 1-row-per-student summaries: – Number of areas student enrolled on & number active on – Number of staff actively associated with those areas – Days from start of term when VLE first & last used – Percentage of total hits between 9am – 9pm – Total number of student logons (distinct sessions) – Total number of document hits by student & by tutors – Total number of chat hits by student & by tutors – Total number of assessment hits by student & by tutors
    • Analytics case study oneAppreciation of 2. Identify Student 1.Appreciate relevant datamore and less the issue demographics sourcessuccessful VLE logspatterns of VLE Exam boarduse outcomes Understanding 3.Summarise 6.Analyse & visualise impacts of new individual data sources VLE on student Categorise VLE usage success & count hits by category 5 Prep the 4 Join on data for common analysis identifiers Join on student ID
    • 4 Join on common identifiersAnalytics case study oneStudentID Demographics aStudentID Categorised VLE usage bStudentID Progressed? c SELECT … FROM a, b, c WHERE a.StudentID = b.StudentID AND b.StudentID = c.StudentIDStudentID Demographics Categorised VLE usage Progressed?
    • Analytics case study one Appreciation of 2. Identify Student 1.Appreciate relevant data more and less the issue demographics sources successful VLE logs patterns of VLE Exam board use outcomes Understanding 3.Summarise 6.Analyse & visualise impacts of new individualInterpret data sourcesoutput of VLE on student Categorise VLE usagestatistical success & count hitstests by category 5 Prep the 4 Join on data for common Code missing analysis identifiers Join on values, collapse student ID categories…
    • 6.Analyse & visualiseAnalytics case study oneRandom forest analysis of factors predicting progression:• Percentage of student usage between 9am and 9pm• Number of day since start of term when VLE last used• Total number of tutors’ document hits• Total number of students’ document hits• Total number of tutors’ chat hits• Total number of student logons (distinct sessions)• Number of days since start of term when VLE first usedhttp://onlinelibrary.wiley.com/doi/10.1002/sres.2130/abstract
    • 6.Analyse & visualiseAnalytics case study oneVisualising partial dependence on the top three predictors 9am-9pm usage Last VLE access Tutors’ doc hitshttp://onlinelibrary.wiley.com/doi/10.1002/sres.2130/abstract
    • Analytics case study one Appreciation of 2. Identify Student 1.Appreciate relevant data more and less the issue demographics sources successful VLE logs patterns of VLE Exam board use outcomes Understanding 3.Summarise 6.Analyse & visualise impacts of new individualInterpret data sourcesoutput of VLE on student Categorise VLE usagestatistical success & count hitstests by category 5 Prep the 4 Join on data for common Code missing analysis identifiers Join on values, collapse student ID categories…
    • 1.Appreciate the issueAnalytics case study oneEnhanced appreciation of relationships between VLE use andstudent success• “At risk of failure” alarm bells – High usage percentage outside 9am-9pm – Early finish or late start in year for VLE use• Categorised VLE use more informative than total hits – Documents / Content – Chat / Dialogue – Assessment• More questions than answers … we’ll be exploring further with our new Moodle VLE
    • 1.Appreciate the issueAnalytics case study twoUnderstanding predictors of NSS overall satisfaction forScience and Engineering courses• Fielding, A.F., P.J.Dunleavy and A.M. Langan (2010) Effective use of the UKs National Student (Satisfaction) Survey (NSS) data in science and engineering subjects. Journal of Further and Higher Education, 33, 347-368.• Concern that focusing on low mean scores without reference to subject benchmarks could be ineffective for improving overall satisfactionMore background…• http://www.gees.ac.uk/events/2010/feedbknss/documents/MarkLanganInsightsintoNSS_La ngan_GEES_Nov2010_submitted.ppt
    • Analytics case study two Appreciation 2. Identify UK NSS 1.Appreciate relevant data of how best to the issue dataset filtered sources improve by JACS3 satisfaction Science & Engineering Understanding predictors satisfaction on 3.Summarise 6.Analyse & individualInterpret visualise Science & data sources Questionoutput of Engineering responsesstatistical summarisedtests courses to %satisfied 5 Prep the 4 Join on data for common Code missing analysis identifiers Not required: values, collapse single dataset categories…
    • 6.Analyse & visualiseAnalytics case study twoRandom forest analysis of Q.s predicting overall satisfaction:Rank Predicting questionnaire item Inc MSE (%)1 Q15 - The course is well organised and is running smoothly 119.892 Q1 - Staff are good at explaining things 71.453 Q4 - The course is intellectually stimulating 66.714 Q14 - Any changes in the course or teaching have been communicated effectively 60.795 Q10 - I have received sufficient advice and support with my studies 55.34…10 Subject 32.35…18 Q7 - Feedback on my work has been prompt 10.4919 Q9 - Feedback on my work has helped me clarify things I did not understand 6.6520 Q5 - The criteria used in marking have been clear in advance 6.6021 Q21 - As a result of the course, I feel confident in tackling unfamiliar problems 3.3222 Q8 - I have received detailed comments on my work 3.04
    • 6.Analyse & visualiseAnalytics case study twoSector predictions of Q22 from Q1-21 responses for Sci EngUnder-performing… Actual Predicted Residual SE1 SE2 SE3 SubjectsOver-performing…
    • Analytics case study two Appreciation 2. Identify UK NSS 1.Appreciate relevant data of how best to the issue dataset filtered sources improve by JACS3 satisfaction Science & Engineering Understanding predictors satisfaction on 3.Summarise 6.Analyse & individualInterpret visualise Science & data sources Questionoutput of Engineering responsesstatistical summarisedtests courses to %satisfied 5 Prep the 4 Join on data for common Code missing analysis identifiers Not required: values, collapse single dataset categories…
    • 1.Appreciate the issueAnalytics case study twoEnhanced understanding of the predictors of satisfaction forScience and Engineering courses• Statistical demonstration of significance of subject variation led to a more sophisticated NSS response at MMU• Course organisation and teaching are significant predictors• Low feedback scores not a significant predictor
    • Analytics to develop understanding 2. Identify 1.Appreciate relevant data the issue sources Refining 3.Summarise 6.Analyse & visualise understanding individual data sources of a problem space 5 Prep the 4 Join on data for common analysis identifiers
    • Next steps with relevant data sources Exam Board VLE logs Outcomes Student & Course Records Assignment Student Submissions Satisfaction 2. Identify 3.Summarise 4 Join on relevant data individual common sources data sources identifiers