• Save
Confidence in Learning Analytics aka. The Pulse of Learning Analytics
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Confidence in Learning Analytics aka. The Pulse of Learning Analytics

on

  • 3,627 views

Presentation of the paper by Drachsler & Greller on Confidence in Learning Analytics given at LAK12 conference, April 30th 2012, Vancouver, Canada...

Presentation of the paper by Drachsler & Greller on Confidence in Learning Analytics given at LAK12 conference, April 30th 2012, Vancouver, Canada

Data and survey available at:
http://bit.ly/la_survey

Statistics

Views

Total Views
3,627
Views on SlideShare
2,470
Embed Views
1,157

Actions

Likes
2
Downloads
0
Comments
1

8 Embeds 1,157

http://www.drachsler.de 937
http://www.scoop.it 79
http://dougclow.wordpress.com 56
http://portal.ou.nl 51
http://dougclow.org 27
http://translate.googleusercontent.com 4
http://webcache.googleusercontent.com 2
http://zorgacademie.ou.nl 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Confidence in Learning Analytics aka. The Pulse of Learning Analytics Presentation Transcript

  • 1. Confidence in Learning AnalyticsUnderstandings and Expectations from the Stakeholders Hendrik Drachsler & Wolfgang Greller LAK12,Vancouver, Canada, 30th April 2012 1
  • 2. Goals of the Presentation LA Framework Survey Findings Conclusions 2
  • 3. A view on Learning AnalyticsThe LearningAnalyticsFramework 3
  • 4. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 5. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 6. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 7. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 8. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 9. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 10. Greller, W., & Drachsler, H., (submitted). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. 4
  • 11. Stakeholders vs. LA FrameworkOpinions fromthe stakeholderstoward thedimensions of theLA framework 5
  • 12. Learning Analytics Questionnaire Extrapolate opinions from different target groups: (a) what is the current understandings and the expectations on LA (b) is there a common understanding of LA 6
  • 13. Learning Analytics Questionnaire • 4 weeks available • 156 people after clean up • 121 people full records Data and survey available at: http://bit.ly/la_survey 7
  • 14. Participants 5.8% 11.0% Higher Education8.4% K-12 Vocational Others 74.8% 8
  • 15. Participants - Roles % 50 44% Teachers 36% Researchers37.5 L.Designers 26% Managers 25 16%12.5 0 s s s s er er er er Te ac h ea rch sig n na g Roles es L .D e Ma R 9
  • 16. Participants - ReachResponses from 31 countries [UK (38), US (30), NL (22)] 10
  • 17. Stakeholders data datasubjects clients 11
  • 18. Stakeholders (a) who was expected to benefit the most from learning analytics Teachers Parents Institutions Learners 12
  • 19. Stakeholders (a) who was expected to benefit the most from learning analytics Outcomes: 1. Teachers 2. Learners 3. Institutions 4. Parents Teachers Parents Institutions Learners 12
  • 20. Stakeholders (b) how much will learning analytics influence bilateral relationships? Teachers Parents Institutions 13 Learners
  • 21. Stakeholders (b) how much will learning analytics influence bilateral relationships? Outcomes: 1. Teacher-student 84% 2. Student-teacher 63% 3. Student-student 46% 4. Teacher-teacher 41% Teachers Parents Institutions 13 Learners
  • 22. ObjectivesReflection(Glahn, 2009) 14
  • 23. ObjectivesReflection Prediction(Glahn, 2009) 14
  • 24. Objectives The importance of 3 generic objectives: (a) reflection (b) prediction (c) unveil hidden information 15
  • 25. ObjectivesIn which way learning analytics will change educationalpractice in particular areas? n = 119 11% no changes at all 43% small changes 45% extensive changes 16
  • 26. ObjectivesIn which way learning analytics will change educationalpractice in particular areas? Item=2: Timely information n 119 about learning 11% no changes at all 43% small changes Item 8: extensive changes 45% Better insights by institutions in their courses Item 5: Easier grading Item 6: Objective assessment 16
  • 27. Educational DataDrachsler, H., et al. (2010). Issues and Considerations regarding Sharable Data Sets forRecommender Systems in Technology Enhanced Learning. 1st Workshop RecommnderSystems in Technology Enhanced Learning (RecSysTEL@EC-TEL 2010) September, 28,2010, Barcelona, Spain. 17
  • 28. Educational Data Drachsler, H., et al. (2010). Issues and Considerations regarding Sharable Data Sets forVerbert, K., Manouselis, in Technology Enhanced Learning.E. (accepted).Recommnder Recommender Systems N., Drachsler, H., and Duval, 1st Workshop Dataset-driven Systems in Technology Enhanced Learning (RecSysTEL@EC-TEL 2010) September, 28,Research to Support Learning and Knowledge Analytics. Journal of Educational Technology 2010, Barcelona, Spain.& Society. 17
  • 29. Educational Data Researcher: 1. Added context information (n=43, means 3.42) Teacher: 1. Added context information (n=52, means 3.42) Manager: 1. Sharing within the institution (n=16, means 3.63) 2. Anonymisation (n=19, means 3.53) 18
  • 30. TechnologiesManouselis, N., Drachsler, H., Verbert, K., and Duval, E. (to appear). RecommenderSystems for Learning. Berlin:Springer 19
  • 31. TechnologiesPredictionManouselis, N., Drachsler, H., Verbert, K., and Duval, E. (to appear). RecommenderSystems for Learning. Berlin:Springer 19
  • 32. Technologies ReflectionPeter Kraker, ClaudiaWagner, FleurJeanquartier, StefanieN. Lindstaedt (2011):On the Way to aScience Intelligence:Visualizing TEL Tweetsfor Trend DetectionSixth EuropeanConference onTechnology EnhancedLearning (EC-TEL 2011)Manouselis, N., Drachsler, H., Verbert, K., and Duval, E. (to appear). RecommenderSystems for Learning. Berlin:Springer 19
  • 33. TechnologiesTrust in accurate and appropriate LA results ...1.View on learning progress2. Predict learning resource 3. Assessment 4.View on engagement 5. Compare learners 6. Prediction of peers 7. Prediction of learner performance 20
  • 34. Constraints1.Legal protection2.Privacy3.Ethics4.Ownership 21
  • 35. ConstraintsEffect size of LA on ... not at all a little very much privacy ethics data data trans- ownership openness parency
  • 36. Competences 23
  • 37. Competences 1.E-literacy 2.Interpretation skills 3.Self-directedness 4.Ethical understanding 23
  • 38. Competences 24
  • 39. Competences Item 3: Critical reflection Item 7: Self-directedness Item 1: Numerical skills Item 5: Ethical thinking 24
  • 40. Competences Item 3: Critical reflection Item 7: Self-directednessthat 70.2% (n=85) believed learners were NOT competent enough to Item 1: Numerical skills independently learn from learning analytics Item 5: Ethical thinking 24
  • 41. Limitationspicture by Stephan Downes http://www.flickr.com/photos/stephen_downes/3808714505 25
  • 42. Limitations1. Dominance of responsesfrom the HE sector2. Absence of students3. Low awareness of LA; Only surveyed innovators and early adopters4. Mainly opinions from western cultures 26
  • 43. Survey summaryMain findingsof the surveyaccording tothe LAframework 27
  • 44. + Stakeholders1.Main beneficiaries of LA are learnersand teachers followed by organisations2.Biggest benefits would be gained in theteacher-to-student relationship3.Learners require teacher support tolearn from LA 28
  • 45. + Stakeholders1.Main beneficiaries of LA are learnersand teachers followed by organisations2.Biggest benefits would be gained in theteacher-to-student relationship LA asto3.Learners require teacher support tio n?learn from LA i nnova 28
  • 46. + Objectives1. Reflection support is main objective fromthe stakeholders view ...2. ...by revealing hidden information aboutlearners 29
  • 47. + Objectives1. Reflection support is main objective fromthe stakeholders view ...2. ...by revealing hidden information aboutlearners Reflection support only for teacher student relationship? 29
  • 48. + Educational Data1. Context information from learners and the learningprocess2. Anonymisation is the second most important dataattribute3. Willingness to share if data is anonymised 30
  • 49. + Educational Data1. Context information from learners and the learningprocess2. Anonymisation is the second most important dataattribute3. Willingness to share if data is anonymised Can we achieve a collection of reference datasets? 30
  • 50. + Technologies1. Trust in LA algorithms is not well developed2. High confidence on gaining a comprehensiveview of the learning progress 31
  • 51. + Technologies1. Trust in LA algorithms is not well developed2. High confidence on gaining a comprehensiveview of the learning progress How accurate can we measure a learning progress? 31
  • 52. + Constraints1. Data ownership is the most important topic2. LA lead to breaches of privacy but privacy and ethical aspectsare of lesser importance3. Many organisations have ethical boards and guidelines in place 32
  • 53. + Constraints1. Data ownership is the most important topic2. LA lead to breaches of privacy but privacy and ethical aspectsare of lesser importance3. Many organisations have ethical boards and guidelines in place Do we need new policies for data ownership and privacy? 32
  • 54. + Competences1. Skepticism that LA will lead to more independence oflearners to control their learning process2. Training need to guide students to more self-directednessand critical reflection 33
  • 55. + Competences1. Skepticism that LA will lead to more independence oflearners to control their learning process2. Training need to guide students to more self-directednessand critical reflection Do we need mandatory courses on statistics for the edu. sector? 33
  • 56. Future R&D Future R&Dpicture by Tom Raftery http://www.flickr.com/photos/traftery/4773457853 34
  • 57. Future R&D1. Encourage the LA communityto gain additional insights fromour dataset: http://bit.ly/la_survey2. Look for partners to surveypublic bodies like schoolfoundations and governmentalinstitutions3. Compare LA in differenteducational cultures and extendsurvey to eastern countries 35
  • 58. Future R&DGroup Concept Mappingto identifyconsensus aboutparticular issuesin learninganalytics ! 36
  • 59. Many Thanks::Questions? This presentation is available at: slideshare.com/Drachsler Email: hendrik.drachsler@ou.nl Email: wolfgang.greller@ou.nl Supporting projects: 37