Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Visual Learning Pulse - Final Thesis presentation

2,399 views

Published on

The final presentation of the master thesis project Visual Learning Pulse: Flow Prediction and Feedback in Self-Regulated Learning, a project collaboration between the Department of Data Science and Knowledge Engineering of the Univeristy of Maastricht and the Welten Institute of the Open University in the Netherlands.

TITLE:
Visual Learning Pulse: Flow Prediction and Feedback in Self-regulated Learning

ABSTRACT:
Visual Learning Pulse is a Master thesis research project developed in cooperation with the Welten Institute, the Research Centre for Learning, Teaching and Technology at the Open University of the Netherlands, and partially nanced by the European project Learning Analytics Community Exchange (LACE). Visual Learning Pulse explores whether physiological and physical data such as heart rate, step count and weather data if correlated with learning activity data can be used to predict learning success in self-regulated learning settings.

To verify this hypothesis an experiment was opportunely designed, consisting of three phases, lasting six weeks and involving nine participants, each of them wearing a Fitbit HR wrist band and having their application usage recorded during their learning and working activities throughout the day. An ad-hoc infrastructure for longitudinal and multi-modal data was designed and implemented. The data from di erent sources were stored using the Experience API (xAPI) data standard in a cloud distributed database called Learning Record Store.

The participants (doctoral students at the Open Universiteit) - were asked to rate their learning experience through an Activity Rating Tool indicating their perceived level of productivity, stress, challenge and abilities. These self-reported performance indicators were used as training labels for the two algorithms employed for prediction of time series data: the Vector Autoregression and Linear Mixed E ect Model.

A major task of the thesis consisted of developing the software application to pre-process, perform the analysis and generate the predictions on real time, in order to provide timely feedback to the users about their learning performances. Although not showing high overall accuracy, the prediction models were successfully learnt and used in production: in the third phase of the experiment, two visualisations mechanisms were used, the Learner Dashboard and the Feedback Cubes.

In addition, a conceptual paper of Visual Learning Pulse, illustrating setup and overall the rationale was presented at the Learning Analytics & Knowledge conference 2016 in Edinburgh, Scotland and was included in CEUR workshop proceedings.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Visual Learning Pulse - Final Thesis presentation

  1. 1. UM supervisors Kurt Driessens Pietro Bonizzi OU supervisors Hendrik Drachsler Maren Scheffel Maastricht, 29th June 2016 Daniele DI MITRI presents MSc Thesis in Artificial Intelligence
  2. 2. Visual Learning Pulse – Final thesis presentation 2 What was done - visual 21/09/2015 Internship starts 21/12/2015 Internship ends Design pre-test Experim ent Implement 29/06/2016 Thesis ends Report 01/02/2016 Thesis starts Paper submitted to LAK conference Analysis,literature 18-25/06/16JTEL summerschool 25-29/04/16 LAK conference Coding 31/03/16 Announcing Presentation Trainingphase Validation phase Exploitation phase Reporting 17/05/16 11/04/16 30/05/16 8 months of work
  3. 3. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation What was done - numbers 3 1 publication 2 conferences 2 software apps 6 presentations 9 blog posts 20+ meetings 1800 lines of code
  4. 4. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Learning Analytics & Knowledge Conference 2016 4 Di Mitri, Scheffel, Drachsler, Börner, Ternier 2016 - Learning Pulse : using Wearable Biosensors and Learning Analytics to Investigate and Predict Learning Success in Self - regulated Learning.
  5. 5. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 5 Background, meaning, vision 5
  6. 6. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Data deluge in Education 6
  7. 7. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 7 Data-driven approach Picture from tincanapi.com
  8. 8. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Self-regulated learners need support 8 Self-Regulated Learning → no guidance → no feedback → no support
  9. 9. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Related work 9 Signals, Purdue University Student success, University North Dakota S3, Desire To Learn
  10. 10. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Dimensions of Learning 10
  11. 11. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Machine Learning with Human Learning 11 y = f(X) Learning performance Predictive Model Input space
  12. 12. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 12 Own Approach 12
  13. 13. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 13 RESEARCH QUESTION Can we predict learning success out of physiological, activity and weather data? 13
  14. 14. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 14 Participants ● 9 PhD students at Welten institute ● Different disciplines ● Different OS
  15. 15. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Experiment Timeline 15 11th to 29th April 2016 1st phase: “Training” Participants rate their activity 17nd to 27th May 2016 2nd phase: “Validation” Participants rate their activity + Feedback visualization 30th May to 3th June 2016 3rd phase: “Exploitation” Individual and group Feedback visualization
  16. 16. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 16 Input space Context Body Activities Body: physiological (heart-rate) and physical responses (steps) - from Fitbit HR Activities: applications used during learning from RescueTime Context: weather data from OpenWeatherMap
  17. 17. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 17 Hypothesis space: the Flow Csikszentmihalyi, 1972 Theoretical Empirical
  18. 18. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Activity Rating Tool 18 Productivity How productive was last activity? Stress How stressful was last activity? Challenge How challenging was last activity? Abilities How prepared did you feel for the activity? FLOW Participants rate hourly, from 7AM to 7PM A scalable web app! Client: Bootstrap + Jquery Sever: GoogleApp + Python “Very easy to use!”
  19. 19. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 19 VLP Data Model
  20. 20. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 20 Scheffel, M., Ternier, S., & Drachsler, H. (2016). The Dutch xAPI Specification for Learning Activities http://bit.ly/DutchXAPIreg Experience API Data storing format for the Learning Record Store
  21. 21. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 21 The Data journey
  22. 22. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation A complicated Architecture 22
  23. 23. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Data collection 23 ● PULL data from the 3rd party APIs ● Make the xAPI triples ● PUSH data in the LRS ● It’s scalable! ● No collisions ● It’s fast ● It’s Interoperable Learning Pulse Server + Learning Record Store
  24. 24. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Data Processing application 24 Script in Python on a VM which processes data in real time
  25. 25. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 25 Transformed dataset ● Time Series: tabular representation ● 5 minutes intervals ● Enough samples now! ● Easier view for Machine Learning ● Signal resampling needed 8728 observations X 29 attributes
  26. 26. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 26 (Issue 1) Extract features from TS Heart Rate Variability and Heart Rate Entropy… didn’t work SOLUTION ● Mean of the signal ● Maximum ● Minimum ● Standard Deviation ● Average change Heart-ratesignalfor15mins
  27. 27. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 27 (Issue 2) Reduce sparsity Rule based grouping of applications Subjects can be compared Applications used are too sparse Let’s create application categories
  28. 28. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 28 (Issue 3) Ladder effect Trade-off: number of samples vs How much bother people NO SOLUTION
  29. 29. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 29 (Issue 4) Dependency constraint Independence constraint Knowing one value of et for one observation does not help us to guess value of et+1 yt = α + βX t + et cov(et ,et+1 ) = 0 FIXED Effect RANDOM Effect SOLUTION follows...
  30. 30. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Approach 1) Vector Auto Regression 30 x0 x1 x2 x... an t0 x x x ... x t1 x x x ... x t... ... ... ... ... ... tp x x x ... x tp+1 ? ? ? ? ? tp+2 ? ? ? ? ? PAST PRESENT FUTURE Time intervals PREPROCESS Timeseries were LOGged LIMITATIONS ● Participants need to be treated separately ● Doesn’t work with categorical data ● Doesn’t work with random effects
  31. 31. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Approach 2) Mixed Effect Linear Model 31 x0 x1 x2 ... xn-1 xn g y t0 x x x ... x x 1 y t1 x x x ... x x 1 y t2 x x x ... x x 2 y t... ... ... ... ... ... ... 2 y tp- 1 x x x x x x 3 y tp ? ? --- --- --- --- x ? Random EffectsFixed Effects Group Tried both Python and R implementations Used R-squared for goodness-test LIMITATIONS ● Poor results ● Convergence time ● Mono-output ● Algebra errors
  32. 32. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 32 Issue: high inter-subject variability i.e. Participants have rated very differently
  33. 33. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Visualisations 33 33
  34. 34. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 34 Learner Dashboard 9 personal Dashboard + 1 public
  35. 35. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 35 *Börner, Tabuenca, Storm, Happe, and Specht. 2015Feedback Cubes
  36. 36. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Conclusions 36 36
  37. 37. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 37 Limitations ● Low accuracy of prediction RQ-answer: YES but prediction accuracy can be improved. ● Real-time issues Fitbit synchronisations, Virtual Machine performance ● 3rd party API constraints ● No great solution for sparse data (manual grouping)
  38. 38. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 38 Achievements ● Real-time system works ● Data collection was seamless ● Good dataset for experiments (will be open sourced) ● Useful insights IoT in Learning ● Reusable architecture
  39. 39. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Future ideas 39 39
  40. 40. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 40 Modelling sparse data (idea) a1 a2 a3 a4 a5 a6 Ft Ft+1 Ft+2 Hidden Flow values Ft+3 Random sampling a7 a8 Visible applications Hidden Markov Chains + Random sampling
  41. 41. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 41 Thank you! Q&A “Life can only be understood backwards, but it must be lived forwards” - Kierkegaard

×