Download as PDF, PPTX
Uploaded byDaniele Di Mitri
Digital Learning Projection - Learning state estimation from multimodal learning experiences.
The document presents a doctoral consortium focused on learning state estimation using multimodal data, highlighting the importance of collecting and analyzing diverse data to understand learning processes. It outlines research challenges, questions, and proposed experiments aimed at developing a framework for predicting learning states and providing effective feedback. Key aspects include addressing data collection, sensor fusion, and the architectural design necessary for achieving these goals.
More Related Content
Similar to Digital Learning Projection - Learning state estimation from multimodal learning experiences.
Digital Learning Projection - Learning state estimation from multimodal learning experiences.
- 1. Doctoral consortium LAK17 14thMarch 2017, Vancouver, Canada Digital Learning Projection Daniele DI MITRI ddm@ou.nl Learning state estimation from multimodal learning experiences.
- 2. Pagina 2 Where doeslearning happen? “Online learning does not happen online, it happens where the learner is. It can’t happen where the learner isn’t.” - Peter Goodyear
- 3. Background takeaways 1. Learningdata available are not enough 2. Learning happens everywhere: ubiquitous and incidental 3. Sensors can collect multimodal data 4. Machine learning can help analysing data 5. Automatically learn how to estimate “learning states” 6. It can generate feedback and recommend actions Pagina 3
- 4. Machine Learning approach Pagina4 Learning States Multi modal data Learning state estimation
- 5. Pagina 5 Proposed Framework Blueprint ofCognitive Inference Back-track the intangible by projecting the tangible.
- 6. Input space Pagina 6 Multimodal Data Involuntary EEG/ Focus HeartRate Sweat Deliberated Step count Gaze direction Head position Hands position • Observable behavior! • Taxonomy • Several way to divide events – Deliberated vs involuntary – Deterministic vs stochastic (random) – Endogenous vs exogenous – Interactive vs Reflective
- 7. Output space Pagina 7 Canbe defined as: • Learning Gain • Learning Progress • Learning Performance The spaces can be divided into segments that are the Learning States Affective- Behaviour- Cognition (ABC) Learning Gains Project Open University U.K. https://twitter.com/LearningGains 0 10 20 30 40 50 60 70 80 Affective (emotions) Cognition (feelings) Behaviour (actions) Learning State (idea) LS1 LS2 LS3
- 8. Research Challenges C1 –Availability of Labels – defining and sampling the output space. C2 – Architectural Design– designing an architecture which collects different multiple heterogeneous modalities C3 – Sensor Fusion – aligning the multimodal data for analysis and prediction C4 – Appropriate Feedback – designing some sort of feedback which maximises learning Pagina 8
- 9. Research Questions • Q1– Learning States Is it possible to represent the learning process into numerable learning states which can be predicted? • Q2 – Data Collection What are the requirements for a multimodal sensor fusion architecture? • Q3 – Data Analysis Do multimodal data streams combined with learner’s action sequences improve the prediction of learning states? • Q4 – Feedback Generation How can we use multimodal data to generate feedback to support learning? Pagina 9
- 10. Research Tasks Pagina 10 •T1 – Literature Review on multimodal data for learning • T2 – 1st experiment Learning Pulse • T3 – 2nd experiment WEKIT prototype • T4 – 3rd experiment using WEKIT for Learning States
- 11. Mapping tasks withquestions & challenges Pagina 11 Q1: Learning States Q3: Data Analysis Q2: Data Collection Literature Review 1st Experiment 2nd Experiment 3rd Experiment C3:Sensor Fusion C2: Architecture Design C1: Availability Labels Tasks Questions Challenges C4: Appropriate Feedback Q4: Feedback Generation
- 12. Literature review: LearningBlueprint • Taxonomy of Multimodal Data for Learning • Target: JCAL special issue on MMLA • Look for 1. similar experiments 2. techniques used to collect data 3. multimodal data chosen in related studies 4. learning performance indicators used 5. data analysis approaches used 6. results obtained. Pagina 12
- 13. 1st Experiment: LearningPulse LAK17 Pagina 13 Di Mitri, D., Scheffel, Drachsler, H., M., Börner, D., & Specht, M. (2017). Learning Pulse : a machine learning approach for predicting performance in self-regulated learning using multimodal data.
- 14. 2nd Experiment: WEKIT Pagina14 • Wearable Enhanced Knowledge Intensive Training • Main task now: designing & building prototype • Design of a multimodal architecture • For more info: https://wekit.eu/
- 15. 3rd Experiment: FirstAid Training with manikins • full monitoring of the learning environment • clear start and end • performance measurement • practical learning over more cognitive ones • close set of actions. • High practical significance! Pagina 15
- 16. 3rd Experiment: OptionB) Indigenous shelter building • Proposal of collaboration • Land & Learning Indigenous Technology Experience (LLITE) with Canadian partners • Fit for WEKIT AR technology • 3D holograms can be used • Case scenario must be defined well Pagina 16
- 17. Unique Selling Points •Generative approach vs data already available • Machine learning approach learn from history – Input space: multimodal data – Output space: notion of Learning State • “What to do with data?” vs “what data can be collected?” • Real-time data collection & analysis vs ex-post • Random Action sequences • AR technologies Pagina 17
- 18. S.W.O.T. Pagina 18 Strengths Weaknesses Datadriven Not enough data Opportunities Threats Personalisation Garbage-in-garbage-out
- 19. Ethics & Privacy- ‘2084’ • Out of scope of my PhD • But still really relevant! • Great opportunity vs great risks • Expand the LA Framework • Follow-up of the project Pagina 19
- 20. Q&A Thanks for listening! DanieleDi Mitri ddm@ou.nl @dimstudi0 Pagina 20