Astute symposium 2013-10-10_smart_sensors_userstate_josesaez_santiagofernandez


Published on

Astute symposiume 10/10/2013 - Smart sensors userstate

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Astute symposium 2013-10-10_smart_sensors_userstate_josesaez_santiagofernandez

  2. 2. INTRODUCTION Higher data flow WHY? Data Complex services Actions Embedded Systems are becoming more and more complex Different applications Astute project Error Information Support Advice Confirm 2
  3. 3. INTRODUCTION ASTUTE project aims to improve usability of embedded systems by using user state and context situation capture to provide pro-active decision support via multi-modal interfaces. CONTEXT CAPTURE CONTEXT MODELING ENGINE PROACTIVE DECISION ENGINE MULTI-MODAL HMI Pro-active decision support system based on human centered design able to support user intentions while keeping him in control. Astute project 3
  4. 4. USER AWARENESS PHYSICAL ENVIRONMENT CONDITIONS •Information Priority PROCESSING DATA INPUTS PHYSIOLOGIC MEASURES Astute project •Decision Support USER STATE •Data delivery •Keep in control 4
  6. 6. USER STATE EEG: Electroencephalography is the measurement of electrical activity resulting from ionic currents flows in brain neurons using multiple electrodes placed on the scalp. It is commonly used in medicine for diagnostic applications, like epilepsy, encephalopathies or sleep disorders, by analyzing its spectral content. Further applications include EEG average analysis for cognitive sciences by analyzing response to time-locked events and stimulus. STRESS FATIGUE Astute project RELAXATION FATIGUE CONCENTRATION DISEASES 6
  7. 7. USER STATE ECG: Electrocardiography is the transthoracic measurement of electrical activity in heart using different electrodes attached to user’s skin. It is commonly used in medicine to measure heartbeats rate, size and position of the different heart chambers and any effect of external source on heart. Although ECG information is limited to physiological user status, it is useful to complement EEG data to complement obtained information increasing system performance and reliability. How this information is merged? Astute project 7
  8. 8. RATIONALE & MOTIVATION • Cognitive-affective states are relevant in the realisation of tasks that: – Manage a large volume of information in the interface/system/process – are a cognitive challenge – Are critical (urgency, safety, health, sports) – Involve people (human resources, leadership, coaching, social and personal relationships) • Availability of information on user state facilitates interface/system/process pro-activeness, which is accomplished via decision support built on top of data-/knowledge-based models, providing: – Alarms, feedback – Recommendations for adapting the interface/system/process Astute project 8
  9. 9. RATIONALE & MOTIVATION • Present work provides a description of research and development of a user state diagnostic system, within the broader context of the European Artemis ASTUTE project ASTUTE aims to develop advanced and innovative pro-active HMI supported by reasoning engine system, for improving the way the human being deals with complex and huge information quantities in different operative conditions and contexts. • A number of previous projects generally used a limited range of sensors network, mainly focused on autonomous psychophysiological information, and used concrete context scenarios • More technical effort may be done to incorporate measures of brain activity, and thus to delineate a full picture of brain-body reaction • A probabilistic model has been developed, given its capability to handle uncertainty in sources information and inference, and sound mathematical framework. Astute project 9
  10. 10. OVERVIEW Context User Profile Web services User state Sensors Mobile Astute project Mobile Cloud App Cloud App 10
  11. 11. DATA CAPTURE Frequency bands α, β, θ,... Calibration & normalisation Heart rate EEG raw data Calibration & normalisation To user state model User’s profile Web services connectivity Astute project 11
  12. 12. OUR COGNITIVE MODEL • Three user states that are relevant in working environments and safetycritical tasks: – stress, mental workload and fatigue • A fourth state, namely inaptitude, is derived as a combination of the three aforementioned user-states (stress, mental workload and fatigue): – this is input to decision support, where recommendation for assistance will be given Astute project 12
  13. 13. OUR COGNITIVE MODEL • Input used for diagnosis of user state comes from – brain activity (EEG) and heart activity sensors • This is complemented by selected – predictive factors from context (context complexity, task workload) and – user profile (experience, age, fitness), – extracted from each user-case Astute project 13
  14. 14. OUR COGNITIVE MODEL As the number of parents of a node increase, conditional probability tables (CPT) become larger. Limit relations between nodes if there’s conditional independence Use canonical distributions NoisyOR is used in our model Adapt distributions with statistical analysis of a large amount of training data p(Symptom2 | Disease1, Disease2) Symptom2 Disease1 False True False False 80 % 20 % False True 40 % 60 % True False 30 % 70 % True Astute project Disease2 True 20 % 80 % 14
  15. 15. OUR COGNITIVE MODEL • This first model can be extended with “contrasts” – for verifying the need for assistance via the state inaptitude. – In the end, this strategy improves the decision reliability Astute project 15
  16. 16. OUR COGNITIVE MODEL • Assistance recommendation is provided – alarms, voice, warning messages, etc. – assistance is based on the expected utility in order to advice the user Astute project 16
  17. 17. USAGE EXAMPLE Astute project 17
  18. 18. USAGE EXAMPLE Astute project 18
  19. 19. SCHEMATIC ARCHITECTURE Astute project 19
  20. 20. FUTURE WORK • additional sensors – for example to complement existing information or to cope with the unavailability of specific sensors in particular settings • further input parameters extracted from existing sensors • improved pre-processing and fine-tuning of features can enhance robustness • new user’s cognitive and affective states • further exploitation of user’s state contrasts and assistance • exploring a dynamic version for anticipation or prediction of the user’s state • adaptability to users via training procedures and to improve the prediction capacity of the cognitive model • mobility and user profiling and personalization are key to our system and deserve significant attention Astute project 20
  21. 21. CONCLUSIONS • we’ve shown present and future work within the framework of ASTUTE project • emphasis is on providing an integrated solution to monitor and adapt the user’s state to the task demands in complex contexts – to design and implement a probabilistic cognitive model that is predicting the user’s state based on complex context use-cases – a set of sensors is diagnosing such user’s states based of brain and heart rate evidence • this solution is partially overcome by other projects, however we add value in increasing the set of sensors, range of user’s states in real and intensive scenarios Astute project 21
  22. 22. THANKS FOR YOUR ATTENTION! Astute project 22