Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

idalab seminar #19 - Dr. Thorsten O. Zander

104 views

Published on

Can computers learn to understand their users in a way we intuitively understand other people? The short answer is: Yes, they can. Teaching computers to adapt to the human mind is exactly what Brain-Computer Interfacing is about.

Based on EEG – measuring electromagnetic potentials generated by brain activity on the scalp – Brain-Computer Interfaces (BCIs) can automatically detect and react to changes in the state of its user’s mind. This ‘thought reading’ can be used to directly control a computer – as if we were using a computer mouse, while not using a single muscle in our body. In addition to explicit, intentional control, BCI can be used for implicit communication from the user to the computer. The computer can see and learn how our brain responds to changes in the environment and build – step by step – a model of its user’s mind. It can then continuously adapt to our intentions, ideas and concepts and support the ongoing Human-Computer Interaction.
I call this approach Neuroadaptivity and see it as a path to a convergence of human and machine intelligence. In that way, a neuroadaptive computer can indeed learn to gain an understanding of its user.

I call this approach Neuroadaptivity and see it as a path to a convergence of human and machine intelligence. In that way, a neuroadaptive system – a computer – can indeed learn to gain an understanding of its user.

Published in: Data & Analytics
  • accessibility Books Library allowing access to top content, including thousands of title from favorite author, plus the ability to read or download a huge selection of books for your pc or smartphone within minutes.........ACCESS WEBSITE Over for All Ebooks ..... (Unlimited) ......................................................................................................................... Download FULL PDF EBOOK here { http://bit.ly/2m6jJ5M } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

idalab seminar #19 - Dr. Thorsten O. Zander

  1. 1. Dr. Thorsten O. Zander Agency for Data Science Machine learning & AI Mathematical modelling Data strategy Towards Neuroadaptivity: How we can connect computers directly to the human mind idalab seminar #19 | June 6th 2019
  2. 2. How we can connect computers directly to the human mind? Dr. Thorsten O. Zander Towards Neuroadaptivity
  3. 3. Brain-Computer Interfaces Communicating to a machine with no need for muscular activity Using a brain sensing mechanism (e.g. EEG), Machine Learning/Signal Processing and Neuroscience
  4. 4. Supporting severely disabled people
  5. 5. BCI for impaired users See: Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., ... & Flor, H. (1999). A spelling device for the paralysed. Nature, 398(6725), 297.
  6. 6. The P300 speller See: Farwell, L. A., & Donchin, E. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and clinical Neurophysiology, 70(6), 510- 523. https://physionet.org/pn4/erpbci/
  7. 7. The P300 speller (video)
  8. 8. An overview of BCI research for non- impaired users
  9. 9. Direct Control / Active BCI Direct control Direct Control Is defined as the intentional directing of commands at the interface of a computer system, which then follows the instructions.
  10. 10. Basketparadigm Steering a cursor by Motor Imagery Zander, T. O., & Krol, L. R. (2017). Team PhyPA: Brain-computer interfacing for everyday human-computer interaction. Periodica Polytechnica Electrical Engineering and Computer Science, 61(2), 209- 216.
  11. 11. Basket-Paradigm (PhyPA lab 2005) R L R L R L R L  Goal of this game is to hit the target field with a ball by using imagined movements 2. Online Session1.Calibration Measurement
  12. 12. BRAINFLIGHT Controlling a flight simulator Zander, T. O., & Krol, L. R. (2017). Team PhyPA: Brain-computer interfacing for everyday human-computer interaction. Periodica Polytechnica Electrical Engineering and Computer Science, 61(2), 209-216.
  13. 13. Direct BCI-control of a flight simulator (Team PhyPA & TU Munich, 2014)
  14. 14. Results
  15. 15. Human-Computer Interaction • Control room photo: ABB Ability™Symphony® Plus power plant automationsystem; Credit: ABB AG. • Mousephoto:Doug Engelbart's Prototype"X-Y positionindicator for a display system", 1964;SRI International / CC BY-SA 3.0 • Keyboard photo: Sholesand Glidden typewriter prototype1873;public domain 2018 1964 Fitts PM (ed) (1951) Human engineering for an effective air navigation and traffic control system. National Research Council, Washington, DC
  16. 16. Direct Control / Active BCI Implicit Control Implicit Control is defined as an automatic state change of a technical system based at least in part on an evaluation of the user’s current state, without any actual commands to that effect being intentionally communicated to the system by the user. Implicit Control / Passive BCI
  17. 17. Intentions Passive Brain-Computer-Interfaces 18/80 Mental Workload Emotions Context-related responses Interpretations Zander, T. O., & Kothe, C. (2011). Towards passive Brain–Computer Interfaces: Applying Brain– Computer Interface technology to Human–Machine Systems in general. Journal of Neural Engineering, 8(2), 025005. …
  18. 18. Example for Neuroadaptive Technology ? ?? Music-Recommendation System
  19. 19. Workload-adaptive interfaces Zander, T. O., & Krol, L. R. (2017). Team PhyPA: Brain-computer interfacing for everyday human-computer interaction. Periodica Polytechnica Electrical Engineering and Computer Science, 61(2), 209-216.
  20. 20. (De-)populating displays according to current workload
  21. 21. M(eye)ndtris Krol, L. R., Freytag, S. C., & Zander, T. O. (2017, November). Meyendtris: A hands- free, multimodal Tetris clone using eye tracking and passive BCI for intuitive neuroadaptive gaming. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (pp. 433-437). ACM.
  22. 22. Neurogaming – M(eye)ndtris 23/72 Dwell for “immediate drop” Dwell to rotate Gaze indicator – block will move there Falling Speed modulated by relaxation of the player Set blocks will be destroyed if player’s EEG shows an error response Eye Tracker Passive BCI
  23. 23. Example: Neurogaming – M(eye)ndtris 24/72
  24. 24. Implicit control over a cursor Zander, T. O., Krol, L. R., Birbaumer, N. P., & Gramann, K. (2016). Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity. Proceedings of the National Academy of Sciences (PNAS), 113(52), 1489814903.
  25. 25. Implicit Cursor Control Stimuli
  26. 26. Implicit Cursor Control Reinforcement Learning: Directional proabilities are adapted after each jump based on BCI output.
  27. 27. Implicit Cursor Control
  28. 28. Implicit Cursor Control Results
  29. 29. Implicit Cursor Control Neurophysiological interpretation - We get responses to both, jumps interpreted as being good and bad - Linear correpsondence between angle of deviation and amplitude - Localized (ICA) at mPFC, showing a realtion to Predictive Coding.  Modelling of higher order cognition through cognitive probing.
  30. 30. Conclusion
  31. 31. 1950-2000 Development of measurements techniques 2000-2010 2013-today Near future Zander, T. O., Lehne, M., Ihme, K., Jatzev, S., Correia, J., Kothe, C., ... & Nijboer, F. (2011). A dry EEG-system for scientific research and brain–computer interfaces. Frontiers in neuroscience, 5, 53. Zander, T.O., Andreessen, L.M., Berg, A., Bleuel, B., Pawlitzki, J., Zawallich, L., Krol, L.R., Gramann, K. (2017), Evaluation of a dry EEG system for application of passive Brain-Computer Interfaces in autonomous driving, Frontiers in neuroscience, 6, 65.
  32. 32. Towards Universal BCIs The machine learning based approach to Brain-Computer Interfaces o Individual calibration that is task specific Goal: Subject-/Task-Independent classification
  33. 33. • Brain-Computer Interfaces provide insight into human brain activity to a technical system. • Machines can learn to react to changes in the cognitive or affective state of their users. • Step by step, an artificial intelligence can learn to understand a human “intuitively”. Conclusion From user-state detection to user modelling
  34. 34. • The resulting models can be used to learn more about human neuroscience. • Neuroadaptive technology can improve our daily life and make our interaction with technology safer, more efficient and more fun. • The machine get’s insight into intimate parts of our brain activity and hence ourselves, potentially without our awareness! Conclusion Impact on our societies
  35. 35. TEAM PhyPA 2019 ` Laurens Lena Juliane Thorsten
  36. 36. Thorsten Zander thorsten@zanderlaboratories.com

×