Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Takumi Kodama, Shoji Makino and Tomasz M. Rutkowski*
Life Science Center of TARA, University of Tsukuba
1
*The University ...
1: Introduction - What’s BCI?
● Brain Computer Interface (BCI)
○ Neurotechnology
○ Exploits user intention ONLY using brai...
1: Introduction - ALS Patients
● Amyotrophic lateral sclerosis (ALS) patients
○ Have difficulty to move their muscle by th...
● Tactile (Touch) P300-based BCI paradigm
○ Predict user’s intentions with finding P300 responses
○ P300 responses are evo...
● Development of a new tactile BCI paradigm
● Propose new communication option toward ALS
patients for improving their Qua...
● Full-body Tactile P300-based BCI (fbBCI) [1]
○ Applied six vibrotactile stimulus patterns to user’s back
○ User can use ...
● Full-body Tactile P300-based BCI (fbBCI)
2: Method - Demonstration
7
https://www.youtube.com/watch?v=sn6OEBBKsPQ
2: Method - fbBCI specification
8
Subject
Robot
Brain waves
Commands
2: Method - Signal Acquisition
9
● Event related potential (ERP) interval
○ captures 800 ms long after vibrotactile stimul...
2: Method - Signal Processing
10
● Bandpass Filtering & Downsampling
Ch○○
Before:
After:
fs = 512 [Hz]
fs’ = 128 [Hz]
● Concatenating feature vectors
2: Method - Feature Extraction
In fbBCI:
fs’ = 128 [Hz]
Vlength = ceil(128・0.8) = 103
11
…...
● Non-linear SVM (Gaussian Kernel)
2: Method - Classification (1)
12
where γ = 1/VlengthALL (c = 1)
hyperplane
● Training the classifier
2: Method - Classification (2)
13
VT1
VT2
VlengthALL VlengthALL
VN1
VN2
Classifier (2cls)
VTmax
...
● Evaluation with trained classifier
○ P300 responses exist or not?
2: Method - Classification (3)
14
VT1
VlengthALL
・
・
V...
2: Method - Experimental settings
15
Condition Details
Number of users (mean age) 10 (21.9 years old)
Number of trials 1 t...
● P300 responses were confirmed (> 4 μV) in every channel
3: Result - ERP (P300) responses
16
Target
Non-Target
3: Result - Classification accuracy
17
● Mean classification average: 59.83 %
4: Discussion and conclusions
18
● The effectiveness of fbBCI modality was
confirmed
○ Classification accuracy : 59.83 % b...
● [1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for
Direct Brain-robot Control. In: Proceedings of ...
20
Thank you for listening!
● Training the classifier
2: Method - Classification (2)
21
VT1
VT2
VlengthALL VlengthALL
VN1
VN2
VTmax
・
・
・
・
・
・
VNmax
...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
2: Method - Evaluation phase
● How to predict user’s intention with trained classifier?
○ Correct example
28
ω1 : Target
C...
2: Method - Evaluation phase
29
ω1 : Target
Classifier
(2cls)
1 × 10
35.1 %
Target 6
Session: 6/6
ω1 : Target
Classifier
(...
2: Method - Evaluation phase
Target 11/6
5
Target 2
Target 3
3
5
● Calculate stimulus pattern classification accuracy
○ Ho...
Upcoming SlideShare
Loading in …5
×

Toward a QoL improvement of ALS patients: Development of the Full-body P300-based Tactile Brain-Computer Interface

148 views

Published on

Kodama T, Makino S, Rutkowski TM. Toward a QoL improvement of ALS patients: Development of the Full-body P300-based Tactile Brain--Computer Interface. In: Proceedings of 2016 AEARU Young Researchers International Conference (YRIC-2016). University of Tsukuba; 2016. p. 5.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Toward a QoL improvement of ALS patients: Development of the Full-body P300-based Tactile Brain-Computer Interface

  1. 1. Takumi Kodama, Shoji Makino and Tomasz M. Rutkowski* Life Science Center of TARA, University of Tsukuba 1 *The University of Tokyo, Tokyo, Japan Toward a QoL improvement of ALS patients: Development of the Full-Body P300-based Tactile Brain-Computer Interface Toward a QoL improvement of ALS patients: Development of the Full-Body P300-based Tactile Brain-Computer Interface @2016 AEARU Young Researchers International Conference@2016 AEARU Young Researchers International Conference
  2. 2. 1: Introduction - What’s BCI? ● Brain Computer Interface (BCI) ○ Neurotechnology ○ Exploits user intention ONLY using brain waves 2
  3. 3. 1: Introduction - ALS Patients ● Amyotrophic lateral sclerosis (ALS) patients ○ Have difficulty to move their muscle by themselves ○ BCI could be a communicating tool for ALS patients 3 http://www.businessinsider.com/an-eye-tracking-interface-helps-als-patients-use-computers-2015-9 Dr. Hawkins
  4. 4. ● Tactile (Touch) P300-based BCI paradigm ○ Predict user’s intentions with finding P300 responses ○ P300 responses are evoked by touch stimuli 1: Introduction - Research Approach 41, Stimulate touch sensories 2, Classify brain response A B A B 3, Predict user intention 92.0% 43.3% A B Target Non-Target P300 brainwave response
  5. 5. ● Development of a new tactile BCI paradigm ● Propose new communication option toward ALS patients for improving their Quality of Life 1: Introduction - Research Purpose 5
  6. 6. ● Full-body Tactile P300-based BCI (fbBCI) [1] ○ Applied six vibrotactile stimulus patterns to user’s back ○ User can use fbBCI with their body laying down 2: Method - Our Approach 6[1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct Brain-robot Control. In: Proceedings of the Sixth International Brain-Computer Interface Meeting. Asilomar Conference Center, Pacific Grove, CA USA: Verlag der Technischen Universitaet Graz; 2016. p. 68.
  7. 7. ● Full-body Tactile P300-based BCI (fbBCI) 2: Method - Demonstration 7 https://www.youtube.com/watch?v=sn6OEBBKsPQ
  8. 8. 2: Method - fbBCI specification 8 Subject Robot Brain waves Commands
  9. 9. 2: Method - Signal Acquisition 9 ● Event related potential (ERP) interval ○ captures 800 ms long after vibrotactile stimulus onsets ○ will be converted to feature vectors with their potentials In fbBCI (default settings): fs = 512 [Hz] ERPinterval = 800 [ms] = 0.8 [sec] Vlength = ceil(512・0.8) = 410 Vlength VCh○○ p[0] … p[Vlength - 1] Vlength = ceil( fs・ERPinterval) where fs [Hz] , ERPinterval [sec] Ch○○
  10. 10. 2: Method - Signal Processing 10 ● Bandpass Filtering & Downsampling Ch○○ Before: After: fs = 512 [Hz] fs’ = 128 [Hz]
  11. 11. ● Concatenating feature vectors 2: Method - Feature Extraction In fbBCI: fs’ = 128 [Hz] Vlength = ceil(128・0.8) = 103 11 … Vlength VCz … Vlength VPz … Vlength VCP6 … … … … ……V ex.) VlengthALL = Vlength・8 = 103・8 = 824 VlengthALL … Ch1 Ch2 Ch8
  12. 12. ● Non-linear SVM (Gaussian Kernel) 2: Method - Classification (1) 12 where γ = 1/VlengthALL (c = 1) hyperplane
  13. 13. ● Training the classifier 2: Method - Classification (2) 13 VT1 VT2 VlengthALL VlengthALL VN1 VN2 Classifier (2cls) VTmax ・ ・ ・ ・ ・ ・ VNmax VTmax = 60 / ne VNmax = 60 / ne Random choose as many as Tmax } Non-TargetTarget
  14. 14. ● Evaluation with trained classifier ○ P300 responses exist or not? 2: Method - Classification (3) 14 VT1 VlengthALL ・ ・ VTmax = 10 Target? or Non-Target? Classifier (2cls) Test data
  15. 15. 2: Method - Experimental settings 15 Condition Details Number of users (mean age) 10 (21.9 years old) Number of trials 1 training + 5 tests Stimulus frequency of exciters 40 Hz EEG recording system g.USBamp active electrodes EEG system EEG sampling rate 512 Hz Vibration stimulus length 100 ms Inter-stimulus Interval (ISI) 400 ~ 430 ms
  16. 16. ● P300 responses were confirmed (> 4 μV) in every channel 3: Result - ERP (P300) responses 16 Target Non-Target
  17. 17. 3: Result - Classification accuracy 17 ● Mean classification average: 59.83 %
  18. 18. 4: Discussion and conclusions 18 ● The effectiveness of fbBCI modality was confirmed ○ Classification accuracy : 59.83 % by GSVM ○ Expect to improve a QoL for ALS patients ● However, more analyses would be required ○ Only 10 healthy users has tried yet ○ Need higher accuracies for practical applications
  19. 19. ● [1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct Brain-robot Control. In: Proceedings of the Sixth International Brain-Computer Interface Meeting: BCI Past, Present, and Future. Asilomar Conference Center, Pacific Grove, CA USA: Verlag der Technischen Universitaet Graz; 2016. p. 68. ● [2] Kodama T, Shimizu K, Makino S, Rutkowski TM. Tactile Brain–computer Interface Based on Classification of P300 Responses Evoked by Spatial Vibrotactile Stimuli Delivered to the User’s Full Body. In: Asia-Pacific Signal and Information Processing Association, 2016 Annual Summit and Conference (APSIPA ASC 2016). APSIPA. Jeju, Korea: IEEE Press; 2016. p. (accepted, in press). ● [3] Kodama T, Shimizu K, Makino S, Rutkowski TM. Full–body Tactile P300–based Brain–computer Interface Accuracy Refinement. In: Proceedings of the International Conference on Bio-engineering for Smart Technologies (BioSMART 2016). Dubai, UAE: IEEE Press; 2016. p. (accepted, in press). Publications 19
  20. 20. 20 Thank you for listening!
  21. 21. ● Training the classifier 2: Method - Classification (2) 21 VT1 VT2 VlengthALL VlengthALL VN1 VN2 VTmax ・ ・ ・ ・ ・ ・ VNmax ・ ・ ・ VTmax = 60 / ne VNmax = 300 / ne Classifier (2cls) Non-TargetTarget
  22. 22. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 22 ω1 : Target Classifier (2cls) Target 1 1 2 34 5 6 1 6 5 4 3 2 ω2 : Non-Target × 10 × 10 × 10 × 10 × 10 × 10 Session: 1/6
  23. 23. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 23 ω1 : Target Classifier (2cls) Target 2 1 2 34 5 6 1 × 10 2 × 10 Session: 2/6 6 5 4 3 2 ω2 : Non-Target × 20 × 20 × 20 × 20 × 10 1 × 10
  24. 24. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 24 ω1 : Target Classifier (2cls) Target 3 1 2 34 5 6 ω2 : Non-Target Session: 3/6 1 × 10 2 × 10 6 5 4 3 2 × 30 × 30 × 30 × 20 × 20 1 × 20 3 × 10
  25. 25. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 25 ω1 : Target Classifier (2cls) Target 4 1 2 34 5 6 ω2 : Non-Target Session: 4/6 1 × 10 2 × 10 6 5 4 3 2 × 40 × 40 × 30 × 30 × 30 1 × 30 3 × 10 4 × 10
  26. 26. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 26 ω1 : Target Classifier (2cls) Target 5 1 2 34 5 6 ω2 : Non-Target Session: 5/6 1 × 10 2 × 10 6 5 4 3 2 × 50 × 40 × 40 × 40 × 40 1 × 40 3 × 10 4 × 10 5 × 10
  27. 27. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 27 ω1 : Target Classifier (2cls) Target 6 1 2 34 5 6 ω2 : Non-Target Session: 6/6 1 × 10 2 × 10 6 5 4 3 2 × 50 × 50 × 50 × 50 × 50 1 × 50 3 × 10 4 × 10 5 × 10 6 × 10 60 300
  28. 28. 2: Method - Evaluation phase ● How to predict user’s intention with trained classifier? ○ Correct example 28 ω1 : Target Classifier (2cls) 1 × 10 72.6 % Target 1 Session: 1/6 ω1 : Target Classifier (2cls) 2 × 10 24.4 % ω1 : Target Classifier (2cls) 3 × 10 56.3 % ω1 : Target Classifier (2cls) 4 × 10 44.1 % ω1 : Target Classifier (2cls) 5 × 10 62.9 % ω1 : Target Classifier (2cls) 6 × 10 39.8 % 1 2 34 5 6
  29. 29. 2: Method - Evaluation phase 29 ω1 : Target Classifier (2cls) 1 × 10 35.1 % Target 6 Session: 6/6 ω1 : Target Classifier (2cls) 2 × 10 48.1 % ω1 : Target Classifier (2cls) 3 × 10 69.2 % ω1 : Target Classifier (2cls) 4 × 10 54.3 % ω1 : Target Classifier (2cls) 5 × 10 50.9 % ω1 : Target Classifier (2cls) 6 × 10 64.3 % 1 2 34 5 6 ● How to predict user’s intention with trained classifier? ○ Wrong example
  30. 30. 2: Method - Evaluation phase Target 11/6 5 Target 2 Target 3 3 5 ● Calculate stimulus pattern classification accuracy ○ How many sessions could the user classify targets? Target 4 Target 5 Target 6 2 4 Result 1 Session 2/6 3/6 4/6 5/6 6/6 1 Trial Classification accuracy rate: 4/6 = 0.667 ⇒ 66.7 [%] Correct Correct Wrong Correct Correct Wrong Target Status

×