Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Tactile P300-based
Brain-computer Interface
Accuracy Improvement
201420642 Takumi Kodama
Multimedia Laboratory, Departme...
1: Introduction - What’s BCI?
● Brain Computer Interface (BCI)
○ Neurotechnology
○ Exploits user intention ONLY using brai...
1: Introduction - ALS Patiens
● Amyotrophic lateral sclerosis (ALS) patients
○ Have difficulty to move their muscle by the...
● Tactile (Touch) P300-based BCI paradigm
○ P300 responses were evoked by external (touch) stimuli
○ Predict user’s intent...
● Full-body Tactile P300-based BCI (fbBCI) [1]
○ Applied six vibrotactile stimulus patterns to user’s back
○ User can use ...
● Full-body Tactile P300-based BCI (fbBCI)
1: Introduction - Demonstration
6
https://www.youtube.com/watch?v=sn6OEBBKsPQ
● P300 responses were confirmed (> 4 μV) in each channel
1: Introduction - fbBCI results (1)
7
Target
Non-Target
● Problem: Not high Online classification accuracies
○ SWLDA : 53.67 % (10 users average)
● not enough online results to a...
● Improve the fbBCI classification accuracies
● Affirm the potential validity of proposed fbBCI
modality
1: Introduction -...
● Test several signal preprocessing combinations ①
○ Downsampling
○ Epoch averaging
● Classify with three different machin...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6)...
2: Method - Evaluation phase
● How to predict user’s intention with trained classifier?
○ Correct example
17
ω1 : Target
C...
2: Method - Evaluation phase
18
ω1 : Target
Classifier
(2cls)
1 × 10
35.1 %
Target 6
Session: 6/6
ω1 : Target
Classifier
(...
2: Method - Evaluation phase
Target 11/6
5
Target 2
Target 3
3
5
● Calculate stimulus pattern classification accuracy
○ Ho...
2: Method - Signal Acquisition
20
● Event related potential (ERP) interval
○ Captures 800 ms long after vibrotactile stimu...
2: Method - Signal Preprocessing(2)
21
● Downsampling
○ ERPs were decimated by
2 (256 Hz), 4 (128 Hz), 8 (64 Hz),
16 (32 H...
22
● Epoch averaging
○ ERPs were averaged using 2, 5
or 10 ERPs, or no averaging
○ To reject background noise
ne = 1 ne = ...
● Concatenating feature vectors
2: Method - Feature Extraction
ex.)
fs = 256 [Hz] (nd = 2)
Vlength = ceil(256・0.8) = 205
2...
● Machine learning methods
○ SWLDA
○ Linear SVM
… K(u,v’) = u v’
○ Non-linear SVM (Gaussian)
… K(u,v’) = exp(-γ||u-v|| )
γ...
● Training the classifier
2: Method - Classification (2)
25
VT1
VT2
VlengthALL VlengthALL
VN1
VN2
VTmax
・
・
・
・
・
・
VNmax
...
● Training the classifier
2: Method - Classification (2)
26
VT1
VT2
VlengthALL VlengthALL
VN1
VN2
Classifier (2cls)
VTmax
...
● Evaluation with trained classifier
○ Same nd and ne were applied
2: Method - Classification (3)
27
VT1
VlengthALL
・
・
VT...
● SWLDA classification accuracies
○ BEST: 56.33 % (nd = 4, ne = 1)
3: Results - SWLDA
28
Number of epoch averaging (ne)
Si...
● Linear SVM classification accuracies
○ BEST: 57.33 % (nd = 16, ne = 10)
3: Results - Linear SVM
29
Signal decimation (nd...
● Non-linear SVM classification accuracies
○ BEST: 59.83 % (nd = 4, ne = 1)
3: Results - Non-linear SVM
30
Signal decimati...
4: Discussion and conclusions
31
● fbBCI classification accuracy has been improved
○ Both nd and ne combinations were test...
● [1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct
Brain-robot Control. In: Proceedings of ...
● [5] Shimizu K, Kodama T, Jurica P, Cichocki A, Rutkowski TM. Tactile BCI
Paradigms for Robots’ Control. In: 6th Conferen...
34
Thanks for the listening!
● Bandpass filtering
○ Set at 0.1 ~ 30 Hz (fixed)
○ To limit noise of exciters (40 Hz)
2: Method - Signal Preprocessing(1)...
Upcoming SlideShare
Loading in …5
×

A Tactile P300-based Brain-Computer Interface Accuracy Improvement

175 views

Published on

Kodama T, Makino S, Rutkowski TM. A Tactile P300-based Brain-Computer Interface Accuracy Improvement. In: The Mid Term Presentation for Master's Degree, Department of Computer Science - Graduate School of Systems and Information Engineering, University of Tsukuba, Tsukuba, Japan, 2016.

Published in: Science
  • Be the first to comment

  • Be the first to like this

A Tactile P300-based Brain-Computer Interface Accuracy Improvement

  1. 1. A Tactile P300-based Brain-computer Interface Accuracy Improvement 201420642 Takumi Kodama Multimedia Laboratory, Department of Computer Science Supervisors: Shoji Makino and Tomasz M. Rutkowski* 1 *The university of Tokyo, Tokyo, Japan @ Midterm Presentation for Master’s Degree on July, 2016
  2. 2. 1: Introduction - What’s BCI? ● Brain Computer Interface (BCI) ○ Neurotechnology ○ Exploits user intention ONLY using brain waves 2
  3. 3. 1: Introduction - ALS Patiens ● Amyotrophic lateral sclerosis (ALS) patients ○ Have difficulty to move their muscle by themselves ○ BCI could be a communicating tool for ALS patients 3 http://www.businessinsider.com/an-eye-tracking-interface-helps-als-patients-use-computers-2015-9 Dr. Hawkins
  4. 4. ● Tactile (Touch) P300-based BCI paradigm ○ P300 responses were evoked by external (touch) stimuli ○ Predict user’s intentions with finding P300 responses 1: Introduction - Research Approach 41, Stimulate touch sensories 2, Classify brain response A B A B 3, Predict user intention 92.0% 43.3% A B Target Non-Target P300 brainwave response
  5. 5. ● Full-body Tactile P300-based BCI (fbBCI) [1] ○ Applied six vibrotactile stimulus patterns to user’s back ○ User can use fbBCI with their body laying down 1: Introduction - Our Method 5[1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct Brain-robot Control. In: Proceedings of the Sixth International Brain-Computer Interface Meeting. Asilomar Conference Center, Pacific Grove, CA USA: Verlag der Technischen Universitaet Graz; 2016. p. 68.
  6. 6. ● Full-body Tactile P300-based BCI (fbBCI) 1: Introduction - Demonstration 6 https://www.youtube.com/watch?v=sn6OEBBKsPQ
  7. 7. ● P300 responses were confirmed (> 4 μV) in each channel 1: Introduction - fbBCI results (1) 7 Target Non-Target
  8. 8. ● Problem: Not high Online classification accuracies ○ SWLDA : 53.67 % (10 users average) ● not enough online results to assert fbBCI validity ... 1: Introduction - fbBCI results (2) 8
  9. 9. ● Improve the fbBCI classification accuracies ● Affirm the potential validity of proposed fbBCI modality 1: Introduction - Research Purpose 9
  10. 10. ● Test several signal preprocessing combinations ① ○ Downsampling ○ Epoch averaging ● Classify with three different machine learning methods ② ○ SWLDA ○ Linear SVM ○ Non-linear SVM (Gaussian kernel) 2: Method - Conditions 10 CommandBrainwave ① ②
  11. 11. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 11 ω1 : Target Classifier (2cls) Target 1 1 2 34 5 6 1 6 5 4 3 2 ω2 : Non-Target × 10 × 10 × 10 × 10 × 10 × 10 Session: 1/6
  12. 12. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 12 ω1 : Target Classifier (2cls) Target 2 1 2 34 5 6 1 × 10 2 × 10 Session: 2/6 6 5 4 3 2 ω2 : Non-Target × 20 × 20 × 20 × 20 × 10 1 × 10
  13. 13. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 13 ω1 : Target Classifier (2cls) Target 3 1 2 34 5 6 ω2 : Non-Target Session: 3/6 1 × 10 2 × 10 6 5 4 3 2 × 30 × 30 × 30 × 20 × 20 1 × 20 3 × 10
  14. 14. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 14 ω1 : Target Classifier (2cls) Target 4 1 2 34 5 6 ω2 : Non-Target Session: 4/6 1 × 10 2 × 10 6 5 4 3 2 × 40 × 40 × 30 × 30 × 30 1 × 30 3 × 10 4 × 10
  15. 15. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 15 ω1 : Target Classifier (2cls) Target 5 1 2 34 5 6 ω2 : Non-Target Session: 5/6 1 × 10 2 × 10 6 5 4 3 2 × 50 × 40 × 40 × 40 × 40 1 × 40 3 × 10 4 × 10 5 × 10
  16. 16. ● How to train the P300-based BCI classifier? ○ Each stimulus pattern were given 10 times randomly ○ Altogether 360 (60×6) times for a training classifier 2: Method - Training phase 16 ω1 : Target Classifier (2cls) Target 6 1 2 34 5 6 ω2 : Non-Target Session: 6/6 1 × 10 2 × 10 6 5 4 3 2 × 50 × 50 × 50 × 50 × 50 1 × 50 3 × 10 4 × 10 5 × 10 6 × 10 60 300
  17. 17. 2: Method - Evaluation phase ● How to predict user’s intention with trained classifier? ○ Correct example 17 ω1 : Target Classifier (2cls) 1 × 10 72.6 % Target 1 Session: 1/6 ω1 : Target Classifier (2cls) 2 × 10 24.4 % ω1 : Target Classifier (2cls) 3 × 10 56.3 % ω1 : Target Classifier (2cls) 4 × 10 44.1 % ω1 : Target Classifier (2cls) 5 × 10 62.9 % ω1 : Target Classifier (2cls) 6 × 10 39.8 % 1 2 34 5 6
  18. 18. 2: Method - Evaluation phase 18 ω1 : Target Classifier (2cls) 1 × 10 35.1 % Target 6 Session: 6/6 ω1 : Target Classifier (2cls) 2 × 10 48.1 % ω1 : Target Classifier (2cls) 3 × 10 69.2 % ω1 : Target Classifier (2cls) 4 × 10 54.3 % ω1 : Target Classifier (2cls) 5 × 10 50.9 % ω1 : Target Classifier (2cls) 6 × 10 64.3 % 1 2 34 5 6 ● How to predict user’s intention with trained classifier? ○ Wrong example
  19. 19. 2: Method - Evaluation phase Target 11/6 5 Target 2 Target 3 3 5 ● Calculate stimulus pattern classification accuracy ○ How many sessions could the user classify targets? Target 4 Target 5 Target 6 2 4 Result 1 Session 2/6 3/6 4/6 5/6 6/6 1 Trial Classification accuracy rate: 4/6 = 0.667 ⇒ 66.7 % Correct Correct Wrong Correct Correct Wrong Target Status
  20. 20. 2: Method - Signal Acquisition 20 ● Event related potential (ERP) interval ○ Captures 800 ms long after vibrotactile stimulus onsets ○ will be converted to feature vectors with their potentials ex.) fs = 512 [Hz] ERPinterval = 800 [ms] = 0.8 [sec] Vlength = ceil(512・0.8) = 410 Vlength VCh○○ p[0] … p[Vlength - 1] Vlength = ceil( fs・ERPinterval) where fs [Hz] , ERPinterval [sec] Ch○○
  21. 21. 2: Method - Signal Preprocessing(2) 21 ● Downsampling ○ ERPs were decimated by 2 (256 Hz), 4 (128 Hz), 8 (64 Hz), 16 (32 Hz) or kept intact (512 Hz) ○ To reduce vector length Vlength nd = 2 (256 Hz) nd = 16 (32 Hz) Ch○○ Ch○○
  22. 22. 22 ● Epoch averaging ○ ERPs were averaged using 2, 5 or 10 ERPs, or no averaging ○ To reject background noise ne = 1 ne = 10 Ch○○ Ch○○ 2: Method - Signal Preprocessing(3)
  23. 23. ● Concatenating feature vectors 2: Method - Feature Extraction ex.) fs = 256 [Hz] (nd = 2) Vlength = ceil(256・0.8) = 205 23 … Vlength VCz … Vlength VPz … Vlength VCP6 … … … … ……V ex.) VlengthALL = Vlength・8 = 205・8 = 1640 VlengthALL … Ch1 Ch2 Ch8
  24. 24. ● Machine learning methods ○ SWLDA ○ Linear SVM … K(u,v’) = u v’ ○ Non-linear SVM (Gaussian) … K(u,v’) = exp(-γ||u-v|| ) γ = 1/VlengthALL , c = 1 2: Method - Classification (1) 24 T 2
  25. 25. ● Training the classifier 2: Method - Classification (2) 25 VT1 VT2 VlengthALL VlengthALL VN1 VN2 VTmax ・ ・ ・ ・ ・ ・ VNmax ・ ・ ・ VTmax = 60 / ne VNmax = 300 / ne Classifier (2cls) Non-TargetTarget
  26. 26. ● Training the classifier 2: Method - Classification (2) 26 VT1 VT2 VlengthALL VlengthALL VN1 VN2 Classifier (2cls) VTmax ・ ・ ・ ・ ・ ・ VNmax VTmax = 60 / ne VNmax = 60 / ne Random choose as many as Tmax } Non-TargetTarget
  27. 27. ● Evaluation with trained classifier ○ Same nd and ne were applied 2: Method - Classification (3) 27 VT1 VlengthALL ・ ・ VTmax = 10 / ne Target? or Non-Target? Classifier (2cls) Test data
  28. 28. ● SWLDA classification accuracies ○ BEST: 56.33 % (nd = 4, ne = 1) 3: Results - SWLDA 28 Number of epoch averaging (ne) Signal decimation (nd)
  29. 29. ● Linear SVM classification accuracies ○ BEST: 57.33 % (nd = 16, ne = 10) 3: Results - Linear SVM 29 Signal decimation (nd) Number of epoch averaging (ne)
  30. 30. ● Non-linear SVM classification accuracies ○ BEST: 59.83 % (nd = 4, ne = 1) 3: Results - Non-linear SVM 30 Signal decimation (nd) Number of epoch averaging (ne)
  31. 31. 4: Discussion and conclusions 31 ● fbBCI classification accuracy has been improved ○ Both nd and ne combinations were tested ○ 53.67 % in previous reported results ⇒ 59.83 % by non-linear SVM (nd = 4, ne = 1) ○ 57.33 % by linear SVM and 56.33 % by SWLDA ● The potential validity of fbBCI modality was reconfirmed ○ Expect to improve a QoL for ALS patients ● However, more analyses would be required ○ Only 10 healthy users of fbBCI paradigm ○ Need higher accuracies for a practical application
  32. 32. ● [1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct Brain-robot Control. In: Proceedings of the Sixth International Brain-Computer Interface Meeting: BCI Past, Present, and Future. Asilomar Conference Center, Pacific Grove, CA USA: Verlag der Technischen Universitaet Graz; 2016. p. 68. ● [2] Kodama T, Shimizu K, Makino S, Rutkowski TM. Tactile Brain–computer Interface Based on Classification of P300 Responses Evoked by Spatial Vibrotactile Stimuli Delivered to the User’s Full Body. In: Asia-Pacific Signal and Information Processing Association, 2016 Annual Summit and Conference (APSIPA ASC 2016). APSIPA. Jeju, Korea: IEEE Press; 2016. p. (submitted). ● [3] Kodama T, Shimizu K, Makino S, Rutkowski TM. Full–body Tactile P300–based Brain–computer Interface Accuracy Refinement. In: Proceedings of the International Conference on Bio-engineering for Smart Technologies (BioSMART 2016). Dubai, UAE: IEEE Press; 2016. p. (submitted). ● [4] Kodama T, Makino S, Rutkowski TM. Toward a QoL improvement of ALS patients: Development of the Full-body P300-based Tactile Brain--Computer Interface. In: Proceedings of The AEARU Young Researchers International Conference (YRIC-2016). University of Tsukuba; 2016. p. (submitted). Publications 32
  33. 33. ● [5] Shimizu K, Kodama T, Jurica P, Cichocki A, Rutkowski TM. Tactile BCI Paradigms for Robots’ Control. In: 6th Conference on Systems Neuroscience and Rehabilitation 2015 (SNR 2015). Tokorozawa, Japan; 2015. p. 28. ● [6] Rutkowski TM, Shimizu K, Kodama T, Jurica P, Cichocki A, Shinoda H. Controlling a Robot with Tactile Brain-computer Interfaces. In: Abstracts of the 38th Annual Meeting of the Japan Neuroscience Society - Neuroscience 2015. BMI/BCI. Kobe, Japan: Japan Neuroscience Society; 2015. p. 2P332. ● [7] Shimizu K, Aminaka D, Kodama T, Nakaizumi C, Jurica P, Cichocki A, et al. Brain-robot Interfaces Using Spatial Tactile and Visual BCI Paradigms - Brains Connecting to the Internet of Things Approach. In: The International Conference on Brain Informatics & Health - Type II Paper: Proceedings 2015. London, UK: Imperial College London; 2015. p. 9-10. ● [8] Rutkowski TM, Shimizu K, Kodama T, Jurica P, Cichocki A. Brain--robot Interfaces Using Spatial Tactile BCI Paradigms - Symbiotic Brain-robot Applications. In: Symbiotic Interaction. vol. 9359 of Lecture Notes in Computer Science. Switzerland: Springer International Publishing; 2015. p. 132-137. Publications (co-author) 33
  34. 34. 34 Thanks for the listening!
  35. 35. ● Bandpass filtering ○ Set at 0.1 ~ 30 Hz (fixed) ○ To limit noise of exciters (40 Hz) 2: Method - Signal Preprocessing(1) 35 Ch○○

×