Kodama T, Makino S, Rutkowski TM. A Tactile P300-based Brain-Computer Interface Accuracy Improvement. In: The Mid Term Presentation for Master's Degree, Department of Computer Science - Graduate School of Systems and Information Engineering, University of Tsukuba, Tsukuba, Japan, 2016.

1. A Tactile P300-based
Brain-computer Interface
Accuracy Improvement
201420642 Takumi Kodama
Multimedia Laboratory, Department of Computer Science
Supervisors: Shoji Makino and Tomasz M. Rutkowski*
1
*The university of Tokyo, Tokyo, Japan
@ Midterm Presentation for Master’s Degree on July, 2016

2. 1: Introduction - What’s BCI?
● Brain Computer Interface (BCI)
○ Neurotechnology
○ Exploits user intention ONLY using brain waves
2

3. 1: Introduction - ALS Patiens
● Amyotrophic lateral sclerosis (ALS) patients
○ Have difficulty to move their muscle by themselves
○ BCI could be a communicating tool for ALS patients
3
http://www.businessinsider.com/an-eye-tracking-interface-helps-als-patients-use-computers-2015-9
Dr. Hawkins

4. ● Tactile (Touch) P300-based BCI paradigm
○ P300 responses were evoked by external (touch) stimuli
○ Predict user’s intentions with finding P300 responses
1: Introduction - Research Approach
41, Stimulate touch sensories 2, Classify brain response
A
B
A
B
3, Predict user intention
92.0% 43.3%
A B
Target
Non-Target
P300 brainwave response

5. ● Full-body Tactile P300-based BCI (fbBCI) [1]
○ Applied six vibrotactile stimulus patterns to user’s back
○ User can use fbBCI with their body laying down
1: Introduction - Our Method
5[1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct Brain-robot Control. In: Proceedings of the Sixth International
Brain-Computer Interface Meeting. Asilomar Conference Center, Pacific Grove, CA USA: Verlag der Technischen Universitaet Graz; 2016. p. 68.

6. ● Full-body Tactile P300-based BCI (fbBCI)
1: Introduction - Demonstration
6
https://www.youtube.com/watch?v=sn6OEBBKsPQ

7. ● P300 responses were confirmed (> 4 μV) in each channel
1: Introduction - fbBCI results (1)
7
Target
Non-Target

8. ● Problem: Not high Online classification accuracies
○ SWLDA : 53.67 % (10 users average)
● not enough online results to assert fbBCI validity ...
1: Introduction - fbBCI results (2)
8

9. ● Improve the fbBCI classification accuracies
● Affirm the potential validity of proposed fbBCI
modality
1: Introduction - Research Purpose
9

10. ● Test several signal preprocessing combinations ①
○ Downsampling
○ Epoch averaging
● Classify with three different machine learning methods ②
○ SWLDA
○ Linear SVM
○ Non-linear SVM (Gaussian kernel)
2: Method - Conditions
10
CommandBrainwave
① ②

11. ● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6) times for a training classifier
2: Method - Training phase
11
ω1 : Target
Classifier (2cls)
Target 1
1
2
34
5
6
1
6
5
4
3
2
ω2 : Non-Target
× 10
× 10
× 10
× 10
× 10
× 10
Session: 1/6

12. ● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6) times for a training classifier
2: Method - Training phase
12
ω1 : Target
Classifier (2cls)
Target 2
1
2
34
5
6
1 × 10
2 × 10
Session: 2/6
6
5
4
3
2
ω2 : Non-Target
× 20
× 20
× 20
× 20
× 10
1 × 10

13. ● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6) times for a training classifier
2: Method - Training phase
13
ω1 : Target
Classifier (2cls)
Target 3
1
2
34
5
6 ω2 : Non-Target
Session: 3/6
1 × 10
2 × 10
6
5
4
3
2
× 30
× 30
× 30
× 20
× 20
1 × 20
3 × 10

14. ● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6) times for a training classifier
2: Method - Training phase
14
ω1 : Target
Classifier (2cls)
Target 4
1
2
34
5
6 ω2 : Non-Target
Session: 4/6
1 × 10
2 × 10
6
5
4
3
2
× 40
× 40
× 30
× 30
× 30
1 × 30
3 × 10
4 × 10

15. ● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6) times for a training classifier
2: Method - Training phase
15
ω1 : Target
Classifier (2cls)
Target 5
1
2
34
5
6 ω2 : Non-Target
Session: 5/6
1 × 10
2 × 10
6
5
4
3
2
× 50
× 40
× 40
× 40
× 40
1 × 40
3 × 10
4 × 10
5 × 10

16. ● How to train the P300-based BCI classifier?
○ Each stimulus pattern were given 10 times randomly
○ Altogether 360 (60×6) times for a training classifier
2: Method - Training phase
16
ω1 : Target
Classifier (2cls)
Target 6
1
2
34
5
6 ω2 : Non-Target
Session: 6/6
1 × 10
2 × 10
6
5
4
3
2
× 50
× 50
× 50
× 50
× 50
1 × 50
3 × 10
4 × 10
5 × 10
6 × 10
60 300

17. 2: Method - Evaluation phase
● How to predict user’s intention with trained classifier?
○ Correct example
17
ω1 : Target
Classifier
(2cls)
1 × 10
72.6 %
Target 1
Session: 1/6
ω1 : Target
Classifier
(2cls)
2 × 10
24.4 %
ω1 : Target
Classifier
(2cls)
3 × 10
56.3 %
ω1 : Target
Classifier
(2cls)
4 × 10
44.1 %
ω1 : Target
Classifier
(2cls)
5 × 10
62.9 %
ω1 : Target
Classifier
(2cls)
6 × 10
39.8 %
1
2
34
5
6

18. 2: Method - Evaluation phase
18
ω1 : Target
Classifier
(2cls)
1 × 10
35.1 %
Target 6
Session: 6/6
ω1 : Target
Classifier
(2cls)
2 × 10
48.1 %
ω1 : Target
Classifier
(2cls)
3 × 10
69.2 %
ω1 : Target
Classifier
(2cls)
4 × 10
54.3 %
ω1 : Target
Classifier
(2cls)
5 × 10
50.9 %
ω1 : Target
Classifier
(2cls)
6 × 10
64.3 %
1
2
34
5
6
● How to predict user’s intention with trained classifier?
○ Wrong example

19. 2: Method - Evaluation phase
Target 11/6
5
Target 2
Target 3
3
5
● Calculate stimulus pattern classification accuracy
○ How many sessions could the user classify targets?
Target 4
Target 5
Target 6
2
4
Result
1
Session
2/6
3/6
4/6
5/6
6/6
1 Trial
Classification accuracy rate:
4/6 = 0.667
⇒ 66.7 %
Correct
Correct
Wrong
Correct
Correct
Wrong
Target Status

20. 2: Method - Signal Acquisition
20
● Event related potential (ERP) interval
○ Captures 800 ms long after vibrotactile stimulus onsets
○ will be converted to feature vectors with their potentials
ex.)
fs = 512 [Hz]
ERPinterval = 800 [ms] = 0.8 [sec]
Vlength = ceil(512・0.8) = 410
Vlength
VCh○○
p[0]
…
p[Vlength - 1]
Vlength = ceil( fs・ERPinterval)
where fs [Hz] , ERPinterval [sec]
Ch○○

21. 2: Method - Signal Preprocessing(2)
21
● Downsampling
○ ERPs were decimated by
2 (256 Hz), 4 (128 Hz), 8 (64 Hz),
16 (32 Hz) or kept intact (512 Hz)
○ To reduce vector length Vlength
nd = 2 (256 Hz) nd = 16 (32 Hz)
Ch○○ Ch○○

22. 22
● Epoch averaging
○ ERPs were averaged using 2, 5
or 10 ERPs, or no averaging
○ To reject background noise
ne = 1 ne = 10
Ch○○ Ch○○
2: Method - Signal Preprocessing(3)

23. ● Concatenating feature vectors
2: Method - Feature Extraction
ex.)
fs = 256 [Hz] (nd = 2)
Vlength = ceil(256・0.8) = 205
23
…
Vlength
VCz …
Vlength
VPz …
Vlength
VCP6
…
…
… … ……V
ex.) VlengthALL = Vlength・8 = 205・8 = 1640
VlengthALL
…
Ch1 Ch2 Ch8

24. ● Machine learning methods
○ SWLDA
○ Linear SVM
… K(u,v’) = u v’
○ Non-linear SVM (Gaussian)
… K(u,v’) = exp(-γ||u-v|| )
γ = 1/VlengthALL , c = 1
2: Method - Classification (1)
24
T
2

25. ● Training the classifier
2: Method - Classification (2)
25
VT1
VT2
VlengthALL VlengthALL
VN1
VN2
VTmax
・
・
・
・
・
・
VNmax
・
・
・
VTmax = 60 / ne VNmax = 300 / ne
Classifier (2cls)
Non-TargetTarget

26. ● Training the classifier
2: Method - Classification (2)
26
VT1
VT2
VlengthALL VlengthALL
VN1
VN2
Classifier (2cls)
VTmax
・
・
・
・
・
・
VNmax
VTmax = 60 / ne VNmax = 60 / ne
Random choose
as many as Tmax
}
Non-TargetTarget

27. ● Evaluation with trained classifier
○ Same nd and ne were applied
2: Method - Classification (3)
27
VT1
VlengthALL
・
・
VTmax = 10 / ne
Target? or
Non-Target? Classifier (2cls)
Test data

28. ● SWLDA classification accuracies
○ BEST: 56.33 % (nd = 4, ne = 1)
3: Results - SWLDA
28
Number of epoch averaging (ne)
Signal decimation (nd)

29. ● Linear SVM classification accuracies
○ BEST: 57.33 % (nd = 16, ne = 10)
3: Results - Linear SVM
29
Signal decimation (nd)
Number of epoch averaging (ne)

30. ● Non-linear SVM classification accuracies
○ BEST: 59.83 % (nd = 4, ne = 1)
3: Results - Non-linear SVM
30
Signal decimation (nd)
Number of epoch averaging (ne)

31. 4: Discussion and conclusions
31
● fbBCI classification accuracy has been improved
○ Both nd and ne combinations were tested
○ 53.67 % in previous reported results
⇒ 59.83 % by non-linear SVM (nd = 4, ne = 1)
○ 57.33 % by linear SVM and 56.33 % by SWLDA
● The potential validity of fbBCI modality was reconfirmed
○ Expect to improve a QoL for ALS patients
● However, more analyses would be required
○ Only 10 healthy users of fbBCI paradigm
○ Need higher accuracies for a practical application

32. ● [1] Kodama T, Shimizu K, Rutkowski TM. Full Body Spatial Tactile BCI for Direct
Brain-robot Control. In: Proceedings of the Sixth International Brain-Computer
Interface Meeting: BCI Past, Present, and Future. Asilomar Conference Center,
Pacific Grove, CA USA: Verlag der Technischen Universitaet Graz; 2016. p. 68.
● [2] Kodama T, Shimizu K, Makino S, Rutkowski TM. Tactile Brain–computer
Interface Based on Classification of P300 Responses Evoked by Spatial Vibrotactile
Stimuli Delivered to the User’s Full Body. In: Asia-Pacific Signal and Information
Processing Association, 2016 Annual Summit and Conference (APSIPA ASC 2016).
APSIPA. Jeju, Korea: IEEE Press; 2016. p. (submitted).
● [3] Kodama T, Shimizu K, Makino S, Rutkowski TM. Full–body Tactile P300–based
Brain–computer Interface Accuracy Refinement. In: Proceedings of the International
Conference on Bio-engineering for Smart Technologies (BioSMART 2016). Dubai,
UAE: IEEE Press; 2016. p. (submitted).
● [4] Kodama T, Makino S, Rutkowski TM. Toward a QoL improvement of ALS
patients: Development of the Full-body P300-based Tactile Brain--Computer
Interface. In: Proceedings of The AEARU Young Researchers International
Conference (YRIC-2016). University of Tsukuba; 2016. p. (submitted).
Publications
32

33. ● [5] Shimizu K, Kodama T, Jurica P, Cichocki A, Rutkowski TM. Tactile BCI
Paradigms for Robots’ Control. In: 6th Conference on Systems Neuroscience and
Rehabilitation 2015 (SNR 2015). Tokorozawa, Japan; 2015. p. 28.
● [6] Rutkowski TM, Shimizu K, Kodama T, Jurica P, Cichocki A, Shinoda H.
Controlling a Robot with Tactile Brain-computer Interfaces. In: Abstracts of the 38th
Annual Meeting of the Japan Neuroscience Society - Neuroscience 2015. BMI/BCI.
Kobe, Japan: Japan Neuroscience Society; 2015. p. 2P332.
● [7] Shimizu K, Aminaka D, Kodama T, Nakaizumi C, Jurica P, Cichocki A, et al.
Brain-robot Interfaces Using Spatial Tactile and Visual BCI Paradigms - Brains
Connecting to the Internet of Things Approach. In: The International Conference on
Brain Informatics & Health - Type II Paper: Proceedings 2015. London, UK: Imperial
College London; 2015. p. 9-10.
● [8] Rutkowski TM, Shimizu K, Kodama T, Jurica P, Cichocki A. Brain--robot
Interfaces Using Spatial Tactile BCI Paradigms - Symbiotic Brain-robot Applications.
In: Symbiotic Interaction. vol. 9359 of Lecture Notes in Computer Science.
Switzerland: Springer International Publishing; 2015. p. 132-137.
Publications (co-author)
33

34. 34
Thanks for the listening!

35. ● Bandpass filtering
○ Set at 0.1 ~ 30 Hz (fixed)
○ To limit noise of exciters (40 Hz)
2: Method - Signal Preprocessing(1)
35
Ch○○