This document outlines a research project focused on developing a tactile brain-computer interface (BCI) to facilitate communication for ALS patients who cannot move their muscles. The study utilizes a gaming pad that delivers vibrotactile stimuli to various body parts and evaluates user response through psychophysical and EEG experiments to validate the effectiveness of the BCI. Results indicate that the proposed method shows high performance in command identification and clearly evokes the P300 response associated with target stimuli.

1. Spatial Tactile Brain-Computer Interface Paradigm
by Applying Vibration Stimulus to Large Body Areas
201213082 Multimedia Laboratory !
! Takumi Kodama!
Supervisors:!
! Tomasz M. Rutkowski, Takeshi Yamada, Shigeki Miyabe, Shoji Makino

2. Research Background
"2
❖ Healthy and non-disabled people
− Can communicate their intentions with others easily
❖ Amyotrophic lateral sclerosis (ALS) patients
− Can’t move their muscle by themselves
− Thus they can’t communicate their intentions
So they need device not using muscle like “BCI”
・・・

3. About “Brain-Computer Interface”
BCI could be a communicating tool for ALS patients
"3
Brain waves are
captured
Machine
learning estimates
user intention
Thought based
application control
❖ Brain Computer Interface (BCI)
− Control the device based on brain signals (EEG)
❖ To input BCI commands:
− Not necessary to use your body movements

4. ❖ Stimulus-driven BCI:
− Use brain activity only generated in response to external stimuli
❖ Stimulus-driven BCI has five modes (five senses of human):
− Difference is how to evoke P300 response
Type of Brain-Computer Interface:
I use “tactile (touch) BCI” in my research project
"4
Auditory BCI
Gustatory BCI
Olfactory BCI
Tactile BCI
Visual BCI
Stimulus-driven BCI
Imagery BCI
BCI

5. About “P300 response”
❖ P300 response
− Positive signal that occurs 300 ms after receiving a stimulus
P300 response evoked by applying tactile stimulus to human body
"5
: Target
: non-Target

6. Research Approach
❖ Development of a new tactile BCI paradigm
− Propose new communication option by using tactile BCI
"6
Give user tactile stimuli of body parts with larger distances
Tactile stimuliTactile stimuli

7. About a “gaming pad”
❖ Specification of the gaming pad
−Used to deliver tactile stimuli to any body parts
"7
Only the vibrotactile output to stimulate the users is used
Specification
Name “ZEUS VYBE”
Developer Disney Research
Input Audio signal
Output Sound and vibration
Sound Two speakers inside
Vibration 12 vibration motors inside
Input: !
Audio signal
♪ ♫
Output:!
Sound & Vibration
~
~~
~

8. ❖ Define six vibrotactile stimulus positions in gaming pad
− Stimulus positions are located in spatial body parts
"8
Users identify six vibrotactile positions individually
How to deliver the tactile stimulus
No. Pos. name
1 Left shoulder (LS)
2 Right shoulder (RS)
3 Left waist (LW)
4 Right waist (RW)
5 Left leg (LL)
6 Right leg (RL)

9. How to use the gaming pad
❖ Gaming pad is used in “laying-position”
− Because ALS patients are often bedridden
"9
Users lie on the gaming pad and receive vibrotactile stimuli
“Sitting position” “Laying position” ▲Experiment in laying position

10. ❖ The experiments were carried out in two steps
"10
First: Psychophysical Experiment
#1 Psychophysical
Experiment
Purpose:
!
!
!
!
!
Experimental Procedure
! ? !
!
#2 EEG Experiment
!
Purpose:
!
!
!
!
・To check whether the P300 response
could be evoked after vibrotactile stimuli
・To check whether the users could
perceive the tactile stimulation correctly

11. ❖ Flow of the experiment in each sequences
"11
#1 Psychophysical Experiment
3. Users press the
button when feel the
target stimulus
!
!
!
!
!
!
!
press!
1. Present the target
stimulus
!
!
!
!
!
!
Target!
Right Waist
Measure the reaction time from stimulus onset to button press
2. Give six random
stimulus
!
!
!
!
!
3
1
6
5
2
non-Target
non-Target
non-Target
non-Target
non-Target
Target!4

12. #1 Psychophysical Experiment
One Sequence [11sec]
Vibrate!
③
Vibrate!
①
Vibrate!
⑥
Vibrate!
⑤
Vibrate!
④
Vibrate!
②
time [sec]
1. Present Target Stimulus [5sec] 2. Random Stimulus Trial [6sec]
Vibrate!
④
Vibrate
Example
[1sec]
Break Time
[2sec]
6 7 8 9 101 2 3 4 50
Vibration Each Stimulus [1sec × 6]
"12
Present Target
[2sec]
❖ The detail flow of one sequence
press!

13. ❖ Experimental condition
"13
#1 Psychophysical Experiment
Condition Detail
Subjects 7 persons (3 males and 4 females)
Type of stimulus Only tactile (vibration)
Use device Gaming pad “ZEUS VYBE”
Stimulus positions
6 (Left Shoulder,Right Shoulder , Left
Waist, Right Waist, Left Leg, Right Leg)
Number of sequences 120
Stimulus duration 300 ms
Inter-stimulus interval (ISI) 700 ms

14. "14
Next: EEG Experiment
Experimental Procedure
❖ The experiments were carried out in two steps
#1 Psychophysical
Experiment
Purpose:
!
!
!
!
!
! ? !
!
#2 EEG Experiment
!
Purpose:
!
!
!
!
・To check whether the P300 response
could be evoked after vibrotactile stimuli
・To check whether the users could
perceive the tactile stimulation correctly

15. ❖ Flow of the experiment in each sessions
1. Present the target
stimulus
!
!
!
!
!
!
"15
Classify the user’s brain responses in each target stimulus
#2 EEG Experiment
2. Give 60 random
stimulus (10 Targets + 50
non-Targets)
!
!
!
!
!
3. Users pay attention to
the target stimulus
(Generate P300 response)
!
!
!
!
!
！
1
6
5
2
-
-
-
-
6
3
5
-
-
-
-
3
5
2
1
-
-
-
-
…
-
Target!
Right Waist
4
!
4
!
4
!
4
!
4
!
2
3

16. ❖ Experimental condition (additional & changed item)
"16
#2 EEG Experiment
Condition Detail
EEG recording system g.USBamp active EEG electrodes system
Number of channels 16
EEG electrode position
Cz Pz P3 P4 C3 C4 CP5 CP6
P1 P2 POz C1 C2 FC1 FC2 FCz
Number of sessions 5
Target stimulus length 250 ms
Inter-stimulus interval (ISI) 350~370 ms

17. ❖ Confusion matrix of all users’ response
− Correct rate of response for each target stimulus
"17
#1 Psychophysical Experiment Results
Response
Target
1 2 3 4 5 6
123456

18. ❖ Boxplots of all subjects’ response
− Average and distribution of the response time for each target stimulus
"18
#1 Psychophysical Experiment Results
Command
Responsetime[ms]
300 ms
200300400500600700800
Median value
Minimum value
Maximum value
1/4 value
3/4 value
1 2 3 4 5 6

19. ❖ EEG experiment SWLDA the accuracy for each of the 7 subjects
0%
20%
40%
60%
80%
#1 #2 #3 #4 #5 #6 #7Ave.
42.7%
50.0%
73.4%
16.7%
46.7%
43.3%
30.0%
38.9%
"19
#2 EEG Experiment Results
0%
20%
40%
60%
80%
100%
#1 #2 #3 #4 #5 #6 #7 Ave.
76.2%
100%100%
50%
66.7%
83.3%
50%
83.3%
Accuracy[%]
Average accuracy rates Max accuracy rates
Users Users
Chance level:
16.7%

20. ❖ EEG mean result in Cz channels
"20
: Target : non-Target
#2 EEG Experiment Results

21. ❖ EEG mean result in Cz channels
"20
: Target : non-Target
#2 EEG Experiment Results

22. ❖ Information transfer rate (ITR)
− Commonly used to assess the performance of BCI
"21
#2 EEG Experiment Result
R: the number of bits / selection;
N: the number of classes (N=6 in this experiment);
P: the classifier accuracy (each best result in this experiment);
V: the classification speed in selections / minute (V=5.56 in
this experiment);

23. ❖ EEG experiment ITR results of 7 uses
"22
Subject Max accuracy [%] ITR [bit/min]
#1 83.3 4.30
#2 50 1.18
#3 83.3 4.30
#4 66.7 2.48
#5 50 1.18
#6 100 7.18
#7 100 7.18
#2 EEG Experiment Result

24. ❖ The effectiveness of proposed method was proven as follows:
− This approach could be applicable easily for ALS patients
− The BCI command identification performance of each
target stimulus has been very high
− The P300 response appears very clearly in the all tested
electrode channels
"23
Conclusions