This document summarizes a technical seminar on using convolutional neural networks for P300 detection in brain-computer interfaces. The seminar covers an introduction to brain-computer interfaces and the P300 signal, describes existing P300 detection systems and the convolutional neural network approach, and presents the network architecture, learning process, evaluation results on two datasets showing improved detection rates over other methods, and conclusions. The seminar demonstrates that the convolutional neural network approach outperforms existing methods for P300 detection, especially with a limited number of electrodes or training epochs.

1. Technical seminar
on
Convolutional Neural Networks
for
P300 Detection with Application to
Brain-Computer Interfaces
Date: 21-03-2012
Presented By:
Deepa D. Shedi
1KS08CS026
8th sem
CSE

2. OVERVIEW
 Introduction
 The P300 Speller Matrix and Detection
 Database
 Existing Systems
 Convolutional Neural Network
 Input Normalization
 Neural Network Topology
 Learning
 Classifiers
 Result for P300 Detection
 Network Analysis
 Character Recognition Rate
 Information Transfer Rate
 Discussion
 Conclusion

3. INTRODUCTION
Brain Computer Interface :
A Brain-Computer Interface (BCI) is a specific type of human-
computer interface that enables the direct communication
between human and computers by analyzing brain
measurements.
Eelectroencephalogram(EEG):
It is a measure of brain’s voltage fluctuations as detected from
scalp electrodes. It captures typical patterns of P300 signals.
P300 :
EPRs are voltage fluctuations in the EEG induced with in the
brain that are time locked to sensory or motor events.
The P300 is positive bump in the ERP named so because it
starts about 300 milliseconds after an event. 1

4. THE P300 SPELLER
MATRIX AND DETECTION
The two classification problems.
 P300 detection.
 Character Recognition.
Fig 1: P300 detection Fig 2 : Character recognition.
2

5. DATABASE
Data set contains a complete record of P300 evoked
potentials from two subjects.
The subject was presented with a matrix of size 6X6.
2 out of 12 intensifications for rows/columns.
The number of samples for both databases and for each
subject is presented.
Table 1: Database Size for Each Subject
3

6. EXISTING SYSTEMS
This section describes some of the best techniques that have been
proposed during the III BCI competition.
 Support Vector Machine(SVM)
 Band-pass Filtering
 Frequency Filtering And Principal Component Analysis (PCA)
 Gradient Boosting Method
 Component Classifier Linear Discriminant Analysis (LDA)
4

7. CONVOLUTIONAL NEURAL
NETWORK
The classifiers that are used for the detection of P300
responses are based on a convolutional neural network
(CNN).
Neural network is a multilayer perceptron (MLP).
Neural network is used for object recognition and
handwriting character recognition.
A classifier based on a CNN seems to be a good approach
for EEG classification
5

8. INPUT NORMALIZATION
Steps in Normalization:
Step 1: Subsampling of
EEG signal to reduce the
size of the data to analyze
and divided by two.
Step 2: Bandpass filtering
of signal to keep
only relevant frequencies.
Normalized Signal :
Fig. 3. Electrode map.
6

9. NEURAL NETWORK
TOPOLOGY
Fig. 4. Neural Network Architecture.
7

10. CONTD.
The network topology is described as follows:
L0: The input layer. Ii,j with 0 ≤ i < Nelec and 0 ≤ j < Nt.
L1: The first hidden layer is composed of Ns maps. We define
L1Mm, the map number m. Each map of L1 has the size Nt.
L2: The second hidden layer is composed of 5Ns maps. Each
map of L2 has six neurons.
L3: The third hidden layer is composed of one map of 100
neurons. This map is fully connected to the different maps of L2.
L4: The output layer. This layer has only one map of two
neurons, which represents the two classes of the problem (P300
and no P300). This layer is fully connected to L3. 8

11. LEARNING
A neuron in the network is defined by n(l, m, j).
This sigmoid function for only one map in the layer:
Convolution of the input signal for first two hidden layers:
The classical sigmoid function is used for the two
last layers:
9

12. LEARNING
σm1(j) represents the scalar product between a set of input
neurons and the weight connections between these neurons
and the neuron number j in the map m in the layer l.
σm1(j) for the four layers.
For L1 :
This layer aims at finding the best electrodes combination for
the classification.
For L2 :
This layer translates subsampling and temporal filters.
10

13. LEARNING
For L3 :
In this layer, each neuron has NsNf +1 input weights. L3
contains 100(5*6*Ns) input connections.
For L4 :
Each neuron of L4 is connected to each neuron of L3.
11

14. LEARNING
Learning rate ϒ for layers L1 and L2 :
Learning rate ϒ for layers L1 and L2 :
The detection of a P300 wave :
12

15. CLASSIFIERS
Single classifiers:
• CNN-1
• CNN-2a
• CNN-2b
• CNN-3
Multiclassifiers :
• MCNN-1
• MCNN-2
• MCNN-3
13

16. RESULT FOR P300 DETECTION
Results of the P300 Detection for Subject A :
Table 2: Results Obtained After the P300 detection of Subject A.
Results of the P300 Detection for Subject B :
14
Table 3: Results Obtained After the P300 detection of Subject B.

17. RESULT FOR P300 DETECTION
Measures for evaluating the quality of results :
Subject B allows getting better results for the classification.
15

18. NETWORK ANALYSIS
Spatial filters obtained with CNN-1 for subject A.
Fig. 5. Spatial filters obtained with CNN-1 for subject A.
Spatial filters obtained with CNN-1 for subject B.
16
Fig.6. Spatial filters obtained with CNN-1 for subject B.

19. NETWORK ANALYSIS
Electrode Ranking :
Table 4: Ranking of Electrodes
Cumulated probabilities of the P300 detection:
The coordinate of the character are defined by :
17

20. CHARACTER RECOGNITION
RATE
Table 5:Character Recognition Rate (in Percent) for the Different Classifiers 18

21. INFORMATION TRANSFER
RATE
Information Transfer Rate :
T is the time needed to recognize one character. T is
defined by :
Best recognition rate when only 10 epochs are used.
19

22. INFORMATION TRANSFER
RATE
Fig. 7. ITR (in bits per minute) in relation to the number of epochs.
20

23. DISCUSSION
The database has two main interests.
• It forces the system to reach the limit of the P300
detection.
• It is an excellent challenge for the machine learning
community.
Steady-state visual evoked potentials (SSVEPs), the user
has to focus on some visual Stimuli.
The interest of convolutional neural networks is double.
• It allows a high performance in the classification.
• They can allow deeper analysis of brain activity.
21

24. DISCUSSION
In the P300 speller, it is possible that the subject may not
have always focused on the expected target.
Table 6:Confusion of Character Recognition
Comparison
Table 7:Comparison of the Recognition Rate and the ITR with Other Results. 22

25. CONCLUSION
This model is based on a convolutional neural network.
It outperforms the best method in two situations:
•When the number of electrodes is restricted to 8.
• when the number of considered epochs is 10.
The recall of the P300 detection is the main feature that
dictates the overall performance of the P300 speller.
The detection of P300 waves remains a very challenging
problem for both the machine learning and neuroscience
communities.
23

26. REFERENCES
C.W. Anderson, S.V. Devulapalli, and E.A. Stolz, “Determining Mental State from
EEG Signals Using Parallel Implementations of Neural Networks,” Proc. IEEE
Workshop Neural Networks for Signal in Processing, pp. 475-483, 1995.
K. Chellapilla, S. Puri, and P.Y. Simard, “High Performance Convolutional Neural
Networks for Document Processing,” Proc.10th Int’l Workshop Frontiers in
Handwriting Recognition, 2006.
B. Blankertz, K.-R. Muller, G. Curio, T.M. Vaughan, G. Schalk, J.R.Wolpaw, A.
Schlogl, C. Neuper, G. Pfurtscheller, T. Hinterberger, M. Schroder, and N. Birbaumer,
“The BCI Competition 2003: Progress and Perspectives in Detection and
Discrimination of EEG Single Trials,” IEEE Trans. Biomedical Eng., vol. 51, no. 6, pp.
1044-1051, June 2004.
D.J. Krusienski, E.W. Sellers, D. McFarland,T.M. Vaughan, and J.R. Wolpaw,
“Toward Enhanced P300 Speller Performance,” J. Neuroscience Methods, vol. 167,
pp. 15-21, 2008.
G. Schalk, D.J. McFarland, T. Hinterberger, N. Birbaumer, and J. Wolpaw, “BCI2000:
A General-Purpose Brain-Computer Interface(BCI) System,” IEEE Trans. Biomedical
Eng., vol. 51, no. 6, pp. 1034-1043, June 2004.

27. Thank You!!
Questions???
27