This e-book gives you a sneak peak into how the classification of right hand movements via EEG could contribute to the development of a brain-computer interface. The Azoft R&D department, along with Sergey Alyamkin and Expasoft provide detailed data from research done for the "Grasp-and-Lift EEG Detection" competition organized by Kaggle. You’ll learn why the deep learning algorithms can be effective in various types of signal classifications and how to apply convolutional neural networks for a specific task such as identifying hand motions from EEG recordings. See more details on our website: http://rnd.azoft.com/classification-eeg-signals-brain-computer-interface/

1. Classification of EEG
Signals for Brain-Computer
Interface
www.azoft.com
Software Development

2. Software Development
Introduction
One of the current issues in medical science today is the classification
of signals recorded from the brain via electroencephalography (EEG, which
is an electrophysiological monitoring method to record electrical activity
of the brain). Classification of cerebral cortex signals makes it possible
to diagnose various neurological diseases (i.e. epilepsy), as well as create
a brain-computer interface for managing robotic devices, including
prosthetics for physically disabled patients.
Azoft R&D team, together with Sergey Alyamkin and Expasoft, decided
to research the topic of building a brain-computer interface and participated
in the «Grasp-and-Lift EEG Detection» competition organized by Kaggle.
According to competition rules, participants were given 2 months to identify
and classify various movements of the right hand with the lowest error rate
using EEG.
The Project Task
To develop a model which helps classify various right-hand movements
using EEG.
The Project Plan
1. Studying the Biological Aspects of the Project
2. Data Pre-processing
3. Identification of Brain Signal Appearance
4. Choosing an Appropriate Machine Learning Algorithm
5. Comparing Received Models and Sending the Best Result

3. Implementation
Studying Biological Aspects of the Project
The human brain serves as the center of the nervous system and can be
divided into several levels (Image 1), each of which has its own functions:
1st level — the cortex — responsible for managing sensory and motor
functions, as well as handling complex cognitive processes.
2nd level — basal ganglia — responsible for managing consensual
movements and regulating muscle tone.
3rd level — hippocampus, hypophysis, hypothalamus, callosal gyrus, corpus
amygdaloideum - mainly responsible for managing emotional reactions
and states, and endocrine regulation.
4th level (the lowest) — reticular formation and other structures of brain
stem — responsible for managing vegetative processes. [1]
Software Development
Image 1. The general structure of the brain.

4. Software Development
The image above illustrates the following brain structures: 1) cortex;
2) thalamus; 3) corpus callosum; 4) optic nerve; 5) eye; 6) amygdaloid body;
7) hypothalamus; 8) hypophysis; 9) hippocampus; 10) pons; 11) cerebellum;
12) spinal medulla; 13) medulla oblongata; 14) reticular formation.
As for the anatomic structure, the brain consists of two hemispheres — right
and left.
Each hemisphere includes five lobes of cerebrum (Image 2):
1. frontal lobe;
2. parietal lobe;
3. occipital lobe;
4. temporal lobe;
5. insular lobe, insula.
Image 2. The lobes of cerebrum
All the data (anatomical, physiological, and clinical) proves that the cerebral
cortex plays a leading role in the cerebral organization of physical processes.
The cerebral cortex is most differentiated by structure and function areas
of the brain.
The cerebral cortex is divided into the following elements:
• Paleocortex
• Archicortex
• Mesocortex
• Neocortex

5. Software Development
The neocortex of the human brain is most structurally complex - it takes up
96% of all the hemispherical surface. For this reason, we will primarily look
at this new part of the brain.
All neocortex areas are constructed uniformly. The most typical for a human
is the new six-layered cortex (Image 3), although the layer number is different
in various brain parts. Layers differ from each other in their thickness, neuron
structure, and arrangement.
The cerebral cortex is heterogeneous and even in one hemisphere has
different cell compositions (Image 3). It helps to identify cytoarchitectonic
areas of the cortex - the similarly organized centers.
Сytoarchitectonics is a study that looks at properties of various cerebral
cortex structures, focusing primarily on cellular composition. In particular,
cytoarchitectonics studies distinctive features of various cortex formations
and their general cell structure characteristics: size and shape of cell elements,
their layer distribution, their location density throughout the whole cortex
cross-section and in certain layers, their division to the sublayers, the presence
of certain cell forms in various layers, as well as cell distribution in vertical
direction.
Image 3. The scheme of neuro and cytoarchitectonic structure of several cortex zones
The borders of cytoarchitectonic areas coincide with functionally specific
neocortex areas, which are responsible for various sense organs. Different

6. Software Development
studies have been conducted to broaden our knowledge about
the human brain’s functional areas, for example via direct cortex stimulation
by microelectrodes, positron emission tomography, or other methods using
comparison of local destructions of cortex areas with observable behavior
fluctuations. [2]
Let’s consider sensorimotor centers of the cerebral cortex in detail.
Image 4: Sensorimotor centers of the cerebral cortex of the human brain (according
to scientific studies)
As shown in the image above, the sensorimotor centers of the human brain
are: 1) root of the tongue; 2) larynx; 3) palate; 4) lower jaw; 5) tongue; 6) lower
part of the face; 7) upper part of the face; 8) neck; 9) fingers; 10) hand;
11) arm from shoulder to wrist; 12) shoulder; 13) shoulder blade; 14) chest;
15) abdomen; 16) lower leg; 17) knee; 18) thigh; 19) toes; 20) the big toe;
21) four toes; 22) foot; 23) face; 24) pharynx. As shown in the image above,
the motor area of the cortex is located between the blue and red lines,
whereas the sensorimotor area is between the red and green lines. [3]
Finally, we learned that the main brain activity associated with hand movement
is in the range from 7 to 30 Hz.

7. Software Development
Data Preprocessing
Filtering
The main objects of the study in the EEG signal were eye movement, the
movement of the electrodes, the contraction of muscles of the head and
the heart, and 50-60Hz network interference. Compared to useful signals,
interference caused by movement of eyes, muscles, and electrodes, are
arranged at lower frequencies (from 0.1 Hz to 6 Hz). Therefore, we decided to
use a bandpass filter with the bandwidth of 7 to 30 Hz.
According to the competition, the developed system should be causal. In
other words, you cannot take data from the future in the specific moment
of classification. In addition, following the competition rules, the signal
classification will be precise if it enters the interval ±150 milliseconds, which
equals to ±75 samplings. From there the maximal order of the bandpass filter
might be 75*2 = 150.
It is important to note that the digital filtering contributed the least distortion
in the signal, so we chose a filter with the finite impulse response. In addition,
we used a Kaiser window with beta = 1 for eliminating the Gibbs effect.
Image 5: Amplitude frequency response and phase response characteristics of the derived
filter

8. Software Development
Image 6: Group delay of the derived filter
As is seen from the derived amplitude-response (blue color)
and dynamic-response (red color) characteristics (Image 5), frequencies that
are lower than 7 and higher than 30 strain out, and the phase response
characteristic is linear. Also, the group delay of the filter equals 75 (Image 6)
across the whole frequency line, which satisfies our initial requirements
of the filter.
Downsampling
As for decimation, which is the process of reducing the sampling rate
of a signal, we dropped it from 500 to 62.5 Hz. Since the maximum useful
frequency is 30 Hz, taking into consideration the Nyquist theorem,
the sampling rate must be greater than 60 Hz. During downsampling
a bandpass filter with the bandwidth of 7 to 30 Hz helps to avoid aliasing.
Identification of Brain Signal Appearance
We used several different approaches to identify typical signs of brain signals
for classifications applicable to machine learning. This was necessary
in order to reduce the amount of input data, exclude redundant information,
and increase the accuracy of brain signal classification. The testing methods
included principal component analysis, electrode selection from the biological
viewpoint, and wavelet transform.
First of all, we decided to try one of the most popular methods of identifying
typical signs - principal component analysis.

9. Software Development
Principal Component Analysis
Principal component analysis has the following main idea related
to the project task: the same signal can come from the cortex to several
different electrodes (Image 7).
Image 7: Mixing signals on the way from the cortex to electrodes located on the head
If we assume that signals are linear mixing, then the signal vector
from detectors:
where – column vector 32x1 – signals from detectors;
– column vector 10x1 – signals from the cortex;
– matrix 32x10 – unknown.
We can transform this correspondence to:
where – Moore-Penrose pseudoinverse.
If we know a matrix , we can set to the classificator entry not the given raw
data but the improved data .

10. Software Development
We tried to calculate a matrix using SVD (singular value decomposition).
All 32 signals band together to the signal matrix X. Every signal is a matrix
column. Matrix X has the size Nx32. This matrix went through a singular value
decomposition
,
where – unitary matrixes, - diagonal matrix with all the elements
being positive and sorted out in descending order. has a size Nx32,
– 32x32, – 32x32.
The first few matrix columns have to be the same deep signals, which
appear in the brain when a patient begins to move a hand. The first few
columns are the main components. Usually, artifacts are located in the last
matrix columns.
When we implemented data analysis with SVD, we didn’t find a correlation
between signals from the first matrix columns and the moments when a
patient began to move a hand. For this reason, we had to stop using principal
component analysis. It was assumed that deep brain signals are mutually
orthogonal in calculating compulsory matrix by means of SVD – singular
value decomposition. In practice, it turned out that they were not mutually
orthogonal. Besides, in the experiment we had 32 detectors, which is not
enough for good SVD. It would be better to use ICA (independent component
analysis) for nonorthogonal signals. Unfortunately, we ran out of time and
didn’t try ICA.

11. Software Development
Electrode Selection From the Biological Viewpoint
When choosing electrodes, our decision was based on their spatial
arrangement. In the following picture, the electrodes that provided data
are labeled with the numbers from 1 to 32.
Image 8: Schematic illustration of electrodes on the surface of the head
There is a phenomenon called hemispheric asymmetry. Along with other
meanings, it implies that the left hemisphere is responsible for the right side
of the body, with the right hemisphere responsible for the left side of the
body. Test subjects moved their right hand, which means a signal appeared
in the left hemisphere. Based on this and according to Image 4, we chose the
following electrodes: FC5, FC1, T7, C3, Cz, CP5, and CP1.

12. Software Development
Wavelet Transform
Having chosen the required electrodes, we decided to focus on wavelet
transform as our key method of brain signal identification.
Wavelet transform of one-dimensional signals is widespread in different areas
of science - from seismic data processing to speech processing. This approach
has a range of advantages compared to spectral analysis in the case
of unsteady pulse signals.
The choice of a specific wavelet kind and type depends primarily on analyzed
signals and analysis tasks. Today there are no clear methods which could help
choose the optimal wavelet for a particular task. That’s why an expert often
has to decide what wavelet function to use.
There is an implied principle of wavelet choice: the basic wavelet functions
look has to be similar to the typical manipulated signal.
Gauss wavelet was chosen as the basic function gaus2:
Image 9. Gauss wavelet gaus2

13. Software Development
Image 10. The result of wavelet transform for the EEG signal
Using this approach, we can analyze the fine structure of signals by means of
their simultaneous localization based on both frequency and time.
Choosing an Appropriate Machine Learning Algorithm
When choosing the best machine learning algorithm for the task, we decided
to apply the familiar convolutional neural networks (CNN), which are a part of
deep learning algorithms.
Сonvolutional Neural Networks
Сonvolutional neural networks are among the most commonly used neural
network types, especially for tasks that involve two-dimensional signals (i.e.
images). Although, these networks can also work with regular signals.

14. Software Development
Image 11: The example of CNN work with a one-dimensional signal
We set 32 one-dimensional signals to the CNN entry over a particular period
of time.
Up to some experience, we identified that 4096 samples (up to downsampling)
yield the lowest error rate in the final model.
Finally, we got the following model (Image 12):

15. Software Development
Image 12: The derived model: CL is the convolutional layer, MP is map pooling, FC is fully
connected layer, FM is feature masks.
The derived model enabled us to calculate the area under ROC curve,
which equals 0.91983.
The increase in the number of masks, convolutional layers and changes
of kernel didn’t bring an improvement. The area under the ROC curve varies
between ~0.88 and ~0.91.
Random Forest
The second experimental algorithm was the so-called “random forest”,
since it is one of the most popular methods for classification today.
This algorithm consists of a set of decisions trees, and each of them gets its
own dataset from the total dataset.
For better understanding of how the algorithm works let’s consider
the example of the decision tree for an abstract task with two features
(Image 13).
Initially elements from two different classes are on the plane. We divide
them in a way to separate one class from the other by one horizontal

16. Software Development
or vertical line. Next we divide the derived planes in the same way before we
get the probability of dominant class higher than a certain limit or the unit
depth comes to a maximum, or the element quantity becomes lower than
a certain limit.
Image 13: The example of the decision tree for abstract task with two features
The decision tree is not a very stable and reliable algorithm. The algorithm’s
high accuracy of classification and flexibility are attainable due to a set
of decision trees.
We ran out of time to try a combination of “wavelet transform + random
forest” because wavelet transform is very time-consuming.
Recurrent Neural Network
Recurrent neural networks (RNN) are the neural networks with feedback
connections. Biological neural networks are recurrent. They have memory.
Humans, as well as animals, have memory. Feedback connections gives
the network the opportunity to obtain memory.
That allows it to not convey signal history to the network. Moreover,

17. Software Development
the memory depth to the past can be determined during learning.
For the recurrent neural network, we used the raw data without removing
atrifacts, because RNN does it. Therefore, the recurrent neural network has 32
entries.
We ran multiple experiments with the recurrent neural network, tested various
architectures and delays in feedback connections. The network didn’t learn.
There was a problem with memory – we could not create a complex network
with more than two layers. Then we came to the conclusion that we need
to try a special type of neural network – LSTM (long short term memory).
LSTM has inside specific memory units, which are made on the base
of controlled backlink with a coefficient of 1.
Image 14. Memory unit of the LSTM recurrent neural network
The decision of whether to turn the backlink on or off is the responsibility
of the other neural network part. Turning off the backlink leads to erasing
information in the memory core. Thus, the network decides
when to remember, use or forget the information.
Nevertheless, the complexity of the LSTM algorithm requires deeper
investigation than it was possible under the competition circumstances.
For these reasons, we decided to stop at the current stage, without obtaining
LSTM testing results.

18. Software Development
Conclusion
Finally, our R&D team was able to obtain high quality classification of EEG
signals during the process of hand movements. Thanks to the quality
parameter “area under ROC curve”, which was derived as a result of using
convolutional neural networks, we verified that deep learning algorithms are
effective in various types of signal classification. Therefore, we can conclude
that convolutional neural networks are one of the greatest deep learning
methods for the development of brain-computer interfaces. In simple words,
show us your EEG signals and we’ll figure out what you were doing at the
moment of the EEG recording.
Literature
[1] Homskaya E.D. Neiropsychologiya [Neuropsychology]. St. Petersburg,
Piter, 2005. 496 p. (in Russian)
[2] Kuo C-C, Luu P, Morgan KK, Dow M, Davey C, et al. Localizing
Movement-Related Primary Sensorimotor Cortices with Multi-Band EEG
Frequency Changes and Functional MRI (2014). Available at:
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0112103
(Accessed 3 October 2015)
[3] Saveliev S.V. Vozniknovenie mozga cheloveka [The emergence of the
human brain]. Moscow, Vedi, 2010. 324 p. (in Russian)
[4] H. P. Martinez, Y. Bengio, G. N. Yannakakis Learning deep physiological
models of affect. Computational Intelligence Magazine, IEEE, Vol. 8, No. 2.
(May 2013), pp. 20-33. Available at:
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6496209 (Accessed
4 October 2015)
[5] Model Random Forest dlya classificacii, realizaciya na c# (The Random
Forest model for classification, c# implementation). Available at:
https://habrahabr.ru/post/215453/ (Accessed 7 October 2015)