Downloaded 10 times
![Event Related Desynchronization (ERD)
▰ Mu [8–13 Hz] and Beta [13–30 Hz] bands
▰ Always correlated with ERS
11
(Neuper et al ,2006)](https://image.slidesharecdn.com/finalseminarpresentation-171108195629/75/A-Brain-Computer-Interface-Speller-for-Smart-Devices-11-2048.jpg)
![Event Related Desynchronization (ERD)
▰ Mu [8–13 Hz] and Beta [13–30 Hz] bands
▰ Always correlated with ERS
12
(Neuper et al ,2006)](https://image.slidesharecdn.com/finalseminarpresentation-171108195629/75/A-Brain-Computer-Interface-Speller-for-Smart-Devices-12-2048.jpg)
![Pre-Processing
▰ Common Average Reference (CAR) to eliminate external noise
▰ Z-score to adjust values measured from different channels on different scales to a
notionally common scale
▰ Band-pass filter [4-41] Hz to remove eye artifacts and get domain that contains
valuable ERD/RDS information
▰ Down-sample to speedup subsequent processing.
16
Pre-
Processing
Feature
Extraction
Dimensionality
Reduction
Classification
EEG Data Command](https://image.slidesharecdn.com/finalseminarpresentation-171108195629/75/A-Brain-Computer-Interface-Speller-for-Smart-Devices-16-2048.jpg)
![35
Command
AE 157
BP 1
8-15
BP 2
12-21
BP 3
20-30
LDA 157 Evaluation 157
..
1
2
.
.
157
AE 2 LDA 2 Evaluation 2
AE 1 LDA 1 Evaluation 1
Pre-Processing
EEG Data
Dimensionality ReductionFeature Extraction Classification Evaluation
..
..
Best
Sample
Feature
EEG Signal Flow in Motor Imagery System
[672528x22]
[168132 x 22]
[22 x 228] [228x 45] [228]](https://image.slidesharecdn.com/finalseminarpresentation-171108195629/75/A-Brain-Computer-Interface-Speller-for-Smart-Devices-35-2048.jpg)
Uploaded byMahmoud Helal
A Brain Computer Interface Speller for Smart Devices
This document presents a motor imagery-based brain-computer interface speller for mobile devices. It introduces a motor imagery structure model consisting of pre-processing, feature extraction, dimensionality reduction, and classification blocks. It develops an autoencoder-based dimensionality reduction method and compares it to PCA. It also develops a Hex-O-Spell mobile application using motor imagery to spell words. Results show the autoencoder approach achieves better performance than PCA. Testing on three subjects demonstrates the utility of the Hex-O-Spell mobile application. Future work involves enhancing the methods and application.
More Related Content
Similar to A Brain Computer Interface Speller for Smart Devices
A Brain Computer Interface Speller for Smart Devices
- 1. A Brain-Computer InterfaceSpeller for Smart Devices Mahmoud A. Helal Supervised by : Dr. Mohamed Taher and Dr. Seif Eldawlatly Computer and Systems Engineering Department, Faculty of Engineering Ain Shams University Cairo, Egypt
- 2. Agenda 2 INTRODUCTION ▰Brain-Computer Interfaces (BCI) ▰Electroencephalography(EEG) ▰BCI Applications ▰BCI Mobile Applications ▰10-20 System ▰Event Related Desynchronization (ERD) CONTRIBUTIONS ▰Motor Imagery Structure Model ▰Hex-O-Spell Mobile Application CONCLUSION AND FUTURE WORK
- 3.
- 4. Brain-Computer Interface (BCI) Acommunication system that facilitates controlling external devices by recording, processing and analyzing signals detected from the brain neural activity 4 INVASIVE NON-INVASIVE
- 5. Electroencephalography (EEG) Record electricalactivity along the scalp produced by the firing of neurons over a short period of time 5 EEG Signal Command Feedback
- 6.
- 7. BCI Mobile Applications 7 NeuroPhoneSystem Campbell, A., T. Choudhury, et al. (2010)
- 8. BCI Mobile Applications 8 BCIMessenger Li, Y., J. Zhang, et al. (2009)
- 9. BCI Mobile Applications 9RunAppand ImgView Elsawy, A. S. and S. Eldawlatly (2015)
- 10. 10 – 20System ▰ Introduced by the International Federation of Societies for Electroencephalography and Clinical Neurophysiology ▰ Method used to describe electrodes position over the scalp 10
- 11. Event Related Desynchronization(ERD) ▰ Mu [8–13 Hz] and Beta [13–30 Hz] bands ▰ Always correlated with ERS 11 (Neuper et al ,2006)
- 12. Event Related Desynchronization(ERD) ▰ Mu [8–13 Hz] and Beta [13–30 Hz] bands ▰ Always correlated with ERS 12 (Neuper et al ,2006)
- 13. Raw EEG datafor right and left hand imaginary 13 (μV)(μV) (μV)(μV)
- 14.
- 15. Motor Imagery StructureModel ▰ Consists of 4 main blocks: ▻ Pre-Processing removing artifacts ▻ Feature Extraction extract information from raw signals ▻ Dimensionality Reduction reduce data dimensionality to speedup classification ▻ Classification classify the reduced dimension data 15 Pre- Processing Feature Extraction Dimensionality Reduction ClassificationEEG Data Command
- 16. Pre-Processing ▰ Common AverageReference (CAR) to eliminate external noise ▰ Z-score to adjust values measured from different channels on different scales to a notionally common scale ▰ Band-pass filter [4-41] Hz to remove eye artifacts and get domain that contains valuable ERD/RDS information ▰ Down-sample to speedup subsequent processing. 16 Pre- Processing Feature Extraction Dimensionality Reduction Classification EEG Data Command
- 17.
- 18. Feature Extraction -Band power 18 Pre- Processing Feature Extraction Dimensionality Reduction Classification EEG Data Command ▰Apply 5th order Butterworth band-pass filter using the frequency bands ▰Square every sample in the output signal to obtain power 𝑝 𝑡 = 𝑥2 (𝑡) ▰Average power 𝑝 𝑛 = 1 𝑤 𝑘=0 𝑤 𝑝(𝑛 − 𝑘) ▰Log 𝑙𝑜𝑔10 𝑝(𝑛)
- 19. ▰Principal Component Analysis(PCA) is a classical technique in dimensionality reduction ▰The goal is to project data on the minimum number of principal components that represent the maximum amount of variance in the data 𝒛 = 𝒙 . 𝒘 Dimensionality Reduction – Principal Component Analysis (PCA) 19 Pre- Processing Feature Extraction Dimensionality Reduction Classification EEG Data Command Feature 1 Feature2
- 20. ▰An Autoencoder (AE)is a neural network with three or more layers, an input layer and an output layer ▰The main target of AE is to reconstruct its input on its output nodes ▰Number of nodes of the hidden layer is less than the number for the input layer Dimensionality Reduction - AutoEncoder 20 Pre- Processing Feature Extraction Dimensionality Reduction Classification EEG Data Command x zh Input layer Hidden layer Output layer Encoder Decoder Feature Data Compressed Features
- 21. Classification – LinearDiscriminant Analysis (LDA) 21 Pre- Processing Feature Extraction Dimensionality Reduction Classification EEG Data Command ▰ LDA is most commonly used as a data classification technique in BCI ▰ One-versus-all method is used to classify 4 classes 𝐽 𝑤 = 𝑤 𝑡 . 𝑆 𝑏 . 𝑤 𝑤 𝑡 . 𝑆 𝑤 . 𝑤 ▰ 𝑆 𝑏 is the between class scatter matrix 𝑆 𝑏 = 𝑘=1 𝑚 𝑛 𝑘(µ 𝑘 − µ)(µ 𝑘 − µ) 𝑇 ▰ 𝑆 𝑤 is the within-class scatter matrix Feature 1 Feature2
- 22. ▰Based on Bayes’Theorem ▰Naive Bayes classifier assumes independence between features 𝑃 𝑐| 𝑓 = 𝑃 𝑓| 𝑐 . 𝑃(𝑐) 𝑃(𝑓) 𝑃 𝑐| 𝑓 is the posterior probability of class c given feature f, . 𝑃(𝑐) is the prior probability of class c, 𝑃 𝑓| 𝑐 is the likelihood which is the probability of feature f given class c Classification – Naive Bayes Classifier (NBC) 22 Pre- Processing Feature Extraction Dimensionality Reduction Classification EEG Data Command
- 23. RESULTS DataSet , Evaluationand Performance of Methods 23
- 24. ▰BCI Competition IV(2008) - Dataset 2a ▰9 subjects. 2 sessions were recorded on 2 days ▰Single session consisted of 288 trials over 22 EEG channels ▰The subject to perform one of the following motor imagery tasks: left hand, right hand, both feet, or tongue BCI Competition Dataset 24
- 25. Kappa Evaluation 25 Predicted Right Hand Left Hand Feet Tongue Actual RightHand 𝑫 𝟏𝟏 𝑫 𝟏𝟐 𝑫 𝟏𝟑 𝑫 𝟏𝟒 Left Hand 𝑫 𝟐𝟏 𝑫 𝟐𝟐 𝑫 𝟐𝟑 𝑫 𝟐𝟒 Feet 𝑫 𝟑𝟏 𝑫 𝟑𝟐 𝑫 𝟑𝟑 𝑫 𝟑𝟒 Tongue 𝑫 𝟒𝟏 𝑫 𝟒𝟐 𝑫 𝟒𝟑 𝑫 𝟒𝟒 ▰ kappa is calculated by equation 𝑘 = 𝑝0 − 𝑝 𝑒 1 − 𝑝 𝑒 ▰ Observed proportionate agreement between raters 𝑝0 = 1 𝑛 𝑖=1 𝑐 𝐷𝑖𝑖 ▰ Expected agreement on the same data 𝑝 𝑒 = 1 𝑛2 𝑖=1 𝑐 𝐷𝑖+ . 𝐷+𝑖
- 26. ▰BCI Competition IV(2008) - Dataset 2a ▰9 subjects. 2 sessions were recorded on 2 days ▰Single session consisted of 288 trials over 22 EEG channels ▰The subject to perform one of the following motor imagery tasks: left hand, right hand, both feet, or tongue BCI Competition Dataset 26
- 27. ▰ Technique usedto estimate the optimal number of reduced dimensions in dimensionality reduction methods or optimal classifiers used. ▰ It is commonly used to prevent overfitting the classifier Cross-Validation 27
- 28. ▰Examine different frequencybands ranges in order to determine the most effective frequency bands in classification process Results: Feature Extraction 28 # Bands (Hz) # features kappa 1 8,13 22 0.449074 2 8-15 22 0.472737 3 10-13 22 0.431070 4 5-10 22 0.316358 5 9-12 22 0.415123 # Bands (Hz) # features kappa 1 8-15, 13-20 44 0.518519 2 8-15, 18-25 44 0.517490 3 8-15, 12-21 44 0.534979 4 8-15, 20-28 44 0.513889 5 8-15, 20-25 44 0.503086 # Bands (Hz) # features Kappa 1 8-15, 12-21, 20-25 66 0.509259 2 8-15, 12-21, 22-29 66 0.529835 3 8-15, 12-21, 24-27 66 0.524177 4 8-15, 12-21, 20-30 66 0.538580
- 29. ▰Cross-validation is usedby dividing the training data to 80% training and 20% validation selected randomly, then repeat this process 10-times Results: Classification 29 Classifier Kappa 1 NBC 0.45 2 LDA 0.51
- 30. ▰Compare the performanceof AE and PCA for the same number of reduced dimensions. range of 45 to 65 with a step of 5 averaged across subjects Results: Performance over Number of Dimensions 30 Dimensionality Reduction Kappa 1 Without 0.51 2 PCA 0.52 3 AutoEncoder 0.56
- 31. ▰Subject-dependent optimal parametersused for each method (number of hidden neurons in AE and number of principal components in PCA) Results: AutoEncoder versus PCA 31 Dimensionality Reduction Kappa 1 Without 0.51 2 PCA 0.52 3 AutoEncoder 0.55
- 32. ▰Examine the performancewhen sigmoid activation function is used compared to linear activation function averaged across subjects. Results: AE Activation Function 32 AE Activation Kappa 1 Linear 0.53 2 Sigmoid 0.55
- 33. Results: AutoEncoder versusCompetition Results 33 # contributor Subjects Mean 1 2 3 4 5 6 7 8 9 1 Kai Keng Ang 0.68 0.42 0.75 0.48 0.40 0.27 0.77 0.75 0.61 0.57 2 Mahmoud Helal 0.76 0.33 0.74 0.53 0.18 0.32 0.78 0.71 0.64 0.55 3 Liu Guangquan 0.69 0.34 0.71 0.44 0.16 0.21 0.66 0.73 0.69 0.52 4 Wei Song 0.38 0.18 0.48 0.33 0.07 0.14 0.29 0.49 0.44 0.31 5 Damien Coyle 0.46 0.25 0.65 0.31 0.12 0.07 0.00 0.46 0.42 0.30 6 Jin Wu 0.41 0.17 0.39 0.25 0.06 0.16 0.34 0.45 0.37 0.29
- 34. ▰Results in channels9, 10 and 11 have the highest value of the weights ▰Channels correspond to C1, Cz and C2 which cover the left, central and right areas of the motor cortex Analysis of AE Weights 34
- 35. 35 Command AE 157 BP 1 8-15 BP2 12-21 BP 3 20-30 LDA 157 Evaluation 157 .. 1 2 . . 157 AE 2 LDA 2 Evaluation 2 AE 1 LDA 1 Evaluation 1 Pre-Processing EEG Data Dimensionality ReductionFeature Extraction Classification Evaluation .. .. Best Sample Feature EEG Signal Flow in Motor Imagery System [672528x22] [168132 x 22] [22 x 228] [228x 45] [228]
- 36. HEX-O-SPELL Algorithm and MobileApplication 36
- 37. ▰Five letters ineach six Hexs ▰Right Feet : select ▰Right Hand : Return Hex-O-Spell : Algorithm 37
- 38. Hex-O-Spell : MobileApplication 38 SPELLING PHASE User try to write word TRAINING PHASE Application asks user to imagne right hand / right feet movement
- 39. Hex-O-Spell : MobileApplication 39
- 40. ▰Test on threesubject train 50 trial (25 right hand, 25 right feet) 80% training data 20% testing data Hex-O-Spell : Results 40 Subject Gender Best PCA Training Accuracy Cross- Validation Accuracy 1 Subject A Male 22 96.66% 90% 2 Subject B Female 28 100% 70% 3 Subject C Male 26 96.66% 80%
- 41.
- 42.
- 43. MOTOR IMAGERY SYSTEM ▻Different methods implemented and tested ▻ Autoencoder dimension reduction method is introduced, and compared with traditional and classical method PCA ▻ Results suggest using autoencoders with sigmoid activation functions achieve better performance compared to using linear PCA Conclusion HEX-O-SPELL ▻ BCI Hex-O-Spell application is developed. ▻ Results for three different subjects demonstrate the utility of the application 43
- 44. MOTOR IMAGERY SYSTEM ▻Enhance Methods automated detection for best frequency bands in band power, Employing deep autoencoder architectures (stacked and denoising autoencoders) ▻ Other Methods wavelet packet decomposition (WPD) in feature extraction. ▻ Add new methods common spatial patterns (CSP). Future Work 44
- 45. Future Work 45 HEX-O-SPELL ▻ Enhancingalgorithm : allow the arrow to rotate clock wise and counter clock wise to speedup character selection process) ▻ Add Features autocomplete word, type integers (0-9), or symbols (&, # …) or operate in multiple languages
- 46. PUBLICATIONS ▰ M. A.Helal, S. Eldawlatly and M. Taher, "Using Autoencoders for Feature Enhancement in Motor Imagery Brain-Computer Interfaces," 2017 13th IASTED International Conference on Biomedical Engineering (BioMed), Innsbruck, Austria, pp. 89-93, 2017. ▰ M. A. Helal, S. Eldawlatly and M. Taher, "A Brain-Computer Interface Hex-O-spell Application for Mobile Devices,” in preparation. 46
- 47.