SlideShare a Scribd company logo
1 of 48
Investigating EEG for Saliency and
Segmentation Applications in Image
Processing
Eva Mohedano

1
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

2
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

3
1- PROBLEM STATEMENT
Design a system based on a Brain Computer Interface (BCI) wich measure
Electroencephalography (EEG) signals to answer the following questions:
1 - Are the EEG signals to compute Saliency Maps?
2 - Are the EEG signals useful to images segmentation?

EEG Signals
Visual stimulus
BCI

Data Processing
4
1- PROBLEM STATEMENT
Design a system based on a Brain Computer Interface (BCI) wich measure
Electroencephalography (EEG) signals to answer the following questions:
1 - Are the EEG signals to compute Saliency Maps?
2 - Are the EEG signals useful to images segmentation?

Visual stimulus
BCI

Data Processing
5
1- PROBLEM STATEMENT
Design a system based on a Brain Computer Interface (BCI) wich measure
Electroencephalography (EEG) signals to answer the following questions:
1 - Are the EEG signals to compute Saliency Maps?
2 - Are the EEG signals useful to images segmentation?

Visual stimulus
BCI

Data Processing
6
1- PROBLEM STATEMENT
1 - Are the EEG signals to compute Saliency Maps?
Motivation:
- New way to compute maps of the atention of the image
based directly in the reaction of the brain and not in the
features of the images (Niebur and Koch (1996) algorithm).

Visual stimulus
BCI

Data Processing
7
1- PROBLEM STATEMENT
2 - Are the EEG signals useful to images segmentation?
Motivation:
- Reduce the user interaction to the minimun expression.
- Measure the brain reaction at local scale of the image.

Visual stimulus
BCI

Data Processing
8
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

9
2- RELATED WORK
2.1 – BCI in image processing applications

The oddball paradigm

10
2- RELATED WORK
2.1 – BCI in image processing applications

The oddball paradigm

P300

11
2- RELATED WORK
2.1 – BCI in image processing applications

The oddball paradigm

P300

• Speed rate around 10Hz
• Usually experiements centered to
find target images not target regions
12
2- RELATED WORK

Event-Related Potential

•8 electrodes placed mainly in the
posterior points on the scalp.
• Which is consistent with the
discriminating
activity
typically
produced by a P300 ERP.
[Optimising the Number of Channels in
EEG-Augmented Image Search. Graham
Healy]

13
2- RELATED WORK

Event-Related Potential

•8 electrodes placed mainly in
How to present the scalp. to the
image
posterior points on the
generate and is consistent signal?
• Which detect this with the
discriminating
activity
produced by a P300 ERP.

typically

[Optimising the Number of Channels in
EEG-Augmented Image Search. Graham
Healy]

14
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

15
3- LOCAL EXPLORATION OF THE IMAGE
First Design: Sliding Window

http://www.youtube.com/watch?v=bKTGKVx58Ps

16
3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 1
• Eyes movement affect to the EEG signals – Introduce Artifacts to the signal

Opened eyes / Closed eyes. Image from the slides Dr. Ranjith Polusani
17
3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 2
• Progressive inspection may not generate a useful reaction in the EEG waves.
Suggestions meeting Thomas Ward and Nima Bidgely Shamlo :
- Follow Oddball Paradigm and perform RSVP od the windows

P300

SNAP - Simulation and Neuroscience
Application Platform
18
3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 3
•Syncronitzation Problem

19
3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 4
•Size of the object / window
Grabcut Dataset – Objects of different size

Suggestions meeting Thomas Ward and Nima Bidgely Shamlo :
To use images with an homogeneus background with a salient object.
The number of distractors (windows with background) must be higher
than the number of targets (windows with object).
20
3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 5
•What am I seeing?

21
3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 5
•What am I seeing?

22
3- LOCAL EXPLORATION OF THE IMAGE

CHALLENGES

SOLUTIONS

1 - Eyes movement

Display a fixed window on the screen

2 - Progressive inspection

SNAP to perform a random RSVP

3 - Syncronitzation Problem

First test with flashes to find ERPS

4 - Size of the object / window

Generate my own dataset

5 - What am I seeing?
• Is it just noise?
• Am I able to detect something?

Real time visualitzation of the signal

23
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

24
http://www.youtube.com/watch?v=KsgtvQkOE
lQ&feature=youtu.be

4- EXPERIMENTAL SET-UP
Second Design: Starting from the easiest case
a) Device Calibration
I. Real time visualization of Alpha waves
II. Detecting ERPS

b) Synthetic Images
I. RSVP synthetic images fitted in the window.

c) RSVP real images

25
4- EXPERIMENTAL SET-UP
a) Device Calibration

?
Is the device well
connected?

Is the syncronitzation
method correct ?

Am I able to detect
something?

26
4- EXPERIMENTAL SET-UP
a) Device Calibration

SIGNAL EXPECTED - Closed eyes – Alpha waves (8-12 Hz)

Closed-eye EEG alpha waves (10-20 channels Pz-Top, Fz-Bottom) extracted from http://blog.grahamhealy.com/
27
4- EXPERIMENTAL SET-UP
a) Device Calibration

SIGNAL OBTAINED

5 seconds Closed Eyes

Amplitude (uV)

40
20
0
-20
-40

0

0.5

1

1.5

0

0.5

1

1.5

2

2.5
3
3.5
Time (sec)
5 seconds Opened Eyes

4

4.5

5

4

4.5

5

Amplitude (uV)

40
20
0
-20
-40

2

2.5
3
Time (sec)

3.5

28
4- EXPERIMENTAL SET-UP
a) Device Calibration

Finding ERPS response after a white flash
SIGNAL EXPECTED: After the flash P100 and a negative peak between 150-200ms

Presenting a serie of white flashes (2 seconds between the flashes)
29
4- EXPERIMENTAL SET-UP
a) Device Calibration

Finding ERPS response after a white flash
SIGNAL OBTAINED: 60 Flashes to get the response.
Averaged ERP waveform per channel

Channel

P100 (ms)

1

130

320

2

90

210

3

90

210

4

10

220

5

110

220

6

90

210

7

10

220

8

100

22

Mean

10

N1 (ms)

80

23

Amplitude (uV)

5

0

-5

-10

-15

0

100

200

300

400

500
600
Time (ms)

700

800

900

1000

30
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

31
5- SIGNAL PROCESSING OF EEG SIGNALS
Data adquisition

100 windows per Image

1 Target
99 Distractors
32
5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Single trial
One Image

1

2

3

4

5

6

7

8
33
5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Single trial - PROBLEM

- Signal very noisy
- Single Targets and Single
Distractors very similar

34
5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Feature
96 Distractors
Mean Absolute Amplitude
looks different

96 Targets

Energy from 0 to
600ms
Feature for the window
presented

35
5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Averaged trials

1 averaged target
99 averaged
distractors

Energy from
0-600 ms

36
5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Averaged trials
SINGLE

AVERAGED

1 x 32 single repeats

1 averaged target

99 x 32 single
repeats

99 averaged
distractors

1 x 32 window repeats

1 averaged target

99 x 32 single
repeats

Average
by 32

99 averaged
distractors

1 x 32 window repeats

1 averaged target

99 x 32 single
repeats

99 averaged
distractors
37
5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Averaged trials
1 averaged target

Problem:
Too few (target) samples
for training

99 averaged
distractors
1 averaged target
99 averaged
distractors
1 averaged target
99 averaged
distractors
38
5- SIGNAL PROCESSING OF EEG SIGNALS
Bootstrapping
SINGLE

AVERAGED

1 x 32 single repeats

1 x 32 averaged target

99 x 32 single
repeats

99 x 1
averaged distractors

1 x 32 window repeats

99 x 32 single
repeats
1 x 32 window repeats

99 x 32 single
repeats

1 x 32 averaged target

Bootstrapping

99 x 1
averaged distractors
1 x 32 averaged target

99 x 1
averaged distractors39
5- SIGNAL PROCESSING OF EEG SIGNALS
Classification

Problem:
Unbalanced dataset for
binary classification

AVERAGED
1 x 32 averaged target

99 x 1
averaged distractors
1 x 32 averaged target

99 x 1
averaged distractors
1 x 32 averaged target

99 x 1
averaged distractors40
5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
AVERAGED

AVERAGED

1 x 32 averaged targets

99 x 1
averaged distractors

1 x 32 averaged targets

Subsample

1 x 32 averaged targets

99 x 1
averaged distractors

1 x 32 averaged targets

Subsample

1 x 32 averaged targets

99 x 1
averaged distractors

32 x 1 avgd distractors

32 x 1 avgd distractors

1 x 32 averaged targets

Subsample

32 x 1 avgd distractors
41
5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
AVERAGED

HISTOGRAM

1 x 32 averaged targets

32 x 1 avgd distractors

1 x 32 averaged targets
32 x 1 avgd distractors

1 x 32 averaged targets
32 x 1 avgd distractors
42
5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
AVERAGED
1 x 32 averaged targets

32 x 1 avgd distractors

1 x 32 averaged targets
32 x 1 avgd distractors

SVMTRAIN
(linear
kernel)

Classifier

1 x 32 averaged targets
32 x 1 avgd distractors
43
5- SIGNAL PROCESSING OF EEG SIGNALS
Classification

100 x 32 single repeats

Average
by 32

100 avgd windows
44
5- SIGNAL PROCESSING OF EEG SIGNALS
Classification

100 x 32 single repeats

Average
by 32

100 avgd samples

Classifier

SVM
PREDICT
45
5- SIGNAL PROCESSING OF EEG SIGNALS
8 samples feature vectors
Cross validation approach 3 train + 1 test
CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions

47
6- CONCLUSIONS
- Results from sythetic images provide and evidence that BCI devices could be
used to located an object into an image.

- Simplicity of the system: Energy value from 8 channels to train SVM with
lineal kernel.

Future work
-Study the impact of the number of repetitions.
- Extract better features.
-Analize data from real images.

-Tool to evaluate and compare the EEG mask (ROC, Jaccard index)

48

More Related Content

Similar to Eva Mohedano, "Investigating EEG for Saliency and Segmentation Applications in Image Processing"

Time-series forecasting of indoor temperature using pre-trained Deep Neural N...
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Time-series forecasting of indoor temperature using pre-trained Deep Neural N...
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Francisco Zamora-Martinez
 
AlphaZero and beyond: Polygames
AlphaZero and beyond: PolygamesAlphaZero and beyond: Polygames
AlphaZero and beyond: PolygamesOlivier Teytaud
 
Qualcomm research-imagenet2015
Qualcomm research-imagenet2015Qualcomm research-imagenet2015
Qualcomm research-imagenet2015Bilkent University
 
Emotiv Epoc/EEG/BCI
Emotiv Epoc/EEG/BCIEmotiv Epoc/EEG/BCI
Emotiv Epoc/EEG/BCISuhail Khan
 
Emotiv Epoc/BCi/EEG
Emotiv Epoc/BCi/EEGEmotiv Epoc/BCi/EEG
Emotiv Epoc/BCi/EEGSuhail Khan
 
RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES
RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES
RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES Touradj Ebrahimi
 
Rob livesley intro to bci
Rob livesley intro to bciRob livesley intro to bci
Rob livesley intro to bciataccessgroup
 
The John Henry lens design challenge
The John Henry lens design challengeThe John Henry lens design challenge
The John Henry lens design challengeDave Shafer
 
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...IEEEBEBTECHSTUDENTPROJECTS
 
Generating a time shrunk lecture video by event
Generating a time shrunk lecture video by eventGenerating a time shrunk lecture video by event
Generating a time shrunk lecture video by eventYara Ali
 
Restoring vision to the blind: The challenge of sight recovery technologies
Restoring vision to the blind: The challenge of sight recovery technologiesRestoring vision to the blind: The challenge of sight recovery technologies
Restoring vision to the blind: The challenge of sight recovery technologiesMichael Beyeler
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用CHENHuiMei
 
Defense Presentation
Defense PresentationDefense Presentation
Defense PresentationSahil Chaubal
 

Similar to Eva Mohedano, "Investigating EEG for Saliency and Segmentation Applications in Image Processing" (20)

Talk Gencesp
Talk GencespTalk Gencesp
Talk Gencesp
 
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
 
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Time-series forecasting of indoor temperature using pre-trained Deep Neural N...
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...
 
AlphaZero and beyond: Polygames
AlphaZero and beyond: PolygamesAlphaZero and beyond: Polygames
AlphaZero and beyond: Polygames
 
Qualcomm research-imagenet2015
Qualcomm research-imagenet2015Qualcomm research-imagenet2015
Qualcomm research-imagenet2015
 
Emotiv epoc final
Emotiv epoc  finalEmotiv epoc  final
Emotiv epoc final
 
Emotiv Epoc/EEG/BCI
Emotiv Epoc/EEG/BCIEmotiv Epoc/EEG/BCI
Emotiv Epoc/EEG/BCI
 
Emotiv Epoc/BCi/EEG
Emotiv Epoc/BCi/EEGEmotiv Epoc/BCi/EEG
Emotiv Epoc/BCi/EEG
 
RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES
RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES
RECENT ADVANCES IN BRAIN-COMPUTER INTERFACES
 
Rob livesley intro to bci
Rob livesley intro to bciRob livesley intro to bci
Rob livesley intro to bci
 
The John Henry lens design challenge
The John Henry lens design challengeThe John Henry lens design challenge
The John Henry lens design challenge
 
Inroduction to BCI
Inroduction to BCIInroduction to BCI
Inroduction to BCI
 
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...
 
Bci
BciBci
Bci
 
Bci
BciBci
Bci
 
Generating a time shrunk lecture video by event
Generating a time shrunk lecture video by eventGenerating a time shrunk lecture video by event
Generating a time shrunk lecture video by event
 
EEG course.pptx
EEG course.pptxEEG course.pptx
EEG course.pptx
 
Restoring vision to the blind: The challenge of sight recovery technologies
Restoring vision to the blind: The challenge of sight recovery technologiesRestoring vision to the blind: The challenge of sight recovery technologies
Restoring vision to the blind: The challenge of sight recovery technologies
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
 
Defense Presentation
Defense PresentationDefense Presentation
Defense Presentation
 

More from Universitat Politècnica de Catalunya

The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...Universitat Politècnica de Catalunya
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoUniversitat Politècnica de Catalunya
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Universitat Politècnica de Catalunya
 
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosGeneration of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosUniversitat Politècnica de Catalunya
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Universitat Politècnica de Catalunya
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Universitat Politècnica de Catalunya
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Universitat Politècnica de Catalunya
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Universitat Politècnica de Catalunya
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Universitat Politècnica de Catalunya
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Universitat Politècnica de Catalunya
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Universitat Politècnica de Catalunya
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Universitat Politècnica de Catalunya
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Universitat Politècnica de Catalunya
 

More from Universitat Politècnica de Catalunya (20)

Deep Generative Learning for All
Deep Generative Learning for AllDeep Generative Learning for All
Deep Generative Learning for All
 
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
 
The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
 
Open challenges in sign language translation and production
Open challenges in sign language translation and productionOpen challenges in sign language translation and production
Open challenges in sign language translation and production
 
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosGeneration of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
 
Discovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in MinecraftDiscovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in Minecraft
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...
 
Intepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural NetworksIntepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural Networks
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
 
Curriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object SegmentationCurriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object Segmentation
 
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
 

Recently uploaded

Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 

Recently uploaded (20)

Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 

Eva Mohedano, "Investigating EEG for Saliency and Segmentation Applications in Image Processing"

  • 1. Investigating EEG for Saliency and Segmentation Applications in Image Processing Eva Mohedano 1
  • 2. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 2
  • 3. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 3
  • 4. 1- PROBLEM STATEMENT Design a system based on a Brain Computer Interface (BCI) wich measure Electroencephalography (EEG) signals to answer the following questions: 1 - Are the EEG signals to compute Saliency Maps? 2 - Are the EEG signals useful to images segmentation? EEG Signals Visual stimulus BCI Data Processing 4
  • 5. 1- PROBLEM STATEMENT Design a system based on a Brain Computer Interface (BCI) wich measure Electroencephalography (EEG) signals to answer the following questions: 1 - Are the EEG signals to compute Saliency Maps? 2 - Are the EEG signals useful to images segmentation? Visual stimulus BCI Data Processing 5
  • 6. 1- PROBLEM STATEMENT Design a system based on a Brain Computer Interface (BCI) wich measure Electroencephalography (EEG) signals to answer the following questions: 1 - Are the EEG signals to compute Saliency Maps? 2 - Are the EEG signals useful to images segmentation? Visual stimulus BCI Data Processing 6
  • 7. 1- PROBLEM STATEMENT 1 - Are the EEG signals to compute Saliency Maps? Motivation: - New way to compute maps of the atention of the image based directly in the reaction of the brain and not in the features of the images (Niebur and Koch (1996) algorithm). Visual stimulus BCI Data Processing 7
  • 8. 1- PROBLEM STATEMENT 2 - Are the EEG signals useful to images segmentation? Motivation: - Reduce the user interaction to the minimun expression. - Measure the brain reaction at local scale of the image. Visual stimulus BCI Data Processing 8
  • 9. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 9
  • 10. 2- RELATED WORK 2.1 – BCI in image processing applications The oddball paradigm 10
  • 11. 2- RELATED WORK 2.1 – BCI in image processing applications The oddball paradigm P300 11
  • 12. 2- RELATED WORK 2.1 – BCI in image processing applications The oddball paradigm P300 • Speed rate around 10Hz • Usually experiements centered to find target images not target regions 12
  • 13. 2- RELATED WORK Event-Related Potential •8 electrodes placed mainly in the posterior points on the scalp. • Which is consistent with the discriminating activity typically produced by a P300 ERP. [Optimising the Number of Channels in EEG-Augmented Image Search. Graham Healy] 13
  • 14. 2- RELATED WORK Event-Related Potential •8 electrodes placed mainly in How to present the scalp. to the image posterior points on the generate and is consistent signal? • Which detect this with the discriminating activity produced by a P300 ERP. typically [Optimising the Number of Channels in EEG-Augmented Image Search. Graham Healy] 14
  • 15. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 15
  • 16. 3- LOCAL EXPLORATION OF THE IMAGE First Design: Sliding Window http://www.youtube.com/watch?v=bKTGKVx58Ps 16
  • 17. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGE 1 • Eyes movement affect to the EEG signals – Introduce Artifacts to the signal Opened eyes / Closed eyes. Image from the slides Dr. Ranjith Polusani 17
  • 18. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGE 2 • Progressive inspection may not generate a useful reaction in the EEG waves. Suggestions meeting Thomas Ward and Nima Bidgely Shamlo : - Follow Oddball Paradigm and perform RSVP od the windows P300 SNAP - Simulation and Neuroscience Application Platform 18
  • 19. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGE 3 •Syncronitzation Problem 19
  • 20. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGE 4 •Size of the object / window Grabcut Dataset – Objects of different size Suggestions meeting Thomas Ward and Nima Bidgely Shamlo : To use images with an homogeneus background with a salient object. The number of distractors (windows with background) must be higher than the number of targets (windows with object). 20
  • 21. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGE 5 •What am I seeing? 21
  • 22. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGE 5 •What am I seeing? 22
  • 23. 3- LOCAL EXPLORATION OF THE IMAGE CHALLENGES SOLUTIONS 1 - Eyes movement Display a fixed window on the screen 2 - Progressive inspection SNAP to perform a random RSVP 3 - Syncronitzation Problem First test with flashes to find ERPS 4 - Size of the object / window Generate my own dataset 5 - What am I seeing? • Is it just noise? • Am I able to detect something? Real time visualitzation of the signal 23
  • 24. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 24
  • 25. http://www.youtube.com/watch?v=KsgtvQkOE lQ&feature=youtu.be 4- EXPERIMENTAL SET-UP Second Design: Starting from the easiest case a) Device Calibration I. Real time visualization of Alpha waves II. Detecting ERPS b) Synthetic Images I. RSVP synthetic images fitted in the window. c) RSVP real images 25
  • 26. 4- EXPERIMENTAL SET-UP a) Device Calibration ? Is the device well connected? Is the syncronitzation method correct ? Am I able to detect something? 26
  • 27. 4- EXPERIMENTAL SET-UP a) Device Calibration SIGNAL EXPECTED - Closed eyes – Alpha waves (8-12 Hz) Closed-eye EEG alpha waves (10-20 channels Pz-Top, Fz-Bottom) extracted from http://blog.grahamhealy.com/ 27
  • 28. 4- EXPERIMENTAL SET-UP a) Device Calibration SIGNAL OBTAINED 5 seconds Closed Eyes Amplitude (uV) 40 20 0 -20 -40 0 0.5 1 1.5 0 0.5 1 1.5 2 2.5 3 3.5 Time (sec) 5 seconds Opened Eyes 4 4.5 5 4 4.5 5 Amplitude (uV) 40 20 0 -20 -40 2 2.5 3 Time (sec) 3.5 28
  • 29. 4- EXPERIMENTAL SET-UP a) Device Calibration Finding ERPS response after a white flash SIGNAL EXPECTED: After the flash P100 and a negative peak between 150-200ms Presenting a serie of white flashes (2 seconds between the flashes) 29
  • 30. 4- EXPERIMENTAL SET-UP a) Device Calibration Finding ERPS response after a white flash SIGNAL OBTAINED: 60 Flashes to get the response. Averaged ERP waveform per channel Channel P100 (ms) 1 130 320 2 90 210 3 90 210 4 10 220 5 110 220 6 90 210 7 10 220 8 100 22 Mean 10 N1 (ms) 80 23 Amplitude (uV) 5 0 -5 -10 -15 0 100 200 300 400 500 600 Time (ms) 700 800 900 1000 30
  • 31. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 31
  • 32. 5- SIGNAL PROCESSING OF EEG SIGNALS Data adquisition 100 windows per Image 1 Target 99 Distractors 32
  • 33. 5- SIGNAL PROCESSING OF EEG SIGNALS Preprocessing: Single trial One Image 1 2 3 4 5 6 7 8 33
  • 34. 5- SIGNAL PROCESSING OF EEG SIGNALS Preprocessing: Single trial - PROBLEM - Signal very noisy - Single Targets and Single Distractors very similar 34
  • 35. 5- SIGNAL PROCESSING OF EEG SIGNALS Preprocessing: Feature 96 Distractors Mean Absolute Amplitude looks different 96 Targets Energy from 0 to 600ms Feature for the window presented 35
  • 36. 5- SIGNAL PROCESSING OF EEG SIGNALS Preprocessing: Averaged trials 1 averaged target 99 averaged distractors Energy from 0-600 ms 36
  • 37. 5- SIGNAL PROCESSING OF EEG SIGNALS Preprocessing: Averaged trials SINGLE AVERAGED 1 x 32 single repeats 1 averaged target 99 x 32 single repeats 99 averaged distractors 1 x 32 window repeats 1 averaged target 99 x 32 single repeats Average by 32 99 averaged distractors 1 x 32 window repeats 1 averaged target 99 x 32 single repeats 99 averaged distractors 37
  • 38. 5- SIGNAL PROCESSING OF EEG SIGNALS Preprocessing: Averaged trials 1 averaged target Problem: Too few (target) samples for training 99 averaged distractors 1 averaged target 99 averaged distractors 1 averaged target 99 averaged distractors 38
  • 39. 5- SIGNAL PROCESSING OF EEG SIGNALS Bootstrapping SINGLE AVERAGED 1 x 32 single repeats 1 x 32 averaged target 99 x 32 single repeats 99 x 1 averaged distractors 1 x 32 window repeats 99 x 32 single repeats 1 x 32 window repeats 99 x 32 single repeats 1 x 32 averaged target Bootstrapping 99 x 1 averaged distractors 1 x 32 averaged target 99 x 1 averaged distractors39
  • 40. 5- SIGNAL PROCESSING OF EEG SIGNALS Classification Problem: Unbalanced dataset for binary classification AVERAGED 1 x 32 averaged target 99 x 1 averaged distractors 1 x 32 averaged target 99 x 1 averaged distractors 1 x 32 averaged target 99 x 1 averaged distractors40
  • 41. 5- SIGNAL PROCESSING OF EEG SIGNALS Classification AVERAGED AVERAGED 1 x 32 averaged targets 99 x 1 averaged distractors 1 x 32 averaged targets Subsample 1 x 32 averaged targets 99 x 1 averaged distractors 1 x 32 averaged targets Subsample 1 x 32 averaged targets 99 x 1 averaged distractors 32 x 1 avgd distractors 32 x 1 avgd distractors 1 x 32 averaged targets Subsample 32 x 1 avgd distractors 41
  • 42. 5- SIGNAL PROCESSING OF EEG SIGNALS Classification AVERAGED HISTOGRAM 1 x 32 averaged targets 32 x 1 avgd distractors 1 x 32 averaged targets 32 x 1 avgd distractors 1 x 32 averaged targets 32 x 1 avgd distractors 42
  • 43. 5- SIGNAL PROCESSING OF EEG SIGNALS Classification AVERAGED 1 x 32 averaged targets 32 x 1 avgd distractors 1 x 32 averaged targets 32 x 1 avgd distractors SVMTRAIN (linear kernel) Classifier 1 x 32 averaged targets 32 x 1 avgd distractors 43
  • 44. 5- SIGNAL PROCESSING OF EEG SIGNALS Classification 100 x 32 single repeats Average by 32 100 avgd windows 44
  • 45. 5- SIGNAL PROCESSING OF EEG SIGNALS Classification 100 x 32 single repeats Average by 32 100 avgd samples Classifier SVM PREDICT 45
  • 46. 5- SIGNAL PROCESSING OF EEG SIGNALS 8 samples feature vectors Cross validation approach 3 train + 1 test
  • 47. CONTENT 1 - Problem statement 2 – Related Work 3 – Local exploration of the image 4 – Experimental set-up 5 – Signal Processing of EEG Signals 6 – Conclusions 47
  • 48. 6- CONCLUSIONS - Results from sythetic images provide and evidence that BCI devices could be used to located an object into an image. - Simplicity of the system: Energy value from 8 channels to train SVM with lineal kernel. Future work -Study the impact of the number of repetitions. - Extract better features. -Analize data from real images. -Tool to evaluate and compare the EEG mask (ROC, Jaccard index) 48