1
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
The Use of Non-Conventional
Means for...
2
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Observation
• Human brain is still a ...
3
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Approach
• Use human brain as a co-pr...
4
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
BCI versus social tagging
• Social ta...
5
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
A couple interesting illustrations
• ...
6
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Curiosity cloning in deep space explo...
7
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Curiosity cloning in deep space explo...
8
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Specialist versus naïve subjects EEG
...
9
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
BCI for Emotional tagging
AnalysisF
r...
10
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Music: The Finest Language of Emotion
11
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Emotion representation
12
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
EEG Single Trial Classification
• Go...
13
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Results
a
b
a) Single trial classifi...
14
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Challenges
• More mature and more ef...
15
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Challenges
• Multimodal approach
EEG...
16
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Signal aquistion
Emotiv Epoc
Dry EEG...
17
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
Future biosignals acquisition
Mit Me...
18
Multimedia Signal Processing Group
Swiss Federal Institute of Technology, Lausanne
More during discussions
Upcoming SlideShare
Loading in …5
×

My talk at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

1,250 views
1,173 views

Published on

My talk slides at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,250
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

My talk at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

  1. 1. 1 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne The Use of Non-Conventional Means for Media Content Analysis and Understanding - Brain Signals - Touradj Ebrahimi
  2. 2. 2 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Observation • Human brain is still a more efficient processor for some tasks when compared to computers – Media content analysis and understanding
  3. 3. 3 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Approach • Use human brain as a co-processor for advanced content analysis – Social networks applied to content analysis and annotation – Brain Computer Interface
  4. 4. 4 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne BCI versus social tagging • Social tagging – Explicit – Verbal • BCI – Implicit – Non-verbal
  5. 5. 5 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne A couple interesting illustrations • Curiosity cloning • Emotional tagging
  6. 6. 6 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Curiosity cloning in deep space exploration • Pure pattern matching, Scientific Richness Index or other classifiers are programmed to find what we already know: the expected. Q: Can we code the interest in the unexpected? (Scientific) Curiosity?
  7. 7. 7 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Curiosity cloning in deep space exploration • An alternative to explicit and specific definition of what we are looking for ...(e.g., dust-devils) • Present to experts (e.g., experts on mars geology) a lot of images and rate them • Images/Rating pairs can form a training set for a classifier • The classifier could be programmed on a rover • Robot’s processor would be a “clone” of the scientist’s interest, curiosity, expertise
  8. 8. 8 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Specialist versus naïve subjects EEG Averaged PZ electrodes for subjects. Top left: specialist, Others: naïve subjects. Horizontal axis is the time after stimulus onset and vertical axis amplitude of the P300 signal. Scientifically interesting Target (red), Non-Target (blue) , Non-obvious target (dashed black) stimulus.
  9. 9. 9 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne BCI for Emotional tagging AnalysisF r Biosignals acquisition EEG signals Feature vector Single window Classification Classifier Single window Aggregation
  10. 10. 10 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Music: The Finest Language of Emotion
  11. 11. 11 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Emotion representation
  12. 12. 12 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne EEG Single Trial Classification • Goal: Predict valence, arousal and stance for each video. • Threshold subjects' arousal/valence/stance into two classes (e.g. positive or negative arousal) • Extract features using common spatial patterns algorithms • Use linear SVM classifier for classification. • Segment each video into 10 samples and test using leave-one- video-out cross-validation.
  13. 13. 13 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Results a b a) Single trial classification b) aggregated result
  14. 14. 14 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Challenges • More mature and more efficient solutions – Fortunately, our community is good at it!
  15. 15. 15 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Challenges • Multimodal approach EEG Sensor cap Plethysmograph (bloodflow) Galvanic skin response Heart rate Temperature sensor Respiration sensor
  16. 16. 16 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Signal aquistion Emotiv Epoc Dry EEG electrode headset Designed for games TU Eindhoven concept Dry EEG electrode diadem For rehabilitation EEG baseball cap Dry EEG electrode hat For everyday use OCZ NIA Dry EMG+EEG electrode headband For gaming Integration into headphones
  17. 17. 17 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Future biosignals acquisition Mit MediaLab HandWave Bluetooth GSR sensors For man-machine interaction SenseWear Measures temperature, heat flux, GSR, movement For medical applications Fraunhofer institute EmoGlove Measures heart rate/GSR/temperature For man-machine interaction Biosensor mouse Measures GSR in the thumb For stress sensing in offices
  18. 18. 18 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne More during discussions

×