Your SlideShare is downloading. ×
My talk at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

My talk at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

1,057
views

Published on

My talk slides at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

My talk slides at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,057
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 1 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne The Use of Non-Conventional Means for Media Content Analysis and Understanding - Brain Signals - Touradj Ebrahimi
  • 2. 2 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Observation • Human brain is still a more efficient processor for some tasks when compared to computers – Media content analysis and understanding
  • 3. 3 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Approach • Use human brain as a co-processor for advanced content analysis – Social networks applied to content analysis and annotation – Brain Computer Interface
  • 4. 4 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne BCI versus social tagging • Social tagging – Explicit – Verbal • BCI – Implicit – Non-verbal
  • 5. 5 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne A couple interesting illustrations • Curiosity cloning • Emotional tagging
  • 6. 6 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Curiosity cloning in deep space exploration • Pure pattern matching, Scientific Richness Index or other classifiers are programmed to find what we already know: the expected. Q: Can we code the interest in the unexpected? (Scientific) Curiosity?
  • 7. 7 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Curiosity cloning in deep space exploration • An alternative to explicit and specific definition of what we are looking for ...(e.g., dust-devils) • Present to experts (e.g., experts on mars geology) a lot of images and rate them • Images/Rating pairs can form a training set for a classifier • The classifier could be programmed on a rover • Robot’s processor would be a “clone” of the scientist’s interest, curiosity, expertise
  • 8. 8 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Specialist versus naïve subjects EEG Averaged PZ electrodes for subjects. Top left: specialist, Others: naïve subjects. Horizontal axis is the time after stimulus onset and vertical axis amplitude of the P300 signal. Scientifically interesting Target (red), Non-Target (blue) , Non-obvious target (dashed black) stimulus.
  • 9. 9 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne BCI for Emotional tagging AnalysisF r Biosignals acquisition EEG signals Feature vector Single window Classification Classifier Single window Aggregation
  • 10. 10 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Music: The Finest Language of Emotion
  • 11. 11 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Emotion representation
  • 12. 12 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne EEG Single Trial Classification • Goal: Predict valence, arousal and stance for each video. • Threshold subjects' arousal/valence/stance into two classes (e.g. positive or negative arousal) • Extract features using common spatial patterns algorithms • Use linear SVM classifier for classification. • Segment each video into 10 samples and test using leave-one- video-out cross-validation.
  • 13. 13 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Results a b a) Single trial classification b) aggregated result
  • 14. 14 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Challenges • More mature and more efficient solutions – Fortunately, our community is good at it!
  • 15. 15 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Challenges • Multimodal approach EEG Sensor cap Plethysmograph (bloodflow) Galvanic skin response Heart rate Temperature sensor Respiration sensor
  • 16. 16 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Signal aquistion Emotiv Epoc Dry EEG electrode headset Designed for games TU Eindhoven concept Dry EEG electrode diadem For rehabilitation EEG baseball cap Dry EEG electrode hat For everyday use OCZ NIA Dry EMG+EEG electrode headband For gaming Integration into headphones
  • 17. 17 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Future biosignals acquisition Mit MediaLab HandWave Bluetooth GSR sensors For man-machine interaction SenseWear Measures temperature, heat flux, GSR, movement For medical applications Fraunhofer institute EmoGlove Measures heart rate/GSR/temperature For man-machine interaction Biosensor mouse Measures GSR in the thumb For stress sensing in offices
  • 18. 18 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne More during discussions