Your SlideShare is downloading. ×
  • Like
  • Save
My talk at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

My talk at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

  • 1,033 views
Published

My talk slides at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

My talk slides at the ACM Multimedia 2010 panel on The Use of Non-conventional Means for Media Content Analysis and Understanding

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,033
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. 1 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne The Use of Non-Conventional Means for Media Content Analysis and Understanding - Brain Signals - Touradj Ebrahimi
  • 2. 2 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Observation • Human brain is still a more efficient processor for some tasks when compared to computers – Media content analysis and understanding
  • 3. 3 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Approach • Use human brain as a co-processor for advanced content analysis – Social networks applied to content analysis and annotation – Brain Computer Interface
  • 4. 4 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne BCI versus social tagging • Social tagging – Explicit – Verbal • BCI – Implicit – Non-verbal
  • 5. 5 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne A couple interesting illustrations • Curiosity cloning • Emotional tagging
  • 6. 6 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Curiosity cloning in deep space exploration • Pure pattern matching, Scientific Richness Index or other classifiers are programmed to find what we already know: the expected. Q: Can we code the interest in the unexpected? (Scientific) Curiosity?
  • 7. 7 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Curiosity cloning in deep space exploration • An alternative to explicit and specific definition of what we are looking for ...(e.g., dust-devils) • Present to experts (e.g., experts on mars geology) a lot of images and rate them • Images/Rating pairs can form a training set for a classifier • The classifier could be programmed on a rover • Robot’s processor would be a “clone” of the scientist’s interest, curiosity, expertise
  • 8. 8 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Specialist versus naïve subjects EEG Averaged PZ electrodes for subjects. Top left: specialist, Others: naïve subjects. Horizontal axis is the time after stimulus onset and vertical axis amplitude of the P300 signal. Scientifically interesting Target (red), Non-Target (blue) , Non-obvious target (dashed black) stimulus.
  • 9. 9 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne BCI for Emotional tagging AnalysisF r Biosignals acquisition EEG signals Feature vector Single window Classification Classifier Single window Aggregation
  • 10. 10 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Music: The Finest Language of Emotion
  • 11. 11 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Emotion representation
  • 12. 12 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne EEG Single Trial Classification • Goal: Predict valence, arousal and stance for each video. • Threshold subjects' arousal/valence/stance into two classes (e.g. positive or negative arousal) • Extract features using common spatial patterns algorithms • Use linear SVM classifier for classification. • Segment each video into 10 samples and test using leave-one- video-out cross-validation.
  • 13. 13 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Results a b a) Single trial classification b) aggregated result
  • 14. 14 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Challenges • More mature and more efficient solutions – Fortunately, our community is good at it!
  • 15. 15 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Challenges • Multimodal approach EEG Sensor cap Plethysmograph (bloodflow) Galvanic skin response Heart rate Temperature sensor Respiration sensor
  • 16. 16 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Signal aquistion Emotiv Epoc Dry EEG electrode headset Designed for games TU Eindhoven concept Dry EEG electrode diadem For rehabilitation EEG baseball cap Dry EEG electrode hat For everyday use OCZ NIA Dry EMG+EEG electrode headband For gaming Integration into headphones
  • 17. 17 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne Future biosignals acquisition Mit MediaLab HandWave Bluetooth GSR sensors For man-machine interaction SenseWear Measures temperature, heat flux, GSR, movement For medical applications Fraunhofer institute EmoGlove Measures heart rate/GSR/temperature For man-machine interaction Biosensor mouse Measures GSR in the thumb For stress sensing in offices
  • 18. 18 Multimedia Signal Processing Group Swiss Federal Institute of Technology, Lausanne More during discussions