SlideShare a Scribd company logo
1 of 10
Download to read offline
EUMSSI	
  team	
  at	
  the	
  MediaEval
Person	
  Discovery	
  Challenge	
  2016
Nam	
  Le,	
  Jean-­‐Marc	
  Odobez,	
  Sylvain	
  Meignier
{nle,	
  odobez}@idiap.ch
sylvain.meignier@univ-­‐lemans.fr
Overview
07/12/2016
Olivier  Truchot
Marisol Turaine
Video	
  OCR	
  and	
  NER3
07/12/2016
Original
Image
Text region
detection
Text
extraction
Text
recognition
Hypothesis
merging
• Multiple	
  image	
  segmentations	
  of	
  the	
  same	
  region	
  
è all	
  results	
  are	
  compared	
  and	
  aggregated	
  over	
  time	
  
è several	
  hypotheses	
  è high	
  recall
• NER	
  based	
  on	
  MITIE	
  with	
  heuristics.
Face	
  diarization4
07/12/2016
DPM
CRF-multi-target
Face	
  clustering Hierarchical clustering
shots
Face	
  tracking
Face	
  detection
Talking	
  face	
  detection5
07/12/2016
Face	
  track
9	
  directions	
  of	
  optical	
  flows
PCA	
  ⇒ 𝒙 𝒕
x% x& x'(&
LSTM LSTM LSTM…
x& x) x'
Mean	
  Pooling Classifier
ℎ%
ℎ&
h'(&
DW	
  dataset	
  for	
  talking	
  face	
  &	
  dubbing:	
  http://bit.ly/dw-­‐dubbing
• LIUM	
  diarization tool:	
  
www-­‐lium.univ-­‐lemans.fr/en/content/liumspkdiarization
• Input:	
  a	
  video
• Output:	
  homogeneous	
  segments	
  
Speaker	
  diarization6
07/12/2016
Result	
  ranking7
07/12/2016
• Direct naming: maximize co-occurrences between clusters and
named entities.
− Face naming: name 𝑁-
.
and talking score 𝑡 𝑁-
.
− Speaker naming: name 𝑁-
0
and equal score 1.0
• For one shot 𝑠 : 𝑄6 = 	
  ∅
• Names which face agrees with speaker naming rank highest:
− If ∃𝑁;
0
/𝑁-
.
= 𝑁;
0
: 𝑄6 	
  ← 𝑁-
.
, 2.0 + 𝑡 𝑁-
.
• Otherwise, face naming has higher rank:
− If ∄𝑁;
0
/𝑁-
.
= 𝑁;
0
: 𝑄6 	
  ← 𝑁-
.
, 1.0 + 𝑡 𝑁-
.
− If ∄𝑁-
0
/𝑁-
.
= 𝑁;
0
: 𝑄6 	
  ← 𝑁-
0
, 1.0
Result	
  ranking8
07/12/2016
Shot	
  1 Shot	
  2 Shot	
  3 Shot	
  4
Query: Results:	
  2	
  – 4	
  – 1	
  -­‐ 3	
  
Submissions9
07/12/2016
MAP@1 MAP@10 MAP@100
Sub.	
  (1) 30.3 22.0 21.0
Sub.	
  (2) 58.6 42.9 42.0
Sub. (3) 64.2 53.1 52.1
Sub.	
  (4) 68.3 56.2 54.7
Sub.	
  (5) 79.2 65.2 63.4
Face	
  diarization Baseline	
  OCR-­‐NER Face	
  namingSub.	
  (1)
Face	
  diarization Our	
  OCR-­‐NER Face	
  namingSub.	
  (2)
Face	
  diarization Our	
  OCR-­‐NER
Talking	
  
face	
  naming
Sub.	
  (3)
Face	
  diarization OCR-­‐NER Talking	
  face	
  
naming
+
Speaker	
  naming
Sub.	
  (4)
Speaker	
  
diarization
OCR-­‐NER
Sub.	
  (4)	
  +	
  Sub.	
  (1)	
  +	
  Baseline	
  2Sub.	
  (5)
12/7/16
The	
  End10

More Related Content

Viewers also liked

Viewers also liked (12)

MediaEval 2016 - COSMIR and the OpenMIC Challenge: A Plan for Sustainable Mus...
MediaEval 2016 - COSMIR and the OpenMIC Challenge: A Plan for Sustainable Mus...MediaEval 2016 - COSMIR and the OpenMIC Challenge: A Plan for Sustainable Mus...
MediaEval 2016 - COSMIR and the OpenMIC Challenge: A Plan for Sustainable Mus...
 
MediaEval 2016 - HUCVL Predicting Interesting Key Frames with Deep Models
MediaEval 2016 - HUCVL Predicting Interesting Key Frames with Deep ModelsMediaEval 2016 - HUCVL Predicting Interesting Key Frames with Deep Models
MediaEval 2016 - HUCVL Predicting Interesting Key Frames with Deep Models
 
MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop
MediaEval 2016 - IR Evaluation: Putting the User Back in the LoopMediaEval 2016 - IR Evaluation: Putting the User Back in the Loop
MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop
 
MediaEval 2016 - ININ Submission to Zero Cost ASR Task
MediaEval 2016 - ININ Submission to Zero Cost ASR TaskMediaEval 2016 - ININ Submission to Zero Cost ASR Task
MediaEval 2016 - ININ Submission to Zero Cost ASR Task
 
MediaEval 2016 - Simula Team @ Context of Experience Task
MediaEval 2016 - Simula Team @ Context of Experience TaskMediaEval 2016 - Simula Team @ Context of Experience Task
MediaEval 2016 - Simula Team @ Context of Experience Task
 
MediaEval 2016 - BUT Zero-Cost Speech Recognition
MediaEval 2016 - BUT Zero-Cost Speech RecognitionMediaEval 2016 - BUT Zero-Cost Speech Recognition
MediaEval 2016 - BUT Zero-Cost Speech Recognition
 
MediaEval 2016 - UNIFESP Predicting Media Interestingness Task
MediaEval 2016 - UNIFESP Predicting Media Interestingness TaskMediaEval 2016 - UNIFESP Predicting Media Interestingness Task
MediaEval 2016 - UNIFESP Predicting Media Interestingness Task
 
MediaEval 2016 - UPMC at MediaEval2016 Retrieving Diverse Social Images Task
MediaEval 2016 - UPMC at MediaEval2016 Retrieving Diverse Social Images TaskMediaEval 2016 - UPMC at MediaEval2016 Retrieving Diverse Social Images Task
MediaEval 2016 - UPMC at MediaEval2016 Retrieving Diverse Social Images Task
 
MediaEval 2016 - Emotion in Music Task: Lessons Learned
MediaEval 2016 - Emotion in Music Task: Lessons LearnedMediaEval 2016 - Emotion in Music Task: Lessons Learned
MediaEval 2016 - Emotion in Music Task: Lessons Learned
 
MediaEval 2016 - TUD-MMC Predicting media Interestingness Task
MediaEval 2016 - TUD-MMC Predicting media Interestingness TaskMediaEval 2016 - TUD-MMC Predicting media Interestingness Task
MediaEval 2016 - TUD-MMC Predicting media Interestingness Task
 
MediaEval 2016 - ETH-CVL: Textual-Visual Embeddings and Video2GIF for Video I...
MediaEval 2016 - ETH-CVL: Textual-Visual Embeddings and Video2GIF for Video I...MediaEval 2016 - ETH-CVL: Textual-Visual Embeddings and Video2GIF for Video I...
MediaEval 2016 - ETH-CVL: Textual-Visual Embeddings and Video2GIF for Video I...
 
MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task
MediaEval 2016: A Multimodal System for the Verifying Multimedia Use TaskMediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task
MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task
 

Similar to MediaEval 2016 - EUMSSI Team at the MediaEval Person Discovery Challenge

"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"..."How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
Edge AI and Vision Alliance
 
"Approaches for Vision-based Driver Monitoring," a Presentation from PathPart...
"Approaches for Vision-based Driver Monitoring," a Presentation from PathPart..."Approaches for Vision-based Driver Monitoring," a Presentation from PathPart...
"Approaches for Vision-based Driver Monitoring," a Presentation from PathPart...
Edge AI and Vision Alliance
 
LiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptx
LiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptxLiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptx
LiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptx
VishnuRajuV
 

Similar to MediaEval 2016 - EUMSSI Team at the MediaEval Person Discovery Challenge (20)

MedGIFT projects in medical imaging
MedGIFT projects in medical imagingMedGIFT projects in medical imaging
MedGIFT projects in medical imaging
 
MediaEval 2015 - Multimodal Person Discovery in Broadcast TV
MediaEval 2015 - Multimodal Person Discovery in Broadcast TVMediaEval 2015 - Multimodal Person Discovery in Broadcast TV
MediaEval 2015 - Multimodal Person Discovery in Broadcast TV
 
Decision-Point Panorama-Based Indoor Navigation
Decision-Point Panorama-Based Indoor NavigationDecision-Point Panorama-Based Indoor Navigation
Decision-Point Panorama-Based Indoor Navigation
 
Face recognition svm+pca
Face recognition   svm+pcaFace recognition   svm+pca
Face recognition svm+pca
 
Open and Collaborative Software for Digital Pathology
Open and Collaborative Software for Digital Pathology Open and Collaborative Software for Digital Pathology
Open and Collaborative Software for Digital Pathology
 
Introduction to Visual Analysis
Introduction to Visual AnalysisIntroduction to Visual Analysis
Introduction to Visual Analysis
 
Perception and Quality of Immersive Media
Perception and Quality of Immersive MediaPerception and Quality of Immersive Media
Perception and Quality of Immersive Media
 
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"..."How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
"How Image Sensor and Video Compression Parameters Impact Vision Algorithms,"...
 
MediaEval 2017 - Satellite Task: Visual and textual analysis of social media ...
MediaEval 2017 - Satellite Task: Visual and textual analysis of social media ...MediaEval 2017 - Satellite Task: Visual and textual analysis of social media ...
MediaEval 2017 - Satellite Task: Visual and textual analysis of social media ...
 
"Approaches for Vision-based Driver Monitoring," a Presentation from PathPart...
"Approaches for Vision-based Driver Monitoring," a Presentation from PathPart..."Approaches for Vision-based Driver Monitoring," a Presentation from PathPart...
"Approaches for Vision-based Driver Monitoring," a Presentation from PathPart...
 
Case Study: Lecture Capture goes Campus-Wide at Creighton University with Pan...
Case Study: Lecture Capture goes Campus-Wide at Creighton University with Pan...Case Study: Lecture Capture goes Campus-Wide at Creighton University with Pan...
Case Study: Lecture Capture goes Campus-Wide at Creighton University with Pan...
 
Video Retrieval of Specific Persons in Specific Locations
Video Retrieval of Specific Persons in Specific LocationsVideo Retrieval of Specific Persons in Specific Locations
Video Retrieval of Specific Persons in Specific Locations
 
Elderly Assistance- Deep Learning Theme detection
Elderly Assistance- Deep Learning Theme detectionElderly Assistance- Deep Learning Theme detection
Elderly Assistance- Deep Learning Theme detection
 
An analysis of_machine_and_human_analytics_in_classification
An analysis of_machine_and_human_analytics_in_classificationAn analysis of_machine_and_human_analytics_in_classification
An analysis of_machine_and_human_analytics_in_classification
 
Facial Expression Recognition
Facial Expression Recognition Facial Expression Recognition
Facial Expression Recognition
 
LiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptx
LiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptxLiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptx
LiDeng-BerlinOct2015-ASR-GenDisc-4by3.pptx
 
Burnaev and Notchenko. Skoltech. Bridging gap between 2D and 3D with Deep Lea...
Burnaev and Notchenko. Skoltech. Bridging gap between 2D and 3D with Deep Lea...Burnaev and Notchenko. Skoltech. Bridging gap between 2D and 3D with Deep Lea...
Burnaev and Notchenko. Skoltech. Bridging gap between 2D and 3D with Deep Lea...
 
Face Recognition Methods based on Convolutional Neural Networks
Face Recognition Methods based on Convolutional Neural NetworksFace Recognition Methods based on Convolutional Neural Networks
Face Recognition Methods based on Convolutional Neural Networks
 
Towards Machine Comprehension of Spoken Content
Towards Machine Comprehension of Spoken ContentTowards Machine Comprehension of Spoken Content
Towards Machine Comprehension of Spoken Content
 
Scottish Urban Air Quality Steering Group - Modelling & Monitoring Workshop -...
Scottish Urban Air Quality Steering Group - Modelling & Monitoring Workshop -...Scottish Urban Air Quality Steering Group - Modelling & Monitoring Workshop -...
Scottish Urban Air Quality Steering Group - Modelling & Monitoring Workshop -...
 

More from multimediaeval

Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...
Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...
Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...
multimediaeval
 

More from multimediaeval (20)

Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal...
Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal...Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal...
Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal...
 
HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table...
HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table...HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table...
HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table...
 
Sports Video Classification: Classification of Strokes in Table Tennis for Me...
Sports Video Classification: Classification of Strokes in Table Tennis for Me...Sports Video Classification: Classification of Strokes in Table Tennis for Me...
Sports Video Classification: Classification of Strokes in Table Tennis for Me...
 
Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention...
Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention...Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention...
Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention...
 
Essex-NLIP at MediaEval Predicting Media Memorability 2020 Task
Essex-NLIP at MediaEval Predicting Media Memorability 2020 TaskEssex-NLIP at MediaEval Predicting Media Memorability 2020 Task
Essex-NLIP at MediaEval Predicting Media Memorability 2020 Task
 
Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a V...
Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a V...Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a V...
Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a V...
 
Fooling an Automatic Image Quality Estimator
Fooling an Automatic Image Quality EstimatorFooling an Automatic Image Quality Estimator
Fooling an Automatic Image Quality Estimator
 
Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable C...
Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable C...Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable C...
Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable C...
 
Pixel Privacy: Quality Camouflage for Social Images
Pixel Privacy: Quality Camouflage for Social ImagesPixel Privacy: Quality Camouflage for Social Images
Pixel Privacy: Quality Camouflage for Social Images
 
HCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-Matching
HCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-MatchingHCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-Matching
HCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-Matching
 
Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...
Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...
Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...
 
HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ ...
HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ ...HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ ...
HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ ...
 
Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Int...
Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Int...Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Int...
Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Int...
 
Deep Conditional Adversarial learning for polyp Segmentation
Deep Conditional Adversarial learning for polyp SegmentationDeep Conditional Adversarial learning for polyp Segmentation
Deep Conditional Adversarial learning for polyp Segmentation
 
A Temporal-Spatial Attention Model for Medical Image Detection
A Temporal-Spatial Attention Model for Medical Image DetectionA Temporal-Spatial Attention Model for Medical Image Detection
A Temporal-Spatial Attention Model for Medical Image Detection
 
HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Netw...
HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Netw...HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Netw...
HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Netw...
 
Fine-tuning for Polyp Segmentation with Attention
Fine-tuning for Polyp Segmentation with AttentionFine-tuning for Polyp Segmentation with Attention
Fine-tuning for Polyp Segmentation with Attention
 
Bigger Networks are not Always Better: Deep Convolutional Neural Networks for...
Bigger Networks are not Always Better: Deep Convolutional Neural Networks for...Bigger Networks are not Always Better: Deep Convolutional Neural Networks for...
Bigger Networks are not Always Better: Deep Convolutional Neural Networks for...
 
Insights for wellbeing: Predicting Personal Air Quality Index using Regressio...
Insights for wellbeing: Predicting Personal Air Quality Index using Regressio...Insights for wellbeing: Predicting Personal Air Quality Index using Regressio...
Insights for wellbeing: Predicting Personal Air Quality Index using Regressio...
 
Use Visual Features From Surrounding Scenes to Improve Personal Air Quality ...
 Use Visual Features From Surrounding Scenes to Improve Personal Air Quality ... Use Visual Features From Surrounding Scenes to Improve Personal Air Quality ...
Use Visual Features From Surrounding Scenes to Improve Personal Air Quality ...
 

Recently uploaded

GENETICALLY MODIFIED ORGANISM'S PRESENTATION.ppt
GENETICALLY MODIFIED ORGANISM'S PRESENTATION.pptGENETICALLY MODIFIED ORGANISM'S PRESENTATION.ppt
GENETICALLY MODIFIED ORGANISM'S PRESENTATION.ppt
SyedArifMalki
 

Recently uploaded (20)

An Overview of Active and Passive Targeting Strategies to Improve the Nano-Ca...
An Overview of Active and Passive Targeting Strategies to Improve the Nano-Ca...An Overview of Active and Passive Targeting Strategies to Improve the Nano-Ca...
An Overview of Active and Passive Targeting Strategies to Improve the Nano-Ca...
 
Taphonomy and Quality of the Fossil Record
Taphonomy and Quality of the  Fossil RecordTaphonomy and Quality of the  Fossil Record
Taphonomy and Quality of the Fossil Record
 
NUMERICAL Proof Of TIme Electron Theory.
NUMERICAL Proof Of TIme Electron Theory.NUMERICAL Proof Of TIme Electron Theory.
NUMERICAL Proof Of TIme Electron Theory.
 
X-rays from a Central “Exhaust Vent” of the Galactic Center Chimney
X-rays from a Central “Exhaust Vent” of the Galactic Center ChimneyX-rays from a Central “Exhaust Vent” of the Galactic Center Chimney
X-rays from a Central “Exhaust Vent” of the Galactic Center Chimney
 
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRingsTransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
 
GENETICALLY MODIFIED ORGANISM'S PRESENTATION.ppt
GENETICALLY MODIFIED ORGANISM'S PRESENTATION.pptGENETICALLY MODIFIED ORGANISM'S PRESENTATION.ppt
GENETICALLY MODIFIED ORGANISM'S PRESENTATION.ppt
 
TEST BANK for Organic Chemistry 6th Edition.pdf
TEST BANK for Organic Chemistry 6th Edition.pdfTEST BANK for Organic Chemistry 6th Edition.pdf
TEST BANK for Organic Chemistry 6th Edition.pdf
 
Adaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte CarloAdaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte Carlo
 
Manganese‐RichSandstonesasanIndicatorofAncientOxic LakeWaterConditionsinGale...
Manganese‐RichSandstonesasanIndicatorofAncientOxic  LakeWaterConditionsinGale...Manganese‐RichSandstonesasanIndicatorofAncientOxic  LakeWaterConditionsinGale...
Manganese‐RichSandstonesasanIndicatorofAncientOxic LakeWaterConditionsinGale...
 
A Scientific PowerPoint on Albert Einstein
A Scientific PowerPoint on Albert EinsteinA Scientific PowerPoint on Albert Einstein
A Scientific PowerPoint on Albert Einstein
 
Heads-Up Multitasker: CHI 2024 Presentation.pdf
Heads-Up Multitasker: CHI 2024 Presentation.pdfHeads-Up Multitasker: CHI 2024 Presentation.pdf
Heads-Up Multitasker: CHI 2024 Presentation.pdf
 
Vital Signs of Animals Presentation By Aftab Ahmed Rahimoon
Vital Signs of Animals Presentation By Aftab Ahmed RahimoonVital Signs of Animals Presentation By Aftab Ahmed Rahimoon
Vital Signs of Animals Presentation By Aftab Ahmed Rahimoon
 
Polyethylene and its polymerization.pptx
Polyethylene and its polymerization.pptxPolyethylene and its polymerization.pptx
Polyethylene and its polymerization.pptx
 
ABHISHEK ANTIBIOTICS PPT MICROBIOLOGY // USES OF ANTIOBIOTICS TYPES OF ANTIB...
ABHISHEK ANTIBIOTICS PPT MICROBIOLOGY  // USES OF ANTIOBIOTICS TYPES OF ANTIB...ABHISHEK ANTIBIOTICS PPT MICROBIOLOGY  // USES OF ANTIOBIOTICS TYPES OF ANTIB...
ABHISHEK ANTIBIOTICS PPT MICROBIOLOGY // USES OF ANTIOBIOTICS TYPES OF ANTIB...
 
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptxSaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
 
GBSN - Biochemistry (Unit 8) Enzymology
GBSN - Biochemistry (Unit 8) EnzymologyGBSN - Biochemistry (Unit 8) Enzymology
GBSN - Biochemistry (Unit 8) Enzymology
 
Costs to heap leach gold ore tailings in Karamoja region of Uganda
Costs to heap leach gold ore tailings in Karamoja region of UgandaCosts to heap leach gold ore tailings in Karamoja region of Uganda
Costs to heap leach gold ore tailings in Karamoja region of Uganda
 
MSCII_ FCT UNIT 5 TOXICOLOGY.pdf
MSCII_              FCT UNIT 5 TOXICOLOGY.pdfMSCII_              FCT UNIT 5 TOXICOLOGY.pdf
MSCII_ FCT UNIT 5 TOXICOLOGY.pdf
 
EU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdfEU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdf
 
GBSN - Microbiology (Unit 5) Concept of isolation
GBSN - Microbiology (Unit 5) Concept of isolationGBSN - Microbiology (Unit 5) Concept of isolation
GBSN - Microbiology (Unit 5) Concept of isolation
 

MediaEval 2016 - EUMSSI Team at the MediaEval Person Discovery Challenge

  • 1. EUMSSI  team  at  the  MediaEval Person  Discovery  Challenge  2016 Nam  Le,  Jean-­‐Marc  Odobez,  Sylvain  Meignier {nle,  odobez}@idiap.ch sylvain.meignier@univ-­‐lemans.fr
  • 3. Video  OCR  and  NER3 07/12/2016 Original Image Text region detection Text extraction Text recognition Hypothesis merging • Multiple  image  segmentations  of  the  same  region   è all  results  are  compared  and  aggregated  over  time   è several  hypotheses  è high  recall • NER  based  on  MITIE  with  heuristics.
  • 4. Face  diarization4 07/12/2016 DPM CRF-multi-target Face  clustering Hierarchical clustering shots Face  tracking Face  detection
  • 5. Talking  face  detection5 07/12/2016 Face  track 9  directions  of  optical  flows PCA  ⇒ 𝒙 𝒕 x% x& x'(& LSTM LSTM LSTM… x& x) x' Mean  Pooling Classifier ℎ% ℎ& h'(& DW  dataset  for  talking  face  &  dubbing:  http://bit.ly/dw-­‐dubbing
  • 6. • LIUM  diarization tool:   www-­‐lium.univ-­‐lemans.fr/en/content/liumspkdiarization • Input:  a  video • Output:  homogeneous  segments   Speaker  diarization6 07/12/2016
  • 7. Result  ranking7 07/12/2016 • Direct naming: maximize co-occurrences between clusters and named entities. − Face naming: name 𝑁- . and talking score 𝑡 𝑁- . − Speaker naming: name 𝑁- 0 and equal score 1.0 • For one shot 𝑠 : 𝑄6 =  ∅ • Names which face agrees with speaker naming rank highest: − If ∃𝑁; 0 /𝑁- . = 𝑁; 0 : 𝑄6  ← 𝑁- . , 2.0 + 𝑡 𝑁- . • Otherwise, face naming has higher rank: − If ∄𝑁; 0 /𝑁- . = 𝑁; 0 : 𝑄6  ← 𝑁- . , 1.0 + 𝑡 𝑁- . − If ∄𝑁- 0 /𝑁- . = 𝑁; 0 : 𝑄6  ← 𝑁- 0 , 1.0
  • 8. Result  ranking8 07/12/2016 Shot  1 Shot  2 Shot  3 Shot  4 Query: Results:  2  – 4  – 1  -­‐ 3  
  • 9. Submissions9 07/12/2016 MAP@1 MAP@10 MAP@100 Sub.  (1) 30.3 22.0 21.0 Sub.  (2) 58.6 42.9 42.0 Sub. (3) 64.2 53.1 52.1 Sub.  (4) 68.3 56.2 54.7 Sub.  (5) 79.2 65.2 63.4 Face  diarization Baseline  OCR-­‐NER Face  namingSub.  (1) Face  diarization Our  OCR-­‐NER Face  namingSub.  (2) Face  diarization Our  OCR-­‐NER Talking   face  naming Sub.  (3) Face  diarization OCR-­‐NER Talking  face   naming + Speaker  naming Sub.  (4) Speaker   diarization OCR-­‐NER Sub.  (4)  +  Sub.  (1)  +  Baseline  2Sub.  (5)