FIVR: Fine-Grained Incident Video Retrieval. Presentation by Giorgos Kordopatis-Zilos, Symeon Papadopoulos, Ioannis Patras, Ioannis Kompatsiaris. CERTH-ITI

1. FIVR: Fine-Grained Incident Video Retrieval
Giorgos Kordopatis-Zilos Symeon Papadopoulos Ioannis Patras Ioannis Kompatsiaris

2. Fine-grained Incident Video Retrieval (FIVR)
Problem statement
• Retrieve all videos that depict the same incident given a query video
• Types of video associations with respect to an incident
• Duplicate videos
• Videos of the same incident
• Complementary viewpoint videos
• Different time interval videos
Application scenarios
• Multimedia verification, Copyright protection
• Video/movie production, Video forensic analysis
• Event reconstruction/mashup, News media analysis and reporting

3. Motivation & contribution
Motivation
• Composition of a large-scale challenging dataset
• User-generated videos
• Large number of real-world events of the same nature
• Challenging queries with several associated videos
Contribution
• FIVR introduction and the definition of three video associations
• Creation of a large-scale dataset (FIVR-200K) consisting of 225,960 videos
• Development of a process for the collection, annotation, and selection queries
• Comprehensive experimental study

4. Video retrieval paradigm
Query video Retrieved videos
Video database

5. Timeline of an incident
Query Video
Complementary Scene VideoIncident Scene Video

6. Video examples
Query Video
Complementary
Scene Video
Duplicate
Scene Video
Incident
Scene Video

7. Dataset collection

8. Dataset collection
Crawling
• Crawling of Wikipedia's Current Event1
• Collection of the major news events
• Crawled period: January 1st 2013 to December 31st 2017
1 https://en.wikipedia.org/wiki/Portal:Current_events

9. Dataset collection
Major news events
• News events are associated with a topic, headline, text, date, and hyperlinks
Event topics
Armed conflicts
and attacks
Disasters and
accidents
Science and
technology
Business and
economics
Sports Politics Religion Media

10. Dataset collection
Filtering
• Retain all news events with topic related to ‘Armed conflicts and attacks’ or ‘Disasters and
accidents’

11. Dataset collection
Querying
• Collect videos from YouTube
• Query YouTube APIs2 with news event headlines
• Limit the results between news event date and one week after event date
• Limit to videos with duration less than 5 minutes
2 https://developers.google.com/youtube/

12. Dataset collection
FIVR-200K composition
• Crawled news events from January 2013 to June 2017
• Collected 9,431 news events
• Retained 4,687 news events after filtering
• Collected videos from YouTube platform
• Total 225,960 videos (48 videos/event)

13. Query selection
Video Similarity
• Calculate video similarity combining visual and textual ones
Connected Components
• Build a graph based on the video similarity
• Extract the connected components
• Filter out components
• Low uploader ratio
• Contain videos uploaded in different time
Query selection
• Retain only short videos, i.e. less than 90 seconds
• Select the video that was published earliest
• Selection of 100 queries from the largest components

14. Annotation process
Annotation labels
• Near-Duplicate (ND)
• Duplicate Scene (DS)
• Complementary Scene (CS)
• Incident Scene (IS)
• Distractors (DI)

15. Experimental setup
Evaluation metrics
• mean Average Precision (mAP)
• interpolated Precision-Recall (PR) curve
Benchmarked methods
• Five state-of-the-art video retrieval approaches
• Six feature extraction methods (handcrafted and deep learning)
Retrieval tasks
• Three evaluation setups to simulate different scenarios of the problem
• Each setup accepts different labels

16. Benchmarked approaches

17. Comprehensive experiments

18. Comprehensive experiments

19. Comprehensive experiments

20. Comprehensive experiments

21. Thank you!
FIVR-200K publicly available in:
http://ndd.iti.gr/fivr/
https://github.com/MKLab-ITI/FIVR-200K
With the support of:
Get in touch:
Giorgos Kordopatis-Zilos: georgekordopatis@iti.gr / @g_kordo