• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Towards Using Semantic Features for Near-Duplicate Video Detection
 

Towards Using Semantic Features for Near-Duplicate Video Detection

on

  • 1,210 views

Towards Using Semantic Features for Near-Duplicate Video Detection.

Towards Using Semantic Features for Near-Duplicate Video Detection.

Paper presented at the ICME 2010 Workshop on Visual Content Identification and Search in Singapore.

Statistics

Views

Total Views
1,210
Views on SlideShare
1,209
Embed Views
1

Actions

Likes
3
Downloads
0
Comments
0

1 Embed 1

https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Towards Using Semantic Features for Near-Duplicate Video Detection Towards Using Semantic Features for Near-Duplicate Video Detection Presentation Transcript

    • Towards using Semantic Features for Near-duplicate Video Detection Workshop on Visual Content Identification and Search Singapore – July 23, 2010 Hyun-seok Min , Wesley De Neve, Yong Man Ro Image and Video Systems Lab Department of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) email: hsmin@kaist.ac.kr
    • Outline semantic video signature 3. Experiments semantic concepts discussion methodology 4. Conclusions and future work background challenges 1. Introduction 2. Towards semantic-based NDVC detection /22
    • Outline semantic video signature 3. Experiments semantic concepts discussion methodology 4. Conclusions and future work background challenges 1. Introduction 2. Towards semantic-based NDVC detection /22
    • Background
      • Increasing amount of online digital video content
        • easy-to-use multimedia devices
        • cheap storage and bandwidth
      • Increasing number of near-duplicate video clips
        • digital video content can be easily edited and redistributed
      • Near-duplicate video clips or NDVCs may infringe copyright or clutter search results
        • need for identifying NDVCs
      /22 Search results on YouTube for the query “I will survive Jesus” A significant number of search results are near-duplicates
    • NDVC Definition
      • Basic definition
        • identical or approximately identical videos
          • were the subject of at least one transformation
          • transformations preserve semantic information
        • addresses copyright infringement
      • Extended definition
        • includes a user-centric component
          • not considered as NDVCs by users: identical video clips that contain relevant complementary information
          • considered as NDVCs by users: video clips that are not alike, but that are visually similar and semantically related
        • addresses cluttered search results
      /22
    • NDVC Examples /22 NDVC transformation (cam cording, subtitles) transformation (blur) original videos
    • Video Signature
      • What is a video signature ?
        • represents a video clip with a unique set of features
      • Conventional video signatures
        • often created by extracting low-level visual features from video frames
      /22 video clip … video signature feature extraction
    • Video Signature based on Low-level Visual Features
      • Problem
        • near-duplicates are often not visually similar
      /22 original video NDVC transformation (cam cording, subtitles) … … video signature video signature Visual match? No!
    • Outline semantic video signature 3. Experiments semantic concepts discussion methodology 4. Conclusions and future work background challenges 1. Introduction 2. Towards semantic-based NDVC detection /22
    • Video Signature based on Semantic Features
      • Observation
        • near-duplicates often contain similar semantics
      10/22 original video NDVC transformation (cam cording, subtitles) Semantic concepts: Semantic concepts: Semantic match? Yes! indoor, man, face, … indoor, man, face, …
    • Semantic Video Signature
      • Represents the temporal variation of several semantic concepts in a video clip
        • allows dealing with a limited concept vocabulary
      semantic video signature (binary-valued matrix) 1 1 0 1 1 sky ... 0 0 1 0 0 indoor ... keyframes (one for each shot) ... 0 1 0 0 1 architecture ... ...
    • Outline semantic video signature 3. Experiments semantic concepts discussion methodology 4. Conclusions and future work background challenges 1. Introduction 2. Towards semantic-based NDVC detection /22
    • Methodology (1/2)
      • Concept detection uses Support Vector Machines
      • 3 criteria were used to select semantic concepts
        • represent visual concepts
        • need to be highly popular/common
        • detection is easy and reliable
      • Selected semantic concepts
        • ‘ gravel’, ‘park’, ‘pavement’, ‘road’, ‘rock’, ‘sand’, ‘sidewalk’, ‘face’, ‘people’, ‘indoor-light’, ‘field’, ‘peak’, ‘wood’, ‘night’, ‘street-light’, ‘flowers’, ‘leaves’, ‘trees’, ‘cloudy’, ‘sunny’, ‘sunset’, ‘brick’, ‘arch’, ‘buildings’, ‘wall’, ‘windows’, ‘beach’, ‘high-wave’, ‘low-wave’, ‘still water’, ‘mirrored water’, and ‘snow’
      /22
    • Methodology (2/2)
      • Database
        • reference video and query video set
          • MUSCLE-VCD-2007 set
        • size
          • reference video set: 101 videos
          • query video set: 15 videos
      • Low-level visual features
        • color
          • color structure (CS), color layout (CL), and scalable color (SC)
        • texture
          • homogeneous texture (HT), edge histogram (EH)
      14/22
    • Robustness Against Transformations (1/2)
      • Requirement
        • NDVCs should have a highly similar semantic signature
      • Transformations applied
        • blur, picture-in-picture, brightness
      • Example transformation
      /22 Original frame Blurred frames using various strengths (filter size: 5, 9, 13, 17, 21)
    • Robustness Against Transformations (2/2)
      • Observation
        • semantic video signature is robust against the content transformations applied
      /22 similarity rates computed for our semantic video signature are about 90% for all transformations applied
    • Robustness Against Key Frame Selection
      • Requirement
        • video signatures should be robust against different key frame selection strategies
          • key frames from the original video and an NDVC may differ
      • Feature variation for varying key frames
      • Observation
        • semantic information does not change significantly throughout a shot, whereas low-level visual features do
      /22 CS CL SC EH HT Semantic features Variation 8.31 1.69 7.11 4.80 4.05 0.90
    • Uniqueness (1/2)
      • Requirement
        • two semantically different videos should have a different semantic video signature
        • two semantically identical videos should have the same semantic video signature
      • Uniqueness was investigated by varying
        • the number of semantic concepts used
        • the number of shots in a query video clip
      /22
    • U niqueness (2/2)
      • Observation
        • NDVC detection performance increases when
          • the number of semantic concepts describing a shot increases
          • the number of shots in a query video clip increases
      /22 NDVC detection performance increases as the number of shots in a query video sequence increases, even when a limited semantic concept vocabulary is in use N correct / N queries
    • Outline semantic video signature 3. Experiments semantic concepts discussion methodology 4. Conclusions and future work background challenges 1. Introduction 2. Towards semantic-based NDVC detection /22
    • Conclusions and Future Work
      • Semantic video signatures for NDVC detection
        • exploit the temporal variation of semantic concepts
        • show a high level of robustness against
          • transformations of the video content
          • different key frame selection strategies
        • show a high degree of uniqueness
          • even when a limited semantic concept vocabulary is in use
      • Future research
        • scalability of semantic concept detection
        • comparison with the use of local features
        • re-ranking of video search results
      /22
    • Thank you! Any questions? /22