Near-Duplicate Video Detection Using Temporal Patterns of Semantic Concepts


Published on

Near-Duplicate Video Detection Using Temporal Patterns of Semantic Concepts.

Paper presented at The IEEE International Symposium on Multimedia (ISM2009) in San Diego, California, USA.

Published in: Technology
1 Comment
1 Like
  • Interesting article, thanks. The area of video fingerprinting and duplicate video search seems to became popular nowdays. I saw some other papers related to this and even commercial products. Thus Duplicate Video Search definitelly uses some video fingerprinting to detect duplicates on user's PC. Volicon uses video fingerprinting to monitor TV broadcasts. Zeitera and Video Fingerprinting provide SDKs.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Near-Duplicate Video Detection Using Temporal Patterns of Semantic Concepts

  1. 1. Near-Duplicate Video Detection UsingTemporal Patterns of Semantic Concepts<br />IEEE International Symposium on Multimedia<br />San Diego, California, USADecember 14-16, 2009<br />Hyun-seok Min, Jaeyoung Choi, Wesley De Neve, Yong Man Ro<br />Image and Video Systems Lab<br />Department of Electrical Engineering<br />Korea Advanced Institute of Science and Technology (KAIST)<br />Daejeon, Republic of Korea<br />
  2. 2. Overview<br />Introduction<br />Near-duplicates<br />Semantic video signatures<br />Experimental results<br />Conclusions<br />2 /20<br />
  3. 3. Introduction<br />Importance of duplicate video detection<br />prevents cluttering of search results<br />prevents copyright infringement<br />3 /20<br />Search results for the query “I will survive Jesus”<br />A significant number of search results are near-duplicates!<br />
  4. 4. Definition of Near-duplicates<br />Identicalor approximately identical videos<br />photometric variations<br />e.g., change of color and lighting<br />editing operations<br />e.g., insertion of captions, logos, and borders<br />speed changes<br />e.g., addition or removal of frames<br />semantic concepts<br />e.g., ‘road’, ‘sand’, ‘snow’ , ….<br />4 /20<br />
  5. 5. Examples of Near-duplicates<br />original videos<br />near-duplicates<br />transformation<br />(cam cording, insertion of subtitles)<br />5 /20<br />transformation<br />(blur)<br />
  6. 6. Video Signatures<br /><ul><li>What is a video signature?
  7. 7. represents a video segment with a unique set of features
  8. 8. Conventional video signatures
  9. 9. often created by extracting low-level visual features fromvideo frames</li></ul>6 /20<br />video content<br />featureextraction<br />video signature<br />…<br />
  10. 10. Use of Low-level Visual Features forCreating a Video Signature<br />Problem<br />near-duplicates may not be visually similar<br />original video<br />near-duplicate<br />transformation<br />(cam cording, insertion of subtitles)<br />Visual match? No!<br />video signature<br />video signature<br />…<br />…<br />7 /20<br />
  11. 11. Semantic Similarity<br />Observation<br />near-duplicates often contain similar semantics<br />original video<br />near-duplicate<br />transformation<br />(cam cording, insertion of subtitles)<br />Semantic match? Yes!<br />Semantic concepts:<br />Semantic concepts:<br />indoor, man, face, …<br />indoor, man, face, …<br />8 /20<br />
  12. 12. Use of Semantic Concepts forCreating a Video signature<br />Semantic concept detection<br />traditionally used for classifying video clips into several predefined concepts<br />Problem<br />limited number of semantic concepts can be detected<br />Solution<br />use of temporal variation of semantic concepts<br />different from video sequence to video sequence<br />9 /20<br />
  13. 13. Semantic Video Signature Creation (1/2)<br />Semantic video signature creation<br />A1<br />A2<br />A3<br />…<br />…<br />Semantic video signature<br />V<br />video<br />shots<br />key frames<br />…<br />concept classification<br />classifier for ‘Street’<br />classifier for ‘Beach’<br />classifier for ‘Tree’<br />Ai<br />AN<br />N: the number of shots<br />M: the number of predefined semantics<br />Ci: ith predefined semantic concept<br />semantic video signature<br />si<br />…<br />sN<br />s2<br />s1<br />…<br />10 /20<br />
  14. 14. Semantic Video Signature Creation (2/2)<br />11 /20<br />original video<br />near-duplicate<br />transformation<br />…<br />…<br />…<br />…<br />Semantic video signature of original video<br />Semantic video signature of near-duplicate<br />
  15. 15. Matching Procedure<br />12 /20<br />Semantic video signature of near-duplicate<br />Semantic video signature of original video<br />
  16. 16. Experimental Setup (1/3)<br />Reference database<br />video sequences taken from TRECVID2007<br />over 9 hours of video data<br />format: MPEG-1<br />resolution: 352X288<br />frame rate: 25 frame per second (fps)<br />Screenshots<br />13 /20<br />
  17. 17. Experimental Setup (2/3)<br />Creation of query video (near-duplicate) set <br />number of query video sequences<br />64 in total<br />average length of the query video sequences<br />3 minutes<br />Process for generating query video sequences <br />original video<br />sampling<br />subvideoof original video<br />transformation<br />query video<br />14 /20<br />
  18. 18. Experimental Setup (3/3)<br />Transformations used<br />spatial transformations<br />Gaussian blur<br />logo insertion <br />letter-box<br />resizing<br />temporal transformations<br />change of frame rate<br />original<br />15 /20<br />
  19. 19. Experimental Results: Spatial Transformation (1/2)<br />16 /20<br />The precision increases as the threshold value decreases, while in turn, the recall value decreases.<br />blur<br />letter-box<br />
  20. 20. Experimental Results: Spatial Transformation (2/2)<br />17 /20<br />Ordinal measurement does not work well with logo insertion, compared to the proposed method.<br />
  21. 21. Experimental Results: Temporal Transformation<br />18 /20<br />The proposed method is robust against temporally modified video sequences. <br />
  22. 22. Conclusions<br />Proposed the use of semantic video signatures for near-duplicate video detection<br />relies on a number of semantic concepts detected along the temporal axis<br />Experimental results<br />indicate that the use of a semantic video signature looks promising<br />Future work <br />improving the accuracy of semantic concept detection<br />use of additional semantic concepts<br />19 /20<br />
  23. 23. Any questions or comments?<br />20/20<br />