3. Video data Sound and Vision The Netherlands Institute for Sound and Vision news magazine, science news, news reports, documentaries, educational programming, and archival video BBC rushes unedited material All materials in MPEG-1.. yep!)
5. Content-based copy detection copyright control business intelligence advertisement tracking law enforcement investigations
6. Video transformation Picture in picture (The original video is inserted in front of a background video) Insertions of pattern Strong reencoding Change of gamma Decrease in quality Blur, change of gamma, frame dropping, contrast, compression, ratio, white noise Post production Crop, shift, contrast, caption (text insertion), flip (vertical mirroring), insertion of pattern, Picture in Picture (the original video is in the background) Change to randomly choose 1 transformation from each of the 3 main categories.
7. AT&T Research at TRECVID 2009Content-based Copy Detection Applications discovering copyright infringement of multimedia content monitoring commercial air time querying video by example Approaches digital video watermarking content based copy detection (CBCD).
9. Content based sampling Shot boundary detection (SBD) Adopts a “divide and conquer” strategy Six independent detectors: Cut, fade in, fade out, fast dissolve (less than 5 frames), dissolve and motion Each detector is a finite state machine (FSM) FSMs depent on two types of visual features: Intra-frame (only one frame) Inter-frame (current frame+previous frame)
15. Transformation detection andnormalization for query keyframe Query Keyframe Normalization Equalize and blur the query keyframe to overcome the effect of change of Gamma and white noise transformations.
16. Transformation detection andnormalization for query keyframe And we have 10 types of query keyframe: original, letterbox removed, PiP scaled, equalized, blurred and flipped versions of these five types
18. Reference keyframe transformation Only 2 transformations Half-resolution rescaling For compared with the detected PiP region in the query keyframes Strong re-encoding For dealing with the strong re-encoded query keyframes. And we have 3 types of reference keyframe
21. Scale-invariant feature transform SIFT Extraction It’s main feature for locating video copies Locating the keypoints that have local maximum Difference of Gaussian values both in scale and in space. (specified by location, scale and orientation) Computing a descriptor for each keypoint. The descriptor is the gradient orientation histogram, which is a 128 dimension feature vector.
23. Locality sensitive hashing (LSH) The basic idea hash the input items so that similar items are mapped to the same buckets with high probability a – random vector following a Gaussian distribution with zero mean and unit variance w – preset bucket size b – in range [0,w]
25. Indexing and search by LSH Sort LSH values independency Save with SIFT identifications in separate index file SIFT identifications: (String) Reference video ID Keyframe ID SIFT ID
27. Keyframe level query refinement Two issues: the original SIFT matching by Euclidian distance is not reliable it‘s possible that two SIFT features that are far away mapped to the same LSH value
32. Count the number of keypoints in the reference whose transformed to the coordinates of their matching keypoints in the query keyframe. These keypoints are called inliers
41. CBCD result generation Query video ID Reference video ID Information of copied reference video segment Starting frame of copied segment in the query video Decision score
42. CBCD Evaluation Results Dataset 1407 short query videos 838 reference videos 208 non-reference videos Extract For entire reference video set 268,000 keyframes 57,000,000 SIFT features For entire query video set 18,000 keyframes 2,600,000 SIFT features
