The MPEG-7 Video Signature
Tools for Content Recognition
What is MPEG-7?
• "Multimedia Content Description Interface“
• ISO/IEC standard by MPEG (Moving Picture Experts Group)
• Providing meta-data for multimedia
• MPEG-1, -2, -4: make content available;
MPEG-7: makes content accessible, retrievable, filterable,
manageable (via device / computer).
• Multi-degrees of interpretation of information’s meaning
• Support as broad a range of applications as possible.
• A compatible (with existing tech) and extensible standard.
Why The MPEG-7 Video
Signature Tools ?
• Handling billions of videos.
• Efficiently search for a copy of a specific
piece of video content.
• Duplicate video clip detection in large
• Finding out edited or modified version
and embedded in a longer piece of video
Application’s domains (incomplete)
• Broadcast media selection (e.g., radio channel, TV channel).
• Digital libraries (e.g., film, video, audio and radio archives).
• E-Commerce (e.g., personalized advertising).
• Education (e.g., repositories of multimedia courses, multimedia
search for support material).
• Home Entertainment (e.g., management of personal multimedia
collections, including manipulation of content, e.g. karaoke).
• Journalism (e.g. searching speeches of a certain politician using his
name, his voice or his face).
• Multimedia directory services (e.g. yellow pages, G.I.S).
• Surveillance and remote sensing.
• Database management and deduplication
Requirements of the proposed
2) Robustness to editing operations
4) Fast matching
5) Fast extraction
7) Non alteration of the content
8) Self-containment of the signatures
9) Coding independence
11)Accurate temporal localization of duplicated and
There are four parts to the standard in video signature:
1) The descriptor extraction and decoding, along with its
descriptor definition language
2) A reference software implementation and source code
for the video signature tools
3) The conditions and dataset for ensuring conformance to
4) An exemplary pair wise matching and localization
scheme, as used during the MPEG-7 evaluation
Main benefits to the different
1) Follows a systematic peer-reviewed evaluation
process, leading to the adoption of the best
technologies from various proposals
2) the video signature tools enable
interoperability, i.e., they allow different users and
systems to talk to each other in terms of descriptors.
Video Signature Extraction and
The video signature comprises two parts:
1) Fine signatures
2) Coarse signatures
3) Video Signature Organization
Video Signature Matching and
• The matching between two video signatures v1 and v2 is carried out
in three stages
1) Coarse Signature Matching
2) Temporal Parameter Estimation
3) Localization and Verification
• Achieves high levels of robustness to
common video editing operations
• Accurately detect and localize a piece of
video content embedded in a longer piece
of unrelated video content
1) Stavros Paschalakis, Kota Iwamoto, Paul Brasnett, Nikola Sprljan, Ryoma
Oami, Toshiyuki Nomura, Akio Yamada, Miroslaw Bober, “The MPEG-7 Video
Signature Tools for Content Identification”, IEEE TRANSACTIONS ON CIRCUITS
AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 22, NO. 7, JULY 2012.
2) A. Hampapur, K. Hyun, and R. Bolle, “Comparison of sequence matching
techniques for video copy detection,” in Proc. Conf. Storage Retrieval Media
Databases, 2002, pp. 194–201.
3) J. Law-To, L. Chen, A. Joly, I. Laptev, O. Buisson, V. Gouet-Brunet, N.
Boujemaa, and F. Stentiford, “Video copy detection: A comparative study,” in Proc.
6th ACM Int. Conf. Image Video Retrieval, Jul. 2007, pp. 371–378.
4) H. T. Shen, X. Zhou, Z. Huang, J. Shao, and X. Zhou, “UQLIPS: A real-time near-
duplicate video clip detection system,” in Proc. 33rd Int. Conf. Very Large Data
Bases, Sep. 2007, pp. 1374–1377.
5) S. Paisitktiangkrai, T. Mei, J. Zhang, and X.-S. Hua, “Scalable clip-based near-
duplicate video detection with ordinal measure,” in Proc. ACM Int. Conf. Image
Video Retrieval, Jul. 2010, pp. 121–128.