Near-Duplicate Videos

Let’s say you’re looking for the Bush attack video…

… and you get 11,100 results.

… after 40 minutes... watching the videos listed on the first page you notice > 50% are similar, i.e. NDVC 27% in average [Wu et al., 2007]

NDVC technical definition Identical or approximately identical videos, that differ in some feature: file formats, encoding parameters photometric variations (color, lighting changes ) overlays ( caption, logo, audio commentary ) editing operations ( frames add/remove ) semantic similarity NDVC are videos that are “ essentially the same ”

Identical or approximately identical videos, that differ in some feature:

file formats, encoding parameters

photometric variations (color, lighting changes )

overlays ( caption, logo, audio commentary )

editing operations ( frames add/remove )

semantic similarity

… like this

Two challenges: There is no agreement on a single definition of NDVC NDVC are mostly considered as redundant content that has to be removed from the system

There is no agreement on a single definition of NDVC

NDVC are mostly considered as redundant content that has to be removed from the system

Human Perception of Mauro Cherubini Rodrigo de Oliveira Nuria Oliver Near Duplicate Videos

What kind of NDVC? Malicious (i.e., spam produced by a single user) Copyright infringement (e.g., pirated music videos) User-edited content : videos that complement the original material with additional information

Recently NDVC detection algorithm

Recently NDVC detection algorithm

Why not? NDVC detection algorithm ?

Methodology 2 large-scale online surveys ( n= 1003) 7 pairs of NDVC (differing in 1 feature) Subjects were asked about: Similarity Preference

2 large-scale online surveys ( n= 1003)

7 pairs of NDVC (differing in 1 feature)

Subjects were asked about:

Similarity

Preference

NDVC technical definition Identical or approximately identical videos, that differ in some features: photometric variations (color, lighting changes ) overlays ( caption, logo, audio commentary ) editing operations ( frames add/remove ) And … semantic similarity (e.g., two deer grazing grass in two different forests)

Identical or approximately identical videos, that differ in some features:

photometric variations (color, lighting changes )

overlays ( caption, logo, audio commentary )

editing operations ( frames add/remove )

And …

semantic similarity

(e.g., two deer grazing grass in two different forests)

Audio Quality NDVC Preference Stereo, 44 Khz Mono, 11 Khz

Image Quality NDVC Preference

Audio content (overlay) Preference NDVC

Visual + audio content (length) Preference Not NDVC

Visual content (editing) Not NDVC Want both

Similar semantics , different videos (similar visual info) NDVC Want both

Similar semantics , different videos (similar audio info) Not NDVC Preference

Implications for Design User-centric NDVC definition NDVC are approximately identical videos that might differ in audio/image quality , or overlays . Conversely, identical videos with relevant complementary information (changing clip length or scenes) are not considered as NDVC. Furthermore, users perceive as near-duplicate videos that are not alike but that are visually similar and semantically related .

User-centric NDVC definition

Implications for Design Clustering Groups sharing video, audio, semantic content Ranking based on user-submitted query Highlight the most representative

Clustering

Groups sharing video, audio, semantic content

Ranking based on user-submitted query

Highlight the most representative

Implications for Design Feature and user adaptation Boost ranking based on general observations More content Better image/audio quality … Boost ranking based on personalization Abilities (e.g., auditory skills) Task (e.g., video producer vs. movie enthusiastic) Search query

Feature and user adaptation

Boost ranking based on general observations

More content

Better image/audio quality

…

Boost ranking based on personalization

Abilities (e.g., auditory skills)

Task (e.g., video producer vs. movie enthusiastic)

Search query

Future Work NDVC’s differing in more than 1 low-level feature Propose ways to visualize the NDVCs Study effects of user’s goals while searching videos

NDVC’s differing in more than 1 low-level feature

Propose ways to visualize the NDVCs

Study effects of user’s goals while searching videos

A Human-Centric stance in Multimedia research Biomimetics Crowdsourcing Psychophysical experiments

Thank you! Mauro Cherubini Rodrigo de Oliveira Nuria Oliver [email_address] [email_address] [email_address]