Content-enriched Classifier
C t t
i h d Cl
ifi
for Web Video Classification
o
eb deo C ass cat o
By
Bin Cui & Ce Zhang
Dep...
Outline
• Introduction
• Current Approaches
• Proposed Approach
– Content -enriched Classifier
– Content-enriched Similari...
Introduction
• In video sharing services, the user
services
browses the web by categories.

• Real time categorization pla...
Web Video Processing

Video Title
User Description

CS848 Winter 2011
4
Web Video Classification
Problems
o Although text features and content features are
complementary but utilizing content fe...
Current Approaches

CS848 Winter 2011
6
Proposed Approach
1- Content-Enriched Similarity:
Using visual clues of web videos to obtain more reasonable
semantic rela...
Proposed Approach (cont.)
3- Semantic kernels will be computed using the following
formula:
f
l

4- Multi-Kernel Enhanceme...
Content enriched
Content-enriched Classifier
Training Data
(Test
Features)

Classifier
Content-Enriched
Semantic Kernel

B...
Content Enriched
Content-Enriched Similarity
Generally, two words are similar if they appear in the
y,
y pp
same cluster, ...
CES Classifier Algorithm

CS848 Winter 2011
11
Experimental Results
• Experimental Settings
p
g
– Datasets:
• Two real-life datasets are collected from ‘YouTube’ between...
Word Similarity Approach

• The relation discovered by CSE are meaningful and
agree with common sense.
• The classificatio...
Classification effectiveness

• Classification Performance on different frameworks.
• F-score: accuracy measure for classi...
Effectiveness Per Category

The scores of content classifier have been excluded
because their performance is much worse th...
Performance on Multi-kernel
Multi kernel

• This table shows the results on classification
effectiveness with multi-kernel...
Conclusions
• Novel Framework that exploits visual content
and text features to facilitate online video
categorization is ...
PRESENTED
APPROACH
EXTEND
Camera Motion Model

To enhance the presented approach, we will study the feasibility of
using C
i
Camera M ti
Motion M d ...
Questions

Thank You
CS848 Winter 2011
21
Upcoming SlideShare
Loading in …5
×

Content-enriched Classifier for Web Video Classification

476 views

Published on

Content-enriched Classifier for Web Video Classification

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
476
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Content-enriched Classifier for Web Video Classification

  1. 1. Content-enriched Classifier C t t i h d Cl ifi for Web Video Classification o eb deo C ass cat o By Bin Cui & Ce Zhang Dept. Dept of CS Peking University Gao Cong School of CE, Nanyang Technological University, Singapore Presented by Ahmed Ibrahim SIGIR 2010
  2. 2. Outline • Introduction • Current Approaches • Proposed Approach – Content -enriched Classifier – Content-enriched Similarity – CSE Classifier Algorithm • Experimental Results • Conclusions & Critique • Proposed approach extension
  3. 3. Introduction • In video sharing services, the user services browses the web by categories. • Real time categorization plays a key roll for organizing, browsing, and retrieving online video. CS848 Winter 2011 3
  4. 4. Web Video Processing Video Title User Description CS848 Winter 2011 4
  5. 5. Web Video Classification Problems o Although text features and content features are complementary but utilizing content features in video classification stage is computationally expensive. o Text classification cannot use the rich information contained in video content. o Text description characteristics limits the classification performance of semantic similarity based on WordNet (and / or) term co-occurrence. co occurrence CS848 Winter 2011 5
  6. 6. Current Approaches CS848 Winter 2011 6
  7. 7. Proposed Approach 1- Content-Enriched Similarity: Using visual clues of web videos to obtain more reasonable semantic relations among words which called Content-Enriched Similarity (CES) between words. y( ) 2- Content-Enriched Nonlinear Classifier – At the training stage, a nonlinear SVM classifier is built to stage explore the semantic similarity between words using CES. – At the classification stage, this classifier classifies a new video g , using its text features (but not its content features). CS848 Winter 2011 7
  8. 8. Proposed Approach (cont.) 3- Semantic kernels will be computed using the following formula: f l 4- Multi-Kernel Enhancement: Given several kernels created using different word pair-wise similarity matrices for multiple kernel optimization. CS848 Winter 2011 8
  9. 9. Content enriched Content-enriched Classifier Training Data (Test Features) Classifier Content-Enriched Semantic Kernel Building Classifier Contentenriched word similarity Extract CES Finding the hyperplane in Content-Enriched Content Enriched Kernel Space Training Data (Test + Content Features) Applying Classifier Testing Data g (Test Features) CES: Content-Enriched Similarity CS848 Winter 2011 9
  10. 10. Content Enriched Content-Enriched Similarity Generally, two words are similar if they appear in the y, y pp same cluster, within which the videos are similar in terms of content. Extract Visual Content Features Video Database (5149 videos) “K” clusters = 100 K-means Clustering Cl t i Project ‘tf’ into cluster space ‘VS’ video-cluster relation matrix CS848 Winter 2011 10
  11. 11. CES Classifier Algorithm CS848 Winter 2011 11
  12. 12. Experimental Results • Experimental Settings p g – Datasets: • Two real-life datasets are collected from ‘YouTube’ between Sept 23 & 24 of 2009, YT923 (5149 videos) & YT924(4447 videos). , ( ) ( ) • They categorized both datasets into 15 Categorize. – Preprocessing : Feature Extraction: • Text features are extracted from videos include (video titles and descriptions). • Words are stemmed using WordNet stemmer. • Stop words are manually removed. p y • The following visual content features (color, texture & edges) are extracted . CS848 Winter 2011 12
  13. 13. Word Similarity Approach • The relation discovered by CSE are meaningful and agree with common sense. • The classification results reflect the superiority of proposed methods. CS848 Winter 2011 13
  14. 14. Classification effectiveness • Classification Performance on different frameworks. • F-score: accuracy measure for classification which can be calculated using . • Macro-F: average of F-score for each category. • Micro-F: average of F-score for all decisions Micro F: F score decisions. CS848 Winter 2011 14
  15. 15. Effectiveness Per Category The scores of content classifier have been excluded because their performance is much worse than the text. CS848 Winter 2011 15
  16. 16. Performance on Multi-kernel Multi kernel • This table shows the results on classification effectiveness with multi-kernel solution multi kernel CS848 Winter 2011 16
  17. 17. Conclusions • Novel Framework that exploits visual content and text features to facilitate online video categorization is presented. • Content-enriched Semantic Kernel which extracts word relationship by clustering the video with visual content feature is proposed. CS848 Winter 2011 17
  18. 18. PRESENTED APPROACH EXTEND
  19. 19. Camera Motion Model To enhance the presented approach, we will study the feasibility of using C i Camera M ti Motion M d l as a video content f t Model id t t feature on th the classification performance and efficiency using CC_WEB_VIDEO. CS848 Winter 2011 20
  20. 20. Questions Thank You CS848 Winter 2011 21

×