This document proposes a content-based video recommendation system using low-level visual features to address the new item problem. It extracts features like average shot length, color variance, and motion from video content to classify videos into genres. An evaluation of the system used 120 videos across 4 genres and achieved a classification accuracy of 73.33%. The results showed this approach performed better than baselines in addressing the new item problem. Future work could analyze larger datasets and incorporate additional modalities like audio.

1. Toward Building a
Content-based Video Recommendation
System based on Low-level Features
Yashar Deldjoo
Mehdi Elahi
Massimo Quadrana
Paolo Cremonesi
Corresponding journal ar8cle:
Deldjoo, Yashar; Elahi, Mehdi; Cremonesi, Paolo; Garzo?o, Franca; Piazzolla, Pietro;
Quadrana, Massimo; ",Content-Based Video Recommenda1on System Based on Stylis1c
Visual Features, Journal on Data Seman.cs, 1-15, 2016, Springer

2. Outline
• Introduction
• New item problem
• LL Feature base Recommendation
• Evaluation and Results
• Future work

3. tools that support users decision making by suggesting
products that can be interesting to them.
Examples of Recommender Systems:
Recommender Systems:

4. Is typical done by predicting unknown ratings, by exploiting
the content of items or/and ratings given by users.
Recommendation:
3
1
2
5
2
3
4

5. when a new item is added to the catalogue and we don’t
have information about it, e.g., no rating is available.
New Item Problem:
New Item
3
1
?
2
5
2
?
3
4
?

6. Extreme New Item Problem
We have absolutely no information about an item.
Example: An unknown video content is uploaded by a unknown
user and there is no metadata available.
?
? ?
?
?
? ?
?
?
?

7. How to make recommendation?

8. Video Content
§ There exist 3 main modalities
in a video.
Visual
Audio
Text

9. Video Content
§ There exist 3 main modalities
in a video.
§ There exists many fearures in
each modality.
Visual
Audio
Text
Visual
Feaures
Audio
Feaures
Textual
Features

10. Video Content
§ There exist 3 main modalities
in a video.
§ There exists many fearures in
each modality.
§ Our focus Visual features
Visual
Audio
Text
Visual
Feaures
Audio
Feaures
Textual
Features

11. Visual Features
Visual
Audio
Text
Feaures
Audio
Feaures
Textual
Features
Visual
Structure
Content

12. Video Structure
Scene: A number of shots that
form a semantic unit.
Shot: All frames within a single
camera action.
Frame: One static image from a
series of static images
constituting a video.
Figure: Hierarchical decomposition and representation of video content,
http://www.scholarpedia.org/article/Video_Content_Structuring

13. Example A:
SHOT1

14. Example A:
SHOT2

15. Example A:
SHOT2
2CameraShots– 1Scene

16. Example B:
SHOT1

17. Example B:
SHOT1
1CameraShot– 1Scene

18. Shot Detection
Figure: A schematic illustration of shot detection
http://www.scholarpedia.org/article/Video_Content_Structuring

19. Visual Features
Average
Shotlength
Shot
Motion
Color
Variance
Lighening
Key

20. Average Shot Length
Idea : Slower paced film (e.g. drama) have larger
average shot length whereas action movies appear to
have shorter average shot length.

21. Comparing Average Shot Length
0 2 4 6 8 10 12 14
Drama
Action
shot 1 shot 2 shot 3 shot 4 shot 5 shot 6 shot 7 shot 8 shot 9
shot 10 shot 11 shot 12 shot 13 shot 14 shot 15 shot 16 shot 17 shot 18

22. Visual Features
Average
Shotlength
Shot
Motion
Color
Variance
Lighening
Key

23. Color Variance
figure
Color
Var
(a) 0
(b) 0
(c) 0
(d) 0
(e) 0.25
(f) 17.8
(g) 1.4e+9

24. Horror Comedy

25. Visual Features
Average
Shotlength
Shot
Motion
Color
Variance
Lighening
Key
Opticalflow

26. Shot Motion
Figure 3: Optical flow of a sample image shown

27. Visual Features
Average
Shotlength
Shot
Motion
Color
Variance
Lighening
Key

28. Video Classfication
• FeatureExtraction

29. Evaluation
• 120 Videos
• 4 Main Genres

30. DissimilarityMetrics

31. Rating Prediction

32. Evaluation

33. Results
N
1 2 3 4 5 6 7 8 9 10
recall
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
LL
HL - Genre
LL+ HL - Genre
Corresponding journal ar8cle:
Deldjoo, Yashar; Elahi, Mehdi; Cremonesi, Paolo; Garzo?o, Franca; Piazzolla, Pietro; Quadrana, Massimo; ",Content-Based
Video Recommenda1on System Based on Stylis1c Visual Features, Journal on Data Seman.cs, 1-15, 2016, Springer

34. Classification
ClassificationAccuracy=73.33%
a b c d Classified as
22 5 2 1 a = Action
1 27 2 0 b= Comedy
6 1 22 1 c= Drama
8 0 5 17 d=Horror

35. Conclusion
• we propose a method to remedy the (extreme) New
Item problem in video recommendation domain
• we assume a more realistic scenario, i.e., an up-
and-running video recommender with thousands of
users
• Result of our experiments shown that we have
achieved excellent performance in comparison with
considered baselines

36. Future Work
• Further analysis with bigger datasets, in order to better
understand the performance differences among the
compared methods.
• Investigation of the impact of different recommendation
algorithms, such as those based on Bayesian, or SVD, on
the performance of our method.
• including additional sources of information, such as, audio
features, in order to farther improve the quality of our
content based recommendation method.

37. Thank you!
Corresponding journal ar8cle:
Deldjoo, Yashar; Elahi, Mehdi; Cremonesi, Paolo; Garzo?o, Franca; Piazzolla, Pietro; Quadrana, Massimo; ",Content-Based
Video Recommenda1on System Based on Stylis1c Visual Features, Journal on Data Seman.cs, 1-15, 2016, Springer