Poster of technical demo presented at ACM Multimedia 2012

1. LikeLines: Collecting Timecode-level Feedback
for Web Videos through User Interactions
Raynor Vliegendhart, Martha Larson, and Alan Hanjalic
Multimedia Information Retrieval Lab, Delft University of Technology
Problem Approach
● Problem: Providing users with a navigable heat map of interesting A Web video player component with a navigable heat map, that:
regions of the video they are watching.
● Uses multimedia content analysis to seed the heat map.
● Motivation: Conventional time sliders do not make the inner structure
● Captures implicit and explicit user feedback at the timecode-level
of the video apparent, making it hard to navigate to the interesting bits.
to refine the heat map.
LikeLines player
play
viewers pause
seek
like
... interaction
session server
content analysis
System Overview Implementation
● Video player component, augmented with: ● Video player component implemented in JavaScript and HTML5.
● Navigable heat map that allows users to jump directly to “hot” ● Out-of-the-box support for YouTube and HTML5 videos.
areas;
<script type="text/javascript">
● Time-sensitive “like” button that allows users to explicitly like var player = new LikeLines.Player('playerDiv', {
particular points in the video. video: 'http://www.youtube.com/watch?v=wPTilA0XxYE',
backend: 'http://backend:9090/'
● Captures user interactions: });
</script>
● Implicit feedback such as playing, pausing and seeking;
● Explicit “likes” expressed by the user. ● Video player component communicates with a back-end server
using JSON(P).
● Combines content analysis and captured user interactions to compute
a video’s heat map. ● Back-end server reference implementation is written in Python.
t (s)
+ t (s)
+ …+ t (s)
Future Work
● For what kinds of video is timecode-level feedback useful?
content analysis user feedback 1 user feedback n
● How should user interactions be interpreted?
● Back-end interaction session server stores and aggregates per video: ● How to fuse timecode-level feedback with content analysis
without encouraging snowball effects?
● All interaction sessions between each user and player;
● Initial multimedia content analysis of the video. ● Can timecode-level data be linked to queries to recommend
relevant jump points?
Source code: ● How to collect a critical mass of timecode-level data by
https://github.com/delftmir/likelines-player incentivizing users to interact with the system?
Contact: R.Vliegendhart@tudelft.nl 20th ACM international conference on Multimedia, Nara, Japan, 2012
@ShinNoNoir