Television Linked To The Webwww.linkedtv.euLotte Belice Baltussen, Roeland Ordelman, Jaap BlomNetherlands Institute for So...
2www.linkedtv.eu LinkedTV: interweaving Web and TV into asingle experience Mixing lean back and lean forward Want to ma...
3www.linkedtv.euScenarios and contentEnriched News Professional news contentproduced by RBB for generalaudience local ne...
4www.linkedtv.euAntiques Interactive: Enriched link demolinkedtv.eu
5www.linkedtv.eudbpedia.org/resource/Jan_SluytersWorkflowlinkedtv.euPaintings by Jan Sluijters“...painting of Jan Sluijter...
6www.linkedtv.eudbpedia.org/resource/Jan_SluytersWorkflowlinkedtv.euPaintings by Jan Sluijters“...painting of Jan Sluijter...
7www.linkedtv.euWhy do we need VideoHypE?, Roeland Ordelman1 Broadcasters want to have some control. Editors have the ex...
8www.linkedtv.euVideoHypE requirements, Roeland Ordelman1 Requirements session with 5 AVRO editorial members +digital sta...
9www.linkedtv.euDemo overview – choose your video, Roeland Ordelman1
10www.linkedtv.euDemo overview – shot overview, Roeland Ordelman1
11www.linkedtv.euDemo overview - chapters, Roeland Ordelman1
12www.linkedtv.euDemo overview – selecting entities, Roeland Ordelman1
13www.linkedtv.euDemo overview – selecting links per entity, Roeland Ordelman1
14www.linkedtv.euDemo overview – editor‘s choices overview, Roeland Ordelman1
15www.linkedtv.euDemo overview – adding links, Roeland Ordelman1
17www.linkedtv.eu1. Implementing remaining requirements.2. How to make the editor tool scalable?• Different programmes, di...
18www.linkedtv.euThanks!, Roeland Ordelman1Find me here:@lottebelice | lbbaltussen@beeldengeluid.nlCredits:Again: other ...
Upcoming SlideShare
Loading in...5
×

EuroITV - VideoHypE

213

Published on

Presentation for the FutureTV session at EuroITV 2013. See http://www.linkedtv.eu/event/futuretv2013/.

Paper presented by Lotte Belice Baltussen of the Netherlands Institute for Sound and Vision.

Lotte Belice Baltussen, Roeland Ordelman, Jaap Blom. VideoHypE: An Editor Tool for Supervised Automatic Video Hyperlinking. In 11th European Interactive TV Conference (EuroITV 2013), June 2013, Como, Italy.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
213
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • (1) Big Q: Do you ever have a question about what you are watching? Would you like related information about something you have seen? Something someone mentioned? One option is to pause the TV, go find your smart phone or pad and start some google search. When you carry out the search, your device doesn't know what you were watching, nor what the specific scene was about. Wouldn't it be nice to have a system that knows what you are watching, knows what you like and that presents links to extra pieces of information to you? Technology only likely to be useful when audience , content and program genre match. (2) slide
  • (1) Currently developing tools using 2 program genres
  • (1) What do we think it will look like? Not like this! Select video. Link to video! http://www.linkedtv.eu/scenarios/hyperlinked-documentary-scenario/
  • Developing individual technologies, using content from 2 program genres, with the aim of creating an end2end prototype. 1. Given a video: segment it to shots detect visual concepts detect faces perform audio analysis (ASR) perform keyword extraction 2. Media fragments => time-related! 3. ANCHORS Named Entities => e.g. Jan Sluyters, disambiguated wtih DBpedia link 4. TARGETS web content from WHITE LIST connected to the ANCHORS
  • [Li, Rizzo, Redondo, Troncy, Wald, Wills 2013: http://www.eurecom.fr/~troncy/Publications/Li_Troncy-lime13.pdf].
  • Black = implemented Orange = mocked-up Blue = to be done
  • Clicking a shot results in a jump to the point of the shot in the video Select different videos that need to be edited
  • Clicking a shot results in a jump to the point of the shot in the video Select different videos that need to be edited
  • * Create and edit chapters (titles + start and end times)
  • * Select specific entity types – in this case ‘ person ’
  • * Select links per entity and add them to the editor ’ s choice.
  • Selected links with corresponding entity types shown in editor ’ s choice section Here, the editor can also add his/her own hyperlinks.
  • Selected links with corresponding entity types shown in editor ’ s choice section Here, the editor can also add his/her own hyperlinks.
  • Transcript of "EuroITV - VideoHypE"

    1. 1. Television Linked To The Webwww.linkedtv.euLotte Belice Baltussen, Roeland Ordelman, Jaap BlomNetherlands Institute for Sound and VisionVideoHypE: An Editor Tool for SupervisedAutomatic Video HyperlinkingEuroITV | Como, Italy – 24 June 2013
    2. 2. 2www.linkedtv.eu LinkedTV: interweaving Web and TV into asingle experience Mixing lean back and lean forward Want to maintain immersion while removing fourthwall illusion Works only for certain genres and audiences Does second screen help or hinder?The LinkedTV visionlinkedtv.eu
    3. 3. 3www.linkedtv.euScenarios and contentEnriched News Professional news contentproduced by RBB for generalaudience local news show rbb Aktuell People, locations and eventscan be linked to previousevents.Antiques Interactive Program watchedpredominantly by 50+ers AVRO:Tussen Kunst en Kitsch (TKK) Objects, locations andexperts provide anchors torelated materials (targets).linkedtv.eu
    4. 4. 4www.linkedtv.euAntiques Interactive: Enriched link demolinkedtv.eu
    5. 5. 5www.linkedtv.eudbpedia.org/resource/Jan_SluytersWorkflowlinkedtv.euPaintings by Jan Sluijters“...painting of Jan Sluijters....”HypervideoanalysisPresentationengineNamed EntityRecognitionEnriching NEs withweb contentGeneration ofmedia fragments
    6. 6. 6www.linkedtv.eudbpedia.org/resource/Jan_SluytersWorkflowlinkedtv.euPaintings by Jan Sluijters“...painting of Jan Sluijters....”HypervideoanalysisPresentationengineNamed EntityRecognitionEnriching NEs withweb contentGeneration ofmedia fragmentsVideoHypE
    7. 7. 7www.linkedtv.euWhy do we need VideoHypE?, Roeland Ordelman1 Broadcasters want to have some control. Editors have the expertise to determine relevance + The user interface can only show a limited amount of links, and there are many Research: sample 805 videos, average 55 NEs per video. 55 NEs = many, manypossible links depending on the size of the white list [Li et al., 2013]NE - person: Jan SluijtersNE – art syle: FauvismNE - person: Emiel AardewerkLink white list 1Link white list 2Link white list 3Link white list 6Link white list 7Link white list etc etcLink white list 4Link white list 5
    8. 8. 8www.linkedtv.euVideoHypE requirements, Roeland Ordelman1 Requirements session with 5 AVRO editorial members +digital staff of TKK. Main outcomes:1. Selecting from list of available episodes2. Create and edit chapters3. Timeline with chapters4. See annotations per chapters, distinction between entity types5. Select annotations + related hyperlinks6. Add selected, approved hyperlinks7. Timeline with enities and hyperlinks8. Add new hyperlinks to media fragments (e.g. shots/chapters/spatial)9. Remove hyperlinks and/or entities (per media fragment or episode)10. Separate user environment for each end user partner11. Adding to white list12. Sorting on entity types13. Collaborative space14. Delete annotations forever15. Level of completion per chapter / episode16. Variable # suggested links depending on time pressureMAX 20-30 minutes per episode!!!
    9. 9. 9www.linkedtv.euDemo overview – choose your video, Roeland Ordelman1
    10. 10. 10www.linkedtv.euDemo overview – shot overview, Roeland Ordelman1
    11. 11. 11www.linkedtv.euDemo overview - chapters, Roeland Ordelman1
    12. 12. 12www.linkedtv.euDemo overview – selecting entities, Roeland Ordelman1
    13. 13. 13www.linkedtv.euDemo overview – selecting links per entity, Roeland Ordelman1
    14. 14. 14www.linkedtv.euDemo overview – editor‘s choices overview, Roeland Ordelman1
    15. 15. 15www.linkedtv.euDemo overview – adding links, Roeland Ordelman1
    16. 16. 17www.linkedtv.eu1. Implementing remaining requirements.2. How to make the editor tool scalable?• Different programmes, different requirements• How to deal with live content?• There is not just one front-end => one-screen solutions, 2nd screens, multiplescreens, etc.1. Personalisation: what ‘bandwith’ is left personalisation?2. Relevance is a crucial issue – how can the system ‘learn’?3. User testing with real editorsVideoHypE – discussion & future work
    17. 17. 18www.linkedtv.euThanks!, Roeland Ordelman1Find me here:@lottebelice | lbbaltussen@beeldengeluid.nlCredits:Again: other authors are Roeland Ordelman and Jaap BlomSlides partly by: Lynda Hardman (CWI)Heaps of thanks to AVRO for so generously letting LinkedTV use their content.Links:www.linkedtv.eutwitter.com/BenGlabswww.avro.nl
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×