VideoHypE: An Editor Tool for Supervised Automatic Video Hyperlinking

  • 507 views
Uploaded on

LinkedTV presentation at EuroITV 2013, 24 June 2013, Como, Italy. Talk delivered by by Lotte Belice Baltussen, S&V.

LinkedTV presentation at EuroITV 2013, 24 June 2013, Como, Italy. Talk delivered by by Lotte Belice Baltussen, S&V.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
507
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • (1) Big Q: Do you ever have a question about what you are watching?
    Would you like related information about something you have seen?
    Something someone mentioned?
    One option is to pause the TV, go find your smart phone or pad and start some google search.
    When you carry out the search, your device doesn't know what you were watching, nor what the specific scene was about.
    Wouldn't it be nice to have a system that knows what you are watching, knows what you like and that presents links to extra pieces of information to you?
    Technology only likely to be useful when audience, content and program genre match.
    (2) slide
  • (1) Currently developing tools using 2 program genres
  • (1) What do we think it will look like? Not like this!
    Select video. Link to video!
    http://www.linkedtv.eu/scenarios/hyperlinked-documentary-scenario/
  • Developing individual technologies, using content from 2 program genres, with the aim of creating an end2end prototype.
    1. Given a video:
    segment it to shots
    detect visual concepts
    detect faces
    perform audio analysis (ASR)
    perform keyword extraction
    2. Media fragments => time-related!
    3. ANCHORS Named Entities => e.g. Jan Sluyters, disambiguated wtih DBpedia link
    4. TARGETS web content from WHITE LIST connected to the ANCHORS
  • [Li, Rizzo, Redondo, Troncy, Wald, Wills 2013: http://www.eurecom.fr/~troncy/Publications/Li_Troncy-lime13.pdf].
  • EXPLAIN CHAPTER!
  • Clicking a shot results in a jump to the point of the shot in the video
    Select different videos that need to be edited
  • Clicking a shot results in a jump to the point of the shot in the video
    Select different videos that need to be edited
  • * Create and edit chapters (titles + start and end times)
  • * Select specific entity types – in this case ‘person’
  • * Select links per entity and add them to the editor’s choice.
  • Selected links with corresponding entity types shown in editor’s choice section
    Here, the editor can also add his/her own hyperlinks.
  • Selected links with corresponding entity types shown in editor’s choice section
    Here, the editor can also add his/her own hyperlinks.

Transcript

  • 1. Television Linked To The Web VideoHypE: An Editor Tool for Supervised Automatic Video Hyperlinking Lotte Belice Baltussen, Roeland Ordelman, Jaap Blom Netherlands Institute for Sound and Vision EuroITV | Como, Italy – 24 June 2013 www.linkedtv.eu
  • 2. The LinkedTV vision www.linkedtv.eu  LinkedTV: interweaving Web and TV into a single experience  Mixing lean back and lean forward    2 Want to maintain immersion while removing fourth wall illusion Works only for certain genres and audiences Does second screen help or hinder? linkedtv.eu
  • 3. Scenarios and content www.linkedtv.eu Enriched News Antiques Interactive  Professional news content produced by RBB for general audience  local news show rbb Aktuell  People, locations and events can be linked to previous events.  Program watched predominantly by 50+ers  AVRO: 3 linkedtv.eu Tussen Kunst en Kitsch (TKK)  Objects, locations and experts provide anchors to related materials (targets).
  • 4. Antiques Interactive: Enriched link demo www.linkedtv.eu 4 linkedtv.eu
  • 5. Workflow www.linkedtv.eu Hypervideo analysis Paintings by Jan Sluijters Generation of media fragments Named Entity Recognition Enriching NEs with web content “...painting of Jan Sluijters....” dbpedia.org/resource/Jan_Sluyters 5 linkedtv.eu Presentation engine
  • 6. Workflow www.linkedtv.eu Hypervideo analysis Paintings by Jan Sluijters Generation of media fragments Named Entity Recognition Enriching NEs with web content VideoHypE “...painting of Jan Sluijters....” dbpedia.org/resource/Jan_Sluyters 6 linkedtv.eu Presentation engine
  • 7. Why do we need VideoHypE? www.linkedtv.eu Broadcasters want to have some control. Editors have the expertise to determine relevance + The user interface can only show a limited amount of links, and there are many     Research: sample 805 videos, average 55 NEs per video. 55 NEs = many, many possible links depending on the size of the white list [Li et al., 2013] ist 1 hite l w Link NE - person: Jan Sluijters , Roeland Ordelman1 NE – art syle: Fauvism NE - person: Emiel Aardewerk Link white list 2 Link white lis Link white list 4 Link white list 5 hite l in k w L ist 6 list 7 Link white Link white lis 7 t3 t etc etc
  • 8. VideoHypE requirements www.linkedtv.eu  Requirements session with 5 AVRO editorial members + digital staff of TKK. Main outcomes: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Separate user environment for each end user partner Selecting from list of available episodes Create and edit chapters Timeline with chapters See annotations per chapters, distinction between entity types Select annotations + related hyperlinks , Roeland Ordelman1 Add selected, approved hyperlinks Remove hyperlinks and/or entities (per media fragment or episode) Timeline with enities and hyperlinks Add new hyperlinks to media fragments (e.g. shots/chapters/spatial) Adding to white list Sorting on entity types => not yet Collaborative space => not yet Delete annotations forever => not yet Level of completion per chapter / episode => not yet Variable # suggested links depending on time pressure MAX 20-30 minutes per episode!!! 8
  • 9. Demo overview – choose your video www.linkedtv.eu , Roeland Ordelman1 9
  • 10. Demo overview – shot overview www.linkedtv.eu , Roeland Ordelman1 10
  • 11. Demo overview - chapters www.linkedtv.eu , Roeland Ordelman1 11
  • 12. Demo overview – selecting entities www.linkedtv.eu , Roeland Ordelman1 12
  • 13. Demo overview – selecting links per entity www.linkedtv.eu , Roeland Ordelman1 13
  • 14. Demo overview – editor‘s choices overview www.linkedtv.eu , Roeland Ordelman1 14
  • 15. Demo overview – adding links www.linkedtv.eu , Roeland Ordelman1 15
  • 16. VideoHypE – discussion & future work www.linkedtv.eu 1. 2. Implementing remaining requirements. How to make the editor tool scalable? • • • 1. 2. 3. Different programmes, different requirements How to deal with live content? There is not just one front-end => one-screen solutions, 2nd screens, multiple screens, etc. Personalisation: what ‘bandwith’ is left personalisation? Relevance is a crucial issue – how can the system ‘learn’? User testing with real editors 17
  • 17. Thanks! www.linkedtv.eu Find me here: @lottebelice | lbbaltussen@beeldengeluid.nl Credits: Again: other authors are Roeland Ordelman and Jaap Blom Slides partly by: Lynda Hardman (CWI) Heaps of thanks to AVRO for so generously letting LinkedTV use their content. , Roeland Ordelman1 Links: www.linkedtv.eu twitter.com/BenGlabs www.avro.nl 18