www.linkedtv.eu
Lotte Belice Baltussen, Roeland Ordelman, Jaap Blom
Netherlands Institute for Sound and Vision
VideoHypE: ...
2
www.linkedtv.eu
Video hyperlinking
§  Topic that is receiving increasing attention
§  Relate video (fragments) to othe...
3
www.linkedtv.eu
Experiencing the web on TV
§  Making the web available on TV
§  Typically this could take form in a Se...
4
www.linkedtv.eu
Dynamic contextually relevant links
§  Current approaches: editorial (manual)
§  What could automation...
5
www.linkedtv.eu
VIDEO HYPerlink Editing tool
§  Automatic link suggestion based on video analysis
§  Segmentation, vis...
6
www.linkedtv.eu
Scenarios and content
	
  
Enriched News
§  Professional news content
produced by RBB for general
audie...
7
www.linkedtv.eu
dbpedia.org/resource/Jan_Sluyters
Link suggestion workflow
linkedtv.eu
Paintings by Jan Sluijters
“...pa...
8
www.linkedtv.eu
VideoHypE & Automatic Link Suggestion
, Roeland Ordelman1
§  Supervision on suggested anchors
§  Desel...
9
www.linkedtv.eu
Additional VideoHypE requirements
, Roeland Ordelman1
§  Requirements elicitation session with 5 editor...
10
www.linkedtv.eu
Demo overview – choose your video
, Roeland Ordelman1
11
www.linkedtv.eu
Demo overview – shot overview
, Roeland Ordelman1
12
www.linkedtv.eu
Demo overview - chapters
, Roeland Ordelman1
13
www.linkedtv.eu
Demo overview – selecting entities
, Roeland Ordelman1
14
www.linkedtv.eu
Demo overview – selecting links per entity
, Roeland Ordelman1
15
www.linkedtv.eu
Demo overview – editor‘s choices overview
, Roeland Ordelman1
16
www.linkedtv.eu
Demo overview – adding links
, Roeland Ordelman1
17
www.linkedtv.eu
•  Status:
•  Implementing remaining requirements
•  Next milestone: user testing with real editors
•  ...
18
www.linkedtv.eu
Thanks!
, Roeland Ordelman1
Roeland Ordelman
Audiovisual Content Access
University of Twente, The Nethe...
Upcoming SlideShare
Loading in …5
×

Intetain presentation on VideoHypE, the LinkedTV video hyperlink editor

493 views

Published on

Intetain presentation on VideoHypE, the LinkedTV video hyperlink editor, July 2013

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
493
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Intetain presentation on VideoHypE, the LinkedTV video hyperlink editor

  1. 1. www.linkedtv.eu Lotte Belice Baltussen, Roeland Ordelman, Jaap Blom Netherlands Institute for Sound and Vision VideoHypE: An Editor Tool for Supervised Automatic Video Hyperlinking Intetain | Mons, Belgium – 03 July 2013
  2. 2. 2 www.linkedtv.eu Video hyperlinking §  Topic that is receiving increasing attention §  Relate video (fragments) to other multimedia sources §  Various perspectives and approaches: §  Semantic/Event Web §  Recommendation: detail-on-demand §  Storytelling/narratives: connected video fragments tell a story §  Summarization: links between video fragments to create summary §  Link up communities by means of special sessions, workshops and benchmarks §  Multimedia Modeling (MMM 2014) – Dublin. Special session on Multimedia Hyperlinking and Retrieval – CfP due 01 August 2013 http://mmm2014.org/special-sessions/ §  MediaEval Benchmark Evaluation on Search & Hyperlinking – www.multimediaeval.org 7/2/13
  3. 3. 3 www.linkedtv.eu Experiencing the web on TV §  Making the web available on TV §  Typically this could take form in a Second Screen App §  “Reference Device”, “Companion Device” §  Second Screen dimensions §  Social §  Channel §  Program §  Provide dynamic, contextually relevant links §  Give background information §  Link to e-shops
  4. 4. 4 www.linkedtv.eu Dynamic contextually relevant links §  Current approaches: editorial (manual) §  What could automation bring in? Automatic link suggestion? §  End-user perspective §  Automation to enable really dynamic/user driven links §  Broadcaster’s perspective §  What are potentially interesting links for users? §  Automatic link suggestion could help §  However, some potential links should be avoided! §  We need control over the links §  Which links “add value” to the program? §  We need to take program context into account §  How to maximize link potential & reduce manual effort? §  …
  5. 5. 5 www.linkedtv.eu VIDEO HYPerlink Editing tool §  Automatic link suggestion based on video analysis §  Segmentation, visual concept/face detection, speech & speaker recognition, entity extraction, … §  Editing tool for: §  Selecting/deselecting/adding (suggested) link anchors §  Selecting/adding link targets §  (optionally) pre-view the results §  Development in use scenarios based on specific programming: §  News §  Antiques Roadshow
  6. 6. 6 www.linkedtv.eu Scenarios and content   Enriched News §  Professional news content produced by RBB for general audience §  local news show rbb Aktuell §  People, locations and events can be linked to previous events. Antiques Interactive §  Program watched predominantly by 50+ers §  AVRO: Tussen Kunst en Kitsch (TKK) §  Objects, locations and experts provide anchors to related materials (targets). linkedtv.eu
  7. 7. 7 www.linkedtv.eu dbpedia.org/resource/Jan_Sluyters Link suggestion workflow linkedtv.eu Paintings by Jan Sluijters “...painting of Jan Sluijters....” Video analysis Presentation engine Named Entity Recognition Linking NEs to web content Generation of media fragments Editing tool
  8. 8. 8 www.linkedtv.eu VideoHypE & Automatic Link Suggestion , Roeland Ordelman1 §  Supervision on suggested anchors §  Deselect undesired links §  Favour ‘authored’ websites (e.g., using white-list) §  Determine relevance of links (e.g., on the basis of knowledge about audience) §  Narrow down amount of links: potentially there are many §  Research: sample 805 videos, average 55 NEs per video. 55 NEs = many, many possible links depending on the size of the white list [Li et al., 2013] NE - person: Jan Sluijters NE – art syle: Fauvism Link white list 1 Link white list 2 Link white list 3 Link white list 4 Link white list 5
  9. 9. 9 www.linkedtv.eu Additional VideoHypE requirements , Roeland Ordelman1 §  Requirements elicitation session with 5 editors 1.  Create and edit chapters 2.  Timeline with chapters 3.  See annotations per chapters, distinction between entity types 4.  Select annotations + related hyperlinks 5.  Add selected, approved hyperlinks 6.  Timeline with enities and hyperlinks 7.  Add new hyperlinks to media fragments (e.g. shots/chapters/spatial) 8.  Remove hyperlinks and/or entities (per media fragment or episode) 9.  Adding to white list 10.  Sorting on entity types
  10. 10. 10 www.linkedtv.eu Demo overview – choose your video , Roeland Ordelman1
  11. 11. 11 www.linkedtv.eu Demo overview – shot overview , Roeland Ordelman1
  12. 12. 12 www.linkedtv.eu Demo overview - chapters , Roeland Ordelman1
  13. 13. 13 www.linkedtv.eu Demo overview – selecting entities , Roeland Ordelman1
  14. 14. 14 www.linkedtv.eu Demo overview – selecting links per entity , Roeland Ordelman1
  15. 15. 15 www.linkedtv.eu Demo overview – editor‘s choices overview , Roeland Ordelman1
  16. 16. 16 www.linkedtv.eu Demo overview – adding links , Roeland Ordelman1
  17. 17. 17 www.linkedtv.eu •  Status: •  Implementing remaining requirements •  Next milestone: user testing with real editors •  How do editors appreciate the tool? •  Overload versus added value of aut. suggestions •  Is the technology mature enough? •  Additional questions: •  How do end-users appreciate the output of the tool? •  Can we make the tool adaptive (`learn’ on the basis of editors input) •  How to make the editor tool scalable? Different programmes, different requirements •  Visualisation of linked content (in tool+application) may also influence selection process •  LinkedTV Demo session @ ~16:00 VideoHypE – discussion & future work
  18. 18. 18 www.linkedtv.eu Thanks! , Roeland Ordelman1 Roeland Ordelman Audiovisual Content Access University of Twente, The Netherlands Netherlands Institute for Sound and Vision Cross-Media Interaction Check out: §  www.linkedtv.eu (project website) §  www.multimediaeval.org (benchmark evaluation on Search & Hyperlinking in conjunction with ACM Multimedia 2013, october 18-19, Barcelona)

×