• Save
LinkedTV: Building the future of television
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


LinkedTV: Building the future of television



User oriented overview of project: "LinkedTV: Building the future of television" shown at Volt!, by Lynda Hardman in Eindhoven, June 3rd 2013.

User oriented overview of project: "LinkedTV: Building the future of television" shown at Volt!, by Lynda Hardman in Eindhoven, June 3rd 2013.



Total Views
Views on SlideShare
Embed Views



1 Embed 113

http://www.linkedtv.eu 113


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Not just TV, but interaction with online media (archives). <br />
  • (1) Big Q: Do you ever have a question about what you are watching? <br /> Would you like related information about something you have seen? <br /> Something someone mentioned? <br /> One option is to pause the TV, go find your smart phone or pad and start some google search. <br /> When you carry out the search, your device doesn&apos;t know what you were watching, nor what the specific scene was about. <br /> Wouldn&apos;t it be nice to have a system that knows what you are watching, knows what you like and that presents links to extra pieces of information to you? <br /> Technology only likely to be useful when audience, content and program genre match. <br /> (2) slide <br />
  • (1) Currently developing tools using 2 program genres <br />
  • (1) What do we think it will look like? Not like this! <br /> Select video. Link to video! <br />
  • Developing individual technologies, using content from 2 program genres, with the aim of creating an end2end prototype. <br /> I’ll go through each of these stages. <br /> Don’t explain anything about each part. <br />
  • (1) Lower the cost of preparing seed video for enrichment with other content <br /> Objects are detected in the video, e.g. painting, face <br /> Words are also detected in the speech, or closed captions <br /> (2) Slide: Web aspects: using and cnotributing to W3C recommendations, much of the s/w open source <br />
  • Linking object or word to concept (painting, face, painter) or Named Entity (Jan_Sluijters, Piet_Mondrian) <br /> in Linked Open Data cloud. Each circle is a data set, representing thousands or millions of concepts. Familiar with wikipedia? DBpedia is machine-readable form? <br /> RDF is the language used on the web for expressing concepts (similar to XML for document structure) <br />
  • A number of tools are available, e.g. allowing the conversion of different sources of metadata to be converted to RDF, Named Entity Recognition. <br />
  • Once we have the concepts & named entities in the Linked Open Data cloud, we can find other content (web pages or audiovisual material) by tracing paths through this cloud.. <br /> Go through the bullets <br /> click through the links <br /> Could also link to concepts or entities in other sets, such as birthplace in GeoNames. <br />
  • Large numbers of related concepts can be found very quickly. <br /> Selection based on connections within video topics and user preferences <br /> Need to find suitable video/information sources for users.If suitable material is scarce then can increase complexity of search (length of path in linked open data cloud). <br /> Broadcasters need to be sure of high quality sources, e.g., wikipedia, ArtcyclopediaWant structure that ensures that user "stays with the program" but has access to related information. <br />
  • Team@CWI is investigating user info needs specific to narrow goals of LinkedTV. <br /> Working together with Beeld Geluid on this. <br />
  • Skip if time short <br />
  • Once we have chosen related content, we need to display it to the end user. <br /> Explore interaction modalities to browse the enrichment content. <br /> We need to understand how involved user wishes to be. <br /> Playout of enriched video on multiple devices and cross-device, e.g. second screen <br /> Providing requirements for first implementation of the presentation layer based on User Interface mock-ups of scenarios <br /> Carrying out user focus groups for second round requirements <br />
  • I hope that this overview of the project makes more sense now! <br />
  • Project is making underlying technology available. <br /> Daniel Ockeloen, Noterik, presenting later this afternoon. <br /> HTML5 for cross-device Web application development <br />
  • In addition to the technology development, also collecting annotations of resources in project database. <br />
  • Demos are available online <br />
  • What next?! <br />
  • May the links be with you! Thank you. <br />

LinkedTV: Building the future of television Presentation Transcript

  • 1. Television Linked To The Web Building the future of television Lynda Hardman Centrum Wiskunde & Informatica www.linkedtv.eu
  • 2. The LinkedTV vision www.linkedtv.eu  Web and TV exist in parallel    86% of TV viewers surf the Web at the same time (Yahoo!, USA) 88% of USA consumers use mobile as second screen (Business Insider, May 2013) Manual efforts to link TV program to additional content (Shazam, IntoNow)  LinkedTV: interweaving Web and TV into a single experience  Mixing lean back and lean forward    2 Want to maintain immersion while removing fourth wall illusion Works only for certain genres and audiences Does second screen help or hinder? linkedtv.eu
  • 3. Scenarios and content www.linkedtv.eu Enriched News Antiques Edutainment  Professional news content produced by RBB for general audience  local news show rbb Aktuell  People, locations and events can be linked to previous events.  Program watched predominantly by 50+ers  AVRO: Tussen Kunst en Kitsch  Presenters, locations and objects provide hooks to related materials. 3 linkedtv.eu
  • 4. Enriched link demonstration www.linkedtv.eu 4 linkedtv.eu
  • 5. How will it work? www.linkedtv.eu Video object and word detection Paintings by Jan Sluijters Connection to concepts Selection of related concepts Selection of related content “...schilderij van Jan Sluijters....” dbpedia.org/resource/Jan_Sluyters 5 linkedtv.eu Presentation engine
  • 6. Concept Detection www.linkedtv.eu Video object and word detection Object detection in video Face detection Text analysis, keyword extraction Multilingual automatic speech recognition Combining results from different modality analyses  Speaker identification  Object re-detection       Output in XML using OpenSource tool: EXMARaLDA http://www.exmaralda.org/ “...schilderij van Jan Sluijters....” 6 linkedtv.eu
  • 7. Concept connections www.linkedtv.eu Connection to concepts  Annotations represented in RDF stored in LinkedTV platform  Use existing vocabularies e.g. schema.org, NERD, LSCOM, DBpedia  Named Entity Recognition using statistical & knowledge-based approaches  Enrichment based on textual and visual analyses dbpedia.org/resource/Jan_Sluyters 7 linkedtv.eu
  • 8. LinkedTV metadata conversion www.linkedtv.eu http://linkedtv.eurecom.fr/metadata/ 8 WP2 - Linking hypervideos to Web content
  • 9. Concept selection www.linkedtv.eu Paintings by Jan Sluijters  Additional information, e.g.  biography of artist  style of painting  Related information, e.g.  artists from same period  paintings in similar style  related styles Leo Gestel has art style Selection of related concepts luminism has art style Piet Mondriaan 9 linkedtv.eu has art style Jan Sluijters
  • 10. Content selection www.linkedtv.eu  Selection based on context of scene, e.g. system knows which topics appear next  User preferences, e.g. closely related information to uninteresting topic is less useful Selection of related content 10  We need to understand user information needs (per genre), e.g. don’t miss important news because of personalisation  Broadcasters don’t want users to drift away from program linkedtv.eu
  • 11. Example user information needs for news www.linkedtv.eu  MORE IN DEPTH SPECIFIC INFORMATION I was wondering just how expensive it is to make all those vaccines, you know? I was just wondering if it is a good idea to even make a lot of it or how hard it is to make.  LOCAL EFFECT OF GLOBAL NEWS – What is the impact in my life? Yes then I would look it up, what is the effect in the Netherlands… (of the bird flu virus)  OVERVIEW OF PAST RELATED EVENTS My search would be something like "South Korea state of war ramp up” (…) I would like to know, what happened two years ago and how they solved it… 11 LinkedTV Berlin, 2013
  • 12. User information needs for news www.linkedtv.eu  Information about how the story ends  Updates when important changes occur  More information about the people involved  Where is that location?  Consult related publications and documentaries  Opinions from different experts  Access related info selected by experts and suggested by broadcasters 12 LinkedTV Berlin, 2013
  • 13. Content display www.linkedtv.eu  Many users watch TV news during dinner!  Use second screen to  give user more control  retain sense of immersion via TV screen  Need to resolve issues of pausing when in group situation  There is no single best solution Presentation engine 13 linkedtv.eu
  • 14. How will it work? www.linkedtv.eu Video object and word detection Paintings by Jan Sluijters Connection to concepts Selection of related concepts Selection of related content “...schilderij van Jan Sluijters....” dbpedia.org/resource/Jan_Sluyters 14 linkedtv.eu Presentation engine
  • 15. How can LinkedTV help you? www.linkedtv.eu 15 linkedtv.eu
  • 16. LinkedTV platform & player www.linkedtv.eu External Platform Network Client Editor LoD Cloud Browser Variant Material selection, Annotation adjustment PC / Tablet HTTP Browser HTML 5 LinkedTV Server HbbTV Variant • Video Analysis • Enrichment • Personalization • Delivery Videos from Scenarios Broadcast Repository • Annotations • Links to external Resources A clearly defined and modularized platform architecture, deployment decisions made, initial REST API 16 linkedtv.eu TV Set Browser HTTP CE-HTML An interactive video player based on Web specifications (HTML5) realising intuitive interaction & enrichment layers
  • 17. www.linkedtv.eu 17 linkedtv.eu
  • 18. LinkedTV in 2013 www.linkedtv.eu  Individual video analysis, annotation, hyperlinking, UI/UX design and personalisation components integrated into a LinkedTV platform with end-to-end content workflow  Provide player for desktop, TV and second screen  Create interactive scenarios (rbbAktuell and Tussen Kunst en Kitsch) to engage with users to further understand their interaction and information needs 18 linkedtv.eu
  • 19. www.linkedtv.eu 19 linkedtv.eu