LinkedTV: Building the future of television


Published on

User oriented overview of project: "LinkedTV: Building the future of television" shown at Volt!, by Lynda Hardman in Eindhoven, June 3rd 2013.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Not just TV, but interaction with online media (archives).
  • (1) Big Q: Do you ever have a question about what you are watching?
    Would you like related information about something you have seen?
    Something someone mentioned?
    One option is to pause the TV, go find your smart phone or pad and start some google search.
    When you carry out the search, your device doesn't know what you were watching, nor what the specific scene was about.
    Wouldn't it be nice to have a system that knows what you are watching, knows what you like and that presents links to extra pieces of information to you?
    Technology only likely to be useful when audience, content and program genre match.
    (2) slide
  • (1) Currently developing tools using 2 program genres
  • (1) What do we think it will look like? Not like this!
    Select video. Link to video!
  • Developing individual technologies, using content from 2 program genres, with the aim of creating an end2end prototype.
    I’ll go through each of these stages.
    Don’t explain anything about each part.
  • (1) Lower the cost of preparing seed video for enrichment with other content
    Objects are detected in the video, e.g. painting, face
    Words are also detected in the speech, or closed captions
    (2) Slide: Web aspects: using and cnotributing to W3C recommendations, much of the s/w open source
  • Linking object or word to concept (painting, face, painter) or Named Entity (Jan_Sluijters, Piet_Mondrian)
    in Linked Open Data cloud. Each circle is a data set, representing thousands or millions of concepts. Familiar with wikipedia? DBpedia is machine-readable form?
    RDF is the language used on the web for expressing concepts (similar to XML for document structure)
  • A number of tools are available, e.g. allowing the conversion of different sources of metadata to be converted to RDF, Named Entity Recognition.
  • Once we have the concepts & named entities in the Linked Open Data cloud, we can find other content (web pages or audiovisual material) by tracing paths through this cloud..
    Go through the bullets
    click through the links
    Could also link to concepts or entities in other sets, such as birthplace in GeoNames.
  • Large numbers of related concepts can be found very quickly.
    Selection based on connections within video topics and user preferences
    Need to find suitable video/information sources for users.If suitable material is scarce then can increase complexity of search (length of path in linked open data cloud).
    Broadcasters need to be sure of high quality sources, e.g., wikipedia, ArtcyclopediaWant structure that ensures that user "stays with the program" but has access to related information.
  • Team@CWI is investigating user info needs specific to narrow goals of LinkedTV.
    Working together with Beeld Geluid on this.
  • Skip if time short
  • Once we have chosen related content, we need to display it to the end user.
    Explore interaction modalities to browse the enrichment content.
    We need to understand how involved user wishes to be.
    Playout of enriched video on multiple devices and cross-device, e.g. second screen
    Providing requirements for first implementation of the presentation layer based on User Interface mock-ups of scenarios
    Carrying out user focus groups for second round requirements
  • I hope that this overview of the project makes more sense now!
  • Project is making underlying technology available.
    Daniel Ockeloen, Noterik, presenting later this afternoon.
    HTML5 for cross-device Web application development
  • In addition to the technology development, also collecting annotations of resources in project database.
  • Demos are available online
  • What next?!
  • May the links be with you! Thank you.
  • LinkedTV: Building the future of television

    1. 1. Television Linked To The Web Building the future of television Lynda Hardman Centrum Wiskunde & Informatica
    2. 2. The LinkedTV vision  Web and TV exist in parallel    86% of TV viewers surf the Web at the same time (Yahoo!, USA) 88% of USA consumers use mobile as second screen (Business Insider, May 2013) Manual efforts to link TV program to additional content (Shazam, IntoNow)  LinkedTV: interweaving Web and TV into a single experience  Mixing lean back and lean forward    2 Want to maintain immersion while removing fourth wall illusion Works only for certain genres and audiences Does second screen help or hinder?
    3. 3. Scenarios and content Enriched News Antiques Edutainment  Professional news content produced by RBB for general audience  local news show rbb Aktuell  People, locations and events can be linked to previous events.  Program watched predominantly by 50+ers  AVRO: Tussen Kunst en Kitsch  Presenters, locations and objects provide hooks to related materials. 3
    4. 4. Enriched link demonstration 4
    5. 5. How will it work? Video object and word detection Paintings by Jan Sluijters Connection to concepts Selection of related concepts Selection of related content “...schilderij van Jan Sluijters....” 5 Presentation engine
    6. 6. Concept Detection Video object and word detection Object detection in video Face detection Text analysis, keyword extraction Multilingual automatic speech recognition Combining results from different modality analyses  Speaker identification  Object re-detection       Output in XML using OpenSource tool: EXMARaLDA “...schilderij van Jan Sluijters....” 6
    7. 7. Concept connections Connection to concepts  Annotations represented in RDF stored in LinkedTV platform  Use existing vocabularies e.g., NERD, LSCOM, DBpedia  Named Entity Recognition using statistical & knowledge-based approaches  Enrichment based on textual and visual analyses 7
    8. 8. LinkedTV metadata conversion 8 WP2 - Linking hypervideos to Web content
    9. 9. Concept selection Paintings by Jan Sluijters  Additional information, e.g.  biography of artist  style of painting  Related information, e.g.  artists from same period  paintings in similar style  related styles Leo Gestel has art style Selection of related concepts luminism has art style Piet Mondriaan 9 has art style Jan Sluijters
    10. 10. Content selection  Selection based on context of scene, e.g. system knows which topics appear next  User preferences, e.g. closely related information to uninteresting topic is less useful Selection of related content 10  We need to understand user information needs (per genre), e.g. don’t miss important news because of personalisation  Broadcasters don’t want users to drift away from program
    11. 11. Example user information needs for news  MORE IN DEPTH SPECIFIC INFORMATION I was wondering just how expensive it is to make all those vaccines, you know? I was just wondering if it is a good idea to even make a lot of it or how hard it is to make.  LOCAL EFFECT OF GLOBAL NEWS – What is the impact in my life? Yes then I would look it up, what is the effect in the Netherlands… (of the bird flu virus)  OVERVIEW OF PAST RELATED EVENTS My search would be something like "South Korea state of war ramp up” (…) I would like to know, what happened two years ago and how they solved it… 11 LinkedTV Berlin, 2013
    12. 12. User information needs for news  Information about how the story ends  Updates when important changes occur  More information about the people involved  Where is that location?  Consult related publications and documentaries  Opinions from different experts  Access related info selected by experts and suggested by broadcasters 12 LinkedTV Berlin, 2013
    13. 13. Content display  Many users watch TV news during dinner!  Use second screen to  give user more control  retain sense of immersion via TV screen  Need to resolve issues of pausing when in group situation  There is no single best solution Presentation engine 13
    14. 14. How will it work? Video object and word detection Paintings by Jan Sluijters Connection to concepts Selection of related concepts Selection of related content “...schilderij van Jan Sluijters....” 14 Presentation engine
    15. 15. How can LinkedTV help you? 15
    16. 16. LinkedTV platform & player External Platform Network Client Editor LoD Cloud Browser Variant Material selection, Annotation adjustment PC / Tablet HTTP Browser HTML 5 LinkedTV Server HbbTV Variant • Video Analysis • Enrichment • Personalization • Delivery Videos from Scenarios Broadcast Repository • Annotations • Links to external Resources A clearly defined and modularized platform architecture, deployment decisions made, initial REST API 16 TV Set Browser HTTP CE-HTML An interactive video player based on Web specifications (HTML5) realising intuitive interaction & enrichment layers
    17. 17. 17
    18. 18. LinkedTV in 2013  Individual video analysis, annotation, hyperlinking, UI/UX design and personalisation components integrated into a LinkedTV platform with end-to-end content workflow  Provide player for desktop, TV and second screen  Create interactive scenarios (rbbAktuell and Tussen Kunst en Kitsch) to engage with users to further understand their interaction and information needs 18
    19. 19. 19