LinkedTV project overview (2nd year)


Published on

LinkedTV is an EU funded project aiming to seamlessly interlink the TV and Web experiences. This presentation summarizes the results and achievements by the end of the project's second year, covering media analysis, annotation, linking, personalisation and interactive playout, based on two scenarios: Hyperlinked Documentary and Linked News.

For more, visit

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Developing individual technologies, using content from 2 program genres, with the aim of creating an end2end prototype.
    I’ll go through each of these stages.
    Don’t explain anything about each part.
  • (1) Lower the cost of preparing seed video for enrichment with other content
    Objects are detected in the video, e.g. painting, face
    Words are also detected in the speech, or closed captions
    (2) Slide: Web aspects: using and cnotributing to W3C recommendations, much of the s/w open source
  • Linking object or word to concept (painting, face, painter) or Named Entity (Jan_Sluijters, Piet_Mondrian)
    in Linked Open Data cloud. Each circle is a data set, representing thousands or millions of concepts. Familiar with wikipedia? DBpedia is machine-readable form?
    RDF is the language used on the web for expressing concepts (similar to XML for document structure)
  • Once we have the concepts & named entities in the Linked Open Data cloud, we can find other content (web pages or audiovisual material) by tracing paths through this cloud..
    Go through the bullets
    click through the links
    Could also link to concepts or entities in other sets, such as birthplace in GeoNames.
  • Large numbers of related concepts can be found very quickly.
    Selection based on connections within video topics and user preferences
    Need to find suitable video/information sources for users.If suitable material is scarce then can increase complexity of search (length of path in linked open data cloud).
    Broadcasters need to be sure of high quality sources, e.g., wikipedia, ArtcyclopediaWant structure that ensures that user "stays with the program" but has access to related information.
  • Large numbers of related concepts can be found very quickly.
    Selection based on connections within video topics and user preferences
    Need to find suitable video/information sources for users.If suitable material is scarce then can increase complexity of search (length of path in linked open data cloud).
    Broadcasters need to be sure of high quality sources, e.g., wikipedia, ArtcyclopediaWant structure that ensures that user "stays with the program" but has access to related information.
  • May the links be with you! Thank you.
  • LinkedTV project overview (2nd year)

    1. 1. Television Linked To The Web Web and TV seamlessly interlinked: The LinkedTV project Lyndon Nixon, scientific co-ordinator MODUL University GmbH
    2. 2. The future of television  Bring the rich browsing experience people know from the Web to television  Fulfil the convergence of TV on the Web and Web on the TV  Make TV more instructional and informative by uncovering the hidden knowledge in every video frame 2 © 2013, the LinkedTV project
    3. 3. Pre-LinkedTV: Pop Up video! Viewers have always been interested in additional information about what they are watching! Pop Up Video is a VH1 television show that "pops up" bubbles — officially called "info nuggets" — containing trivia, witticisms ... Most episodes of Pop Up Video play four or five music videos each ... The bubbles that pop up in each video generally appear about every 10–15 seconds ... Production costs for each episode total about $30,000 (source: Wikipedia) 3 © 2013, the LinkedTV project
    4. 4. Trend: More TV over the Internet 4 © 2013, the LinkedTV project
    5. 5. Trend: More Internet content on your TV Below: Boxee, hbbTV 5 © 2013, the LinkedTV project
    6. 6. Trend: More SmartTVs, more apps 54 million SmartTVs sold in 2012 will grow to 170 million in 2017; by then 64% of West European households will have at least 1 SmartTV. „Smart-TV device forecasts“, Informa Telecoms & Media Samsung SmartTV app statistics courtesy gadgets/files/2011/05/5m-samsungapps-1.jpeg 6 © 2013, the LinkedTV project
    7. 7. But are Connected / Smart TVs smart? Consumers are using complementary devices to surf alongside TV, not the TV itself Source: Yahoo Advertising Blog January 2011 Part of infographic courtesy Full image at: 7 © 2013, the LinkedTV project
    8. 8. New tech: Social TV  People tweeting about a program while they watch    Need agreement around a #hashtag Shows interest in deeper interaction with the content The Web generation are less interested in the passive TV experience A shade under three-quarters of TV viewers with broadband are surfing the web while they watch, with 38 per cent of them discussing what they're watching on social media. The proportion discussing TV shows rises to 53 per cent if only the 16 to 23-year-old demographic is included. -- Ovum survey, published in TheRegister, 6 October 2011 8 © 2013, the LinkedTV project
    9. 9. New tech: Second screen apps  Second screen content push while watching TV    Relates to the program as an atomic object Controlled by the content provider Restricted to programs who have „opted in“ IntoNow or Shazam identify what you are watching and provide associated content Second screen apps show you related content without disturbing the TV view 9 © 2013, the LinkedTV project
    10. 10. The LinkedTV future  There is a trend towards „interactive images“ on the Web  There is a trend towards „interactive video“ (or „hypervideo“) on the Web ThingLink can embed clickable images in Facebook WireWax is sharing clickable video on YouTube  We develop a new solution for interactive media in a TV-integrated environment: „LinkedTV“ „... a true TV ecosystem must functionally integrate the apps with the television programming service and content being offered to the consumer.“ TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013 10 © 2013, the LinkedTV project
    11. 11. What makes LinkedTV special?  Build upon Web and broadcast specifications and standards   Make sure LinkedTV is part of the whole industry, not another proprietary fragment HTML5, Linked Data, W3C Media, hbbTV, Open IPTV Forum  Reuse the rich wealth of data and services on the open Web   Give added value at low cost Extendible to closed data and services for specialised offers  Automate the annotation and enrichment as far as possible   Broadcasters barrier to entry is the complexity and effort to make hypervideo Current approaches will never scale to the amount of video being produced 11 © 2013, the LinkedTV project
    12. 12. Scenarios Scenario 1: Interactive News Show  Professional news content produced by RBB  Seed content: local news show "rbb Aktuell" Scenario 2: Hyperlinked Documentary  Cultural heritage content from S&V  Seed content: "Tussen Kunst & Kitsch" from AVRO (Antiques Roadshow format) Scenario 3: Media Arts  MEDIADROM competition for defining new ideas for linked television  Concepts will be promoted at Mons: European Capital of Culture 2015. 12 12 © 2013, the LinkedTV project
    13. 13. User trials for LinkedTV usage insight Scenario 1: Scenario 2: Interactive News Show Hyperlinked Documentary  News is often watched together and  Tussen Kunst & over a meal! Kitsch viewers do  Viewers have more want to look up complex information information needs when watching  Objects and their news history of primary  Enriched news interest viewing could be  Bookmark for later, parallel to the main not during the show news, without additional interaction 13 © 2013, the LinkedTV project Also: industry insight on consumption trends  Second screen and social media integration are the key trends  News and documentary are the most interesting formats for enrichment  Additional info should be compact, bookmarkable 13
    14. 14. Scenario for a hyperlinked documentary See the demo: Content courtesy 14 © 2013, the LinkedTV project
    15. 15. Scenario for an interactive news show See the demo: Content courtesy 15 © 2013, the LinkedTV project
    16. 16. How will it work? Video object and text detection Examples of Paintings by Jan Sluijters … and their digital images Connection to concepts Expansion of related concepts Linking to related content “...schilderij van Jan Sluijters....” 16 © 2013, the LinkedTV project Personalisation engine
    17. 17. Concept Detection Video object and text detection LinkedTV works on services using & combining different forms of audiovisual content analysis: – Face detection – Textual analysis (subtitles, transcripts) – Keyword extraction – Multilingual automatic speech recognition – Speaker identification – Concept detection in video – Object re-detection – Shot segmentation This provides us with identification of “...schilderij van Jan Sluijters....” meaningful media fragments and summaries of their content analysis 17 © 2013, the LinkedTV project
    18. 18. Concept Connections Connection to concepts Keywords and text fragments can be connected to uniquely identified, distinct named entities using online entity extraction services (NERD, – Media annotations are represented in RDF and stored in LinkedTV platform – Re-use of existing Web vocabularies e.g., NERD, LSCOM, DBpedia – Named Entity Recognition uses statistical & knowledge-based approaches (aiming at entity meaning disambiguation) – Extended by complementary visual analyses (e.g. a persons name in text with a face detected in a video fragment) 18 © 2013, the LinkedTV project
    19. 19. Concept Expansion Paintings by Jan Sluijters Expansion of related concepts 19 The use of Linked Data in the identification of concepts in LinkedTV means we can expand concepts along different facets, i.e. allow users to explore in terms of their different interests in a given concept. For example, for an artist like Jan Sluijters, LinkedTV can link into:  Additional information, e.g.  biography of artist Leo Gestel has art style  style of painting  Related information, e.g. luminism has art style has art style  artists from same Piet Mondriaan Jan Sluijters period  paintings in similar style  related styles © 2013, the LinkedTV project
    20. 20. Linking to related content … and their digital images Linking to related content 20 LinkedTV provides enrichment services which provide recommendations for related online content (Web pages, images, audio, video) pertaining to the concepts in the TV program: – Fresh Social Web content coming from e.g. Twitter and Facebook – User Generated content coming from e.g. Flickr and YouTube – Whitelist content coming from partner Websites like public broadcasters in Germany or cultural heritage archives in the Netherlands – Extendable and configurable by source and media type © 2013, the LinkedTV project
    21. 21. Personalisation engine Personalisation engine 21 The LinkedTV Platform pushes annotation and enrichment information to the client (LinkedTV Player) – Both annotations and enrichments include further metadata about the topics they relate to, e.g. Jan Sluijters + Amsterdam, BER Airport + Berlin – A Personalisation engine on the Player filters & ranks the annotations and enrichments according to a locally stored user model, e.g. interested in Amsterdam vs interested in Berlin – The user model can be both manually edited online (LUME) or learnt by the LinkedTV system (attention tracker, event tracking) – All personalisation is opt in and private, respecting ethical issues © 2013, the LinkedTV project
    22. 22. Packaged into a LinkedTV Platform The LinkedTV Platform encapsulates the functionality of the previous 4 steps. The main core of the platform is an Application Service Bus which monitors and triggers events across the components, realising the end to end workflow from a video ingestion to enriched playout. Communication between platform components and between platform and client is realised using RESTful APIs. 22 © 2013, the LinkedTV project
    23. 23. Playout on the LinkedTV Player LinkedTV enriched video playout needs a UI for Linked Television  The Springfield multiscreen platform abstracts the video and its enrichments from the device where they are displayed  Viewers can browse the topics of the program along „Who“, „Where“, „What“ axes  Enrichments can be browsed to, or shared, or bookmarked for later 23 © 2013, the LinkedTV project
    24. 24. Future LinkedTV Player UI LinkedTV player UI will evolve with the Platform technology  Richer displays of associated information  HbbTV implementation  Gesture control 24 © 2013, the LinkedTV project
    25. 25. Follow LinkedTV & keep up with the future of television Web: Twitter: @linkedtv 25 © 2013, the LinkedTV project
    26. 26. 26 © 2013, the LinkedTV project