LinkedTV project results at the end of year 2


Published on

LinkedTV project results at the end of year 2

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

LinkedTV project results at the end of year 2

  1. 1. Television Linked To The Web LinkedTV results at the end of the 2nd year Lyndon Nixon, MODUL University 27/28 November 2013, 2nd Project Review, Hilversum
  2. 2. Overview - What is special about LinkedTV? - Scenarios at end of year 2 - How LinkedTV works (at M24) - Thanks 2
  3. 3. What is special about LinkedTV?  There is a trend towards „interactive images“ on the Web  There is a trend towards „interactive video“ (or „hypervideo“) on the Web ThingLink can embed clickable images in Facebook WireWax is sharing clickable video on YouTube  We develop a new solution for interactive media in a TV-integrated environment: „LinkedTV“ „... a true TV ecosystem must functionally integrate the apps with the television programming service and content being offered to the consumer.“ TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013 3
  4. 4. What is special about LinkedTV? Build upon Web and broadcast specifications and standards Make sure LinkedTV is part of the whole industry, not another proprietary fragment HTML5, Linked Data, W3C Media, hbbTV, Open IPTV Forum Reuse the rich wealth of data and services on the open Web Give added value at low cost Extendible to closed data and services for specialised offers Automate the annotation and enrichment as far as possible Broadcasters barrier to entry is the complexity and effort to make hypervideo Current approaches will never scale to the amount of video being produced 4
  5. 5. Scenarios at end of year 2 Scenario 1: Interactive News Show  Professional news content produced by RBB  Seed content: local news show "rbb Aktuell" 5 Scenario 2: Hyperlinked Documentary  Cultural heritage content from S&V  Seed content: "Tussen Kunst & Kitsch" from AVRO (Antiques Roadshow format) Scenario 3: Media Arts  MEDIADROM competition for defining new ideas for linked television  Concepts will be promoted at Mons: European Capital of Culture 2015.
  6. 6. Scenarios at end of year 2 Scenarios have been realised on top of the full end to end workflow of the LinkedTV Platform Main and companion screen experience with the LinkedTV Player Video demos are online 6
  7. 7. How LinkedTV works (at M24) Video object and text detection Examples of Paintings by Jan Sluijters … and their digital images Connection to concepts Expansion of related concepts Linking to related content “...schilderij van Jan Sluijters....” 7 Personalisation engine
  8. 8. Concept detection Video object and text detection LinkedTV works on services using & combining different forms of audiovisual content analysis: – Face detection – Textual analysis (subtitles, transcripts) – Keyword extraction – Multilingual automatic speech recognition – Speaker identification – Concept detection in video – Object re-detection – Shot segmentation This provides us with identification of “...schilderij van Jan Sluijters....” meaningful media fragments and summaries of their content analysis 8
  9. 9. Concept connections Connection to concepts Keywords and text fragments can be connected to uniquely identified, distinct named entities using online entity extraction services (NERD, – Media annotations are represented in RDF and stored in LinkedTV platform – Re-use of existing Web vocabularies e.g., NERD, LSCOM, DBpedia – Named Entity Recognition uses statistical & knowledge-based approaches (aiming at entity meaning disambiguation) – Extended by complementary visual analyses (e.g. a persons name in text with a face detected in a video fragment) 9
  10. 10. Concept expansion Paintings by Jan Sluijters Expansion of related concepts 10 The use of Linked Data in the identification of concepts in LinkedTV means we can expand concepts along different facets, i.e. allow users to explore in terms of their different interests in a given concept. For example, for an artist like Jan Sluijters, LinkedTV can link into:  Additional information, e.g.  biography of artist Leo Gestel has art style  style of painting  Related information, e.g. luminism has art style has art style  artists from same Piet Mondriaan Jan Sluijters period  paintings in similar style  related styles
  11. 11. Linking to related content … and their digital images Linking to related content 11 LinkedTV provides enrichment services which provide recommendations for related online content (Web pages, images, audio, video) pertaining to the concepts in the TV program: – Fresh Social Web content coming from e.g. Twitter and Facebook – User Generated content coming from e.g. Flickr and YouTube – Whitelist content coming from partner Websites like public broadcasters in Germany or cultural heritage archives in the Netherlands – Extendable and configurable by source and media type
  12. 12. Personalisation engine Personalisation engine 12 The LinkedTV Platform pushes annotation and enrichment information to the client (LinkedTV Player) – Both annotations and enrichments include further metadata about the topics they relate to, e.g. Jan Sluijters + Amsterdam, BER Airport + Berlin – A Personalisation engine on the Player filters & ranks the annotations and enrichments according to a locally stored user model, e.g. interested in Amsterdam vs interested in Berlin – The user model can be both manually edited online (LUME) or learnt by the LinkedTV system (attention tracker, event tracking) – All personalisation is opt in and private, respecting ethical issues
  13. 13. Packaged into the LinkedTV Platform The LinkedTV Platform encapsulates the functionality of the previous 4 steps. The main core of the platform is an Application Service Bus which monitors and triggers events across the components, realising the end to end workflow from a video ingestion to enriched playout. Communication between platform components and between platform and client is realised using RESTful APIs. 13
  14. 14. Playout on the LinkedTV Player LinkedTV enriched video playout needs a UI for Linked Television  The Springfield multiscreen platform abstracts the video and its enrichments from the device where they are displayed  Viewers can browse the topics of the program along „Who“, „Where“, „What“ axes  Enrichments can be browsed to, or shared, or bookmarked for later Current year 2 interface 14 Future user interface mockups
  15. 15. Thanks to... 15