• Like
  • Save
 LinkedTV project overview (2nd year)
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

LinkedTV project overview (2nd year)


LinkedTV is an EU funded project aiming to seamlessly interlink the TV and Web experiences. This presentation summarizes the results and achievements by the end of the project's second year, covering …

LinkedTV is an EU funded project aiming to seamlessly interlink the TV and Web experiences. This presentation summarizes the results and achievements by the end of the project's second year, covering media analysis, annotation, linking, personalisation and interactive playout, based on two scenarios: Hyperlinked Documentary and Linked News.

For more, visit www.linkedtv.eu

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • Developing individual technologies, using content from 2 program genres, with the aim of creating an end2end prototype.
    I’ll go through each of these stages.
    Don’t explain anything about each part.
  • (1) Lower the cost of preparing seed video for enrichment with other content
    Objects are detected in the video, e.g. painting, face
    Words are also detected in the speech, or closed captions
    (2) Slide: Web aspects: using and cnotributing to W3C recommendations, much of the s/w open source
  • Linking object or word to concept (painting, face, painter) or Named Entity (Jan_Sluijters, Piet_Mondrian)
    in Linked Open Data cloud. Each circle is a data set, representing thousands or millions of concepts. Familiar with wikipedia? DBpedia is machine-readable form?
    RDF is the language used on the web for expressing concepts (similar to XML for document structure)
  • Once we have the concepts & named entities in the Linked Open Data cloud, we can find other content (web pages or audiovisual material) by tracing paths through this cloud..
    Go through the bullets
    click through the links
    Could also link to concepts or entities in other sets, such as birthplace in GeoNames.
  • Large numbers of related concepts can be found very quickly.
    Selection based on connections within video topics and user preferences
    Need to find suitable video/information sources for users.If suitable material is scarce then can increase complexity of search (length of path in linked open data cloud).
    Broadcasters need to be sure of high quality sources, e.g., wikipedia, ArtcyclopediaWant structure that ensures that user "stays with the program" but has access to related information.
  • Large numbers of related concepts can be found very quickly.
    Selection based on connections within video topics and user preferences
    Need to find suitable video/information sources for users.If suitable material is scarce then can increase complexity of search (length of path in linked open data cloud).
    Broadcasters need to be sure of high quality sources, e.g., wikipedia, ArtcyclopediaWant structure that ensures that user "stays with the program" but has access to related information.
  • May the links be with you! Thank you.


  • 1. Television Linked To The Web Web and TV seamlessly interlinked: The LinkedTV project Lyndon Nixon, scientific co-ordinator MODUL University GmbH lyndon.nixon@modul.ac.at www.linkedtv.eu
  • 2. The future of television www.linkedtv.eu  Bring the rich browsing experience people know from the Web to television  Fulfil the convergence of TV on the Web and Web on the TV  Make TV more instructional and informative by uncovering the hidden knowledge in every video frame 2 © 2013, the LinkedTV project
  • 3. Pre-LinkedTV: Pop Up video! www.linkedtv.eu Viewers have always been interested in additional information about what they are watching! Pop Up Video is a VH1 television show that "pops up" bubbles — officially called "info nuggets" — containing trivia, witticisms ... Most episodes of Pop Up Video play four or five music videos each ... The bubbles that pop up in each video generally appear about every 10–15 seconds ... Production costs for each episode total about $30,000 (source: Wikipedia) 3 © 2013, the LinkedTV project
  • 4. Trend: More TV over the Internet www.linkedtv.eu 4 © 2013, the LinkedTV project
  • 5. Trend: More Internet content on your TV www.linkedtv.eu Below: Boxee, hbbTV 5 © 2013, the LinkedTV project
  • 6. Trend: More SmartTVs, more apps www.linkedtv.eu 54 million SmartTVs sold in 2012 will grow to 170 million in 2017; by then 64% of West European households will have at least 1 SmartTV. „Smart-TV device forecasts“, Informa Telecoms & Media Samsung SmartTV app statistics courtesy http://thenextweb.com/ gadgets/files/2011/05/5m-samsungapps-1.jpeg 6 © 2013, the LinkedTV project
  • 7. But are Connected / Smart TVs smart? www.linkedtv.eu Consumers are using complementary devices to surf alongside TV, not the TV itself Source: Yahoo Advertising Blog http://www.yadvertisingblog.com January 2011 Part of infographic courtesy http://socialitv.web.ua.pt Full image at: http://socialitv.web.ua.pt/index.php/contents/images/ 7 © 2013, the LinkedTV project
  • 8. New tech: Social TV www.linkedtv.eu  People tweeting about a program while they watch    Need agreement around a #hashtag Shows interest in deeper interaction with the content The Web generation are less interested in the passive TV experience A shade under three-quarters of TV viewers with broadband are surfing the web while they watch, with 38 per cent of them discussing what they're watching on social media. The proportion discussing TV shows rises to 53 per cent if only the 16 to 23-year-old demographic is included. -- Ovum survey, published in TheRegister, 6 October 2011 8 © 2013, the LinkedTV project
  • 9. New tech: Second screen apps www.linkedtv.eu  Second screen content push while watching TV    Relates to the program as an atomic object Controlled by the content provider Restricted to programs who have „opted in“ IntoNow or Shazam identify what you are watching and provide associated content Second screen http://randymatheson.com/social-tv-platform-miso-launches-enhanced-experience-for-dexter-premiere/ apps show you related content without disturbing the TV view http://www.designbynotion.com/metamirror-next-generation-tv/ 9 © 2013, the LinkedTV project
  • 10. The LinkedTV future www.linkedtv.eu  There is a trend towards „interactive images“ on the Web  There is a trend towards „interactive video“ (or „hypervideo“) on the Web ThingLink can embed clickable images in Facebook WireWax is sharing clickable video on YouTube  We develop a new solution for interactive media in a TV-integrated environment: „LinkedTV“ „... a true TV ecosystem must functionally integrate the apps with the television programming service and content being offered to the consumer.“ TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013 10 © 2013, the LinkedTV project
  • 11. What makes LinkedTV special? www.linkedtv.eu  Build upon Web and broadcast specifications and standards   Make sure LinkedTV is part of the whole industry, not another proprietary fragment HTML5, Linked Data, W3C Media, hbbTV, Open IPTV Forum  Reuse the rich wealth of data and services on the open Web   Give added value at low cost Extendible to closed data and services for specialised offers  Automate the annotation and enrichment as far as possible   Broadcasters barrier to entry is the complexity and effort to make hypervideo Current approaches will never scale to the amount of video being produced 11 © 2013, the LinkedTV project
  • 12. Scenarios www.linkedtv.eu Scenario 1: Interactive News Show  Professional news content produced by RBB  Seed content: local news show "rbb Aktuell" Scenario 2: Hyperlinked Documentary  Cultural heritage content from S&V  Seed content: "Tussen Kunst & Kitsch" from AVRO (Antiques Roadshow format) Scenario 3: Media Arts  MEDIADROM competition for defining new ideas for linked television  Concepts will be promoted at Mons: European Capital of Culture 2015. 12 12 © 2013, the LinkedTV project
  • 13. User trials for LinkedTV usage insight www.linkedtv.eu Scenario 1: Scenario 2: Interactive News Show Hyperlinked Documentary  News is often watched together and  Tussen Kunst & over a meal! Kitsch viewers do  Viewers have more want to look up complex information information needs when watching  Objects and their news history of primary  Enriched news interest viewing could be  Bookmark for later, parallel to the main not during the show news, without additional interaction 13 © 2013, the LinkedTV project Also: industry insight on consumption trends  Second screen and social media integration are the key trends  News and documentary are the most interesting formats for enrichment  Additional info should be compact, bookmarkable 13
  • 14. Scenario for a hyperlinked documentary www.linkedtv.eu See the demo: http://www.linkedtv.eu/demos/hyperlinkeddocu Content courtesy 14 © 2013, the LinkedTV project
  • 15. Scenario for an interactive news show www.linkedtv.eu See the demo: http://www.linkedtv.eu/demos/linkednews Content courtesy 15 © 2013, the LinkedTV project
  • 16. How will it work? www.linkedtv.eu Video object and text detection Examples of Paintings by Jan Sluijters … and their digital images Connection to concepts Expansion of related concepts Linking to related content “...schilderij van Jan Sluijters....” dbpedia.org/resource/Jan_Sluyters 16 linkedtv.eu © 2013, the LinkedTV project Personalisation engine
  • 17. Concept Detection www.linkedtv.eu Video object and text detection LinkedTV works on services using & combining different forms of audiovisual content analysis: – Face detection – Textual analysis (subtitles, transcripts) – Keyword extraction – Multilingual automatic speech recognition – Speaker identification – Concept detection in video – Object re-detection – Shot segmentation This provides us with identification of “...schilderij van Jan Sluijters....” meaningful media fragments and summaries of their content analysis 17 linkedtv.eu © 2013, the LinkedTV project
  • 18. Concept Connections www.linkedtv.eu Connection to concepts Keywords and text fragments can be connected to uniquely identified, distinct named entities using online entity extraction services (NERD, http://nerd.eurecom.fr) – Media annotations are represented in RDF and stored in LinkedTV platform – Re-use of existing Web vocabularies e.g. schema.org, NERD, LSCOM, DBpedia – Named Entity Recognition uses statistical & knowledge-based approaches (aiming at entity meaning disambiguation) – Extended by complementary visual analyses (e.g. a persons name in text with a face detected in a video fragment) dbpedia.org/resource/Jan_Sluyters 18 linkedtv.eu © 2013, the LinkedTV project
  • 19. Concept Expansion www.linkedtv.eu Paintings by Jan Sluijters Expansion of related concepts 19 The use of Linked Data in the identification of concepts in LinkedTV means we can expand concepts along different facets, i.e. allow users to explore in terms of their different interests in a given concept. For example, for an artist like Jan Sluijters, LinkedTV can link into:  Additional information, e.g.  biography of artist Leo Gestel has art style  style of painting  Related information, e.g. luminism has art style has art style  artists from same Piet Mondriaan Jan Sluijters period  paintings in similar style  related styles linkedtv.eu © 2013, the LinkedTV project
  • 20. Linking to related content www.linkedtv.eu … and their digital images Linking to related content 20 LinkedTV provides enrichment services which provide recommendations for related online content (Web pages, images, audio, video) pertaining to the concepts in the TV program: – Fresh Social Web content coming from e.g. Twitter and Facebook – User Generated content coming from e.g. Flickr and YouTube – Whitelist content coming from partner Websites like public broadcasters in Germany or cultural heritage archives in the Netherlands – Extendable and configurable by source and media type linkedtv.eu © 2013, the LinkedTV project
  • 21. Personalisation engine www.linkedtv.eu Personalisation engine 21 The LinkedTV Platform pushes annotation and enrichment information to the client (LinkedTV Player) – Both annotations and enrichments include further metadata about the topics they relate to, e.g. Jan Sluijters + Amsterdam, BER Airport + Berlin – A Personalisation engine on the Player filters & ranks the annotations and enrichments according to a locally stored user model, e.g. interested in Amsterdam vs interested in Berlin – The user model can be both manually edited online (LUME) or learnt by the LinkedTV system (attention tracker, event tracking) – All personalisation is opt in and private, respecting ethical issues linkedtv.eu © 2013, the LinkedTV project
  • 22. Packaged into a LinkedTV Platform www.linkedtv.eu The LinkedTV Platform encapsulates the functionality of the previous 4 steps. The main core of the platform is an Application Service Bus which monitors and triggers events across the components, realising the end to end workflow from a video ingestion to enriched playout. Communication between platform components and between platform and client is realised using RESTful APIs. 22 © 2013, the LinkedTV project
  • 23. Playout on the LinkedTV Player www.linkedtv.eu LinkedTV enriched video playout needs a UI for Linked Television  The Springfield multiscreen platform abstracts the video and its enrichments from the device where they are displayed  Viewers can browse the topics of the program along „Who“, „Where“, „What“ axes  Enrichments can be browsed to, or shared, or bookmarked for later 23 © 2013, the LinkedTV project
  • 24. Future LinkedTV Player UI www.linkedtv.eu LinkedTV player UI will evolve with the Platform technology  Richer displays of associated information  HbbTV implementation  Gesture control 24 © 2013, the LinkedTV project
  • 25. Follow LinkedTV & keep up with the future of television Web: www.linkedtv.eu Twitter: @linkedtv 25 © 2013, the LinkedTV project www.linkedtv.eu
  • 26. www.linkedtv.eu 26 linkedtv.eu © 2013, the LinkedTV project