Your SlideShare is downloading. ×
Survey of Semantic Media Annotation Tools - towards New Media Applications with Linked Media
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Survey of Semantic Media Annotation Tools - towards New Media Applications with Linked Media

2,294
views

Published on

Semantic annotation of media resources has been a focus in research since many years, the closing of the "semantic gap" being seen as key to signi cant improvements in media retrieval and browsing …

Semantic annotation of media resources has been a focus in research since many years, the closing of the "semantic gap" being seen as key to signi cant improvements in media retrieval and browsing and
enabling new media applications and services. However, current tools and services exhibit varied approaches which do not easily integrate and
act as a barrier to wider uptake of semantic annotation of online multimedia.
In this paper, we outline the Linked Media principles which can help form a consensus on media annotation approaches, survey current media annotation tools against these principles and present two emerging
toolsets which can support Linked Media conformant annotation, closing with a call to future semantic media annotation tools and services to follow the same principles and ensure the growth of a Linked Media
layer of semantic descriptions of online media which can be an enabler to richer future online media services.

Published in: Technology, Education

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,294
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
24
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Television Linked To The Web www.linkedtv.eu Lyndon Nixon, MODUL University Vienna GmbH Raphael Troncy, EURECOM Survey of Semantic Media Annotation Tools for the Web: towards New Media Applications with Linked Media 25 May 2014 LiME Workshop ESWC2014
  • 2. 2 www.linkedtv.eu Overview Why Linked Media? What is Linked Media? Requirements on online media annotation Survey of existing tools LinkedTV services and applications for Linked Media Summary and Outlook Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 3. 3 www.linkedtv.eu Why Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media „It is growing at more than 20% per annum, fuelled by increased demands for new programming and the huge saving it represents compared with shooting new footage. Interactive technology and the Internet will further contribute to the growth of the market as it makes stock footage cheaper and easier to locate and license.“ - http://moneyam.uk-wire.com/cgi-bin/articles/200201020827103514P.html
  • 4. 4 www.linkedtv.eu Why Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Dublin Core Metadata Initiative (DCMI) Metadata Encoding and Transmission Standard (METS)
  • 5. 5 www.linkedtv.eu Why Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 6. 6 www.linkedtv.eu What is Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media • What is being described? • The media object – what is it? Video uploaded to YouTube 2011 ZDF documentary broadcast 1980 Rembrandt self-portrait from 1639 Date? Is Rembrandt a subject of the whole video? Or only a specific temporal segment of the video? Is the whole frame important or only the part with Rembrandt‘s face?
  • 7. 7 www.linkedtv.eu What is Linked Media? Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Media fragment players: http://ninsuna.elis.ugent.be/MFPlayer/html5 http://linkedtv.eurecom.fr/mediafragmentplayer Emerging support in Web browsers: Firefox, Safari, Chrome
  • 8. 8 www.linkedtv.eu What is Linked Media? The object of annotation • What is the description of the media? "Rembrandt" "Rembrandt self- portrait" from 1639 Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 9. 9 www.linkedtv.eu What is Linked Media? Digital identifiers for non- addressable resources • “URIs for concepts” since the Semantic Web • Linked Data has popularized the notion Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 10. 10 www.linkedtv.eu What is Linked Media? 1. Web media descriptions need a common representation of media structure 2. Web media descriptions need a common representation of media content 3. Web media descriptions need to use a media ontology which supports description of both the structure and content of media 4. The descriptions of media in terms of common representations of structure and content are the basis for deriving links across media on the Web (Linked Media) Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 11. 11 www.linkedtv.eu Requirements on online media annotation Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media “Just as Web browsers could never have been successful without a core consensus on the Web page mark up (HTML), Web media can not fulfil its potential without a core consensus on the available descriptive metadata.” (1) tool or service is online and allows to annotate media of different types via URL (2) media can be annotated in terms of its parts and the media fragmentation follows a non-proprietary approach supported by other tools (3) media can be annotated by concepts where a shared understanding is possible across systems (4) media ontology used for the generated descriptions is interoperable across systems
  • 12. 12 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media EuropeanaConnect Media Annotation Suite (http://dme.arcs.ac.at/annotation/) "Developed in the EuropeanaConnect project, it supports image,audio and video. Both DBPedia and Geonames resources can be annotation targets, and are suggested from free text or location references respectively. Annotations can be exported as RDF using a tool- specific vocabulary."
  • 13. 13 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media ImageSnippets (http://www.imagesnippets.com/) Open Video Annotation (http://www.openvideoannotation.o rg/)
  • 14. 14 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Annotorius (http://annotorious.github.io/) Pundit and SemTube (http://thepund.it and http://www.semedia.dibet.univpm.it/semtube/)
  • 15. 15 www.linkedtv.eu Survey: overview of annotation tools Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Web based; open videos from online or the repository; term suggestion; export annotations as file or to repository. ConnectME annotation tool https://github.com/lyndonnixon/ annotationtool Contact lyndon.nixon@modul.ac.at GNU GPL 3.0 license
  • 16. 16 www.linkedtv.eu LinkedTV: background Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media 40% of TV viewers are using a companion device alongside the TV program.* * J. Abreu, P. Almeida, B. Teles, and M. Reis. Viewer behaviors and practices in the (new) television environment. In Proceedings of the 11th European Conference on Interactive TV and Video, EuroITV '13. http://www.linkedtv.eu Ever saw something on TV and wanted to know more about it, but didn‘t even know how to search for it?
  • 17. 17 www.linkedtv.eu LinkedTV technology Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media “...schilderij van Jan Sluijters....” Video object and word detection dbpedia.org/resource/Jan_Sluyters Connection to concepts Paintings by Jan Sluijters Selection of related concepts Selection of related content Presentation engine
  • 18. 18 www.linkedtv.eu Annotation task: automatic approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media http://nerd.eurecom.fr
  • 19. 19 www.linkedtv.eu Media Fragment + Open Annotation + NERD 08.11. 2012 Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 20. 20 www.linkedtv.eu Annotation task: automatic approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
  • 21. 21 www.linkedtv.eu Annotation task: automatic approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media http://linkedtv.eurecom.fr/tv2rdf/
  • 22. 22 www.linkedtv.eu Annotation task: manual approaches Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media The LinkedTV Editor Tool will have a public code release in late 2014 (planned GNU GPL 3.0), but a test installation can be used at http://editortool.linkedtv.eu (Enquiries: jblom@beeldengeluid.nl)
  • 23. 23 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Edward Snowden asks for asylum in Russia (04 / 07 / 2013)
  • 24. 24 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media In which Russian airport is he exactly?  LSCOM:Face  LSCOM:Building ?
  • 25. 25 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media List of Relevant Named Entities (1) Named Entity (2) Filtering and Ranking b) Expanded Entities b) Re-ranked Entities a) Entities from Video
  • 26. 26 www.linkedtv.eu Enrichment task: associating media Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media  http://linkedtv.project.cwi.nl/news/ mainscreen.html  http://linkedtv.project.cwi.nl/news/ Second Screen @ Main Screen @
  • 27. 27 www.linkedtv.eu Outlook Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media Tool Input Fragment support Conceptual annotation Ontology Europeana Connect Audio, image or video. By URL Spatial & temporal (SVG) DBPedia and GeoNames Own vocabulary ImageSnippets Image only. By URL Spatial. Free to import from LOD or create own LIO ontology OpenVideo Annotation Video. Unclear if support URL Only temporal in the demo, unknown syntax Free text or tags Open Annotation Model Annotorius Image only. By URL. Spatial, proprietary syntax DBPedia No ontology (JS data model) Pundit and SemTube Image and video. By URL Spatial, proprietary syntax Freebase. Terms can be imported. Own RDF vocabulary ConnectME Video. By URL Spatial & temporal using MF URIs DBPedia and Geonames ConnectME Ontology LinkedTV TV2RDF and Editor Tool Video. By URL (Editor Tool uses resources on the LinkedTV platform) Spatial & temporal using MF URIs DBPedia. Editor tool suggests DBPedia but allows any URI. LinkedTV Ontology