• Save
Provenance for Multimedia
Upcoming SlideShare
Loading in...5
×
 

Provenance for Multimedia

on

  • 2,309 views

Provenance for Multimedia: COMM, D&S, M3O. Talk given for the W3C Provenance Incubator Group, May 28th 2010.

Provenance for Multimedia: COMM, D&S, M3O. Talk given for the W3C Provenance Incubator Group, May 28th 2010.

Statistics

Views

Total Views
2,309
Views on SlideShare
2,254
Embed Views
55

Actions

Likes
3
Downloads
0
Comments
0

4 Embeds 55

http://www.slideshare.net 46
http://scireadtrascendere.blogspot.com 4
http://scireadtrascendere.blogspot.mx 3
http://luishectorhuamani.blogspot.com 2

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Provenance for Multimedia Provenance for Multimedia Presentation Transcript

  • Provenance for Multimedia: COMM, D&S, M3O Raphaël Troncy <raphael.troncy@eurecom.fr> Multimedia Semantics, EURECOM (FR)
  • Common Scenario: Tagging Approach Reg1 The "Big Three" at the Yalta Conference (Wikipedia)  Localize a region  Draw a bounding box, a circle around a shape  Annotate the content  Interpret the content  Tag: Winston Churchill, UK Prime Minister, Allied Forces, WWII 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -2
  • Common Scenario: SW Approach Reg1 The "Big Three" at the Yalta Conference (Wikipedia)  Localize a region  Draw a bounding box, a circle around a shape  Annotate the content  Interpret the content  Link to knowledge on the Web :Reg1 foaf:depicts dbpedia:Winston_Churchill dbpedia:Winston_Churchill skos:altLabel "Sir Winston Leonard Spencer-Churchill" dbpedia:Winston_Churchill rdf:type foaf:Person 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -3
  • Research Problem Seq4 Reg1 Seq1 The "Big Three" at the Yalta Conference (Wikipedia) A history of G8 violence (video) (© Reuters)  Multimedia objects are complex  MPEG-7  Compound information objects, fragment identification  Semantic annotation  Subjective interpretation, context dependent  D&S | OIO  Linked data principle  Open to reuse existing knowledge  RDF 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -4
  • 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -5
  • COMM: Design Rationale  Approach:  NO 1-to-1 translation from MPEG-7 to OWL/RDF  Need for patterns: use DOLCE, a well designed foundational ontology as a modeling basis  Design patterns:  Ontology of Information Objects (OIO) Formalization of information exchange Multimedia = complex compound information objects  Descriptions and Situations (D&S) Formalization of context Multimedia = contextual interpretation (situation)  Define multimedia patterns that translate MPEG-7 in the DOLCE vocabulary 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -6
  • COMM: Core Functionalities  Most important MPEG-7 functionalities:  Decomposition of multimedia content into segments  Annotation of segments with metadata Administrative metadata: creation & production Content-based metadata: audio/visual descriptors Semantic metadata: interface with domain specific ontologies  Note that all are subjective and context dependent situations 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -7
  • COMM: D&S / OIO Patterns 3 2 1 Definition of design patterns for decomposition and annotation based on D&S and OIO 1. MPEG-7 describes digital data (multimedia information objects) with digital data (annotation) 2. Digital data entities are information objects 3. Decompositions and annotations are situations that satisfy the rules of a method or algorithm 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -8
  • COMM: Decomposition Pattern MPEG- MPEG-7 7 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -9
  • COMM: Semantic Pattern Domain Ontologies 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 10
  • COMM Example: Region Annotation http://en.wikipedia.org/wiki/ Image:Yalta_Conference.jpg dns:realized-by dns:setting core:semantic- core:image-data annotation dns:plays dns:defines foaf:Person loc:region- loc:spatial-mask- core:semantic-label- locator-descriptor role role dns:played-by rdf:type dns:defines dns:played-by http://en.wikipedia.org/wiki/ loc:bounding-box 5 25 10 20 15 15 10 10 5 15"^^xsd:string Churchill data:has-rectangle 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 11
  • Provenance of Annotation: Naïve Approach dbpedia:Winston_Churchill dc:creator foaf:depicts :me :Reg1 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 12
  • Provenance of Annotation: COMM / M3O ex:annotation-pattern defines ex:annotated-concept ex:annotation-concept classifies classifies satisfies ex:big-3-yalta dbpedia:Winston_Churchillin hasSetting ex:annotation-situation 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 13
  • Provenance of Annotation: COMM / M3O ex:annotation-pattern defines ex:annotated-concept ex:annotation-concept ex:applied-method-role dc:creator classifies classifies satisfies classifies classifies ex:big-3-yalta dbpedia:Winston_Churchillin ex:manual-annotation :me hasSetting ex:annotation-situation http://m3o.semantic-multimedia.org/ontology/2010/02/28/provenance.owl 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 14
  •  Low-level features extraction  Classifier: SVM  Input parameters  Threshold 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 15
  • Summary: Provenance in multimedia semantics  Direct approach:  Reification: what is the semantics of what is being reified?  Named Graph: how to represent the parameters used for detecting a face in an image?  Provenance vocabularies:  OPM in RDF [Moreau et al.], http://openprovenance.org/model/opm.owl  Provenir Vocab [Sahoo et al.], http://knoesis.wright.edu/provenir/provenir.owl  Provenance Vocab [Hartig and Zhao], http://purl.org/net/provenance/  prv:createdBy, prv:performedAt, prv:performedBy, etc.  D&S pattern:  Formalized description of a particular context (situation)  Specialized for multimedia in COMM and M3O 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 16