Provenance for Multimedia

  • 1,509 views
Uploaded on

Provenance for Multimedia: COMM, D&S, M3O. Talk given for the W3C Provenance Incubator Group, May 28th 2010.

Provenance for Multimedia: COMM, D&S, M3O. Talk given for the W3C Provenance Incubator Group, May 28th 2010.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,509
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Provenance for Multimedia: COMM, D&S, M3O Raphaël Troncy <raphael.troncy@eurecom.fr> Multimedia Semantics, EURECOM (FR)
  • 2. Common Scenario: Tagging Approach Reg1 The "Big Three" at the Yalta Conference (Wikipedia)  Localize a region  Draw a bounding box, a circle around a shape  Annotate the content  Interpret the content  Tag: Winston Churchill, UK Prime Minister, Allied Forces, WWII 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -2
  • 3. Common Scenario: SW Approach Reg1 The "Big Three" at the Yalta Conference (Wikipedia)  Localize a region  Draw a bounding box, a circle around a shape  Annotate the content  Interpret the content  Link to knowledge on the Web :Reg1 foaf:depicts dbpedia:Winston_Churchill dbpedia:Winston_Churchill skos:altLabel "Sir Winston Leonard Spencer-Churchill" dbpedia:Winston_Churchill rdf:type foaf:Person 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -3
  • 4. Research Problem Seq4 Reg1 Seq1 The "Big Three" at the Yalta Conference (Wikipedia) A history of G8 violence (video) (© Reuters)  Multimedia objects are complex  MPEG-7  Compound information objects, fragment identification  Semantic annotation  Subjective interpretation, context dependent  D&S | OIO  Linked data principle  Open to reuse existing knowledge  RDF 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -4
  • 5. 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -5
  • 6. COMM: Design Rationale  Approach:  NO 1-to-1 translation from MPEG-7 to OWL/RDF  Need for patterns: use DOLCE, a well designed foundational ontology as a modeling basis  Design patterns:  Ontology of Information Objects (OIO) Formalization of information exchange Multimedia = complex compound information objects  Descriptions and Situations (D&S) Formalization of context Multimedia = contextual interpretation (situation)  Define multimedia patterns that translate MPEG-7 in the DOLCE vocabulary 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -6
  • 7. COMM: Core Functionalities  Most important MPEG-7 functionalities:  Decomposition of multimedia content into segments  Annotation of segments with metadata Administrative metadata: creation & production Content-based metadata: audio/visual descriptors Semantic metadata: interface with domain specific ontologies  Note that all are subjective and context dependent situations 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -7
  • 8. COMM: D&S / OIO Patterns 3 2 1 Definition of design patterns for decomposition and annotation based on D&S and OIO 1. MPEG-7 describes digital data (multimedia information objects) with digital data (annotation) 2. Digital data entities are information objects 3. Decompositions and annotations are situations that satisfy the rules of a method or algorithm 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -8
  • 9. COMM: Decomposition Pattern MPEG- MPEG-7 7 28/05/2010 - Provenance for Multimedia - W3C Provenance XG -9
  • 10. COMM: Semantic Pattern Domain Ontologies 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 10
  • 11. COMM Example: Region Annotation http://en.wikipedia.org/wiki/ Image:Yalta_Conference.jpg dns:realized-by dns:setting core:semantic- core:image-data annotation dns:plays dns:defines foaf:Person loc:region- loc:spatial-mask- core:semantic-label- locator-descriptor role role dns:played-by rdf:type dns:defines dns:played-by http://en.wikipedia.org/wiki/ loc:bounding-box 5 25 10 20 15 15 10 10 5 15"^^xsd:string Churchill data:has-rectangle 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 11
  • 12. Provenance of Annotation: Naïve Approach dbpedia:Winston_Churchill dc:creator foaf:depicts :me :Reg1 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 12
  • 13. Provenance of Annotation: COMM / M3O ex:annotation-pattern defines ex:annotated-concept ex:annotation-concept classifies classifies satisfies ex:big-3-yalta dbpedia:Winston_Churchillin hasSetting ex:annotation-situation 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 13
  • 14. Provenance of Annotation: COMM / M3O ex:annotation-pattern defines ex:annotated-concept ex:annotation-concept ex:applied-method-role dc:creator classifies classifies satisfies classifies classifies ex:big-3-yalta dbpedia:Winston_Churchillin ex:manual-annotation :me hasSetting ex:annotation-situation http://m3o.semantic-multimedia.org/ontology/2010/02/28/provenance.owl 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 14
  • 15.  Low-level features extraction  Classifier: SVM  Input parameters  Threshold 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 15
  • 16. Summary: Provenance in multimedia semantics  Direct approach:  Reification: what is the semantics of what is being reified?  Named Graph: how to represent the parameters used for detecting a face in an image?  Provenance vocabularies:  OPM in RDF [Moreau et al.], http://openprovenance.org/model/opm.owl  Provenir Vocab [Sahoo et al.], http://knoesis.wright.edu/provenir/provenir.owl  Provenance Vocab [Hartig and Zhao], http://purl.org/net/provenance/  prv:createdBy, prv:performedAt, prv:performedBy, etc.  D&S pattern:  Formalized description of a particular context (situation)  Specialized for multimedia in COMM and M3O 28/05/2010 - Provenance for Multimedia - W3C Provenance XG - 16