• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond
 

ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond

on

  • 2,962 views

ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond

ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond
Marc Spaniol, Ralf Klamma, Mathias Lux
TRIPLE-I, Graz, Austria, September 7, 2007

Statistics

Views

Total Views
2,962
Views on SlideShare
2,952
Embed Views
10

Actions

Likes
3
Downloads
1
Comments
0

2 Embeds 10

http://beamtenherrschaft.blogspot.com 9
http://eduspaces.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • ,

ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond Presentation Transcript

  • Ralf Klamma , Marc Spaniol Mathias Lux RWTH Aachen University University of Klagenfurt http://www.multimedia-metadata.info I-Media 2007 Graz, September 7, 2007 ImageSemantics : User-Generated Metadata, Content-Based Retrieval & Beyond
  • Agenda
    • Web 2.0 Image Mining
      • Flickr.com
      • Caliph & Emir
    • ImageSemantics
      • Challenges & Requirements
      • Architecture & Algorithms
      • Rule Representation
    • Conclusions and Outlooks
  • Flickr.com
    • Digital Image
    • Retrieval Results for Tag „gardenia“
    Tags: gardenia, green Very dífferent results
  • Caliph & Emir
    • Digital Image:
    • Retrieved images for low-level features
    Visualization of low-level features Not very similar
  • Web 2.0 Image Mining: ImageSemantics
    • Retrieved Images for Low-level Features and Tags
  • Problem Definition (Image Mining)
    • Challenges:
      • Image Search based on Human Semantics
      • Classification of Images
    • Strategies:
      • CBIR
      • Web 2.0 Tagging
    Multimedia semantics as rule-based system }
  • Requirements
    • Download & analyse training sets from Flickr.com
    • Rule extraction for building semantics
    • Semantic Web representation of rules
    • Rule execution in multimedia information system
  • State-of-the-Art
    • MPEG-7 as a multimedia language
      • Metadata
      • V isual descriptors
    • Image Data Mining / Machine Learning
      • K-Means Clustering
      • Hierarchical Clustering
    • Ontologies / Predicate logic
      • RDF
      • OWL
    • Database Support
      • XML database (eXist, IBM DB2 Viper, Oracle 10g/11g)
  • Rule Component Data Management Maschine Learning OWL Representation XML Database Reasoner Testing
  • MPEG-7 Feature Descriptors
    • Metadata / Image URI
    • Tags
    • Visual Descriptors
      • Edge Histogram
      • Scalable Color
      • Color Layout
  • Rule Extraction Algorithm Clustering for low-level features Training data Sub rule extraction for low-level features Clustering for tags in the low-level cluster Sub rule extraction for tags Complete rule extraction
  • OWL Rule Representation
    • Interval_A_B:
    • <owl:Class rdf:about=&quot;Interval_0_100&quot;>
    • <rdfs:subClassOf rdf:resource=&quot;Interval_A_B&quot;/>
    • <Hasminvalue>0<Hasminvalue>
    • <Hasmaxvalue>100</ Hasmaxvalue>
    • </owl:Class>
    • <owl:DatatypeProperty rdf:about=&quot;Hasminvalue&quot;>
    • <rdfs:domain rdf:resource=&quot;Interval_A_B&quot;/>
    • <rdfs:range rdf:resource=&quot;http://www.w3.org/2001/XMLSchema#double&quot;/> </owl:DatatypeProperty>
    •   <owl:DatatypeProperty rdf:about=&quot;Hasmaxvalue&quot;>
    • <rdfs:domain rdf:resource=&quot;Interval_A_B&quot;/>
    • <rdfs:range rdf:resource=&quot;http://www.w3.org/2001/XMLSchema#double&quot;/>
    • </owl:DatatypeProperty>
    0 100
  • OWL Representation of Rules
    • Sub Rule for low-level features:
    • <owl:Class rdf:about=&quot;LowlevelCluster_A&quot;>
    • <Centriod> Values</Centriod>
    • <rdfs:subClassOf>
    • <owl:Restriction>
    • <owl:onProperty>
    • <owl:ObjectProperty rdf:about=&quot;Distance&quot;/>
    • </owl:onProperty>
    • <owl:allValuesFrom>
    • <owl:Class rdf:about=&quot;Interval_A_B&quot;/>
    • </owl:allValuesFrom>
    • </owl:Restriction>
    • </rdfs:subClassOf>
    centroid Low-level Cluster max. Distance min. Distance
  • OWL Representation of Rules
    • Complete rule
    • <owl:Class rdf:about=&quot;Lowlevel_A_TagContent_Cluster&quot;>
    • <Hastag>Tagcontent</Hastag>
    • <rdfs:subClassOf rdf:resource=&quot;LowlevelCluster_A&quot;/>
    • </owl:Class> 
    • <owl:DatatypeProperty rdf:about=&quot;Hastag&quot;>
    • <rdfs:range rdf:resource=&quot;http://www.w3.org/2001/XMLSchema#string&quot;/>
    • <rdfs:domain rdf:resource=&quot; Lowlevel_A_Tag_Cluster&quot;/> </owl:DatatypeProperty>
    • OWL dataset representation
    • <LowlevelCluster_A rdf:about=&quot;260407965_5c177d3703.mp7.xml&quot;>
    • <rdf:type rdf:resource=&quot;Lowlevel_A_Tag1_Cluster &quot;/>
    • <rdf:type rdf:resource=&quot;Lowlevel_A_Tag2_Cluster&quot;/> </LowlevelCluster_A>
    Lowlevel_A_TagContent_Cluster
  • Representation of Rules in First Order Logic
    • Sub rule for low-level features
    • Centroid(F), maxdistance(G)  class(LowlevelCluster_A), with F cluster center and G distance value.
    • Complete rules
    • Tag(Tag1)  class(LowlevelCluster_A) (tag  low-level features: if image is tagged with Tag1, image is probably in cluster LowlevelCluster_A). 
    • class(LowlevelCluster_A)  Tag(Tag1) (low-level features  tag: if image is in cluster LowlevelCluster_A liegt, image is probably tagged with Tag1).
  • Conclusions & Outlook
    • Search & Retrieval of Images by integration of
      • Web 2.0 Tagging (High-level Semantics)
      • CBIR (Low-level Semantics)
    • Rule-based representation of Image Mining Results
      • OWL Syntax
      • Semantic Web Reasoner
    • Outlook:
      • Integration in Multimedia Information System „Virtual Campfire“
      • Integration in Context-aware Mining Suite