• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Image semantic coding using OTB
 

Image semantic coding using OTB

on

  • 976 views

Image semantic coding using OTB

Image semantic coding using OTB
Marie Liénou; TELECOM ParisTech
Marine Campedel; TELECOM ParisTech

Statistics

Views

Total Views
976
Views on SlideShare
769
Embed Views
207

Actions

Likes
0
Downloads
9
Comments
0

4 Embeds 207

http://www.orfeo-toolbox.org 168
http://orfeo-toolbox.org 35
http://translate.googleusercontent.com 3
http://www.slideshare.net 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Image semantic coding using OTB Image semantic coding using OTB Presentation Transcript

    • Competence Centre on Information Extraction and Image Understanding for Earth Observation July 2009 using OTB Télécom ParisTech Marie Liénou - Marine Campedel Image Semantic Coding 1
    • and Image Understanding for Earth Observation SUMMARY Competence Centre on Information Extraction COC Notion of semantic Coding A promising approach Semantic Coding OTB tool Development of Conclusion and an OTB tool perspectives 2
    • and Image Understanding for Earth Observation COC = COmpetence Centre… Competence Centre on Information Extraction  Tripartite agreement between CNES – DLR and Télécom ParisTech  Signed in June 2005  Goal : joint action on image understanding  SAR/Optical, HR and VHR, temporal series  Feature extraction, modeling, indexing, compression, (interactive) classification, interpretation, knowledge representation, reasoning, …  Means  ~ 4 new phds / year  ~10 permanent researchers partially involved  financial support for specific actions (studentships, engineers, post-docs) 3
    • and Image Understanding for Earth Observation Image Semantic coding Competence Centre on Information Extraction Semantic Coding Meaning Compression Understanding Reduce data while Interpretation ensuring informational Image to text? content [Barnard et al., 2003 ; Jeon et al., 2003] [Li et Bretschneider, 2006] Goal: find an image representation able to capture the contained semantics Idea: use text indexing approach + active learning 4
    • and Image Understanding for Earth Observation Image Semantic coding Visual interaction Competence Centre on Information Extraction Manual annotation Feature extraction Where is Quantization semantics? Automatic annotation « visual words » Active learning Indexing Mining 5
    • and Image Understanding for Earth Observation Image Semantic coding vs KIM Competence Centre on Information Extraction « Design and evaluation of HMC for Image Information Mining » Daschiel and Datcu IEEE transaction on multimedia, vol 7, no6, dec. 2005 6
    • and Image Understanding for Earth Observation A promising approach Competence Centre on Information Extraction  Feature extraction  Segmentation, arbitrar regions  “Classical” signature: color, texture, shape descriptors  Experiments: intensity mean and variance in each spectral band  Quantization  K-Means: each estimated cluster corresponds to one “visual word”  K estimated using MDL (Minimum Description Length) descriptor  Bag-of-words signature for semantics identification  Count visual words on image regions which will be annotated  Normalize (tf-idf)  Exploitation using machine learning (SVM, LDA) 7
    • and Image Understanding for Earth Observation A promising approach Competence Centre on Information Extraction  Marie Lienou PhD work (march 2009)  Tested on several VHR (multispectral) images  Compared to other classification approachs (GMM, SVM) Visual word production Feature Classification Quantization Count words extraction SVM, LDA Annotations Feature Classification Majority rule extraction GMM, SVM Low level annotations  Recognition accuracy demonstrated for “semantically complex” classes Ex: “urban area”  LDA = fast + does not need negative examples 8
    • and Image Understanding for Earth Observation OTB tool: cocSemanticCoding Competence Centre on Information Extraction  Feature extraction  Vectorial image with as many components as feature dimension  Exploitation of OTB extractors at each pixel  Quantization  Use of K-Means filter  Bag-of-words signature  Count visual words on image regions which will be annotated  Normalization (tf-idf)  Learning from manual annotation  Fluid interface facilities  Learn LDA from only target samples  Learn SVM from target samples and counter examples  Classify the whole image  Iterate 9
    • Competence Centre on Information Extraction and Image Understanding for Earth Observation OTB tool: cocSemanticCoding 10
    • Competence Centre on Information Extraction and Image Understanding for Earth Observation OTB tool: cocSemanticCoding 11
    • Competence Centre on Information Extraction and Image Understanding for Earth Observation OTB tool: cocSemanticCoding 12
    • Competence Centre on Information Extraction and Image Understanding for Earth Observation 13
    • Competence Centre on Information Extraction and Image Understanding for Earth Observation 14
    • Competence Centre on Information Extraction and Image Understanding for Earth Observation 15
    • and Image Understanding for Earth Observation Competence Centre on Information Extraction Learning and classification tools LDA on occurrence data SVM on TFiDF data (features) Both results can be obtained with same labeling for comparison Difficulty for the user : compute features adapted to the underlying semantics 16
    • and Image Understanding for Earth Observation Conclusion Competence Centre on Information Extraction  OTB useful features  Vectorial image representation  Great diversity of available filters (extractors, classifiers)  New = LDA classifier + estimator  Visualization tools  cocSemanticCoding tool availability  www.tsi.enst.fr/~campedel/  will be updated  Necessity to valorize research results  Engineering process (C++ programming)  Not easy but OTB is a nice initiative to help researchers  In the future: centralize processing tools (in OTB) + easy their exploitation (documentations, interfaces) 17
    • and Image Understanding for Earth Observation Perspectives Competence Centre on Information Extraction  Other COC tools should be integrated in cocSemanticCoding  MDL to estimate of visual words number  new feature extractors (QMF-based texture descriptors)  Feature selection  Complete relevance feedback framework  New approaches for image interpretation  From semantics to knowledge?  Knowledge engineering: modeling (ontologies) + reasoning  Several works on characterizing relations between identified concepts and/or image objects 18