Your SlideShare is downloading. ×
VERIFICATION_&_VALIDATION_OF_A_SEMANTIC_IMAGE_TAGGING_FRAMEWORK_VIA_GENERATION_OF GEOSPATIAL_IMAGERY_GROUND_TRUTH.pptx
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

VERIFICATION_&_VALIDATION_OF_A_SEMANTIC_IMAGE_TAGGING_FRAMEWORK_VIA_GENERATION_OF GEOSPATIAL_IMAGERY_GROUND_TRUTH.pptx

196

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
196
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Verification & Validation of a Semantic Image Tagging Framework via Generation of Geospatial Imagery Ground Truth
    Shaun S. Gleason1, Mesfin Dema2, Hamed Sari-Sarraf2, Anil Cheriyadat1, Raju Vatsavai1, Regina Ferrell1
     
    1Oak Ridge National Laboratory, Oak Ridge, TN
    2Texas Tech University, Lubbock, TX
    1
  • 2. Contents
    Motivation
    Existing Approaches
    Proposed Approach
    Generative Model Formulation
    General Framework
    Preliminary Results
    Conclusions
    2
  • 3. Motivation
    Automated identification of complex facilities in aerial imagery is an important and challenging problem.
    For our application, nuclear proliferation, facilities of interest can be complex.
    Such facilities are characterized by:
    the presence of known structures,
    their spatial arrangement,
    their geographic location,
    and their location relative to natural resources.
    Development, verification, and validation of semantic classification algorithms for such facilities is hampered by the lack of available sample imagery with ground truth.
    3
  • 4. Semantics:
    Set of objects like:
    Switch yard,
    Containment
    Building,
    Turbine
    Generator,
    Cooling
    Towers
    AND
    Their spatial
    arrangement
    => may
    imply a semantic
    label like “nuclear
    power plant”
    Switch
    Yard
    Turbine
    Building
    Cooling
    Towers
    Containment
    Building
    Motivation (cont.)
  • 5. Motivation (cont.)
    5
    Many algorithms are being developed to extract and classify regions of interest from images, such as in [1].
    V & V of the algorithms have not kept pace with their development due to lack of image datasets with high accuracy ground truth annotations.
    The community needs research techniques that can provide images with accurate ground truth annotation at a low cost.
    [1] Gleason SS, et al., “Semantic Information Extraction from Multispectral Geospatial Imagery via a Flexible Framework,” IGARSS, 2010.
  • 6. Existing Approaches
    Manual ground truth annotation of images
    Very tedious for volumes of images
    Highly subjective
    Using synthetic images with corresponding ground truth data
    Digital Imaging and Remote Sensing Image Generation
    ( DIRSIG) [2]
    Capable of generating hyper-spectral images in range of 0.4-20 microns.
    Capable of generating accurate ground truth data.
    Very tedious 3D scene construction stage.
    Incapable of producing training images in sufficient quantities.
    6
    [2] Digital Imaging and Remote Sensing Image Generation (DIRSIG): http://www.dirsig.org/.
  • 7. Existing Approaches
    In [3,4], researchers attempted to partially automate the cumbersome 3D scene construction of the DIRSIG model.
    LIDAR sensor is used to extract 3D objects from a given location.
    Other modalities are used to correctly identify object types.
    3D CAD models of objects and object locations are extracted.
    Extracted CAD models are placed at their respective position to reconstruct the 3D scene and finally to generate synthetic image with corresponding ground truth.
    Availability of 3D model databases, such as Google SketchUp [5], reduces the need for approaches like [3,4].
    7
    [3] S.R. Lach, et al., “Semi-automated DIRSIG Scene Modeling from 3D LIDAR and Passive Imaging Sources”, in Proc. SPIE Laser Radar Technology and Applications XI, vol. 6214,2006.
    [4] P. Gurram, et al., “3D scene reconstruction through a fusion of passive video and Lidar imagery,” in Proc. 36th AIPR Workshop, pp. 133–138, 2007.
    [5] Google SketchUp: http://sketchup.google.com/.
  • 8. Proposed Approach
    To generate synthetic images with ground truth annotation at low cost, we need a system which can learn from few training examples.
    This system must be generative so that one can sample a plausible scene from the model.
    The system must also be capable of producing synthetic images with corresponding ground truth data in sufficient quantity.
    Our contribution to the problem is two-fold.
    We incorporated expert knowledge into the problem with less effort.
    We adapted a generative model to synthetic image generation process.
    8
  • 9. 9
    Knowledge Representation: And-Or Graph
    Nuclear Power Plant
    Reactor
    Turbine Building
    Building
    Switchyard
    Cooling Tower
    CT Type1
    CT Type 2
    [6] S.C. Zhu. and D. Mumford,” A Stochastic Grammar of Images”. Foundation and Trends in Computer Graphics
    and Vision, 2(4): pp .259–362, 2006
  • 10. Generative Model Formulation: Maximum Entropy Principle(MEP)
    Given observed constraints (i.e. the hierarchical and contextual information) of an unobserved distribution f , a probability distribution p which best approximates f is the one with maximum entropy [7,8].
    10
    [7] J. Porway ,et al. “ Learning compositional Models for Object Categories From Small Sample Sets”, 2009
    [8] J. Porway ,et al. “ A Hierarchical and Contextual Model for Aerial Image Parsing”, 2010
  • 11. Gibbs Distribution
    Parameter Learning
    11
    Generative Model Formulation:Optimization of MEP
  • 12. General Framework
    12
  • 13. General Framework
    13
    [5]
    [9]
    [5] Google SketchUp: http://sketchup.google.com/.
    [9] Persistence of Vision Raytracer (POV-Ray): http://www.povray.org/.
  • 14. Preliminary Results
    We are currently working with experts on annotating training images of nuclear power plant sites.
    To demonstrate the idea of the proposed approach, we have used a simple example as a proof-of-principle.
    Using this example, we illustrate how the generative framework can sample plausible scenes, and finally generate synthetic images with corresponding ground truth annotation.
    14
  • 15. Proof-of-Principle:Training Images
    15
  • 16. 16
    Proof-of-Principle:Manually Annotated Training Images
  • 17. 17
    Proof-of-Principle:Manually Annotated Training Images
    Orientation Corrected Images
  • 18. 18
    Proof-of-Principle:Manually Annotated Training Images
    Orientation Corrected Images Followed by Ellipse Fitting
  • 19. 19
    Relationships
  • 20. Synthesized Images
    20
    Before Learning
    After Learning
  • 21. Synthesized Images
    21
    Before Learning
    After Learning
  • 22. 22
    Synthesized Images
    After Learning
    Synthesized Image
  • 23. 23
    Synthesized Images
    Part level ground truth image
    Object level ground truth image
  • 24. Manually Created Example
    24
    3D Google Sketch-Up model of a nuclear plant: Pickering Nuclear Plant, Canada (left), and model manually overlaid on an image (right).
  • 25. Conclusions
    Maximum Entropy model has proven to be an elegant framework to learn patterns from training data and generate synthetic samples having similar patterns.
    Using the proposed framework, generating synthetic images with accurate ground truth annotation comes at relatively low cost.
    The proposed approach is very promising for algorithm verification and validation.
    25
  • 26. Challenges Ahead
    The current model generates some results that do not represent a well-learned configuration of objects.
    We believe that constraint representation using histograms contributes to invalid results, since some values are averaged out while generating histograms.
    To avoid invalid results, we are currently studying a divide-and-conquer strategy by introducing on-the-fly clustering approaches. This separates the bad samples from the good ones, which helps tune the parameters during the learning phase.
    26
  • 27. Acknowledgements
    Funding for this work is provided by the Simulations, Algorithms, and Modeling program within the NA-22 office of the National Nuclear Security Administration, U.S. Department of Energy.
    27
  • 28. Thank You!
    28

×