Fcv taxo chellappa
Upcoming SlideShare
Loading in...5

Fcv taxo chellappa






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Assumption1: can identify flat pixels in full shade and full sun. This is not a bad assumption since there are many ubiquitous materials such as asphalt and concrete have flat spectra Note: will not consider cos(rho) for now, since this is only a constant and not affect shape of spectra. It will be taken care by normalization at the end F is the fraction of sky unblocked

Fcv taxo chellappa Fcv taxo chellappa Presentation Transcript

  • Computer vision for non-visual spectral regimes and non-traditional applications Rama Chellappa UMD
  • Opening remarks – non-visual sensors
    • Sensors in visual regimes are only a small % of the possible sensors
    • Infrared, hyperspectral, LADAR, SAR, FOPEN SAR, mmWave, polarimetric
    • Provide thermal signatures, material properties, 3D, all weather/all day coverage, looking behind trees, walls, …
    • Sensor physics is important.
    • Low signal to clutter ratio.
    • Characterization of image statistics driven by sensor physics
  • Problems addressed using hyperspectral images
    • Sensor design
    • Material classification/segmentation
    • Anomaly detection
      • More than two decades of work
        • ARO MURI (2002-2007) on the science of land-based target signatures.
        • DARPA DDB program, NGA
        • Glenn Healey: pioneer in HSI and computer vision
        • Larry Wolf, Equinox Corporation
    • Unmixing of pixels (Remote sensing)
    • Compressive sensing possibilities (SPC)
    • Numerous books, papers, conferences…
  • Estimating object reflectance
    • Radio transfer function
      • irradiance to the camera
      • reflectance at location (x,y)
      • main light source (e.g. sun light)
      • ambient light source coming from all directions, assuming constant intensity for all directions
      • fraction of unblocked sky from (x,y) view
    • geometry factor
    Nguyen and Chellappa, CVPR 2010 Workshop on Beyond visible spectrum
  • Tracking radiance
  • Reflectance tracking
    • Robust against illumination, abrupt motion
    • Capable of recovering after losing track
  • Detection of land mines
    • Statistical models of clutter (non-Gaussian)
    • Optimal detection methods
    • Sup-pixel detection methods
    • Detection of disturbed earth
      • Detection of mass graves in the Balkans
    Broadwater and Chellappa, PAMI 2007, SP, 2010
  • Radar images
    • Synthetic aperture radar (SAR), foliage-penetrating SAR (FOPEN SAR), …
    • Speckle noise (random fluctuations in the return signal from an object that is no bigger than a single image-processing element)
    • Shape from shading – radar clinometry (USGS, Frankot, Chellappa, AI Journal, 1990)
    • 3D from interferometric SAR (Zebker and Goldstein), stereo SAR
    • Object detection, indexing recognition (DARPA programs MSTAR, SAIP, DDB)
      • MSTAR program (feature extraction, indexing, prediction, recognition) – Typical object recognition framework
      • Features can correspond to scatterers (supported by physics)
      • Low signal to clutter ratio (tree trunks produce stronger returns than objects)
      • Symmetric alpha-stable noise
    • Global hawk, TESAR sensors
    R.T. Frankot and R. Chellappa, Estimation of Surface Topography from SAR Imagery Using Shape from Shading Techniques, in Physics-Based Vision: Shape Recovery, (eds.), L.B. Wolff, S.A. Shafer and G.E. Healey, Jones and Bartlett Publishers, Boston, MA, pp. 62-101, 1992.
  • LADAR images
    • Took off in the mid eighties
    • From machine vision to outdoor conditions
    • Feature (step, crease edges, surfaces,) extraction, matching and recognition (Hypothesis prediction/verification)
      • Fundamental forms I and II (Besl and Jain)
    • Pulsed-Doppler LADAR (2km) for ATR
    • Better resolution at longer ranges
    • Kinect
    R. Chellappa, S. Der and E.J.M. Rignot, Statistical Characterization of FLIR, LADAR and SAR Imagery, in Statistics and Images, K.V Mardia, (ed.), Carfax publishers, Oxfordshire, U.K., pp. 273-312, 1994 .
  • Opening remarks – non-traditional applications
    • Road following, lane tracking and other automotive applications
      • Dickmanns, Pomereleu, DOT, FHWA
    • Computer vision for the blind
      • Navigation, face/expression recognition
    • Analysis of Schlieren images
      • Detection of oblique structures such as shock waves and shear layers
    • Understanding bee dances
    • Industrial inspection
  • Vision for Schlieren data reduction
    • Schlieren Imaging
      • Aerodynamic visualization technique in wind tunnels -> long established
      • Capture density gradients in supersonic flow
      • Shock waves, shear layers and turbulent structures
      • High speed imaging -> data deluge
    • Images complete the picture
      • Offer what surface measurements may not
      • Are we taking advantage of the data collected?
    • Can vision extend analysis capabilities?
      • Additional insight to flow unsteadiness
      • Desire for automation
      • Removal of human subjectivity
    • Recast as a segmentation/feature extraction problem
  • Extraction of oblique structures
    • Oblique flow structures ubiquitous
    • Physically meaningful segmentation
      • Bilateral filter -> isoperimetric cut
    • Shock wave and shear-layer inclination
      • Canny & Hough transform
    • Classification
      • Length & quantifiable bounds
      • Location enforced from labeling
    • Scale implementation for robustness?
      • Hough transform binning
      • Convergence from two Hough grids
    • Success of automation 92-94%
    • Outer shock motion history
    • Recommended for Publication in AIAA Journal
    • Vision: viable analysis tool
  • Waggle dance
    • Orientation of waggle axis  Direction of Food source.(with respect to sun).
    • Intensity of waggle dance  Sweetness of food source.
    • Frequency of waggle  Distance of food source.
    • Parameters of interest in the waggle dance
      • Waggle Axis : Average orientation of Thorax during Waggle.
      • Duration of Waggle : Number of frames of waggle in each segment of the dance.
  • Anatomy/behavior modeling - prior
    • Three major body parts; each body part modeled as an ellipse.
    • Anatomical modeling ensures
      • Physical limits of body parts are consistent; accounts for structural limitations and correlation among orientation of body parts
      • Insects move in the direction of their head.
    Veeraraghavan, Chellappa and Srinivasn, IEEE TPAMI, March 2008
    • Insects display very specific behaviors - priors
    • Modeling behavior explicitly improve
      • Tracking performance
      • Behavioral understanding
    • Position tracking and behavioral analysis in a unified framework.
  • Result
    • Detect Frames of Waggle Dance by looking at
      • Rate of change of Abdomen Orientation
      • Average absolute motion of center of abdomen in the direction perpendicular to the axis of the bee.
    • Parameters of Interest in the Waggle Dance
      • Waggle Axis : Average orientation of Thorax during Waggle.
      • Duration of Waggle : Number of frames of waggle in each segment of the dance.
  • Looking for a screw amid screws (MERL)
  • My advice to the young ones
    • Look for collaborations outside EE, CS
    • Helps with multidisciplinary credentials
    • Will help with winning MURIs: best source of support.
    • There are top Transactions and journals that accept these papers!
    • Computer vision presents immense opportunities outside traditional areas.