Computer vision for non-visual spectral regimes and non-traditional applications Rama Chellappa UMD
Opening remarks – non-visual sensors <ul><li>Sensors in visual regimes are only a small % of the possible sensors </li></u...
Problems addressed using hyperspectral images <ul><li>Sensor design  </li></ul><ul><li>Material classification/segmentatio...
Estimating object reflectance <ul><li>Radio transfer function   </li></ul><ul><ul><li>irradiance to the camera </li></ul><...
Tracking radiance
Reflectance tracking <ul><li>Robust against illumination, abrupt motion </li></ul><ul><li>Capable of recovering after losi...
Detection of land mines <ul><li>Statistical models of clutter (non-Gaussian) </li></ul><ul><li>Optimal detection methods  ...
Radar images <ul><li>Synthetic aperture radar (SAR), foliage-penetrating SAR (FOPEN SAR), … </li></ul><ul><li>Speckle nois...
LADAR images <ul><li>Took off in the mid eighties </li></ul><ul><li>From machine vision to outdoor conditions </li></ul><u...
Opening remarks – non-traditional applications <ul><li>Road following, lane tracking and other automotive applications </l...
Vision for Schlieren data reduction <ul><li>Schlieren Imaging </li></ul><ul><ul><li>Aerodynamic visualization technique in...
Extraction of oblique structures <ul><li>Oblique flow structures ubiquitous </li></ul><ul><li>Physically meaningful segmen...
Waggle dance <ul><li>Orientation of waggle axis    Direction of Food source.(with respect to sun). </li></ul><ul><li>Inte...
Anatomy/behavior modeling - prior <ul><li>Three major body parts; each body part modeled as an ellipse. </li></ul><ul><li>...
Result <ul><li>Detect Frames of Waggle Dance by looking at </li></ul><ul><ul><li>Rate of change of Abdomen Orientation </l...
Looking for a screw amid screws (MERL)
My advice to the young ones <ul><li>Look for collaborations outside EE, CS </li></ul><ul><li>Helps with multidisciplinary ...
Upcoming SlideShare
Loading in...5
×

Fcv taxo chellappa

195

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
195
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Assumption1: can identify flat pixels in full shade and full sun. This is not a bad assumption since there are many ubiquitous materials such as asphalt and concrete have flat spectra Note: will not consider cos(rho) for now, since this is only a constant and not affect shape of spectra. It will be taken care by normalization at the end F is the fraction of sky unblocked
  • Fcv taxo chellappa

    1. 1. Computer vision for non-visual spectral regimes and non-traditional applications Rama Chellappa UMD
    2. 2. Opening remarks – non-visual sensors <ul><li>Sensors in visual regimes are only a small % of the possible sensors </li></ul><ul><li>Infrared, hyperspectral, LADAR, SAR, FOPEN SAR, mmWave, polarimetric </li></ul><ul><li>Provide thermal signatures, material properties, 3D, all weather/all day coverage, looking behind trees, walls, … </li></ul><ul><li>Sensor physics is important. </li></ul><ul><li>Low signal to clutter ratio. </li></ul><ul><li>Characterization of image statistics driven by sensor physics </li></ul>
    3. 3. Problems addressed using hyperspectral images <ul><li>Sensor design </li></ul><ul><li>Material classification/segmentation </li></ul><ul><li>Anomaly detection </li></ul><ul><ul><li>More than two decades of work </li></ul></ul><ul><ul><ul><li>ARO MURI (2002-2007) on the science of land-based target signatures. </li></ul></ul></ul><ul><ul><ul><li>DARPA DDB program, NGA </li></ul></ul></ul><ul><ul><ul><li>Glenn Healey: pioneer in HSI and computer vision </li></ul></ul></ul><ul><ul><ul><li>Larry Wolf, Equinox Corporation </li></ul></ul></ul><ul><li>Unmixing of pixels (Remote sensing) </li></ul><ul><li>Compressive sensing possibilities (SPC) </li></ul><ul><li>Numerous books, papers, conferences… </li></ul>
    4. 4. Estimating object reflectance <ul><li>Radio transfer function </li></ul><ul><ul><li>irradiance to the camera </li></ul></ul><ul><ul><li>reflectance at location (x,y) </li></ul></ul><ul><ul><li>main light source (e.g. sun light) </li></ul></ul><ul><ul><li>ambient light source coming from all directions, assuming constant intensity for all directions </li></ul></ul><ul><ul><li>fraction of unblocked sky from (x,y) view </li></ul></ul><ul><li> geometry factor </li></ul>Nguyen and Chellappa, CVPR 2010 Workshop on Beyond visible spectrum
    5. 5. Tracking radiance
    6. 6. Reflectance tracking <ul><li>Robust against illumination, abrupt motion </li></ul><ul><li>Capable of recovering after losing track </li></ul>
    7. 7. Detection of land mines <ul><li>Statistical models of clutter (non-Gaussian) </li></ul><ul><li>Optimal detection methods </li></ul><ul><li>Sup-pixel detection methods </li></ul><ul><li>Detection of disturbed earth </li></ul><ul><ul><li>Detection of mass graves in the Balkans </li></ul></ul>Broadwater and Chellappa, PAMI 2007, SP, 2010
    8. 8. Radar images <ul><li>Synthetic aperture radar (SAR), foliage-penetrating SAR (FOPEN SAR), … </li></ul><ul><li>Speckle noise (random fluctuations in the return signal from an object that is no bigger than a single image-processing element) </li></ul><ul><li>Shape from shading – radar clinometry (USGS, Frankot, Chellappa, AI Journal, 1990) </li></ul><ul><li>3D from interferometric SAR (Zebker and Goldstein), stereo SAR </li></ul><ul><li>Object detection, indexing recognition (DARPA programs MSTAR, SAIP, DDB) </li></ul><ul><ul><li>MSTAR program (feature extraction, indexing, prediction, recognition) – Typical object recognition framework </li></ul></ul><ul><ul><li>Features can correspond to scatterers (supported by physics) </li></ul></ul><ul><li>FOPEN SAR </li></ul><ul><ul><li>Low signal to clutter ratio (tree trunks produce stronger returns than objects) </li></ul></ul><ul><ul><li>Symmetric alpha-stable noise </li></ul></ul><ul><li>Global hawk, TESAR sensors </li></ul>R.T. Frankot and R. Chellappa, Estimation of Surface Topography from SAR Imagery Using Shape from Shading Techniques, in Physics-Based Vision: Shape Recovery, (eds.), L.B. Wolff, S.A. Shafer and G.E. Healey, Jones and Bartlett Publishers, Boston, MA, pp. 62-101, 1992.
    9. 9. LADAR images <ul><li>Took off in the mid eighties </li></ul><ul><li>From machine vision to outdoor conditions </li></ul><ul><li>Feature (step, crease edges, surfaces,) extraction, matching and recognition (Hypothesis prediction/verification) </li></ul><ul><ul><li>Fundamental forms I and II (Besl and Jain) </li></ul></ul><ul><li>Pulsed-Doppler LADAR (2km) for ATR </li></ul><ul><li>Better resolution at longer ranges </li></ul><ul><li>Kinect </li></ul>R. Chellappa, S. Der and E.J.M. Rignot, Statistical Characterization of FLIR, LADAR and SAR Imagery, in Statistics and Images, K.V Mardia, (ed.), Carfax publishers, Oxfordshire, U.K., pp. 273-312, 1994 .
    10. 10. Opening remarks – non-traditional applications <ul><li>Road following, lane tracking and other automotive applications </li></ul><ul><ul><li>Dickmanns, Pomereleu, DOT, FHWA </li></ul></ul><ul><li>Computer vision for the blind </li></ul><ul><ul><li>Navigation, face/expression recognition </li></ul></ul><ul><li>Analysis of Schlieren images </li></ul><ul><ul><li>Detection of oblique structures such as shock waves and shear layers </li></ul></ul><ul><li>Understanding bee dances </li></ul><ul><li>Industrial inspection </li></ul>
    11. 11. Vision for Schlieren data reduction <ul><li>Schlieren Imaging </li></ul><ul><ul><li>Aerodynamic visualization technique in wind tunnels -> long established </li></ul></ul><ul><ul><li>Capture density gradients in supersonic flow </li></ul></ul><ul><ul><li>Shock waves, shear layers and turbulent structures </li></ul></ul><ul><ul><li>High speed imaging -> data deluge </li></ul></ul><ul><li>Images complete the picture </li></ul><ul><ul><li>Offer what surface measurements may not </li></ul></ul><ul><ul><li>Are we taking advantage of the data collected? </li></ul></ul><ul><li>Can vision extend analysis capabilities? </li></ul><ul><ul><li>Additional insight to flow unsteadiness </li></ul></ul><ul><ul><li>Desire for automation </li></ul></ul><ul><ul><li>Removal of human subjectivity </li></ul></ul><ul><li>Recast as a segmentation/feature extraction problem </li></ul>
    12. 12. Extraction of oblique structures <ul><li>Oblique flow structures ubiquitous </li></ul><ul><li>Physically meaningful segmentation </li></ul><ul><ul><li>Bilateral filter -> isoperimetric cut </li></ul></ul><ul><li>Shock wave and shear-layer inclination </li></ul><ul><ul><li>Canny & Hough transform </li></ul></ul><ul><li>Classification </li></ul><ul><ul><li>Length & quantifiable bounds </li></ul></ul><ul><ul><li>Location enforced from labeling </li></ul></ul><ul><li>Scale implementation for robustness? </li></ul><ul><ul><li>Hough transform binning </li></ul></ul><ul><ul><li>Convergence from two Hough grids </li></ul></ul><ul><li>Success of automation 92-94% </li></ul><ul><li>Outer shock motion history </li></ul><ul><li>Recommended for Publication in AIAA Journal </li></ul><ul><li>Vision: viable analysis tool </li></ul>
    13. 13. Waggle dance <ul><li>Orientation of waggle axis  Direction of Food source.(with respect to sun). </li></ul><ul><li>Intensity of waggle dance  Sweetness of food source. </li></ul><ul><li>Frequency of waggle  Distance of food source. </li></ul><ul><li>Parameters of interest in the waggle dance </li></ul><ul><ul><li>Waggle Axis : Average orientation of Thorax during Waggle. </li></ul></ul><ul><ul><li>Duration of Waggle : Number of frames of waggle in each segment of the dance. </li></ul></ul>
    14. 14. Anatomy/behavior modeling - prior <ul><li>Three major body parts; each body part modeled as an ellipse. </li></ul><ul><li>Anatomical modeling ensures </li></ul><ul><ul><li>Physical limits of body parts are consistent; accounts for structural limitations and correlation among orientation of body parts </li></ul></ul><ul><ul><li>Insects move in the direction of their head. </li></ul></ul>Veeraraghavan, Chellappa and Srinivasn, IEEE TPAMI, March 2008 <ul><li>Insects display very specific behaviors - priors </li></ul><ul><li>Modeling behavior explicitly improve </li></ul><ul><ul><li>Tracking performance </li></ul></ul><ul><ul><li>Behavioral understanding </li></ul></ul><ul><li>Position tracking and behavioral analysis in a unified framework. </li></ul>
    15. 15. Result <ul><li>Detect Frames of Waggle Dance by looking at </li></ul><ul><ul><li>Rate of change of Abdomen Orientation </li></ul></ul><ul><ul><li>Average absolute motion of center of abdomen in the direction perpendicular to the axis of the bee. </li></ul></ul><ul><li>Parameters of Interest in the Waggle Dance </li></ul><ul><ul><li>Waggle Axis : Average orientation of Thorax during Waggle. </li></ul></ul><ul><ul><li>Duration of Waggle : Number of frames of waggle in each segment of the dance. </li></ul></ul>
    16. 16. Looking for a screw amid screws (MERL)
    17. 17. My advice to the young ones <ul><li>Look for collaborations outside EE, CS </li></ul><ul><li>Helps with multidisciplinary credentials </li></ul><ul><li>Will help with winning MURIs: best source of support. </li></ul><ul><li>There are top Transactions and journals that accept these papers! </li></ul><ul><li>Computer vision presents immense opportunities outside traditional areas. </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×