Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Computer vision for non-visual spectral regimes and non-traditional applications Rama Chellappa UMD
Opening remarks – non-visual sensors <ul><li>Sensors in visual regimes are only a small % of the possible sensors </li></u...
Problems addressed using hyperspectral images <ul><li>Sensor design  </li></ul><ul><li>Material classification/segmentatio...
Estimating object reflectance <ul><li>Radio transfer function   </li></ul><ul><ul><li>irradiance to the camera </li></ul><...
Tracking radiance
Reflectance tracking <ul><li>Robust against illumination, abrupt motion </li></ul><ul><li>Capable of recovering after losi...
Detection of land mines <ul><li>Statistical models of clutter (non-Gaussian) </li></ul><ul><li>Optimal detection methods  ...
Radar images <ul><li>Synthetic aperture radar (SAR), foliage-penetrating SAR (FOPEN SAR), … </li></ul><ul><li>Speckle nois...
LADAR images <ul><li>Took off in the mid eighties </li></ul><ul><li>From machine vision to outdoor conditions </li></ul><u...
Opening remarks – non-traditional applications <ul><li>Road following, lane tracking and other automotive applications </l...
Vision for Schlieren data reduction <ul><li>Schlieren Imaging </li></ul><ul><ul><li>Aerodynamic visualization technique in...
Extraction of oblique structures <ul><li>Oblique flow structures ubiquitous </li></ul><ul><li>Physically meaningful segmen...
Waggle dance <ul><li>Orientation of waggle axis    Direction of Food source.(with respect to sun). </li></ul><ul><li>Inte...
Anatomy/behavior modeling - prior <ul><li>Three major body parts; each body part modeled as an ellipse. </li></ul><ul><li>...
Result <ul><li>Detect Frames of Waggle Dance by looking at </li></ul><ul><ul><li>Rate of change of Abdomen Orientation </l...
Looking for a screw amid screws (MERL)
My advice to the young ones <ul><li>Look for collaborations outside EE, CS </li></ul><ul><li>Helps with multidisciplinary ...
Upcoming SlideShare
Loading in …5
×

Fcv taxo chellappa

305 views

Published on

Published in: Technology, Business
  • Be the first to comment

Fcv taxo chellappa

  1. 1. Computer vision for non-visual spectral regimes and non-traditional applications Rama Chellappa UMD
  2. 2. Opening remarks – non-visual sensors <ul><li>Sensors in visual regimes are only a small % of the possible sensors </li></ul><ul><li>Infrared, hyperspectral, LADAR, SAR, FOPEN SAR, mmWave, polarimetric </li></ul><ul><li>Provide thermal signatures, material properties, 3D, all weather/all day coverage, looking behind trees, walls, … </li></ul><ul><li>Sensor physics is important. </li></ul><ul><li>Low signal to clutter ratio. </li></ul><ul><li>Characterization of image statistics driven by sensor physics </li></ul>
  3. 3. Problems addressed using hyperspectral images <ul><li>Sensor design </li></ul><ul><li>Material classification/segmentation </li></ul><ul><li>Anomaly detection </li></ul><ul><ul><li>More than two decades of work </li></ul></ul><ul><ul><ul><li>ARO MURI (2002-2007) on the science of land-based target signatures. </li></ul></ul></ul><ul><ul><ul><li>DARPA DDB program, NGA </li></ul></ul></ul><ul><ul><ul><li>Glenn Healey: pioneer in HSI and computer vision </li></ul></ul></ul><ul><ul><ul><li>Larry Wolf, Equinox Corporation </li></ul></ul></ul><ul><li>Unmixing of pixels (Remote sensing) </li></ul><ul><li>Compressive sensing possibilities (SPC) </li></ul><ul><li>Numerous books, papers, conferences… </li></ul>
  4. 4. Estimating object reflectance <ul><li>Radio transfer function </li></ul><ul><ul><li>irradiance to the camera </li></ul></ul><ul><ul><li>reflectance at location (x,y) </li></ul></ul><ul><ul><li>main light source (e.g. sun light) </li></ul></ul><ul><ul><li>ambient light source coming from all directions, assuming constant intensity for all directions </li></ul></ul><ul><ul><li>fraction of unblocked sky from (x,y) view </li></ul></ul><ul><li> geometry factor </li></ul>Nguyen and Chellappa, CVPR 2010 Workshop on Beyond visible spectrum
  5. 5. Tracking radiance
  6. 6. Reflectance tracking <ul><li>Robust against illumination, abrupt motion </li></ul><ul><li>Capable of recovering after losing track </li></ul>
  7. 7. Detection of land mines <ul><li>Statistical models of clutter (non-Gaussian) </li></ul><ul><li>Optimal detection methods </li></ul><ul><li>Sup-pixel detection methods </li></ul><ul><li>Detection of disturbed earth </li></ul><ul><ul><li>Detection of mass graves in the Balkans </li></ul></ul>Broadwater and Chellappa, PAMI 2007, SP, 2010
  8. 8. Radar images <ul><li>Synthetic aperture radar (SAR), foliage-penetrating SAR (FOPEN SAR), … </li></ul><ul><li>Speckle noise (random fluctuations in the return signal from an object that is no bigger than a single image-processing element) </li></ul><ul><li>Shape from shading – radar clinometry (USGS, Frankot, Chellappa, AI Journal, 1990) </li></ul><ul><li>3D from interferometric SAR (Zebker and Goldstein), stereo SAR </li></ul><ul><li>Object detection, indexing recognition (DARPA programs MSTAR, SAIP, DDB) </li></ul><ul><ul><li>MSTAR program (feature extraction, indexing, prediction, recognition) – Typical object recognition framework </li></ul></ul><ul><ul><li>Features can correspond to scatterers (supported by physics) </li></ul></ul><ul><li>FOPEN SAR </li></ul><ul><ul><li>Low signal to clutter ratio (tree trunks produce stronger returns than objects) </li></ul></ul><ul><ul><li>Symmetric alpha-stable noise </li></ul></ul><ul><li>Global hawk, TESAR sensors </li></ul>R.T. Frankot and R. Chellappa, Estimation of Surface Topography from SAR Imagery Using Shape from Shading Techniques, in Physics-Based Vision: Shape Recovery, (eds.), L.B. Wolff, S.A. Shafer and G.E. Healey, Jones and Bartlett Publishers, Boston, MA, pp. 62-101, 1992.
  9. 9. LADAR images <ul><li>Took off in the mid eighties </li></ul><ul><li>From machine vision to outdoor conditions </li></ul><ul><li>Feature (step, crease edges, surfaces,) extraction, matching and recognition (Hypothesis prediction/verification) </li></ul><ul><ul><li>Fundamental forms I and II (Besl and Jain) </li></ul></ul><ul><li>Pulsed-Doppler LADAR (2km) for ATR </li></ul><ul><li>Better resolution at longer ranges </li></ul><ul><li>Kinect </li></ul>R. Chellappa, S. Der and E.J.M. Rignot, Statistical Characterization of FLIR, LADAR and SAR Imagery, in Statistics and Images, K.V Mardia, (ed.), Carfax publishers, Oxfordshire, U.K., pp. 273-312, 1994 .
  10. 10. Opening remarks – non-traditional applications <ul><li>Road following, lane tracking and other automotive applications </li></ul><ul><ul><li>Dickmanns, Pomereleu, DOT, FHWA </li></ul></ul><ul><li>Computer vision for the blind </li></ul><ul><ul><li>Navigation, face/expression recognition </li></ul></ul><ul><li>Analysis of Schlieren images </li></ul><ul><ul><li>Detection of oblique structures such as shock waves and shear layers </li></ul></ul><ul><li>Understanding bee dances </li></ul><ul><li>Industrial inspection </li></ul>
  11. 11. Vision for Schlieren data reduction <ul><li>Schlieren Imaging </li></ul><ul><ul><li>Aerodynamic visualization technique in wind tunnels -> long established </li></ul></ul><ul><ul><li>Capture density gradients in supersonic flow </li></ul></ul><ul><ul><li>Shock waves, shear layers and turbulent structures </li></ul></ul><ul><ul><li>High speed imaging -> data deluge </li></ul></ul><ul><li>Images complete the picture </li></ul><ul><ul><li>Offer what surface measurements may not </li></ul></ul><ul><ul><li>Are we taking advantage of the data collected? </li></ul></ul><ul><li>Can vision extend analysis capabilities? </li></ul><ul><ul><li>Additional insight to flow unsteadiness </li></ul></ul><ul><ul><li>Desire for automation </li></ul></ul><ul><ul><li>Removal of human subjectivity </li></ul></ul><ul><li>Recast as a segmentation/feature extraction problem </li></ul>
  12. 12. Extraction of oblique structures <ul><li>Oblique flow structures ubiquitous </li></ul><ul><li>Physically meaningful segmentation </li></ul><ul><ul><li>Bilateral filter -> isoperimetric cut </li></ul></ul><ul><li>Shock wave and shear-layer inclination </li></ul><ul><ul><li>Canny & Hough transform </li></ul></ul><ul><li>Classification </li></ul><ul><ul><li>Length & quantifiable bounds </li></ul></ul><ul><ul><li>Location enforced from labeling </li></ul></ul><ul><li>Scale implementation for robustness? </li></ul><ul><ul><li>Hough transform binning </li></ul></ul><ul><ul><li>Convergence from two Hough grids </li></ul></ul><ul><li>Success of automation 92-94% </li></ul><ul><li>Outer shock motion history </li></ul><ul><li>Recommended for Publication in AIAA Journal </li></ul><ul><li>Vision: viable analysis tool </li></ul>
  13. 13. Waggle dance <ul><li>Orientation of waggle axis  Direction of Food source.(with respect to sun). </li></ul><ul><li>Intensity of waggle dance  Sweetness of food source. </li></ul><ul><li>Frequency of waggle  Distance of food source. </li></ul><ul><li>Parameters of interest in the waggle dance </li></ul><ul><ul><li>Waggle Axis : Average orientation of Thorax during Waggle. </li></ul></ul><ul><ul><li>Duration of Waggle : Number of frames of waggle in each segment of the dance. </li></ul></ul>
  14. 14. Anatomy/behavior modeling - prior <ul><li>Three major body parts; each body part modeled as an ellipse. </li></ul><ul><li>Anatomical modeling ensures </li></ul><ul><ul><li>Physical limits of body parts are consistent; accounts for structural limitations and correlation among orientation of body parts </li></ul></ul><ul><ul><li>Insects move in the direction of their head. </li></ul></ul>Veeraraghavan, Chellappa and Srinivasn, IEEE TPAMI, March 2008 <ul><li>Insects display very specific behaviors - priors </li></ul><ul><li>Modeling behavior explicitly improve </li></ul><ul><ul><li>Tracking performance </li></ul></ul><ul><ul><li>Behavioral understanding </li></ul></ul><ul><li>Position tracking and behavioral analysis in a unified framework. </li></ul>
  15. 15. Result <ul><li>Detect Frames of Waggle Dance by looking at </li></ul><ul><ul><li>Rate of change of Abdomen Orientation </li></ul></ul><ul><ul><li>Average absolute motion of center of abdomen in the direction perpendicular to the axis of the bee. </li></ul></ul><ul><li>Parameters of Interest in the Waggle Dance </li></ul><ul><ul><li>Waggle Axis : Average orientation of Thorax during Waggle. </li></ul></ul><ul><ul><li>Duration of Waggle : Number of frames of waggle in each segment of the dance. </li></ul></ul>
  16. 16. Looking for a screw amid screws (MERL)
  17. 17. My advice to the young ones <ul><li>Look for collaborations outside EE, CS </li></ul><ul><li>Helps with multidisciplinary credentials </li></ul><ul><li>Will help with winning MURIs: best source of support. </li></ul><ul><li>There are top Transactions and journals that accept these papers! </li></ul><ul><li>Computer vision presents immense opportunities outside traditional areas. </li></ul>

×