Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Outline
               Introduction
    Salient Point Detection
                 Challenges
                    Results


...
Outline
                            Introduction
                 Salient Point Detection
                              Ch...
Outline    Defining Saliency
                                       Introduction    The Saliency Experience
               ...
Outline   Defining Saliency
                                         Introduction   The Saliency Experience
               ...
Outline   Defining Saliency
           Introduction   The Saliency Experience
Salient Point Detection   Human Visual System...
Outline   Defining Saliency
                            Introduction   The Saliency Experience
                 Salient Poi...
Outline   Defining Saliency
                            Introduction   The Saliency Experience
                 Salient Poi...
Outline   Defining Saliency
                              Introduction   The Saliency Experience
                   Salient...
Outline   Defining Saliency
                           Introduction   The Saliency Experience
                Salient Point...
Outline   Defining Saliency
                           Introduction   The Saliency Experience
                Salient Point...
Outline   Defining Saliency
           Introduction   The Saliency Experience
Salient Point Detection   Human Visual System...
Outline   Defining Saliency
                           Introduction   The Saliency Experience
                Salient Point...
Outline   Defining Saliency
                              Introduction   The Saliency Experience
                   Salient...
Outline   Defining Saliency
                              Introduction   The Saliency Experience
                   Salient...
Outline   Defining Saliency
                              Introduction   The Saliency Experience
                   Salient...
Outline      Defining Saliency
                              Introduction      The Saliency Experience
                   S...
Outline   Defining Saliency
                              Introduction   The Saliency Experience
                   Salient...
Outline       Defining Saliency
                                          Introduction       The Saliency Experience
      ...
Outline   Defining Saliency
                              Introduction   The Saliency Experience
                   Salient...
Outline   Defining Saliency
                          Introduction   The Saliency Experience
               Salient Point D...
Outline   Defining Saliency
                                  Introduction   The Saliency Experience
                      ...
Outline
                                              Mathematical Framework
                               Introduction
 ...
Outline
                                                   Mathematical Framework
                                    Intr...
Outline
                                              Mathematical Framework
                               Introduction
 ...
Outline
                                               Mathematical Framework
                                Introduction...
Outline
                                          Mathematical Framework
                           Introduction
         ...
Outline
                                             Mathematical Framework
                              Introduction
   ...
Outline
                                                 Mathematical Framework
                                  Introduc...
Outline
                                            Mathematical Framework
                             Introduction
     ...
Outline
                                              Mathematical Framework
                               Introduction
 ...
Outline
                                            Mathematical Framework
                             Introduction
     ...
Outline
                                              Mathematical Framework
                               Introduction
 ...
Outline
                            Introduction
                 Salient Point Detection
                              Ch...
Outline
                           Introduction
                Salient Point Detection
                             Chall...
Outline
                         Introduction
              Salient Point Detection
                           Challenges
...
Outline
                           Introduction
                Salient Point Detection
                             Chall...
Outline
                     Introduction
          Salient Point Detection
                       Challenges
            ...
Outline
                                        Introduction
                             Salient Point Detection
        ...
Upcoming SlideShare
Loading in …5
×

Salient Point Detection

2,358 views

Published on

  • Be the first to comment

Salient Point Detection

  1. 1. Outline Introduction Salient Point Detection Challenges Results Salient Point Detection Tyler Karrels Department of Electrical and Computer Engineering University of Wisconsin - Madison April 22, 2009 Tyler Karrels Salient Point Detection
  2. 2. Outline Introduction Salient Point Detection Challenges Results 1 Introduction Defining Saliency The Saliency Experience Human Visual System (HVS) Psychology of Perception Previous Work 2 Salient Point Detection Mathematical Framework Features Clustering Saliency 3 Challenges 4 Results Tyler Karrels Salient Point Detection
  3. 3. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is saliency? Definitions 1 SALIENT: “strikingly conspicuous; prominent; noticeable” American Heritage Dictionary Tyler Karrels Salient Point Detection
  4. 4. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is saliency? Definitions 1 SALIENT: “strikingly conspicuous; prominent; noticeable” American Heritage Dictionary 2 VISUAL SALIENCY: “. . . the distinct subjective perceptual quality which makes some items in the world stand out from their neighbors and immediately grab our attention.” Laurent Itti, Scholarpedia Tyler Karrels Salient Point Detection
  5. 5. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Tyler Karrels Salient Point Detection
  6. 6. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Popout Effect 1 Tyler Karrels Salient Point Detection
  7. 7. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Popout Effect 2 Tyler Karrels Salient Point Detection
  8. 8. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is salient? Tyler Karrels Salient Point Detection
  9. 9. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Conjunction Test 1 Tyler Karrels Salient Point Detection
  10. 10. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Conjunction Test 2 Tyler Karrels Salient Point Detection
  11. 11. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work FORGET (o.0) Tyler Karrels Salient Point Detection
  12. 12. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Conjunction Test 3 Tyler Karrels Salient Point Detection
  13. 13. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is salient? Tyler Karrels Salient Point Detection
  14. 14. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Phase Transition Tyler Karrels Salient Point Detection
  15. 15. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is salient? Tyler Karrels Salient Point Detection
  16. 16. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work The Eye: Physiology Peripheral Vision, Wikipedia Foveal Vision: attended location; line of sight Peripheral Vision: surrounding locations 1-1 photoreceptor to ganglion in Foveal Vision many-1 for Peripheral Vision (low res. compression) 50% Fovea + 50% Peripheral = 100% Data Sent to Brain! Tyler Karrels Salient Point Detection
  17. 17. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work The Eye: Feature Detector Bottom-Up Processing Detects low-level features in parallel, e.g. color, orientation, contrast, . . . Occurs before brain perceives data Feature detectors compete to direct attention to salient locations How do they compete, communicate, and cooperate? Top-Down Processing The brain’s expectations guide attention Tyler Karrels Salient Point Detection
  18. 18. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Helmholtz Principle “. . . whenever some large deviation from randomness occurs, a structure is perceived.” Desolneux, From Gestalt Theory to Image Analysis: A Probabilistic Approach Tyler Karrels Salient Point Detection
  19. 19. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Gestalt Laws Perceptual Grouping Principles Closure Similarity Proximity Symmetry Continuity Common Fate Tyler Karrels Salient Point Detection
  20. 20. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Sha’asua [4] Continuity & Closure Detect edges Form contours: Connect edges Maximize length Minimize total curvature Longer contours, more salient Disregards other gestalt laws & image features Tyler Karrels Salient Point Detection
  21. 21. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Itti [2] Proximity & Similarity Multiple scales encode proximity Center-surround Measures local contrast Fine scale ‘center’ minus course scale ‘surround’ Normalization encodes similarity Feature map combination Itti’s Biological determines success Saliency Map Model Tyler Karrels Salient Point Detection
  22. 22. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Salient Point Detection Not constrained to be biologically plausible Not image-processing; clustering/outlier detection in Rd Pixels are salient, not objects or regions Challenges When is something salient? When not? Can we quantify saliency? Can we relate computer/human performance? Can we improve on previous methods? Will we? Tyler Karrels Salient Point Detection
  23. 23. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Framework Data Pixels {Xi }n i=1 Xi = (yi , xi , ri , gi , bi ) Video? Include time coordinate: X = (y , x, r , g , b, t) Feature Space Feature Maps {Fj }m j=1 Vi = (yi , xi , ri , gi , bi , Fi1 , . . . , Fim ) Feature Vectors {Vi }n i=1 Feature Space Vi ∈ [0, 1]d Tyler Karrels Salient Point Detection
  24. 24. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results The Process Easy As 1,2,3? 1 Create feature maps 2 Cluster points in Rd 3 Choose the salient cluster Tyler Karrels Salient Point Detection
  25. 25. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Proposed Features Feature Maps {Fj }m j=1 Salient Scenarios 1 Intensity 1 Intensity 2 Color 2 Colors [Red, Green, Blue] 3 Orientation 3 Edge orientations [0 ◦ , 45 ◦ , 90 ◦ , 135 ◦ ] 4 Size 4 Scale Description 5 Location 5 Pixel Location How does our data representation affect performance? Tyler Karrels Salient Point Detection
  26. 26. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results 2-D Example Do we really need 2 dimensions? Is 1 sufficient? Tyler Karrels Salient Point Detection
  27. 27. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results 1-D Example Background pixels: no orientation? Horizontal pixels: 0 ◦ or 180 ◦ ? Tyler Karrels Salient Point Detection
  28. 28. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Feature Subset Selection Choosing Salient Dimensions Interpret variance Projections onto feature subspaces Projections Pr[Vi = (v1 , . . . , vd )] empirical distribution I = {i1 , . . . , il } index set Project onto subset I , induce Pr[Vi = (vi1 , . . . , vil )] Minimize the KL Divergence between Empirical and Subset distributions Tyler Karrels Salient Point Detection
  29. 29. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Vertical, Horizontal, Intensity, Red Example Notice Pr[Red, VerticalBar ] ≈ Pr[RedBar ] Tyler Karrels Salient Point Detection
  30. 30. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Clustering in Feature Space Gaussian Mixture Clustering Figueiredo’s algorithm determines best number of clusters [1] Fits distribution in feature space to a mixture of Gaussians Uses EM algorithm, results vary depending on initialization Subspace Clustering Ma’s algorithm provides distortion parameter [3] Based on rate distortion theory Deterministic, same results every time Tight cluster requires low rate Additional clusters increase rate Tyler Karrels Salient Point Detection
  31. 31. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Subspace Clustering Large → few clusters, Small → many clusters As varies large → small, salient clusters emerge Tyler Karrels Salient Point Detection
  32. 32. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Salient Clusters How to determine a cluster’s saliency? Compare clusters for relative notion of saliency Relative cluster size and variance Small relative size indicates uniqueness Small relative variance indicates similarity ‘Distant’ clusters have less in common. Centroid Distance Mahanalobis Distance Outlier detection methods Tyler Karrels Salient Point Detection
  33. 33. Outline Introduction Salient Point Detection Challenges Results Existence of Salient Points Tyler Karrels Salient Point Detection
  34. 34. Outline Introduction Salient Point Detection Challenges Results Quantification of Salient Points Tyler Karrels Salient Point Detection
  35. 35. Outline Introduction Salient Point Detection Challenges Results Performance Tyler Karrels Salient Point Detection
  36. 36. Outline Introduction Salient Point Detection Challenges Results Orientation Test Results Tyler Karrels Salient Point Detection
  37. 37. Outline Introduction Salient Point Detection Challenges Results Google Tyler Karrels - Click Homepage - Click Papers Tyler Karrels Salient Point Detection
  38. 38. Outline Introduction Salient Point Detection Challenges Results References M. A. T. Figueiredo and A. K. Jain. Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 381–396, 2002. L. Itti and C. Koch. Feature combination strategies for saliency-based visual attention systems. Journal of Electronic Imaging, 10:161, 2001. Y. Ma, H. Derksen, W. Hong, and J. Wright. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1546–1562, 2007. A. Sha’asua. Structural saliency: The detection of globally salient structures using a locally connected network, 1988. ID: 1. Tyler Karrels Salient Point Detection

×