Salient Point Detection

1,900 views
1,842 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,900
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Salient Point Detection

  1. 1. Outline Introduction Salient Point Detection Challenges Results Salient Point Detection Tyler Karrels Department of Electrical and Computer Engineering University of Wisconsin - Madison April 22, 2009 Tyler Karrels Salient Point Detection
  2. 2. Outline Introduction Salient Point Detection Challenges Results 1 Introduction Defining Saliency The Saliency Experience Human Visual System (HVS) Psychology of Perception Previous Work 2 Salient Point Detection Mathematical Framework Features Clustering Saliency 3 Challenges 4 Results Tyler Karrels Salient Point Detection
  3. 3. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is saliency? Definitions 1 SALIENT: “strikingly conspicuous; prominent; noticeable” American Heritage Dictionary Tyler Karrels Salient Point Detection
  4. 4. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is saliency? Definitions 1 SALIENT: “strikingly conspicuous; prominent; noticeable” American Heritage Dictionary 2 VISUAL SALIENCY: “. . . the distinct subjective perceptual quality which makes some items in the world stand out from their neighbors and immediately grab our attention.” Laurent Itti, Scholarpedia Tyler Karrels Salient Point Detection
  5. 5. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Tyler Karrels Salient Point Detection
  6. 6. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Popout Effect 1 Tyler Karrels Salient Point Detection
  7. 7. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Popout Effect 2 Tyler Karrels Salient Point Detection
  8. 8. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is salient? Tyler Karrels Salient Point Detection
  9. 9. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Conjunction Test 1 Tyler Karrels Salient Point Detection
  10. 10. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Conjunction Test 2 Tyler Karrels Salient Point Detection
  11. 11. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work FORGET (o.0) Tyler Karrels Salient Point Detection
  12. 12. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Conjunction Test 3 Tyler Karrels Salient Point Detection
  13. 13. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is salient? Tyler Karrels Salient Point Detection
  14. 14. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Phase Transition Tyler Karrels Salient Point Detection
  15. 15. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work What is salient? Tyler Karrels Salient Point Detection
  16. 16. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work The Eye: Physiology Peripheral Vision, Wikipedia Foveal Vision: attended location; line of sight Peripheral Vision: surrounding locations 1-1 photoreceptor to ganglion in Foveal Vision many-1 for Peripheral Vision (low res. compression) 50% Fovea + 50% Peripheral = 100% Data Sent to Brain! Tyler Karrels Salient Point Detection
  17. 17. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work The Eye: Feature Detector Bottom-Up Processing Detects low-level features in parallel, e.g. color, orientation, contrast, . . . Occurs before brain perceives data Feature detectors compete to direct attention to salient locations How do they compete, communicate, and cooperate? Top-Down Processing The brain’s expectations guide attention Tyler Karrels Salient Point Detection
  18. 18. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Helmholtz Principle “. . . whenever some large deviation from randomness occurs, a structure is perceived.” Desolneux, From Gestalt Theory to Image Analysis: A Probabilistic Approach Tyler Karrels Salient Point Detection
  19. 19. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Gestalt Laws Perceptual Grouping Principles Closure Similarity Proximity Symmetry Continuity Common Fate Tyler Karrels Salient Point Detection
  20. 20. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Sha’asua [4] Continuity & Closure Detect edges Form contours: Connect edges Maximize length Minimize total curvature Longer contours, more salient Disregards other gestalt laws & image features Tyler Karrels Salient Point Detection
  21. 21. Outline Defining Saliency Introduction The Saliency Experience Salient Point Detection Human Visual System (HVS) Challenges Psychology of Perception Results Previous Work Itti [2] Proximity & Similarity Multiple scales encode proximity Center-surround Measures local contrast Fine scale ‘center’ minus course scale ‘surround’ Normalization encodes similarity Feature map combination Itti’s Biological determines success Saliency Map Model Tyler Karrels Salient Point Detection
  22. 22. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Salient Point Detection Not constrained to be biologically plausible Not image-processing; clustering/outlier detection in Rd Pixels are salient, not objects or regions Challenges When is something salient? When not? Can we quantify saliency? Can we relate computer/human performance? Can we improve on previous methods? Will we? Tyler Karrels Salient Point Detection
  23. 23. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Framework Data Pixels {Xi }n i=1 Xi = (yi , xi , ri , gi , bi ) Video? Include time coordinate: X = (y , x, r , g , b, t) Feature Space Feature Maps {Fj }m j=1 Vi = (yi , xi , ri , gi , bi , Fi1 , . . . , Fim ) Feature Vectors {Vi }n i=1 Feature Space Vi ∈ [0, 1]d Tyler Karrels Salient Point Detection
  24. 24. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results The Process Easy As 1,2,3? 1 Create feature maps 2 Cluster points in Rd 3 Choose the salient cluster Tyler Karrels Salient Point Detection
  25. 25. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Proposed Features Feature Maps {Fj }m j=1 Salient Scenarios 1 Intensity 1 Intensity 2 Color 2 Colors [Red, Green, Blue] 3 Orientation 3 Edge orientations [0 ◦ , 45 ◦ , 90 ◦ , 135 ◦ ] 4 Size 4 Scale Description 5 Location 5 Pixel Location How does our data representation affect performance? Tyler Karrels Salient Point Detection
  26. 26. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results 2-D Example Do we really need 2 dimensions? Is 1 sufficient? Tyler Karrels Salient Point Detection
  27. 27. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results 1-D Example Background pixels: no orientation? Horizontal pixels: 0 ◦ or 180 ◦ ? Tyler Karrels Salient Point Detection
  28. 28. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Feature Subset Selection Choosing Salient Dimensions Interpret variance Projections onto feature subspaces Projections Pr[Vi = (v1 , . . . , vd )] empirical distribution I = {i1 , . . . , il } index set Project onto subset I , induce Pr[Vi = (vi1 , . . . , vil )] Minimize the KL Divergence between Empirical and Subset distributions Tyler Karrels Salient Point Detection
  29. 29. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Vertical, Horizontal, Intensity, Red Example Notice Pr[Red, VerticalBar ] ≈ Pr[RedBar ] Tyler Karrels Salient Point Detection
  30. 30. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Clustering in Feature Space Gaussian Mixture Clustering Figueiredo’s algorithm determines best number of clusters [1] Fits distribution in feature space to a mixture of Gaussians Uses EM algorithm, results vary depending on initialization Subspace Clustering Ma’s algorithm provides distortion parameter [3] Based on rate distortion theory Deterministic, same results every time Tight cluster requires low rate Additional clusters increase rate Tyler Karrels Salient Point Detection
  31. 31. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Subspace Clustering Large → few clusters, Small → many clusters As varies large → small, salient clusters emerge Tyler Karrels Salient Point Detection
  32. 32. Outline Mathematical Framework Introduction Features Salient Point Detection Clustering Challenges Saliency Results Salient Clusters How to determine a cluster’s saliency? Compare clusters for relative notion of saliency Relative cluster size and variance Small relative size indicates uniqueness Small relative variance indicates similarity ‘Distant’ clusters have less in common. Centroid Distance Mahanalobis Distance Outlier detection methods Tyler Karrels Salient Point Detection
  33. 33. Outline Introduction Salient Point Detection Challenges Results Existence of Salient Points Tyler Karrels Salient Point Detection
  34. 34. Outline Introduction Salient Point Detection Challenges Results Quantification of Salient Points Tyler Karrels Salient Point Detection
  35. 35. Outline Introduction Salient Point Detection Challenges Results Performance Tyler Karrels Salient Point Detection
  36. 36. Outline Introduction Salient Point Detection Challenges Results Orientation Test Results Tyler Karrels Salient Point Detection
  37. 37. Outline Introduction Salient Point Detection Challenges Results Google Tyler Karrels - Click Homepage - Click Papers Tyler Karrels Salient Point Detection
  38. 38. Outline Introduction Salient Point Detection Challenges Results References M. A. T. Figueiredo and A. K. Jain. Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 381–396, 2002. L. Itti and C. Koch. Feature combination strategies for saliency-based visual attention systems. Journal of Electronic Imaging, 10:161, 2001. Y. Ma, H. Derksen, W. Hong, and J. Wright. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1546–1562, 2007. A. Sha’asua. Structural saliency: The detection of globally salient structures using a locally connected network, 1988. ID: 1. Tyler Karrels Salient Point Detection

×