SlideShare a Scribd company logo
Robert Collins
CSE486, Penn State




                          Lecture 06:
                     Harris Corner Detector
                      Reading: T&V Section 4.3
Robert Collins
CSE486, Penn State
                     Motivation: Matchng Problem
         Vision tasks such as stereo and motion estimation require
         finding corresponding features across two or more views.
Robert Collins
CSE486, Penn State
                     Motivation: Patch Matching
      Elements to be matched are image patches of fixed size




   Task: find the best (most similar) patch in a second image
                         ?
                         =




Camps, PSU
Robert Collins

               Not all Patches are Created Equal!
CSE486, Penn State




   Inituition: this would be a good patch for matching, since
   it is very distinctive (there is only one patch in the second
   frame that looks similar).
                      ?
                      =


Camps, PSU
Robert Collins

               Not all Patches are Created Equal!
CSE486, Penn State




 Inituition: this would be a BAD patch for matching, since
 it is not very distinctive (there are many similar patches
 in the second frame)
                      ?
                      =


Camps, PSU
Robert Collins
CSE486, Penn State
                     What are Corners?




     • They are good features to match!
 M.Hebert, CMU
Robert Collins
CSE486, Penn State
                     Corner Points: Basic Idea
       • We should easily recognize the point by looking
         at intensity values within a small window
       • Shifting the window in any direction should yield
         a large change in appearance.




C.Dyer, UWisc
Robert Collins
CSE486, Penn State
                      Appearance Change in
 Interactive         Neighborhood of a Patch
 “demo”
Robert Collins

              Harris Corner Detector: Basic Idea
CSE486, Penn State




                Harris corner detector gives a mathematical
                approach for determining which case holds.


C.Dyer, UWisc
Robert Collins
CSE486, Penn State
                     Harris Detector: Mathematics




C.Dyer, UWisc
Robert Collins
CSE486, Penn State
                     Harris Detector: Intuition




            For nearly constant patches, this will be near 0.
            For very distinctive patches, this will be larger.
            Hence... we want patches where E(u,v) is LARGE.

C.Dyer, UWisc
Robert Collins
CSE486, Penn State
                     Taylor Series for 2D Functions


                                          First partial derivatives



                                          Second partial derivatives



                                          Third partial derivatives

                       (Higher order terms)


      First order approx
Robert Collins
CSE486, Penn State
                     Harris Corner Derivation



                                         First order approx




                                   Rewrite as matrix equation
Robert Collins
CSE486, Penn State
                     Harris Detector: Mathematics




   Windowing function - computing a    Note: these are just products of
   weighted sum (simplest case, w=1)   components of the gradient, Ix, Iy
C.Dyer, UWisc
Robert Collins

             Intuitive Way to Understand Harris
CSE486, Penn State




          Treat gradient vectors as a set of (dx,dy) points
          with a center of mass defined as being at (0,0).

          Fit an ellipse to that set of points via scatter matrix

          Analyze ellipse parameters for varying cases…
Robert Collins
CSE486, Penn State
  M.Hebert, CMU
                  Example: Cases and 2D Derivatives
Robert Collins
CSE486, Penn State
  M.Hebert, CMU
                  Plotting Derivatives as 2D Points
Robert Collins
CSE486, Penn State
                  Fitting Ellipse to each Set of Points



                                      λ1~λ2 = small
  M.Hebert, CMU




                  λ1~λ2 = large       λ1 large; λ2 = small
Robert Collins
CSE486, Penn State
                     Classification via Eigenvalues




C.Dyer, UWisc
Robert Collins
CSE486, Penn State
                     Corner Response Measure




C.Dyer, UWisc
Robert Collins
CSE486, Penn State
                          Corner Response Map
                               lambda1
          (0,0)
                                                R=0


                                                R=28



                                                R=65
                lambda2




                                                R=104



                                                R=142
Robert Collins
CSE486, Penn State
                     Corner Response Map
                                                         R < 0 “Edge”
                                             lambda1
                                                              R=0

                                                              R=28
                                      |R| small
                                      “Flat”
                                                              R=65


                          lambda2                             R=104


                                                              R=142
                                                   R large
                       R < 0 “Edge”




                                                   “Corner”
Robert Collins
CSE486, Penn State
                        Corner Response Example




            Harris R score.
                     Ix, Iy computed using Sobel operator
                     Windowing function w = Gaussian, sigma=1
Robert Collins
CSE486, Penn State
                     Corner Response Example




                              Threshold: R < -10000
                                    (edges)
Robert Collins
CSE486, Penn State
                     Corner Response Example




                              Threshold: > 10000
                                    (corners)
Robert Collins
CSE486, Penn State
                     Corner Response Example




                             Threshold: -10000 < R < 10000
                               (neither edges nor corners)
Robert Collins

              Harris Corner Detection Algorithm
CSE486, Penn State




                     6. Threshold on value of R. Compute nonmax suppression.

M.Hebert, CMU

More Related Content

Similar to Lecture06

Lecture31
Lecture31Lecture31
Lecture31zukun
 
Lecture14
Lecture14Lecture14
Lecture14zukun
 
Lecture20
Lecture20Lecture20
Lecture20zukun
 
Lecture05
Lecture05Lecture05
Lecture05zukun
 
Lecture22
Lecture22Lecture22
Lecture22zukun
 
Lecture21
Lecture21Lecture21
Lecture21zukun
 
Lecture21
Lecture21Lecture21
Lecture21zukun
 
Lecture02
Lecture02Lecture02
Lecture02zukun
 

Similar to Lecture06 (8)

Lecture31
Lecture31Lecture31
Lecture31
 
Lecture14
Lecture14Lecture14
Lecture14
 
Lecture20
Lecture20Lecture20
Lecture20
 
Lecture05
Lecture05Lecture05
Lecture05
 
Lecture22
Lecture22Lecture22
Lecture22
 
Lecture21
Lecture21Lecture21
Lecture21
 
Lecture21
Lecture21Lecture21
Lecture21
 
Lecture02
Lecture02Lecture02
Lecture02
 

More from zukun

My lyn tutorial 2009
My lyn tutorial 2009My lyn tutorial 2009
My lyn tutorial 2009zukun
 
ETHZ CV2012: Tutorial openCV
ETHZ CV2012: Tutorial openCVETHZ CV2012: Tutorial openCV
ETHZ CV2012: Tutorial openCVzukun
 
ETHZ CV2012: Information
ETHZ CV2012: InformationETHZ CV2012: Information
ETHZ CV2012: Informationzukun
 
Siwei lyu: natural image statistics
Siwei lyu: natural image statisticsSiwei lyu: natural image statistics
Siwei lyu: natural image statisticszukun
 
Lecture9 camera calibration
Lecture9 camera calibrationLecture9 camera calibration
Lecture9 camera calibrationzukun
 
Brunelli 2008: template matching techniques in computer vision
Brunelli 2008: template matching techniques in computer visionBrunelli 2008: template matching techniques in computer vision
Brunelli 2008: template matching techniques in computer visionzukun
 
Modern features-part-4-evaluation
Modern features-part-4-evaluationModern features-part-4-evaluation
Modern features-part-4-evaluationzukun
 
Modern features-part-3-software
Modern features-part-3-softwareModern features-part-3-software
Modern features-part-3-softwarezukun
 
Modern features-part-2-descriptors
Modern features-part-2-descriptorsModern features-part-2-descriptors
Modern features-part-2-descriptorszukun
 
Modern features-part-1-detectors
Modern features-part-1-detectorsModern features-part-1-detectors
Modern features-part-1-detectorszukun
 
Modern features-part-0-intro
Modern features-part-0-introModern features-part-0-intro
Modern features-part-0-introzukun
 
Lecture 02 internet video search
Lecture 02 internet video searchLecture 02 internet video search
Lecture 02 internet video searchzukun
 
Lecture 01 internet video search
Lecture 01 internet video searchLecture 01 internet video search
Lecture 01 internet video searchzukun
 
Lecture 03 internet video search
Lecture 03 internet video searchLecture 03 internet video search
Lecture 03 internet video searchzukun
 
Icml2012 tutorial representation_learning
Icml2012 tutorial representation_learningIcml2012 tutorial representation_learning
Icml2012 tutorial representation_learningzukun
 
Advances in discrete energy minimisation for computer vision
Advances in discrete energy minimisation for computer visionAdvances in discrete energy minimisation for computer vision
Advances in discrete energy minimisation for computer visionzukun
 
Gephi tutorial: quick start
Gephi tutorial: quick startGephi tutorial: quick start
Gephi tutorial: quick startzukun
 
EM algorithm and its application in probabilistic latent semantic analysis
EM algorithm and its application in probabilistic latent semantic analysisEM algorithm and its application in probabilistic latent semantic analysis
EM algorithm and its application in probabilistic latent semantic analysiszukun
 
Object recognition with pictorial structures
Object recognition with pictorial structuresObject recognition with pictorial structures
Object recognition with pictorial structureszukun
 
Iccv2011 learning spatiotemporal graphs of human activities
Iccv2011 learning spatiotemporal graphs of human activities Iccv2011 learning spatiotemporal graphs of human activities
Iccv2011 learning spatiotemporal graphs of human activities zukun
 

More from zukun (20)

My lyn tutorial 2009
My lyn tutorial 2009My lyn tutorial 2009
My lyn tutorial 2009
 
ETHZ CV2012: Tutorial openCV
ETHZ CV2012: Tutorial openCVETHZ CV2012: Tutorial openCV
ETHZ CV2012: Tutorial openCV
 
ETHZ CV2012: Information
ETHZ CV2012: InformationETHZ CV2012: Information
ETHZ CV2012: Information
 
Siwei lyu: natural image statistics
Siwei lyu: natural image statisticsSiwei lyu: natural image statistics
Siwei lyu: natural image statistics
 
Lecture9 camera calibration
Lecture9 camera calibrationLecture9 camera calibration
Lecture9 camera calibration
 
Brunelli 2008: template matching techniques in computer vision
Brunelli 2008: template matching techniques in computer visionBrunelli 2008: template matching techniques in computer vision
Brunelli 2008: template matching techniques in computer vision
 
Modern features-part-4-evaluation
Modern features-part-4-evaluationModern features-part-4-evaluation
Modern features-part-4-evaluation
 
Modern features-part-3-software
Modern features-part-3-softwareModern features-part-3-software
Modern features-part-3-software
 
Modern features-part-2-descriptors
Modern features-part-2-descriptorsModern features-part-2-descriptors
Modern features-part-2-descriptors
 
Modern features-part-1-detectors
Modern features-part-1-detectorsModern features-part-1-detectors
Modern features-part-1-detectors
 
Modern features-part-0-intro
Modern features-part-0-introModern features-part-0-intro
Modern features-part-0-intro
 
Lecture 02 internet video search
Lecture 02 internet video searchLecture 02 internet video search
Lecture 02 internet video search
 
Lecture 01 internet video search
Lecture 01 internet video searchLecture 01 internet video search
Lecture 01 internet video search
 
Lecture 03 internet video search
Lecture 03 internet video searchLecture 03 internet video search
Lecture 03 internet video search
 
Icml2012 tutorial representation_learning
Icml2012 tutorial representation_learningIcml2012 tutorial representation_learning
Icml2012 tutorial representation_learning
 
Advances in discrete energy minimisation for computer vision
Advances in discrete energy minimisation for computer visionAdvances in discrete energy minimisation for computer vision
Advances in discrete energy minimisation for computer vision
 
Gephi tutorial: quick start
Gephi tutorial: quick startGephi tutorial: quick start
Gephi tutorial: quick start
 
EM algorithm and its application in probabilistic latent semantic analysis
EM algorithm and its application in probabilistic latent semantic analysisEM algorithm and its application in probabilistic latent semantic analysis
EM algorithm and its application in probabilistic latent semantic analysis
 
Object recognition with pictorial structures
Object recognition with pictorial structuresObject recognition with pictorial structures
Object recognition with pictorial structures
 
Iccv2011 learning spatiotemporal graphs of human activities
Iccv2011 learning spatiotemporal graphs of human activities Iccv2011 learning spatiotemporal graphs of human activities
Iccv2011 learning spatiotemporal graphs of human activities
 

Lecture06

  • 1. Robert Collins CSE486, Penn State Lecture 06: Harris Corner Detector Reading: T&V Section 4.3
  • 2. Robert Collins CSE486, Penn State Motivation: Matchng Problem Vision tasks such as stereo and motion estimation require finding corresponding features across two or more views.
  • 3. Robert Collins CSE486, Penn State Motivation: Patch Matching Elements to be matched are image patches of fixed size Task: find the best (most similar) patch in a second image ? = Camps, PSU
  • 4. Robert Collins Not all Patches are Created Equal! CSE486, Penn State Inituition: this would be a good patch for matching, since it is very distinctive (there is only one patch in the second frame that looks similar). ? = Camps, PSU
  • 5. Robert Collins Not all Patches are Created Equal! CSE486, Penn State Inituition: this would be a BAD patch for matching, since it is not very distinctive (there are many similar patches in the second frame) ? = Camps, PSU
  • 6. Robert Collins CSE486, Penn State What are Corners? • They are good features to match! M.Hebert, CMU
  • 7. Robert Collins CSE486, Penn State Corner Points: Basic Idea • We should easily recognize the point by looking at intensity values within a small window • Shifting the window in any direction should yield a large change in appearance. C.Dyer, UWisc
  • 8. Robert Collins CSE486, Penn State Appearance Change in Interactive Neighborhood of a Patch “demo”
  • 9. Robert Collins Harris Corner Detector: Basic Idea CSE486, Penn State Harris corner detector gives a mathematical approach for determining which case holds. C.Dyer, UWisc
  • 10. Robert Collins CSE486, Penn State Harris Detector: Mathematics C.Dyer, UWisc
  • 11. Robert Collins CSE486, Penn State Harris Detector: Intuition For nearly constant patches, this will be near 0. For very distinctive patches, this will be larger. Hence... we want patches where E(u,v) is LARGE. C.Dyer, UWisc
  • 12. Robert Collins CSE486, Penn State Taylor Series for 2D Functions First partial derivatives Second partial derivatives Third partial derivatives (Higher order terms) First order approx
  • 13. Robert Collins CSE486, Penn State Harris Corner Derivation First order approx Rewrite as matrix equation
  • 14. Robert Collins CSE486, Penn State Harris Detector: Mathematics Windowing function - computing a Note: these are just products of weighted sum (simplest case, w=1) components of the gradient, Ix, Iy C.Dyer, UWisc
  • 15. Robert Collins Intuitive Way to Understand Harris CSE486, Penn State Treat gradient vectors as a set of (dx,dy) points with a center of mass defined as being at (0,0). Fit an ellipse to that set of points via scatter matrix Analyze ellipse parameters for varying cases…
  • 16. Robert Collins CSE486, Penn State M.Hebert, CMU Example: Cases and 2D Derivatives
  • 17. Robert Collins CSE486, Penn State M.Hebert, CMU Plotting Derivatives as 2D Points
  • 18. Robert Collins CSE486, Penn State Fitting Ellipse to each Set of Points λ1~λ2 = small M.Hebert, CMU λ1~λ2 = large λ1 large; λ2 = small
  • 19. Robert Collins CSE486, Penn State Classification via Eigenvalues C.Dyer, UWisc
  • 20. Robert Collins CSE486, Penn State Corner Response Measure C.Dyer, UWisc
  • 21. Robert Collins CSE486, Penn State Corner Response Map lambda1 (0,0) R=0 R=28 R=65 lambda2 R=104 R=142
  • 22. Robert Collins CSE486, Penn State Corner Response Map R < 0 “Edge” lambda1 R=0 R=28 |R| small “Flat” R=65 lambda2 R=104 R=142 R large R < 0 “Edge” “Corner”
  • 23. Robert Collins CSE486, Penn State Corner Response Example Harris R score. Ix, Iy computed using Sobel operator Windowing function w = Gaussian, sigma=1
  • 24. Robert Collins CSE486, Penn State Corner Response Example Threshold: R < -10000 (edges)
  • 25. Robert Collins CSE486, Penn State Corner Response Example Threshold: > 10000 (corners)
  • 26. Robert Collins CSE486, Penn State Corner Response Example Threshold: -10000 < R < 10000 (neither edges nor corners)
  • 27. Robert Collins Harris Corner Detection Algorithm CSE486, Penn State 6. Threshold on value of R. Compute nonmax suppression. M.Hebert, CMU