13 cv mil_preprocessing

387 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
387
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

13 cv mil_preprocessing

  1. 1. Computer vision: models, learning and inference Chapter 13Image preprocessing and feature extraction Please send errata to s.prince@cs.ucl.ac.uk
  2. 2. Preprocessing• The goal of pre-processing is – to try to reduce unwanted variation in image due to lighting, scale, deformation etc. – to reduce data to a manageable size• Give the subsequent model a chance• Preprocessing definition: deterministic transformation to pixels p to create data vector x• Usually heuristics based on experience Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2
  3. 3. Structure• Per-pixel transformations• Edges, corners, and interest points• Descriptors• Dimensionality reduction Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3
  4. 4. Normalization• Fix first and second moments to standard values• Remove contrast and constant additive luminance variations Before After Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4
  5. 5. Histogram EqualizationMake all of the moments the same by forcing the histogram of intensities to be the same Before/ normalized/ Histogram Equalized Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5
  6. 6. Histogram EqualizationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 6
  7. 7. ConvolutionTakes pixel image P and applies a filter FComputes weighted sum of pixel values, whereweights given by filter.Easiest to see with a concrete example Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 7
  8. 8. Blurring (convolve with Gaussian) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8
  9. 9. Gradient Filters• Rule of thumb: big response when image matches filter Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9
  10. 10. Gabor FiltersComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 10
  11. 11. Haar FiltersComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 11
  12. 12. Local binary patternsComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 12
  13. 13. Textons• An attempt to characterize texture• Replace each pixel with integer representing the texture ‘type’ Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13
  14. 14. Computing TextonsTake a bank of filters and apply Cluster in filter space to lots of imagesFor new pixel, filter surrounding region with same bank, and assign to nearest cluster Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 14
  15. 15. Structure• Per-pixel transformations• Edges, corners, and interest points• Descriptors• Dimensionality reduction Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15
  16. 16. Edges (from Elder and Goldberg 2000)Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16
  17. 17. Canny Edge DetectorComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 17
  18. 18. Canny Edge DetectorCompute horizontal and vertical gradient images h and v Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18
  19. 19. Canny Edge Detector Quantize to 4 directionsComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 19
  20. 20. Canny Edge Detector Non-maximal suppressionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 20
  21. 21. Canny Edge Detector Hysteresis ThresholdingComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 21
  22. 22. Harris Corner DetectorMake decision based onimage structure tensor Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22
  23. 23. SIFT DetectorFilter with difference of Gaussian filters at increasing scalesBuild image stack (scale space)Find extrema in this 3D volume Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23
  24. 24. SIFT DetectorIdentified Corners Remove those on Remove those edges where contrast is low Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24
  25. 25. Assign Orientation Orientation assigned by looking at intensity gradients in region around point Form a histogram of these gradients by binning. Set orientation to peak of histogram.Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25
  26. 26. Structure• Per-pixel transformations• Edges, corners, and interest points• Descriptors• Dimensionality reduction Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26
  27. 27. Sift DescriptorGoal: produce a vector that describes the region around the interest point. All calculations are relative to the orientation and scale of the keypoint Makes descriptor invariant to rotation and scale Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27
  28. 28. Sift Descriptor1. Compute image gradients 2. Pool into local histograms 3. Concatenate histograms 4. Normalize histograms Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28
  29. 29. HoG DescriptorComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 29
  30. 30. Bag of words descriptor• Compute visual features in image• Compute descriptor around each• Find closest match in library and assign index• Compute histogram of these indices over the region• Dictionary computed using K-means Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30
  31. 31. Shape context descriptor Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31
  32. 32. Structure• Per-pixel transformations• Edges, corners, and interest points• Descriptors• Dimensionality reduction Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32
  33. 33. Dimensionality ReductionDimensionality reduction attempt to find a low dimensional (orhidden) representation h which can approximately explain thedata x so thatwhere is a function that takes the hidden variable anda set of parameters q.Typically, we choose the function family and then learn hand q from training data Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33
  34. 34. Least Squares CriterionChoose the parameters q and the hidden variables h so that theyminimize the least squares approximation error (a measure ofhow well they can reconstruct the data x). Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34
  35. 35. Simple ExampleApproximate each data example x with a scalar value h.Data is reconstructed by multiplying h by a parameter f andadding the mean vector m.... or even better, lets subtract m from each data example to getmean-zero data Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35
  36. 36. Simple ExampleApproximate each data example x with a scalar value h.Data is reconstructed by multiplying h by a factor f.Criterion: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36
  37. 37. CriterionProblem: the problem is non-unique. If we multiply f by anyconstant a and divide each of the hidden variables h1...I by thesame constant we get the same cost. (i.e. (fa) (hi/a) = fhi)Solution: We make the solution unique by constraining thelength of f to be 1 using a Lagrange multiplier. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37
  38. 38. CriterionNow we have the new cost function:To optimize this we take derivatives with respect to f and hi,equate the resulting expressions to zero and re-arrange. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 38
  39. 39. SolutionTo compute the hidden value, take dot product with the vector f Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 39
  40. 40. SolutionTo compute the hidden value, take dot product with the vector f or where To compute the vector f, compute the first eigenvector of the scatter matrix . Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40
  41. 41. Computing hTo compute the hidden value, take dot product with the vector f Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41
  42. 42. ReconstructionTo reconstruct, multiply the hidden variable h by vector f. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42
  43. 43. Principal Components AnalysisSame idea, but not the hidden variable h is multi-dimensional.Each components weights one column of matrix F so that data isapproximated asThis leads to cost function:This has a non-unique optimum so we enforce the constraintthat F should be a (truncated) rotation matrix and FTF=I Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43
  44. 44. PCA SolutionTo compute the hidden vector, take dot product with eachcolumn of F.To compute the matrix F, compute the first eigenvectors ofthe scatter matrix .The basis functions in the columns of F are called principalcomponents and the entries of h are called loadings Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44
  45. 45. Dual PCAProblem: PCA as described has a major drawback. We need tocompute the eigenvectors of the scatter matrixBut this has size Dx x Dx. Visual data tends to be very highdimensional, so this may be extremely large.Solution: Reparameterize the principal components asweighted sums of the data...and solve for the new variables Y. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45
  46. 46. Geometric InterpretationEach column of F can bedescribed as a weightedsum of the originaldatapoints.Weights given in thecorresponding columns ofthe new variable Y. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46
  47. 47. MotivationSolution: Reparameterize the principal components asweighted sums of the data...and solve for the new variables Y.Why? If the number of datapoints I is less than the number ofobserved dimension Dx then the Y will be smaller than F andthe resulting optimization becomes easier.Intuition: we are not interested in principal components thatare not in the subspace spanned by the data anyway. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47
  48. 48. Cost functionsPrincipal components analysis...subject to FTF=I.Dual principal components analysis...subject to FTF=I or YTXTXY=I. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 48
  49. 49. SolutionTo compute the hidden vector, take dot product with eachcolumn of F=YX. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 49
  50. 50. SolutionTo compute the hidden vector, take dot product with eachcolumn of F=YX.To compute the matrix Y, compute the first eigenvectors ofthe inner product matrix XTX.The inner product matrix has size I x I.If the number of examples I is less than the dimensionality ofthe data Dx then this is a smaller eigenproblem. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 50
  51. 51. K-Means algorithmApproximate data with a set of meansLeast squares criterionAlternate minimization Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 51
  52. 52. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 52

×