1. Introduction to Machine Vision Zeeshan Zia, M.Sc. (EE) [email_address] , [email_address]

4. Machine Vision / Computer Vision / Image Understanding / Photogrammetry Trying to make Computers extract some useful information from images… Recognize faces, finger prints, other biometrics… Detect anamolies in X-Rays, Ultrasounds, … Point out suspicious human activity like fighting (surveillance) Use ‚vision‘ as a sensor in Control Systems… Make truly ‚autonomous‘ and ‚mobile‘ robots possible… Sophisticated special effects in movies… New and friendlier interfaces to Computers and other machines…

6. Demo (Obama-Kanade)

7. Demo (Mount Everest – ETHZ)

8. Demo (Car and Pedestrian detection)

9. Demo (First-person action recognition) My Master thesis – report available on my webpage…

10. Demo (Sport events analysis – TUM)

11. Demo (Gaze Tracking from outside)

12. Demo (Robotic Arm)

13. Demo (India Driving Challenge)

14. Demo (Cell Tracking)

16. LTI/LSI Systems: Convolution Continuous System : h(t) is the system‘s IMPULSE RESPONSE (1D case), x(t) is the input, and y(t) is the output.

17. Convolution : Discrete Case

18. Convolution : 2D discrete

29. Intensity Histograms There can be other histograms also…(histogram of edges!)

31. Project 2: Depth from Stereo Image 2 Image 1 W (X,Y,Z) BaseLine distance y x x y (x2,y2) (x1,y1) Optical Axis

32. Relating Depth to Image Coordinates

33. Relating Depth to Image Coordinates

34. By Similar Triangles

35. How does the computer know which point is which?

38. Cross-Correlation as a distance measure Remember Example 3.6-2 of Bruce Carlson???

45. Segmentation : some examples

46. Figure-Ground Separation

47. Super-Pixels

48. Project 3: Segmentation by Clustering One simple way of performing segmentation is to use clustering algorithms:

51. Clustering Temperature Humidity

52. Clustering Temperature Humidity

53. Clustering Temperature Humidity m 1 m 2 m 3

54. Clustering of pixels in image Make one vector per pixel of the image, X i = [x, y, R, G, B] Apply clustering on these vectors…call all pixels that end up in the same cluster as one segment! WHY ?

55. K-Means Clustering Algorithm Step 1: Determine number of clusters K Step 2: Randomly choose K different mean vectors: m 1 , m 2 , …, m K Step 3: Choose a data vector X i, and calculate Euclidean distance between this vector and all the mean vectors one-by-one. Step 4: Assign this data vector to the cluster with the minimum distance Step 5: Set i to i + 1, and go back to step 3 Step 6: Calculate new mean vectors based on assignments Step 7: Set i = 0, and go back to step 3

56. K-Means Clustering: Example

59. Project 4: Object Recognition

60. Project 4: Object Recognition

61. Project 4: Object Recognition

62. Project 4: Object Recognition Challenges 4: Scale variation

63. Project 4: Object Recognition

64. Project 4: Object Recognition Challenges 6: Background Clutter Find the dustbin…

65. Project 4: Object Recognition Single Object Recognition A solved problem!

66. Project 4: Object Recognition Challenges 7: Intra-class variation

67. Project 4: Object Recognition Challenges 7: Intra-class variation My PhD focus is intra-class and viewpoint invariance – alongwith all the other problems..

68. Project 4 : SIFT Based Object Recognition

69. Scale Invariance

70. Rotational Invariance