Download to read offline
Lecture 1, 2 - An Introduction ot Computer Vision
- 1. O TO COMPUTERVISION Dr. M. Aksam Iftikhar
- 2. WHAT IS COMPUTERVISION? Nice sunset! “Making computers see and understand”
- 3. MAIN OBJECTIVES: THEORY+ ALGORITHMS Development of the theoretical and algorithmic basis by which useful information about the 3D world can be automatically extracted and analyzed from a single or multiple 2D images of the world.
- 4. COMPUTER VISION, ALSO KNOWNAS ... Computational Vision Includes modeling of biological vision Image Understanding Automated scene analysis (e.g., satellite images, robot navigation) Machine Vision Industrial, factory-floor systems for inspection, measurements, part placement, etc.
- 5. CONNECTIONS TO OTHER DISCIPLINES ComputerVision Image Processing Pattern Recognition & Machine Learning Artificial Intelligence Robotics Psychology & Neuroscience Computer Graphics
- 6.
- 7.
- 8. Computer Graphics Image Output: Geometric Models Synthetic Camera Projection,shading, lighting models
- 9.
- 10. THE GOAL OFCOMPUTER VISION • To bridge the gap between pixels and “meaning” What we see What a computer sees Source: S. Narasimhan
- 11.
- 12. WHY IS COMPUTERVISION CHALLENGING? Viewpoint variations Illumination changes Scale changes Deformation Occlusions Background clutter Motion Intra/Inter-class variations …
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 22. THREE PROCESSING LEVELS (1)Low Level (2) Mid Level (3) High Level
- 23.
- 24. LOW LEVEL VISION- EXAMPLES Corner and blob detection Edge detection
- 25. LOW LEVEL VISION- EXAMPLES • Region segmentation
- 26.
- 27. MID LEVEL VISION- EXAMPLES • 3D Reconstruction
- 28. MID LEVEL VISION- EXAMPLES • Structure (i.e., 3D) from motion 3D teacup model reconstructed from a 240-frame video sequence Optical flow
- 29.
- 30. slide credit: Fei-Fei,Fergus & Torralba Scene Interpretation
- 31. Object categorization sky building flag wall banner bus cars bus face street lamp slidecredit: Fei-Fei, Fergus & Torralba
- 32. Qualitative geometric information slanted rigidmoving object horizontal vertical slide credit: Fei-Fei, Fergus & Torralba rigid moving object non-rigid moving object
- 33. Scene and contextcategorization • outdoor • city • traffic • … slide credit: Fei-Fei, Fergus & Torralba
- 35. COMPUTER VISION APPLICATIONS Industrial inspection/qualitycontrol Surveillance and security Face recognition Gesture recognition Space applications Medical image analysis Autonomous vehicles Virtual reality and much more …...
- 36. INDUSTRIAL COMPUTER VISION(MACHINE VISION) Industrial computer vision systems work really well. Make strong assumptions about lighting conditions Make strong assumptions about the position of objects Make strong assumptions about the type of objects COGNEX
- 37. OPTICAL CHARACTER RECOGNITION (OCR) Digitrecognition, AT&T labs http://yann.lecun.com/exdb/lenet/ • Technology to convert scanned docs to text License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition Automatic check processing
- 38.
- 39. LOGIN WITHOUT APASSWORD… Fingerprint scanners on many new laptops, other devices Face recognition systems now beginning to appear more widely http://www.sensiblevision.com/
- 40.
- 41. How the AfghanGirl was Identified by Her Iris Patterns IRIS BIOMETRICS
- 42.
- 43. APPLICATIONS… AUTOMATED IMAGE CAPTIONING Karpathy, Andrej,and Li Fei- Fei. "Deep visual-semantic alignments for generating image descriptions." arXiv preprint arXiv:1412.2306 (2014).
- 44. MOBILE VISUAL SEARCH: GoogleLens – Search what you see
- 45.
- 46. FACE DETECTION Many newdigital cameras now detect faces Canon, Sony, Fuji, …
- 47.
- 48. SMILE DETECTION? Sony Cyber-shot®T70 Digital Still Camera
- 49. HAND GESTURE RECOGNITION SmartHuman-Computer User Interfaces Sign Language Recognition
- 50. Video Analytics… Human BehaviorAnalysis… Categories of human behavior analysis
- 51. VISUAL SURVEILLANCE AND HUMANACTIVITY RECOGNITION Surveillance and security
- 52. Reverse Engineering ofImages… Image Depth Estimation & Surface Reconstruction Self Driving cars / Robot navigation 3D Printing 3D games/ movies Medical images for Robotic surgery
- 53.
- 54.
- 55. SMART CARS: Visionsystems currently in high-end Tesla, BMW, GM, Volvo models. Mobileye
- 56.
- 57.
- 58. AUTOMATIC 3D RECONSTRUCTIONFROM INTERNET PHOTO COLLECTIONS “Statue of Liberty” 3D model Flickr photos “Half Dome, Yosemite” “Colosseum, Rome”
- 59.
- 60. ROBOTICS http://www.robocup.org/ Semantic Robot VisionChallenge http://www.semantic-robot-vision-challenge.org/ http://www.youtube.com/watch?v=GItjILILB50
- 61. VISION IN SPACE •Vision systems used for several tasks – Panorama stitching – 3D terrain modeling – Obstacle detection, position tracking NASA'S Mars Exploration Rover Spirit
- 62. MOVIE SPECIAL EFFECTS Moviespecial effects • Insert synthetic objects in real image sequences; • Change artificially the position or the orientation of a camera;
- 63. MEDICAL IMAGING Skin/Breast CancerDetection 3D imaging MRI, CT Enable surgeons to visualize internal structures through an automated overlay of 3D reconstructions of internal anatomy on top of live video views of a patient. Image guided surgery Grimson et al., MIT
- 64. Applications… Agriculture & LiveStock Remote sensing of crops and vegetables To explore and develop new ways to improve sustainable food production using satellite technologies.
- 65. Applications… GIS… Earth viewers (3Dmodeling)
- 66. Applications… Content Based ImageRetrieval
- 67. Applications… Forensic Science… Original (leftand middle) and Forged using Splicing (right)
- 68. Forensic Science… How forgedimages effect our society? Counterfeiting (currency, identification, licenses, etc.) Evidence tampering Antique faking (for online shopping) Political propaganda Yellow journalism Scientific research (forging results or observations) DEEP FAKE A never ending research…
- 69. Forensic Science… Machine LearningCreated Fake Images o Use of neural networks to create fake images. o People on the right aren’t real; they’re the product of machine learning. o Generative Adversarial Network (GAN) architecture is used to create these images D. Kim, H.-U. Jang, S.-M. Mun, S. Choi, and H.-K. Lee, "Median filtered image restoration and anti-forensics using adversarial networks", IEEE Signal Processing Letters, vol. 25, pp. 278-282, 2018. Powerful editing tools & GAN images are big challenges https://thispersondoesnotexist.
- 70. Forensic Science… Examples ofreal world forgery Obama is leading (Original) Hosni Mubarak is leading R. D. Berenger, and M. Taha, “Technology disruption theory and Middle East media,” In Proceedings of Association for Education in Journalism and Mass Communication, USA, Chicago, 2012
- 71. Forensic Science… Examples ofreal world forgery Iranian missile test (Original) Removed the failed missile M. Nizza, and P. J. Lyons, "In an Iranian Image, a Missile Too Many," The Lede, The New York Times News Blog, July, 10, 2008.
- 72.
- 73. WHAT SKILLS DOYOU NEED TO SUCCEED IN THIS FIELD? Strong programming skills) Good knowledge of Data Structures and Algorithms Good skills in analyzing algorithm performance (i.e., time and memory requirements). Good background in mathematics, especially in: Linear Algebra Probabilities and Statistics Numerical Analysis