RGPV 8SEM Open Elective Note CSE Branch

1. CS-803
803(A) Image Processing and Computer Vision#
Subject In-charge :
Prof Shilpa Sharma
Asst. Prof. CSE / IT Department
MITM, Ujjain (M.P.)

2.  Understand practice and theory of computer vision.
 Elaborate computer vision algorithms, methods and
concepts
 Implement computer vision systems with emphasis
on applications and problem solving

3.  Apply skills for automatic analysis of digital images
to construct representations of physical objects and
scenes.
 Design and implement real-life problems using
Image processing and computer vision.

4.  Reference Text
 1. Robert Haralick and Linda Shapiro, "Computer
and Robot Vision", Vol I, II, Addison-
Wesley, 1993
 2. David A. Forsyth, Jean Ponce, "Computer Vision:
A Modern Approach" Pearson
 3. Milan Sonka,VaclavHlavac, Roger Boyle, "Image
Processing, Analysis, and Machine Vision" Thomson
Learning.

5.  Introduction to computer vision and Image
processing (CVIP): Basics of CVIP, History of CVIP,
 Evolution of CVIP, CV Models, Image Filtering,
Image Representations, Image StatisticsRecognition
Methodology: Conditioning, Labeling, Grouping,
Extracting, and Matching,
 Morphological Image Processing: Introduction,
Dilation, Erosion, Opening, Closing,

6.  Hit-or-Miss transformation, Morphological algorithm
operations on binary images,
 Morphological algorithm operations on gray-scale
images, Thinning, Thickening,
 Region growing, region shrinking.

7.  Computer vision is concerned with modeling and
replicating human vision using computer software
and hardware.
 Formally if we define computer vision then its
definition would be that computer vision is a
discipline that studies how to reconstruct, interrupt
and understand a 3d scene from its 2d images in
terms of the properties of the structure present in
scene.

9. Computer Vision
Make computers understand images and video.
What kind of
scene?
Where are the cars?
How far is the
building?

10. Vision is really hard
• Vision is an amazing feat of natural intelligence
– Visual cortex occupies about 50% of Macaque brain
– More human brain devoted to vision than anything
else
Is that a
queen or a
bishop?

11. Why computer vision matters
Safety Health Security
Comfort Access
Fun

12. brief history of computer vision
• 1966: Minsky assigns computer
vision as an undergrad summer
project
• 1960’s: interpretation of synthetic
worlds
• 1970’s: some progress on
interpreting selected images
Guzman ‘68
Ohta Kanade ‘78
Turk and Pentland ‘91

13. • 1980’s: ANNs come and go; shift toward
geometry and increased mathematical rigor
• 1990’s: face recognition; statistical analysis in
vogue
• 2000’s: broader recognition; large annotated
datasets available; video processing starts

14.  It needs knowledge from the following fields in order to
understand and stimulate the operation of human vision
system.
 Computer Science
 Electrical Engineering
 Mathematics
 Physiology
 Biology
 Cognitive Science

15.  Computer vision is divided into three basic categories
that are as following:
 Low-level vision: includes process image for feature
extraction.
 Intermediate-level vision: includes object recognition
and 3D scene Interpretation
 High-level vision: includes conceptual description of
a scene like activity, intention and behavior.

16.  Computer Vision overlaps significantly with the
following fields:
 Image Processing: it focuses on image manipulation.
 Pattern Recognition: it studies various techniques to
classify patterns.
 Photogrammetry: it is concerned with obtaining
accurate measurements from images.

17.  Image processing studies image to image
transformation. The input and output of image
processing are both images.
 Computer vision is the construction of explicit,
meaningful descriptions of physical objects from
their image. The output of computer vision is a
description or an interpretation of structures in 3D
scene.

18. Computer Vision and Nearby
Fields
• Computer Graphics: Models to Images
• Comp. Photography: Images to Images
• Computer Vision: Images to Models

19.  1) Robotics
 2) Medicine
 3) Security
 4) Transportation
 5) Industrial Automation

20.  Localization-determine robot location automatically
 Navigation
 Obstacles avoidance
 Assembly (peg-in-hole, welding, painting)

21.  Manipulation (e.g. PUMA robot manipulator)
 Human Robot Interaction (HRI): Intelligent robotics
to interact with and serve people

22.  Classification and detection (e.g. lesion or cells
classification and tumor detection)
 2D/3D segmentation
 3D human organ reconstruction (MRI or ultrasound)
 Vision-guided robotics surgery

23.  Biometrics (iris, finger print, face recognition)
 Surveillance-detecting certain suspicious activities or
behaviors

24.  Autonomous vehicle
 Safety, e.g., driver vigilance monitoring

25.  Industrial inspection (defect detection)
 Assembly
 Barcode and package label reading
 Object sorting
 Document understanding (e.g. OCR)

26. How vision is used now
• Examples of state-of-the-art

27. Optical character recognition (OCR)
Digit recognition, AT&T labs
http://www.research.att.com/~yann/
Technology to convert scanned docs to text
• If you have a scanner, it probably came with OCR software
License plate readers
http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

28. Face detection
• Many new digital cameras now detect
faces
– Canon, Sony, Fuji, …

29. Smile detection
Sony Cyber-shot® T70 Digital Still Camera

30. 3D from thousands of images

31. Object recognition (in
supermarkets)
LaneHawk by EvolutionRobotics
“A smart camera is flush-mounted in the checkout lane, continuously
watching for items. When an item is detected and recognized, the
cashier verifies the quantity of items that were found under the basket,
and continues to close the transaction. The item can remain under the
basket, and with LaneHawk,you are assured to get paid for it… “

32. Vision-based biometrics
“How the Afghan Girl was Identified by Her Iris Patterns” Read the story
wikipedia

33. Login without a password…
Fingerprint scanners on
many new laptops,
other devices
Face recognition systems now
beginning to appear more widely
http://www.sensiblevision.com/

34. Object recognition (in mobile
phones)
Point & Find, Nokia
Google Goggles

35. The Matrix movies, ESC Entertainment, XYZRGB, NRC
Special effects: shape capture

36. Pirates of the Carribean, Industrial Light and Magic
Special effects: motion capture

37. Sports
Sportvision first down line
Nice explanation on www.howstuffworks.com
http://www.sportvision.com/video.html

38. Smart cars
• Mobileye
– Vision systems currently in high-end BMW,
GM, Volvo models
– By 2010: 70% of car manufacturers.
Slide content courtesy of Amnon Shashua

39. Google cars
http://www.nytimes.com/2010/10/10/science/10google.html?ref=artificialintelligence

40. Interactive Games: Kinect
• Object Recognition:
http://www.youtube.com/watch?feature=iv&v=fQ59dXOo
63o
• Mario: http://www.youtube.com/watch?v=8CTJL5lUjHg
• 3D: http://www.youtube.com/watch?v=7QrnwoO1-8A
• Robot:
http://www.youtube.com/watch?v=w8BmgtMKFbY

41. Vision in space
Vision systems (JPL) used for several tasks
• Panorama stitching
• 3D terrain modeling
• Obstacle detection, position tracking
• For more, read “Computer Vision on Mars” by Matthies et al.
NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.

42. Industrial robots
Vision-guided robots position nut runners on wheels

43. Mobile robots
http://www.robocup.org/
NASA’s Mars Spirit Rover
http://en.wikipedia.org/wiki/Spirit_rover
Saxena et al. 2008
STAIR at Stanford

44. Medical imaging
Image guided surgery
Grimson et al., MIT
3D imaging
MRI, CT