This document discusses real-time image processing. It begins with an introduction and definitions of real-time and non-real-time processing. It then discusses the requirements for a real-time image processing platform, including high resolution/frame rate video input and low latency. The document outlines some advantages of real-time image processing such as immediate results and automation. It then provides an overview of an object detection system using Viola-Jones detection with integral images, AdaBoost learning, and a cascade classifier structure. Experimental results show the cascade classifier can detect faces in real-time.

3. What is the problem?
What is the RTIP?
Why are we making RTIP?
What we are going to do?

6.  Introduction
 Literature survey
 Requirements
 Difference between real and non real time processing
 General detection system
 System design
 Advantages
 Disadvantages
 Applications
 Future scope
 Definitions in RTIP in different sense
 References

7. Introduction
•What is Real-time Image Processing?
•How it differs from ordinary Image
Processing?
•What is the need for Real time image
processing?

8. Real time processing Non real time processing
 Is continuous
 Also called interactive
processing
 Have deadlines
 Have predictable response
time
 With soft real-times, missing
a deadline indicates that the
system is not working at its
peak
 Is non continuous
 Also called as batch
processing
 Not have deadlines
 Extended over time period
 In batch processing missed
deadlines might mean that
the computer needs more
processing capacity to finish
tasks.

10. When the processing of sound is first calculated
entirely and saved to an audio file (which can be
listened to later) one speaks of non-realtime or
offline synthesis.

11. When the stream of data goes directly to the
audio interface as it is processed, so that there are
only few milliseconds between the processing
and the listening of the synthesized sound, one
speaks of realtime synthesis.

12. Real-Time Image Processing Platform
Requirements:
• High resolution, high frame rate video
input
• Low latency video input
• Low latency operating system scheduling
• High processing performance

13.  Image processing attempts to extract
information from the outside world through
its visual appearance. Therefore adequate
information must be provided to the
processing algorithm by the video input
hardware.
 Broadcast video provides a practical reference
point as most cameras provide images in
formats derived from broadcast standards
regardless of their computer interface
(analog, USB etc).

14. 1. Customer can see the results immediately.
2. Allows you to automate your business. This
is especially important if your time is very
limited.
3. Real-time image processing also helps
eliminate customer errors.
4. Real-time image processing is fast .
5. Real-time image processing is continuous.

15. SYSTEM
GSM MODULE

18. • Binary images {0,1}
• Greyscale Image : [0,1]
• RGB images (Truecolour Image) :

23.  IMAGE ACQUISITION TOOLBOX
 IMAGE PROCESSING TOOLBOX
 COMPUTER VISION TOOLBOX

25.  Introduction
 Image Representation – Integral Image
 Learning Algorithm – AdaBoost
 Cascade Classifier
 Experiment and Result

26. Object Detection
Detection System
Introduction

27. Paul Viola Michael J. Jones
Introduction

28. 24x2
4
Detectio
n
Window
The size of
detection
window will be
enlarged with a
certain scale
Introduction

29. Integral Image
Integral Image Feature
Image
Representation

30. • Image representation
• Feature
• The integral image at
location x,y contains
the sum of the pixels
above and to the left
of x,y
Integral Image | AdaBoost | Cascade
Sum of pixel
value in this
area
(x,y)

31. • Image representation
• Feature
• The integral image at
location x,y contains
the sum of the pixels
above and to the left
of x,y
Integral Image | AdaBoost | Cascade
Sum of pixel
value in this
area
(x,y)
How to calculate D ?
A B
C D
1 2
3 4

32. • Image representation
• Feature
Integral Image | AdaBoost | Cascade
How to calculate D ?
D = (1+4) –
(2+3)
A B
C D
1 2
3 4
1
2
3
4

33. Integral Image | AdaBoost | Cascade
Detection window
A
B
C
D
Detection window
Detection window Detection window
The value of integral image
feature is the difference
between the sum of the
pixels within the two
rectangular regions.
S1
S2
|S1-S2|

34. Integral Image | AdaBoost | Cascade
The size and position of feature can
be different.
• The eye region is
darker than the
upper-cheeks.
• The nose bridge
region is brighter
than the eyes.

35. • Different size and position of integral image
give so many possible features
• How do we obtain the best representing
features possible?
• How can we refrain from wasting time on
image background? (i.e. non-object)
Integral Image | AdaBoost | Cascade
Good Feature Not Good
Feature
AdaBoost…

36. Weak Classifier
Strong Classifier
AdaBoost
AdaBoost

37. • Consist of just 1 feature
• 1 feature that can separate the image
data with error rate less than 0.5
Feature 1
Feature 2
Feature 3
θ 1
θ 2
θ 3
Weak
Classifier 1
Weak
Classifier 2
Weak
Classifier 3
1/10 = 0.1
6/10 = 0.6
3/10 = 0.3
Integral Image | AdaBoost | Cascade

38. • Set of features/weak classifier
Integral Image | AdaBoost | Cascade
 1 1 1
strong
1
1 ( ) ( )
( ) 2
0 otherwise
n n nh h
h

        
 

x x
x
K K
Weak Classifier
How to build the strong classifier?
AdaBoost

39. • T = number of feature we choose
• h(x, f,p,θ)) = A weak classifier
• f = feature
• θ = threshold
• p = polarity indicating the
direction of the inequality (less
than or greater than θ)
Integral Image | AdaBoost | Cascade

40.  AdaBoost starts with a uniform
distribution of “weights” over training
examples.
 Select the classifier with the lowest
weighted error (i.e. a “weak” classifier)
 Increase the weights on the training
examples that were misclassified.
 (Repeat)
 At the end, carefully make a linear
combination of the weak classifiers
obtained at all iterations.
 1 1 1
strong
1
1 ( ) ( )
( ) 2
0 otherwise
n n nh h
h

        
 

x x
x
K K
Slide taken from a presentation by Qing Chen, Discover Lab,
University of Ottawa
Integral Image | AdaBoost | Cascade

41. • By increasing the number of features per classifier,
we:
◦ Increase detection accuracy.
◦ Decrease detection speed.
• Experiments showed that a 200 feature
classifier makes a good face detector:
◦ Takes 0.7 seconds to scan an 384 by 288 pixel
image.
• Problem: Not real time! (At most 0.067 seconds
needed).
Integral Image | AdaBoost | Cascade
Cascade of
Classifier…

42. Cascade
Cascade Classfifier

43. • Classifier is structured as several
layer/hierarchy
• Each layer consist of 1 strong classifier (set of
weak classifiers/features)
• Evaluating all input window
• Ignore non-face window
• Continue for processing “suspected” face
window
Integral Image | AdaBoost | Cascade

44. • The number of feature/weak classifier in
every layer is based on the value of false
positive rate and detection rate determined
by user
• Previous layers have simpler classifier than
next layer
Integral Image | AdaBoost | Cascade
Stage 1 Stage 2 Stage 3 …
Further
Processing
Input Image
Window
Ignore/Rej
ect Input
Ignore/Rej
ect Input
Ignore/Rej
ect Input
Ignore/Rej
ect Input
Y
N
Y
N
Y
N
Y
N

45. Integral Image | AdaBoost | Cascade
Stage 1
(2 feature)
N
Y Stage 2
(2++
feature)

46. Integral Image | AdaBoost | Cascade

47. • Training set
• Face: 4916 labeled frontal faces
• Testing / Real-time test
• Face: MIT + CMU frontal face test set
• Contain: 130 images with 507 labeled frontal faces

48. Cascade Classifier
v.s
Non Cascade
Classifier
Cascade classifier
nearly 10x faster.

49. They use dataset MIT+CMU with 5 images removed
Voting (Using 3 Viola-Jones Classifier with different
parameter)
++- : Face
--+ : Non Face

50. • Informal observation suggests that the face
detector can detect faces that are tilted up to
about ±15 degrees in plane and about ± 45
degrees out of plane
• Harsh backlighting in which the faces are very
dark while the background is relatively light
sometimes causes failures
• Proposed algorithm will also fail on
significantly occluded faces

51.  imaqfind
 imaqhwinfo
 Imaqreset
 Stop
 videoinput
 imaq.VideoDevice

52.  imread Read image from graphics file
 imwrite Write image to graphics file
 imfinfo Information about graphics file
 gray2ind Convert grayscale or binary image
to indexed image
 ind2gray Convert indexed image to
grayscale image
 mat2grayConvert matrix to grayscale image
 rgb2gray Convert RGB image or colormap
to grayscale
 imshow Display image

53.  detector = vision.CascadeObjectDetector

54.  imread- To read an image into Matlab.
 imshow- To show image in a figure.
 imsave- To save the image.
 imwrite(image,‘filename.type’)- To save the
image.
 imageinfo- For image information
 imaqhwinfo- For Information about available
image acquisition hardware

55.  videoinput- Create video input object
 vision.CascadeObjectDetector- Detect objects
using the Viola-Jones algorithm
 preview- To show the preview of camera
 getsnapshot- Take snapshot
 step- To apply algorithm on image
 insertObjectAnnotation- Insert annotation on
detected object

56. a=videoinput('winvideo',1);
preview(a);
i=getsnapshot(a);
imshow(i);

57. FaceDetector=vision.CascadeObjectDetector()
;
BBox=step(FaceDetector,i);
qw=insertObjectAnnotation(i,'rectangle',BBox,
'face');
imshow(qw)

58.  [a,b]=imread('kid
s.tif');
 b(:,1:2)=0;
 imshow(a,b)
 impixelinfo
 figure
 [a,b]=imread('kid
s.tif');
 b(:,1)=0;
 b(:,3)=0;
 imshow(a,b)
 impixelinfo
 figure
 [a,b]=imread('kids.
tif');
 b(:,3)=0;
 b(:,2)=0;
 imshow(a,b)
 impixelinfo
 figure
 [a,b]=imread('kids.
tif');
 imshow(a,b)
 impixelinfo

60. clc
clear
a=videoinput('winvideo',2);
fd=vision.CascadeObjectDetector;
i=getsnapshot(a);
b=step(fd,i);
ia=insertObjectAnnotation(i,'rectangle
',b,'detected faces');
imshow(ia);