The document presents a novel convolutional neural network (CNN) and TensorFlow-based approach for moving object detection and tracking, addressing limitations in traditional methods like background subtraction and optical flow. The proposed algorithm consists of a two-phase system, focusing on object detection through a pre-trained TensorFlow model and subsequent tracking of identified objects with high accuracy metrics (sensitivity of 92.14%, specificity of 91.24%, and overall accuracy of 90.88%). It highlights the efficiency of the CNN model in tracking objects at 150 frames per second and provides quantitative analysis of performance metrics.

1. Moving Object Detection And Tracking
Using Convolutional Neural Networks
April 13, 2021
Presented by-
Nitish Kumar (2011EE09)
Jeny Khan (2011EE06)
Submitted to:
Dr. Maheshkumar H.Kolekar
Associate Professor
Department of Electrical Engineering
Indian Institute of Technology Patna, Bihar

2. Contents
Motivation
introduction
CNN
TensorFlow Object Detection API
Methodology
Object Detection Algorithm
Object Tracking Algorithm
Results of the proposed algorithm
Quantitative Analysis
References
Moving Object Detection And Tracking Using Convolutional Ne

3. Motivation
The background subtraction is affected by mostly
non-stationary background and illumination changes.
This drawback can be removing by the optical flow algorithm
but it is produces false alarm for tracking algorithms under
cluttered conditions.
In most of the cases of background subtraction, the object
trackers are influenced by background information but it lead
to the misclassification.
To overcome this limitation, in this approach a novel and
generalized Tensor flow based object detection and CNN
based object tracking algorithm has been presented.
Moving Object Detection And Tracking Using Convolutional Ne

4. Convolutional Neural Network
A convolutional neural network (CNN, or ConvNet) is a class
of deep neural networks, most commonly applied to analyzing
visual imagery.
Architecture:
Figure: CNN Architecture
A convolutional neural network consists of an input layer,
hidden layers and an output layer. In any feed-forward neural
network, any middle layers are called hidden because their
inputs and outputs are masked by the activation function and
final convolution.
Moving Object Detection And Tracking Using Convolutional Ne

5. CNN Cont’d
Convolutional layers:
Convolutional layers convolve the input and pass its result to the
next layer.
Figure: Convolutional layers
Moving Object Detection And Tracking Using Convolutional Ne

6. CNN Cont’d
Pooling layers:
Pooling layers reduce the dimensions of data by combining the
outputs of neuron clusters at one layer into a single neuron in the
next layer.
There are two common types of pooling in popular use: max and
average.
Figure: Max pooling
Moving Object Detection And Tracking Using Convolutional Ne

7. TensorFlow Object Detection API
The TensorFlow object detection API is the framework for
creating a deep learning network that solves object detection
problems.
There are already pretrained models in their framework which
they refer to as Model Zoo. This includes a collection of
pretrained models trained on the COCO dataset, the KITTI
dataset, and the Open Images Dataset.
AP is averaged over all categories. Traditionally, this is called
“mean average precision” (mAP).
Moving Object Detection And Tracking Using Convolutional Ne

8. METHODOLOGY
The proposed CNN based moving object detection algorithm
consists of two phase: Object detection and tracking.
The generalized block diagram of the proposed system is
shown in Fig:
Figure: Block Diagram of proposed system
In this system, the video is feed to the system as an input.
Frames are extracted for further processing.
Moving Object Detection And Tracking Using Convolutional Ne

9. Object Detection Algorithm
The object detection is explained in detail in below flow:
Figure: TensorFlow Based Object detection flowchart
Moving Object Detection And Tracking Using Convolutional Ne

10. Object Detection Algorithm Cont’d
TensorFlow based object detection API is an open source
platform which make simple to construct, train and detection
models.
firstly the necessary libraries are imported then import the
pre-trained object detection model.
The weights are initializing along with box and tensor class.
After initialization of all the parameters of the tensor flow
model, the image in which object to be detected is read.
Apply the loaded tensor flow model on the image, the
TensorFlow based model test the image and return the
location (x, y, w, h) of the object in the image.
The success rate of this approach is better and it is applicable
to RGB images.
Moving Object Detection And Tracking Using Convolutional Ne

11. Object Tracking Algorithm
Figure: Flowchart for Object Tracking
Moving Object Detection And Tracking Using Convolutional Ne

12. Object Tracking Algorithm Cont’d
After detecting the object, their locations are important to
start the tracking process.
For tracking to be robust, requires object knowledge and
understanding like motion and its variation over time. Tracker
must be able to its model and adopted for new observations.
The model is capable of incorporating the temporal
information. Rather than focusing on the objects in the
testing time, the pre-trained model which is trained on large
variety of objects in real time.
This lightweight model has ability to track the object at the
speed of 150 frames per second.
The initial positions are learned by the model and the same
points are search in the net frames by testing process of CNN
model.
Moving Object Detection And Tracking Using Convolutional Ne

13. Results of the proposed algorithm (cdv sequence)
Moving Object Detection And Tracking Using Convolutional Ne

14. Results of the proposed algorithm (mdv sequence)
Moving Object Detection And Tracking Using Convolutional Ne

15. Quantitative Analysis
The quantitative analysis is performed using sensitivity, specificity
and accuracy parameter. These parameters are calculated using
True Positive (TP), True Negative (TN), False Positive (FP)
and False Negative (FN).
TP: moving object correctly identified moving object.
FP: Stationary object incorrectly identified as moving object
TN: Stationary object correctly identified as Stationary object
FN: moving object incorrectly identified as Stationary object
Moving Object Detection And Tracking Using Convolutional Ne

16. Quantitative Analysis Cont’d
The mathematical representation of the quality metrics is given as:
Sensitivity: It is the ratio of truly object present in the scene
who are correctly identify as an object.
Sensitivity =
TP
TP + FN
Specificity: It is the ratio of truly stationary object present in
the scene that are correctly identify as a stationary object.
Specificity =
TN
TN + FN
Accuracy: Accuracy is the overall performance of the system
including sensitivity and specificity.
Accuracy =
TP + TN
TP + TN + FP + FN
Moving Object Detection And Tracking Using Convolutional Ne

17. Quantitative Analysis Cont’d
CONCLUSION:
The proposed approach achieves the sensitivity of 92.14%,
specificity of 91.24% and accuracy of 90.88%.
The moving object detection is performed using TensorFlow
object detection API. The object detection module robustly
detects the object. The detected object is tracked using CNN
algorithm.
Moving Object Detection And Tracking Using Convolutional Ne

18. References
Mane, Shraddha, and Supriya Mangale. "Moving object
detection and tracking using convolutional neural networks."
2018 Second International Conference on Intelligent
Computing and Control Systems (ICICCS). IEEE, 2018.
Chen, Y, X. Yang, B. Zhong, S. Pan, D. Chen, and H. Zhang,
“Cnn tracker: Online discriminative object tracking via deep
convolutional neural network”. Applied Soft Computing, 2016.
Junda Zhu, Yuanwei Lao, and Yuan F. Zheng, “Object
tracking in structured environment for video surveillance
applications”, IEEE transactions on circuits and systems for
video technology, vol.20, February 2010.
Moving Object Detection And Tracking Using Convolutional Ne

19. Open to ask questions...!
Moving Object Detection And Tracking Using Convolutional Ne