This document summarizes an object detection, tracking, classification, and counting project. The project involved using video from cameras to: 1) Detect objects in video frames using background subtraction and blob analysis. Kalman filters were then used to track objects across frames and reduce noise. 2) Classify objects by color and count them. Shadow detection methods like Gaussian smoothing and thresholding were also applied to filter out shadows. 3) The project aimed to synchronize object counts passing over a bridge with strain gauge and accelerometer readings, to study pedestrian impacts. The document outlines the full algorithm and issues like noise, shadows and tracking.

1. Object Detection,
Tracking, Classification,
and Counting
SHOUNAK MITRA
ADVISOR: PROFESSOR TAT. S. FU, PHD, P.E.
CIVIL AND ENVIRONMENTAL ENGINEERING DEPARTMENT
UNIVERSITY OF NEW HAMPSHIRE
1

2. Overview
 Significance of the project
 Use of camera angles
 Video Demonstration
 Object Detection
 Noise and Shadow issues
 Tracking
 Classification
 Counting
2

3. Project Significance:
 Pedestrian Detection and Counting
 Synchronization of the Objects passing
over the bridge and the readings of
strain gauges and accelerometers.
3
Clip Obtained from Prof. Bell’s lab
(Travis and Griggs)

4. Camera angles obtained from DOT

5. Algorithm 5
Read
Video File
Background
Separation/
Foreground
Detection
Foreground
Filtration
Blob
Analysis
Detect
Boxes
Noise and
Shadow
Issues
Classification
and
Counting
Detection Phase
Processing PhaseFinal Phase

6. Object Detection Flow
VIDEO FRAME FOREGROUND DETECTION
FOREGROUND FILTRATION OBJECT DETECTION AND COUNTING
6

7. Noise Removal
 Preprocessing and Thresholdng:
 Deleting boxes formed at unexpected
locations
 Kalman Filter
7

8. What is Kalman Filter?
 A Kalman filter is an optimal recursive data processing algorithm
 The Kalman filter incorporates all information that can be provided
to it. It processes all available measurements, regardless of their
precision, to estimate the current value of the variables of interest
 Computationally efficient due to its recursive structure
 Assumes that variables being estimated are time dependent
8

9. What does it do?
 Predictor: predicts parameter values ahead of current
measurements
 Noise Reduction: reduces noise introduced by inaccurate
detections
 Tracking: Facilitates the process of association of multiple objects to
their tracks
9

10. Kalman Filter AKA Predictor - Corrector
(1) Project the state ahead
xˆ-
k = Axˆk – 1 + Buk – 1
(2) Project the error covariance ahead
P-
k = APk – 1 AT + Q
10
Measurement Update (“Correct”)
(1) Compute the Kalman gain
K. k = P-
kHT (HP-
kHT + R)–1
(2) Update estimate with measurement zk
xˆk = xˆ-
k + Kk(zk – Hxˆ-
k )
(3) Update the error covariance
Pk = (I – KkH )P-
k
 Time Update State: Responsible for projecting forward in time the current state
and the error covariance estimates to obtain the a priori estimates for the next
time step.
 Measure update state: Responsible for feedback, i.e. for incorporating a new
measurement into the a priori estimate to obtain an improved a posteriori
estimate.

11. Tracking using Kalman Filter 11

12. The Problem of Shadow 12
 Object Misclassification
 Overlapping of Objects
Shadow
Region

13. Shadow Detection Flow
YES NO
13
Specify
Threshold for
Shadow (Sth)
Get Current
Frame Fn
Store
Background
Frame B
Apply Gaussian
Smoothening
(GB & GFn)
Dn =
B/Fn < 1
Multiply by a
factor > 20
(RDn)

14. Shadow Detection Flow
Background in RGB Scale Background in Gray Scale Foreground in Gray Scale
Unfiltered Shadow detection Thresholding of Shadow Filtered Shadow Binary Scale
14

15. Classification
 Color coded classification
 Centroid lying in the color
15

16. Color Coded Classification and
counting
16
DELETED
SHADOW
REGION

17. Demonstration Video 17