SlideShare a Scribd company logo
1 of 39
Download to read offline
OBJECT TRACKING IN VIDEO
Master Thesis Project of Andrea Ferri
20 th October 2016, UPC, Barcelona
Supervised by Jordi Torres and Xavier Giro ”I” Nieto
Summary
1.Project Overview;
2.Methodology;
3.Project Development;
4.Solved Problems;
5.Running Example;
6.Evaluations;
7.Conclusions;
8.References.
2
Project Overview: Goals
Build a working Model for Object Tracking in Video
Object Detection from Video
3
be the fastest adaptable environment
for Machine Learning,
to implement Research and Development
into a different infrastructure architecture
with a great improvement perspective?”
“Can
4
is an Open Source Software
Library for Machine Intelligence
Work in:
Powered by:
Based on:
5
Goals:
6
Goals:
Environment
NEW (November 9, 2015)
àLess than 1 year
àGreat Potentials of Improvement
àGreat Supported Community
àStill Few Available Components
I exploit the usable projects
at the Best! 7
Goals:
Image Database organized according
to the WordNet hierarchy
Used for the:
ILSVRC
8
Goals:
àVID Challenge:
à 30 Moving Object Classes;
à Specific Datasets Provided:
- Train 3862 Snippets;
- Validation 555 Snippets;
- Test 973 Snippets.
9
10
Neural Network
System of programs and data structures which
approximates the operation of the human
brain.
Model
11
Neural Network
Input Neural
Layer
1°Neural
Layer
Output
Neural
Layer
INPUT
INPUT
INPUT
Last Neural
Layer
OUTPUT
N Neural
Layer
Model
12
Back Propagation
Method for training artificial neural networks
used with an optimization method
13
Back Propagation
INPUT LAYER HIDDEN LAYER/S
INPUT
INPUT
INPUT
ERROR
ERROR BACK PROPAGATION
weights
weights
weights
weights
Goals: Model for Object Tracking in Video
t=0 t=1 t=2 t=3
Class:	Dog	conf 0.78 Class:	Dog	conf 0.59 Class:	Dog	conf 0.34 No	Objects
à For each frame:
I. Detect possible objects;
II. Identify possible detections.
à Track them in time and space.
14
Project Overview: Architecture
Model for Object Tracking in Video
à Per-frame Analysis:
I. Detect possible objects;
II. Identify possible detections.
à In Time & Space Analysis
Still-Image
Approach
Post-Processing
Approach
15
Modular Structure
GENERAL
OBJECT
DETECTOR
POST
PROCESSING
TRACKER
IMAGE
CLASSIFIER
(OBJECT CLC)
AIRPLANES
16
Methodology
à Time Constraints of
à Starting from
Fast Learn
Fast Develop
The Power of
the Community
5 month ß
Scratch ß
17
Project Development
Still-Image
Approach
Post
Processing
Approach
POST
PROCESSING
TRACKER
GENERAL
OBJECT
DETECTOR
IMAGE
CLASSIFIER
(OBJECT CLC)
TensorBox
(GitHub Repo)
Inception
(GitHub Repo)
Python
Implementation
18
Still Image Analysis
GENERAL
OBJECT
DETECTOR
TensorBox (GitHub Repo)
OverFeat Model (Pierre Sermanet et al.)
19
Unbalanced
Trained as Single Class on the 30 VID Classes
Lots of
Peaks
20
Trained as Single Class on the 30 VID Classes
Regular
Curve
21
Inception(GitHub Repo)
Inception V3 Model
(Christian Szegedy et al.)
Well Balanced
IMAGE
CLASSIFIER
(OBJECT CLC)
22
Trained as Multi Class on the 30 VID Classes
Really Smooth
23
Post Processing Analysis
Python Implementation
Based on simplification a of
The Slow and Steady Features Analysis
POST
PROCESSING
TRACKER
- Bounding Boxes
- Object movement
Not a Trainable Model
24
Solved Problems
•Environment Installation;
•Libraries Setting;
•Components Training;
•Components Combination;
•Post Processing Implementation;
•Dataset usage. 25
Results: VID ImageNET Challenge
Team name Entry description
Number of ob ject
categories won
mAP
NUIST
cascaded region
regression + tracking
10 0.808292
NUIST
cascaded region
regression + tracking
10 0.803154
CUVideo
4-model ensemble
(Multi-Context .. &
Motion-Guided .. )
9 0.767981
Trimps-Soushen Ensemble 2 1 0.709651
With Provided Data
26
Results: VID ImageNET Challenge
Team name Entry description
Number of object
categories won
mAP
NUIST
cascaded region
regression + tracking
17 0.79593
NUIST
cascaded region
regression + tracking
5 0.781144
Trimps-Soushen Ensemble 6 5 0.720704
ITLab-Inha
An ensemble for
detection, MCMOT for
tracking
3 0.731471
With Additional Data
27
Results: VID ImageNET Challenge
Team name Entry description mAP
CUVideo 4-model ensemble 0.558557
Tracking + With Provided Data
Team name Entry description
Description of
outside data used
mAP
NUIST
cascaded region
regression +
tracking
proposal network is
fine-tuned from
COCO
0.583898
Tracking + With Additional Data
28
Results: Validation Developed Model
0.002263 mAP
Class mAP Class mAP Class mAP
airplane 0 elephant 0 red panda 0
antelope 0 fox 0 sheep 0.0329
bear 0 giant panda 0 snake 0
bicycle 0 hamster 0 squirrel 0
bird 0 horse 0 tiger 0
bus 0 lion 0 train 0
car 0.0002 lizard 0 turtle 0.0615
cattle 0 monkey 0 watercraft 0.0001
dog 0.0006 motorcycle 0.0219 whale 0
domestic cat 0.1492 rabbit 0 zebra 0
29
Modular Structure
GENERAL
OBJECT
DETECTOR
POST
PROCESSING
TRACKER
IMAGE
CLASSIFIER
(OBJECT CLC)
Evaluations: Developed Model
LOW STARTING
ACCURACY
NOT ENOUGH
TO COMPENSATE
30
Class mAP Class mAP Class mAP
airplane 0 elephant -0.0021 red panda 0
antelope 0 fox +0.0843 sheep -0
bear 0 giant panda 0 snake +0.0214
bicycle 0 hamster 0 squirrel 0
bird 0 horse 0 tiger 0
bus +0.0003 lion 0 train +0.0011
car +0.0019 lizard +0.0001 turtle +0.0991
cattle 0 monkey 0 watercraft +0.0003
dog -0 motorcycle -0 whale +0.0002
domestic cat -0.0006 rabbit +0.0003 zebra 0
LOC Validation Results for the G.O.D.
Best Overlap 31
LOC Validation Results for the G.O.D.
Class mAP Class mAP Class mAP
airplane 0 elephant +0.4077 red panda 0
antelope 0 fox 0 sheep +0.0789
bear 0 giant panda 0 snake -0
bicycle 0 hamster 0 squirrel 0
bird 0 horse 0 tiger 0
bus +0.0014 lion +0.0007 train +0.0056
car +0.0091 lizard 0 turtle +0.4935
cattle +0.0002 monkey 0 watercraft +0.0013
dog -0.0001 motorcycle -0 whale +0.0010
domestic cat -0.0103 rabbit 0 zebra 0
Best Intersection Over Union
32
Change Modules Order
GENERAL
OBJECT
DETECTOR
POST
PROCESSING
TRACKER
IMAGE
CLASSIFIER
(OBJECT CLC)
Evaluations: Possible Improvements
IDENTIFY
OBJECTS
30 SPECIFIC
MODELS
TRAINABLE
MODEL
33
be the fastest adaptable environment
for Machine Learning,
to implement Research and Development,
into a different infrastructure architecture
with a great improvement perspective?
Can
Initial Question
34
Conclusions
35
I Started without any clue about
Deep Learning
And Visual Recognition Topic .
àI Finished implementing
a working model
for Object Tracking in Video.
Conclusions
Yes!
36
demonstrate to be adaptable and
with a great improvement
perspectives.
I think
37
THANKS !
References
• Thesis Project GitHub;
• Tensorbox GitHub;
• YOLO GitHub;
• Inception GitHub;
• TensorFlow.
Andrea Ferri: hause.blackwarhol@gmail.com
38
Questions & Answers
39

More Related Content

Similar to Master Thesis Object Tracking in Video with TensorFlow

502021435-12345678Minor-Project-Ppt.pptx
502021435-12345678Minor-Project-Ppt.pptx502021435-12345678Minor-Project-Ppt.pptx
502021435-12345678Minor-Project-Ppt.pptx
shrey4922
 

Similar to Master Thesis Object Tracking in Video with TensorFlow (20)

Python Raster Function - Esri Developer Conference - 2015
Python Raster Function - Esri Developer Conference - 2015Python Raster Function - Esri Developer Conference - 2015
Python Raster Function - Esri Developer Conference - 2015
 
Introduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and PythonIntroduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and Python
 
Introduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and PythonIntroduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and Python
 
NVIDIA 深度學習教育機構 (DLI): Approaches to object detection
NVIDIA 深度學習教育機構 (DLI): Approaches to object detectionNVIDIA 深度學習教育機構 (DLI): Approaches to object detection
NVIDIA 深度學習教育機構 (DLI): Approaches to object detection
 
運用CNTK 實作深度學習物件辨識 Deep Learning based Object Detection with Microsoft Cogniti...
運用CNTK 實作深度學習物件辨識 Deep Learning based Object Detection with Microsoft Cogniti...運用CNTK 實作深度學習物件辨識 Deep Learning based Object Detection with Microsoft Cogniti...
運用CNTK 實作深度學習物件辨識 Deep Learning based Object Detection with Microsoft Cogniti...
 
Tackling Open Images Challenge (2019)
Tackling Open Images Challenge (2019)Tackling Open Images Challenge (2019)
Tackling Open Images Challenge (2019)
 
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
 
502021435-12345678Minor-Project-Ppt.pptx
502021435-12345678Minor-Project-Ppt.pptx502021435-12345678Minor-Project-Ppt.pptx
502021435-12345678Minor-Project-Ppt.pptx
 
Real time pedestrian detection, tracking, and distance estimation
Real time pedestrian detection, tracking, and distance estimationReal time pedestrian detection, tracking, and distance estimation
Real time pedestrian detection, tracking, and distance estimation
 
Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
 
NVIDIA 深度學習教育機構 (DLI): Medical image segmentation using digits
NVIDIA 深度學習教育機構 (DLI): Medical image segmentation using digitsNVIDIA 深度學習教育機構 (DLI): Medical image segmentation using digits
NVIDIA 深度學習教育機構 (DLI): Medical image segmentation using digits
 
Ultra Fast Deep Learning in Hybrid Cloud Using Intel Analytics Zoo & Alluxio
Ultra Fast Deep Learning in Hybrid Cloud Using Intel Analytics Zoo & AlluxioUltra Fast Deep Learning in Hybrid Cloud Using Intel Analytics Zoo & Alluxio
Ultra Fast Deep Learning in Hybrid Cloud Using Intel Analytics Zoo & Alluxio
 
OpenPOWER Workshop in Silicon Valley
OpenPOWER Workshop in Silicon ValleyOpenPOWER Workshop in Silicon Valley
OpenPOWER Workshop in Silicon Valley
 
Building a Better BaaS
Building a Better BaaSBuilding a Better BaaS
Building a Better BaaS
 
Introduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and PythonIntroduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and Python
 
Practical Digital Image Processing 1
Practical Digital Image Processing 1Practical Digital Image Processing 1
Practical Digital Image Processing 1
 
AI Food detector; A model of Generative adversarial network for food Classifier
AI Food detector; A model of Generative adversarial network for food ClassifierAI Food detector; A model of Generative adversarial network for food Classifier
AI Food detector; A model of Generative adversarial network for food Classifier
 
Automated ML Workflow for Distributed Big Data Using Analytics Zoo (CVPR2020 ...
Automated ML Workflow for Distributed Big Data Using Analytics Zoo (CVPR2020 ...Automated ML Workflow for Distributed Big Data Using Analytics Zoo (CVPR2020 ...
Automated ML Workflow for Distributed Big Data Using Analytics Zoo (CVPR2020 ...
 
Global Azure Bootcamp 2016 - Lyon : Keynote Quantum : Concepts explained & Li...
Global Azure Bootcamp 2016 - Lyon : Keynote Quantum : Concepts explained & Li...Global Azure Bootcamp 2016 - Lyon : Keynote Quantum : Concepts explained & Li...
Global Azure Bootcamp 2016 - Lyon : Keynote Quantum : Concepts explained & Li...
 
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
 

Recently uploaded

01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...
01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...
01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...
AshwaniAnuragi1
 
Artificial intelligence presentation2-171219131633.pdf
Artificial intelligence presentation2-171219131633.pdfArtificial intelligence presentation2-171219131633.pdf
Artificial intelligence presentation2-171219131633.pdf
Kira Dess
 
Scouring of cotton and wool fabric with effective scouring method
Scouring of cotton and wool fabric with effective scouring methodScouring of cotton and wool fabric with effective scouring method
Scouring of cotton and wool fabric with effective scouring method
vimal412355
 

Recently uploaded (20)

HOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptx
HOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptxHOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptx
HOA1&2 - Module 3 - PREHISTORCI ARCHITECTURE OF KERALA.pptx
 
01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...
01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...
01-vogelsanger-stanag-4178-ed-2-the-new-nato-standard-for-nitrocellulose-test...
 
Databricks Generative AI Fundamentals .pdf
Databricks Generative AI Fundamentals  .pdfDatabricks Generative AI Fundamentals  .pdf
Databricks Generative AI Fundamentals .pdf
 
handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailing
 
Autodesk Construction Cloud (Autodesk Build).pptx
Autodesk Construction Cloud (Autodesk Build).pptxAutodesk Construction Cloud (Autodesk Build).pptx
Autodesk Construction Cloud (Autodesk Build).pptx
 
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
 
Theory of Time 2024 (Universal Theory for Everything)
Theory of Time 2024 (Universal Theory for Everything)Theory of Time 2024 (Universal Theory for Everything)
Theory of Time 2024 (Universal Theory for Everything)
 
Computer Graphics Introduction To Curves
Computer Graphics Introduction To CurvesComputer Graphics Introduction To Curves
Computer Graphics Introduction To Curves
 
Signal Processing and Linear System Analysis
Signal Processing and Linear System AnalysisSignal Processing and Linear System Analysis
Signal Processing and Linear System Analysis
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
 
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptxS1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
 
What is Coordinate Measuring Machine? CMM Types, Features, Functions
What is Coordinate Measuring Machine? CMM Types, Features, FunctionsWhat is Coordinate Measuring Machine? CMM Types, Features, Functions
What is Coordinate Measuring Machine? CMM Types, Features, Functions
 
Introduction-to- Metrology and Quality.pptx
Introduction-to- Metrology and Quality.pptxIntroduction-to- Metrology and Quality.pptx
Introduction-to- Metrology and Quality.pptx
 
Basics of Relay for Engineering Students
Basics of Relay for Engineering StudentsBasics of Relay for Engineering Students
Basics of Relay for Engineering Students
 
Artificial intelligence presentation2-171219131633.pdf
Artificial intelligence presentation2-171219131633.pdfArtificial intelligence presentation2-171219131633.pdf
Artificial intelligence presentation2-171219131633.pdf
 
Convergence of Robotics and Gen AI offers excellent opportunities for Entrepr...
Convergence of Robotics and Gen AI offers excellent opportunities for Entrepr...Convergence of Robotics and Gen AI offers excellent opportunities for Entrepr...
Convergence of Robotics and Gen AI offers excellent opportunities for Entrepr...
 
History of Indian Railways - the story of Growth & Modernization
History of Indian Railways - the story of Growth & ModernizationHistory of Indian Railways - the story of Growth & Modernization
History of Indian Railways - the story of Growth & Modernization
 
Fundamentals of Structure in C Programming
Fundamentals of Structure in C ProgrammingFundamentals of Structure in C Programming
Fundamentals of Structure in C Programming
 
Fundamentals of Internet of Things (IoT) Part-2
Fundamentals of Internet of Things (IoT) Part-2Fundamentals of Internet of Things (IoT) Part-2
Fundamentals of Internet of Things (IoT) Part-2
 
Scouring of cotton and wool fabric with effective scouring method
Scouring of cotton and wool fabric with effective scouring methodScouring of cotton and wool fabric with effective scouring method
Scouring of cotton and wool fabric with effective scouring method
 

Master Thesis Object Tracking in Video with TensorFlow