The document discusses object tracking using deep learning. It defines object tracking as locating an object across consecutive video frames. It notes applications in security, road safety, and entertainment. Object tracking differs from object detection in that the object class is unknown during training and tracking considers objects across time rather than individual frames. Challenges include objects leaving the screen or changing pose. The document discusses metrics for evaluating trackers, including accuracy and robustness, and surveys popular modern trackers.

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
ObjectTracking with Deep Learning

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
What is object tracking?

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
What is object tracking?
𝑡0: Define target as bounding box of object to track
𝑡 𝑛: Find target in current frame

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Applications: Security and road Safety

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Applications: Sport and entertainment

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Difference with object detection
Class agnostic tracking: the object is not known at training time
Tracking across time: object detection usually apply only to still
frames
Real-time constraints: most trackers are designed for real-time

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Specific challenges
Object goes off-screen Pose change

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Evaluation Metrics: Accuracy
Average overlap between the predicted and ground truth
bounding boxes during successful tracking periods

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Evaluation Metrics: Accuracy
Average overlap between the predicted and ground truth
bounding boxes during successful tracking periods

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Evaluation Metrics: Average Expected Overlap
Average of the average overlaps on a fixed short sequence
length

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Evaluation Metrics: Robustness
How many times the tracker drifted from the target and
had to be reset

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Evaluation Metrics: Robustness
How many times the tracker drifted from the target and
had to be reset

13. Tracker Survey
The Visual Object Tracking VOT2017 Challenge Result,
Kristan et al., 2017

14. Latest (implemented with MXNet)
Siamese Attentional Keypoint Network for High Performance Visual Tracking,
Gao et al., 2019

15. Simple Baseline using Object Detector
SORT: Simple, online, and realtime tracking of multiple objects in a video sequence,
Bewley et al. 2016
- Detect objects using FasterRCNN
- Estimate state of objects (velocity)
- Detect objects using FasterRCNN
- Match previous frame objects with current based on estimated velocity of bounding box

16. Fully-Convolutional Siamese Networks for ObjectTracking
Bertinetto et al., 2016

17. Fully-Convolutional Siamese Networks for ObjectTracking
Bertinetto et al., 2016

18. Fully-Convolutional Siamese Networks for ObjectTracking
Bertinetto et al., 2016

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Go build!
Siamese FC in MXNet Gluon
https://github.com/forschumi/siamfc-mxnet