Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Video analytics at
scale: DL, CV, ML on
Databricks platform
Claudiu Barbura
Director of Engineering
Blueprint Technologies
Agenda
§ Live demo of Video
Analytics solution
§ Solution Architecture
§ Lessons learned
▪ Infrastructure (gpu vs cpu)
▪ O...
Live Demo of Nash from Azure Marketplace
Lessons learned: Infrastructure
Infrastructure
• 1 Linux VM + ADB Cluster + BLOB
• Shift ALL compute from VM to ADB (Res Mgr is key)
• Video generation (f...
Video processing time at price parity
Video processing time at price parity
Lessons learned: Object detection and tracking
Object detection and tracking
• From fasterrcnn_resnet_50 (mAP 37, 21fps) to faster and more accurate efficientdet-d3 (mAP...
Lessons learned: Behavioral patterns, anomalies
Anomaly Detection
• Vehicle Trajectory Anomaly Detection uses tracking for common, average and rare paths
• 'cluster repre...
Behavioral patterns
• Time-scale based smoothing algorithms
• Low-pass filter/Fourier Series/Numpy.hanning
Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.
Video Analytics At Scale: DL, CV, ML On Databricks Platform
Video Analytics At Scale: DL, CV, ML On Databricks Platform
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

Video Analytics At Scale: DL, CV, ML On Databricks Platform

Download to read offline

Live demo and lessons learned building and publishing an advanced video analytics solution in the Azure Marketplace. This is a deep technical dive into the engineering and data science employed throughout, with all challenges encountered by combining Deep Learning and Computer Vision for object detection and tracking, the operational management and tool building efforts for scaling the video processing and insights extraction to large GPU/CPU Databricks clusters and the machine learning required to detect behavioral patterns, anomalies and scene similarities across processed video tracks.

The entire solution was build using open source scala, python, spark 3.0, mxnet, pytorch, scikit-learn as well as Databricks Connect.

  • Be the first to like this

Video Analytics At Scale: DL, CV, ML On Databricks Platform

  1. 1. Video analytics at scale: DL, CV, ML on Databricks platform Claudiu Barbura Director of Engineering Blueprint Technologies
  2. 2. Agenda § Live demo of Video Analytics solution § Solution Architecture § Lessons learned ▪ Infrastructure (gpu vs cpu) ▪ Object detection & tracking ▪ Behavioral patterns, anomalies § Q&A
  3. 3. Live Demo of Nash from Azure Marketplace
  4. 4. Lessons learned: Infrastructure
  5. 5. Infrastructure • 1 Linux VM + ADB Cluster + BLOB • Shift ALL compute from VM to ADB (Res Mgr is key) • Video generation (ffmpeg) bottleneck • Custom docker image (+opencv, pytorch, sklearn …) pushed to ADB cluster (DCS) • CPU arch optimization: torch.set_num_threads(1) (2-5x) • GPU arch optimization: mxnet->pytorch, cuda.benchmark=true, torch.Tensor(value, device='cuda') • Object Detection model as Spark Broadcast variable (2-4x) • Spark 3.0 GPU-aware scheduling to avoid GPU OOM • ADB cluster: GPU vs CPU perf comparison at price parity (3.77x) • 3 x GPU vs 10 x CPU (80 cores) • GPU: 1x Tesla V100 16GB + 6c CPU/112GB(too much!)
  6. 6. Video processing time at price parity
  7. 7. Video processing time at price parity
  8. 8. Lessons learned: Object detection and tracking
  9. 9. Object detection and tracking • From fasterrcnn_resnet_50 (mAP 37, 21fps) to faster and more accurate efficientdet-d3 (mAP 46.8, 22.7 fps) • efficientdet-d3 detector trained on the MS-COCO dataset, classes used are [bus, car, truck] • Detection confidence threshold 40% (configuration) • mxnet->pytorch due to GPU architecture requirements • Batch frames for detection (CPU/GPU) vs tracking (CPU-only) to avoid context switching • Tracking strategies: FairMot vs Our Own • FairMot: default tracker, up to 10x faster … when it tracks correctly • Custom tracker fallback: JDE (Joint Detection and Embedding) + Kalman Filter + TemplateMatch (openCV)
  10. 10. Lessons learned: Behavioral patterns, anomalies
  11. 11. Anomaly Detection • Vehicle Trajectory Anomaly Detection uses tracking for common, average and rare paths • 'cluster representatives' as pseudo-centroids in non-uniform vector length feature space • Rarity is computed as distance from cluster centroid • DBSCAN for short videos • AgglomerativeClustering for long videos
  12. 12. Behavioral patterns • Time-scale based smoothing algorithms • Low-pass filter/Fourier Series/Numpy.hanning
  13. 13. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.

Live demo and lessons learned building and publishing an advanced video analytics solution in the Azure Marketplace. This is a deep technical dive into the engineering and data science employed throughout, with all challenges encountered by combining Deep Learning and Computer Vision for object detection and tracking, the operational management and tool building efforts for scaling the video processing and insights extraction to large GPU/CPU Databricks clusters and the machine learning required to detect behavioral patterns, anomalies and scene similarities across processed video tracks. The entire solution was build using open source scala, python, spark 3.0, mxnet, pytorch, scikit-learn as well as Databricks Connect.

Views

Total views

129

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

5

Shares

0

Comments

0

Likes

0

×