Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Creating Efficient, Flexible and Scalable Cloud Computer Vision Applications: An Introduction," a Presentation from GumGum

35 views

Published on

For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-sant-chu

For more information about embedded vision, please visit:
http://www.embedded-vision.com

Nishita Sant, Computer Vision Manager, and Greg Chu, Senior Computer Vision Scientist, both of GumGum, present the "Creating Efficient, Flexible and Scalable Cloud Computer Vision Applications: An Introduction" tutorial at the May 2019 Embedded Vision Summit.

Given the growing utility of computer vision applications, how can you deploy these services in high-traffic production environments? Sant and Chu present GumGum’s approach to the infrastructure for serving computer vision models in the cloud. They elaborate on a few aspects, beginning with modularity of computer vision models, including handling images and video equivalently, creating module pipelines, and desiging for library agnosticism so we can leverage open source developments.

They also discuss inter-process communication—specifically, the pros and cons of data serialization, and the importance of standardized data formats between training and serving data, which lends itself to automated feedback from serving data for retraining and automated metrics. Finally, they discuss GumGum's approaches to scaling, including a producer/consumer model, scaling triggers and container orchestration. They illustrate these aspects through examples of image and video processing and module pipelines.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

"Creating Efficient, Flexible and Scalable Cloud Computer Vision Applications: An Introduction," a Presentation from GumGum

  1. 1. © 2019 GumGum Creating Efficient, Flexible, and Scalable Cloud Computer Vision Applications: An Introduction Nishita Sant, Greg Chu GumGum May 2019
  2. 2. © 2019 GumGum GumGum Overview GumGum is an artificial intelligence company with a particular focus in computer vision 2
  3. 3. © 2019 GumGum Agenda 3 1. Why Computer Vision in the Cloud? 2. Computer Vision System Design • Modularity • Inter Process Communication • Scaling 3. Putting it All Together 4. Key Takeaways
  4. 4. © 2019 GumGum Why Computer Vision in the Cloud? 4 • Reliability • Latency • Hardware and Software Resources • Power, Size and Cost Constraints • Volume of Data/ Throughput It depends!
  5. 5. © 2019 GumGum 5 Design Principles
  6. 6. © 2019 GumGum Design Principles 6 03 Scalability 01 Modularity/ Flexibility 02 Inter Process Communication
  7. 7. © 2019 GumGum Design Principles: 01 Modularity/Flexibility 7
  8. 8. © 2019 GumGum Design Principles: 01 Modularity/Flexibility 8 Red-Ford-Mustang Not-Red-Ford-Mustang DenseNet Red Ford Mustang
  9. 9. © 2019 GumGum Design Principles: 01 Modularity/Flexibility 9 Blue-BMW-Z4 Not-Blue-BMW-Z4 Blue BMW Z4 DenseNet
  10. 10. © 2019 GumGum Design Principles: 01 Modularity/Flexibility 10 Computer Vision Modules Input ML-based: CNNs, LSTMs, SVMs Traditional: Feature Matching Hybrid: Feature Matching + SVM Heuristic: Design Logic Output • Vehicle • Ford Mustang • Red Queue/Load Balancer Auto Scaling Group
  11. 11. © 2019 GumGum Objects Design Principles: 01 Modularity/Flexibility 11 Computer Vision Module Pipelines Make/Model Dominant Color
  12. 12. © 2019 GumGum Design Principles: 02 Inter Process Communication 12
  13. 13. © 2019 GumGum Design Principles: 02 Inter Process Communication 13 JSON for IPC Property : Car Region : List of points describing a contour Property : Car, Ford, Mustang, Red Property : Tire, Black Sub-region : List of points describing a contour within another contour
  14. 14. © 2019 GumGum Design Principles: 02 Inter Process Communication 14 Data collection/annotation/verification Re/train model Serving
  15. 15. © 2019 GumGum Design Principles: 02 Inter Process Communication 15 Data Serialization Pros Cons Safe contract between producers & consumers Schema changes requires careful effort ~10x reduced file size Overhead for serialization code/registry Cleaner data Schema evolution
  16. 16. © 2019 GumGum Design Principles: 03 Scalability 16
  17. 17. © 2019 GumGum Design Principles: 03 Scalability 17
  18. 18. © 2019 GumGum Design Principles: 03 Scalability 18
  19. 19. © 2019 GumGum Design Principles: 03 Scalability – Scaling Triggers 1. # of messages • e.g. SQS & CloudWatch, ALB 2. # of messages & average CPU utilization • e.g. SQS & custom logic in Lambda 3. Container management solutions • e.g. K8s/ECS 19 / +
  20. 20. © 2019 GumGum Putting It All Together: How we serve our CV at GumGum 20
  21. 21. © 2019 GumGum 21
  22. 22. © 2019 GumGum 22
  23. 23. © 2019 GumGum 23
  24. 24. © 2019 GumGum Multivitamin: A Python framework for serving ML models https://github.com/gumgum/multivitamin 24
  25. 25. © 2019 GumGum 25 Multivitamin: Data Flow within 1 Application
  26. 26. © 2019 GumGum Multivitamin: Code Sample 26
  27. 27. © 2019 GumGum 27
  28. 28. © 2019 GumGum NHL sponsor & placement detection 28 Sample output:
  29. 29. © 2019 GumGum Key Takeaways 29 • Efficiency • Inter Process Communication • Modularity • Scalability • Client Requirements
  30. 30. © 2019 GumGum Thank You 30
  31. 31. © 2019 GumGum Resources • Computer Vision: At the Edge or In the Cloud? It Depends. • Keras Wiki, Keras Documentation, Github – Keras • MXNet • PyTorch • Open Neural Network Exchange • Caffe2 • TensorFlow • Swagger - API Development Tool • Amazon ElasticSearch • Amazon ElasticBeanstalk • Spring Framework for Java Platform • Densly Connected Convolutional Networks • Three reasons why apache avro data serialization is a good choice • Apache Avro Schema 1.8.1
  32. 32. © 2019 GumGum About GumGum 70% Of Fortune 100 Companies 240+ Gumgummers 16 Offices On 16 Continents 99.4% CAGR Since 2012
  33. 33. © 2019 GumGum Computer Vision Applications CONTENT ENHANCES AD Ad creative built to incorporate image content Localized detection of objects, people, or body parts using Computer Vision …in Augmented Advertising
  34. 34. © 2019 GumGum Sports Sponsorship Measurement
  35. 35. © 2019 GumGum Social Listening GUMGUM SOCIAL Ingest social posts with visual content from firehose of Twitter, Instagram, etc. Detect presence and location of brand logos or other objects Analytics dashboard, interact with influencers
  36. 36. © 2019 GumGum Design Principles ACCURACY SPECS CAN BE CASE SPECIFIC Example: Brand Safety in Digital Advertising High Recall/Low Precision = Risks Minimized High Recall/Low Precision = Inventory Reduced
  37. 37. © 2019 GumGum Design Principles : Modularity/Flexibility Inter-Process Communication
  38. 38. © 2019 GumGum Design Principles: Modularity/Flexibility Inter-Process Communication

×