Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Machine Learning with Kubernetes- AWS Container Day 2019 Barcelona

557 views

Published on

In this session, we discuss Machine Learning on Kubernetes and the advantages for running it on AWS. We also show you a demo of the updates to Escalator - a cluster auto scaling component tuned for batch/ML workloads, and EC2 GPU instance auto-scaling.

  • (Unlimited)....ACCESS WEBSITE Over for All Ebooks ................ accessibility Books Library allowing access to top content, including thousands of title from favorite author, plus the ability to read or download a huge selection of books for your pc or smartphone within minutes ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn }
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD TH1S BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK Here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download Full EPUB EBOOK Here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK Here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download EPUB EBOOK Here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download doc EBOOK Here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE Format, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE Format, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE Format, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Machine Learning with Kubernetes- AWS Container Day 2019 Barcelona

  1. 1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Machine Learning with Kubernetes Yaniv Donenfeld AI/ML Solutions Container Services, AWS Mike Crute Sr. Engineer Container Services, AWS
  2. 2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Centerpiece for digital transformation
  3. 3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  4. 4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 P3 & P3dn EC2 C5 FPGAs Greengrass Elastic inference FRAMEWORKS INTERFACES INFRASTRUCTURE Inferentia EC2 G4 The Amazon ML stack: Broadest & deepest set of capabilities
  5. 5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment Hosting ML Frameworks + Infrastructure EC2 P3 & P3dn EC2 C5 FPGAs Greengrass Elastic inference FRAMEWORKS INTERFACES INFRASTRUCTURE Inferentia EC2 G4 The Amazon ML stack: Broadest & deepest set of capabilities
  6. 6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N V I D E O F O R E C A S TT E X T R A C T P E R S O N A L I Z E VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS ML Services Amazon SageMaker Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment Hosting ML Frameworks + Infrastructure EC2 P3 & P3dn EC2 C5 FPGAs Greengrass Elastic inference FRAMEWORKS INTERFACES INFRASTRUCTURE Inferentia EC2 G4 The Amazon ML stack: Broadest & deepest set of capabilities
  7. 7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Amazon ML stack: Broadest & deepest set of capabilities R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N V I D E O F O R E C A S TT E X T R A C T P E R S O N A L I Z E VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS Amazon SageMaker Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment Hosting EC2 P3 & P3dn EC2 C5 FPGAs Greengrass Elastic inference FRAMEWORKS INTERFACES Inferentia EC2 G4 INFRASTRUCTURE
  8. 8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “A little less conversation, a little more action, please” M A C H I N E L E A R N I N G S T O R A G E Amazon Redshift + Redshift Spectrum Amazon QuickSight Amazon EMR Hadoop, Spark, Presto, Pig, Hive…19 total Amazon Athena Amazon Kinesis Amazon Elasticsearch Service AWS Glue A N A L Y T I C S Amazon S3 Standard-IA Amazon S3 Standard Amazon S3 One Zone-IA Amazon Glacier Amazon S3 Intelligent- Tiering N E W Amazon EBS Amazon S3 Glacier Deep Archive N E W R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N V I D E O F O R E C A S TT E X T R A C T P E R S O N A L I Z E V I S I O N S P E E C H L A N G U A G E C H A T B O T S F O R E C A S T I N G R E C O M M E N D A T I O N S Amazon SageMaker Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment Hosting E C 2 P3 & P3 d n E C 2 C 5 FPG A s G r e e n g r a s s E la s t i c i n f e r e n c e F R A M E W O R K S I N T E R F A C E S I n f e r e n t i a E C 2 G 4 I N F R A S T R U C T U R E
  9. 9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N V I D E O F O R E C A S TT E X T R A C T P E R S O N A L I Z E VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS Amazon SageMaker Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment Hosting EC2 P3 & P3dn EC2 C5 FPGAs Greengrass Elastic inference FRAMEWORKS INTERFACES INFRASTRUCTURE Inferentia EC2 G4 Machine Learning using Kubernetes
  10. 10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML Frameworks + Infrastructure EC2 P3 & P3dn EC2 C5 FPGAs Greengrass Elastic inference FRAMEWORKS INTERFACES INFRASTRUCTURE Inferentia EC2 G4 Machine Learning using Kubernetes
  11. 11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Why Machine Learning on Kubernetes? Composability Portability Scalability O N - P R E M I S E S C L O U D http://www.shutterstock.com/gallery-635827p1.html
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  13. 13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Machine Learning Development Workflow ML Workflow
  14. 14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Why Jupyter? Jupyter provides an easy on-ramp to build, train, and deploy ML models Jupyter Notebook allows you to create and share documents that contain live code, equations, visualizations, and narrative text Uses include: • data cleaning and transformation • numerical simulation • statistical modeling • data visualization • machine learning • and much more
  15. 15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Machine Learning Development Workflow
  16. 16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML on K8s—without KubeFlow Credits: @aronchik
  17. 17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML on K8s—with KubeFlow Credits: @aronchik
  18. 18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What’s in KubeFlow?
  19. 19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  20. 20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubeflow Pipelines • Invoke EMR/Spark jobs for preprocessing directly as a pipeline step • Invoke SageMaker jobs to utilize the SageMaker framework from within Kubeflow
  21. 21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Advantages of KubeFlow on AWS EKS cluster provision with External traffic with to manage Lustre file system Centralized and unified K8s logs in TLS and Auth with and for your K8s API server endpoint Detect GPU instance and install
  22. 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  23. 23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TensorFlow Open source library to develop and train ML models Created by Google Brain team Can run on desktop, servers, mobiles, edge devices
  24. 24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS is the platform of choice to run TensorFlow of all TensorFlow workloads in the cloud runs on AWS
  25. 25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Train twice as fast with TensorFlow 65% Scaling efficiency with 256 GPUs S T O C K T E N S O R F L O W 100% Scaling efficiency with 256 GPUS A W S - O P T I M Z E D T E N S O R F L O W
  26. 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Autonomous Vehicle Sensor Technology © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. LIDAR Camera GPS/HD Maps Telematics Control Unit Accelerometer/Gyroscope DSRC Radar Infrared Ultrasonic
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SAE LEVEL NAME CONTROL MONITORING VIGILANCE FALLBACK HANDOFF OPERATING CONDITIONS 0 Unassisted Human 1 Driver Assist Human and AD System Human Human Some 2 Partial Automation AD System Human Human (0 warning) Some 3 Conditional Automation AD System AD System Human (adequate warning, assumed 15 seconds) Some 4 High Automation AD System AD System AD System Some 5 Full Automation AD System AD System AD System All © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SAE LEVEL NAME CONTROL MONITORING VIGILANCE FALLBACK HANDOFF OPERATING CONDITIONS 0 Unassisted Human 1 Driver Assist Human and AD System Human Human Some 2 Partial Automation AD System Human Human (0 warning) Some 3 Conditional Automation AD System AD System Human (adequate warning, assumed 15 seconds) Some 4 High Automation AD System AD System AD System Some 5 Full Automation AD System AD System AD System All
  28. 28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AV Workload Characteristics • Distributed, Multi-GPU, Multi-Node Training jobs • ML Practitioners don’t know/want to deal with infrastructure • Large datasets on S3 require copying to a staging area on the cluster • Data should be shared across nodes and re-used across jobs • Distributed training code can be hard • Capacity provisioning and smart autoscaling across jobs can be complex
  29. 29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AV Workload Characteristics • Distributed, Multi-GPU, Multi-Node Training jobs • ML Practitioners don’t know/want to deal with infrastructure Kubeflow on AWS provides an abstraction to setup your cluster as part of the Kubeflow setup process
  30. 30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AV Workload Characteristics • Distributed, Multi-GPU, Multi-Node Training jobs • Large datasets on S3 require copying to a staging area on the cluster • Data should be shared across nodes and re-used across jobs • New CSI driver for EFS and FSx Lustre – dynamically provision a file system using PVC. • FSx Lustre can shadow an S3 Bucket – no need to copy
  31. 31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AV Workload Characteristics • Distributed, Multi-GPU, Multi-Node Training jobs • Distributed training code can be hard • Use Horovod, a Distributed Training framework for TensorFlow, Keras, PyTorch, and MXNet
  32. 32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AV Workload Characteristics • Distributed, Multi-GPU, Multi-Node Training jobs • Capacity provisioning and smart autoscaling across jobs can be complex • Introducing updates to Escalator - a cluster auto scaling component that is tuned for batch/ML workloads, and EC2 GPU instance autoscaling
  33. 33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DEMO
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  35. 35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AV Workload Characteristics • Large-scale simulation workloads
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. - 200,000 400,000 600,000 800,000 1,000,000 1,200,000 Koala Genomics AV Safety Concurrent CPUs
  38. 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. - 50,000,000 100,000,000 150,000,000 200,000,000 250,000,000 300,000,000 350,000,000 Koala Genomics AV Safety Total Core Hours / Year
  39. 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rank / Name Rmax / Rpeak (Petaflops) 1. Summit 143.500 / 200.795 2. Sierra 94.640 / 125.436 3. Sunway Tahihu Light 93.015 / 125.436 4. Tianhe-2A 61.445 / 100.679 5. Piz Daint 21.230 / 27.154 6. Trinity 20.159 / 41.461 7. AI Bridging Cloud Infrastructure 19.880 / 32.577 8. SuperMUCNG 19.477 / 26.874 9. Titan 17.590 / 27.113 10. Sequoia 17.173 / 20.133 TOP500 – Top 10 Supercomputers in Nov 2018
  40. 40. We’re helping our AV customers run at Supercomputer Scale, targeting the equivalent of one of the Top 10 largest supercomputers in the world.
  41. 41. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Challenges in setting up containers for ML
  42. 42. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS deep learning containers KEY FEATURES Customizable container images Support for TensorFlow, Apache MXNet Single and multi-node training and inference Pre-packaged Docker container images fully configured and validated Best performance and scalability without tuning Works with Amazon EKS, Amazon ECS, and Amazon EC2
  43. 43. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 16 container images Training Inference GPU CPU Python 2.7 Python 3.6
  44. 44. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Getting Started https://github.com/aws-samples/machine-learning-using-k8s/

×