Downloaded 26 times
Uploaded byCobus Bernard
Deep Dive Amazon SageMaker
The document discusses Amazon SageMaker, a fully managed service that allows users to build, train, and deploy machine learning models at scale. It provides an overview of SageMaker's key features like notebooks for preprocessing data and building models, built-in algorithms for common tasks, one-click training of models, hyperparameter tuning, and deployment of trained models onto managed hosting infrastructure. SageMaker aims to make machine learning accessible to every developer by handling the complexities of training and deploying models.
Deep Dive Amazon SageMaker
- 1. Amazon SageMaker Build, Train,and Deploy Machine Learning Models Quickly & Easily, at scale Cobus Bernard Senior Developer Advocate Amazon Web Services © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 2. Agenda © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. • Machine Learning onAWS • Amazon Machine Learningstack • The Machine Learning Lifecycle • SageMaker Overview • Demo • SageMaker features
- 3. Definitions • Supervised, Unsupervised,Reinforcement Learning – Learning methods • Model – The function produced by training • Inference/Predictions – Requests to a model © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 4. Our mission atAWS Put machine learning in the hands of every developer © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 5. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved.
- 6. Amazon SageMaker: LaunchCustomer “ With Amazon SageMaker, we can accelerate our Artificial Intelligence initiatives at scale by building and deploying our algorithms on the platform. We will create novel large-scale machine learning and AI algorithms and deploy them on this platform to solve complex problems that can power prosperity for our customers. ” –Ashok Srivastava, Chief Data Officer, Intuit © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 7. Model hosting (Amazon SageMaker) Nearreal-time fraud detection in AWS using Amazon SageMaker Calculate features Reader Cleanser Processor Data Lookup Training Feature store Model training (Amazon SageMaker) Model Client service Amazon EMR Amazon SageMaker Amazon SageMaker © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 8. ML F RA M E W O R K S & IN F R A S T R U C T U R E A I SE R V IC E S ML SE R V IC E S Data labeling |Pre-built algorithms & notebooks |One-click training and deployment Build, train, and deploy machine learning models fast Easily add intelligence to applications without machine learning skills Vision |Documents |Speech |Language |Chatbots |Forecasting |Recommendations Flexibility & choice, highest-performing infrastructure Support for ML frameworks |Compute options purpose-built for ML © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 9. M L FR A M E W O R K S & IN F R A ST R U C T U R E A I SE R V IC E S I M A G E V I D E O Vision S p eech Chatbots M L SE R V IC E S Fr a m e w o r k s In t e r f a ce s In f r a s t r u c t u r e E C 2 P 3 & P 3 d n E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C I N F E R E N C E Language Forecasting Recommendations R E K O G N I T I O N R E K O G N I T I O N T E X T R A C T P O L L Y T R A N S C R I B E New T R A N S L A T E C O M P R E H E N D L E X C O M P R E H E N D M E D I C A L New F O R E C A S T New P E R S O N A L I Z E New A M A Z O N S A G E M A K E R New G R O U N D T R U T H N O T E B O O K S New A W S M A R K E T P L A C E A L G O R I T H M S R E I N F O R C E M E N T L E A R N I N G New T R A I N I N G O P T I M I Z A T I O N ( N E O ) New D E P L O Y M E N T H O S T I N G New © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 10. Whatdoes the MachineLearning Lifecycle look like? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 11. Data Visualization & Analysis BusinessProblem – ML problem framing Data Collection Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging YesNo DataAugmentation Feature Augmentation The Machine Learning Process Re-training Predictions © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 12. Let’s Dive Deeperinto eachphase © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 13. Data Visualization & Analysis BusinessProblem – ML problem framing Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging YesNo DataAugmentation Feature Augmentation Problem discovery Re-training Data Collection • Help formulate theright questions • Domain Knowledge Predictions © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 14. Data Visualization & Analysis BusinessProblem – ML problem framing Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging YesNo DataAugmentation Feature Augmentation Retraining Data Collection • Need a dataplatform? • Amazon S3 • AWS Glue • Amazon Athena • Amazon EMR • Amazon Redshift Spectrum Integration Predictions © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 15. Data Visualization & Analysis BusinessProblem – ML problem framing Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging YesNo DataAugmentation Feature Augmentation Retraining Data Collection Model Training Predictions • Setup and manage Notebook Environments • Setup and manage Training Clusters • Write Data Connectors • Scale ML algorithms to large datasets • Distribute ML training algorithm to multiple machines • Secure Model artifacts © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 16. Data Visualization & Analysis BusinessProblem – ML problem framing Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging YesNo DataAugmentation Feature Augmentation Retraining Data Collection Model Deployment Predictions • Setup and manageModel Inference Clusters • Manage and ScaleModel Inference APIs • Monitor and DebugModel Predictions • Models versioningand performance tracking • Automate New Model version promotion to production (A/B testing) © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 17. Pre-built notebooks for common problems Built-in, high- performance algorithms DE PLO Y BU I LD Amazon SageMaker Build, train, tune, and host your own models TR A I N © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 18. Authoring & notebooks ETL AccesstoAWS database services Access to AmazonS3 data lake VPC Zero setup notebook instance © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 19. Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter Tuning DE PLO Y BU I LD TR A I N & TU N E Amazon SageMaker Build, train, tune, and host your own models © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 20. Amazon ECR Trainingdata Modelartifacts Training codeHelper code Model Training (on EC2) Training code Amazon SageMaker © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 21. Amazon SageMaker Built-in algorithms Deeplearning frameworks MXNet & Gluon Tensorflow PyTorch Chainer Custom Image Training (single machine or distributed cluster) Data Flexible and scalable model training © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 22. Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter Tuning BU ILD TR A I N & TU N E Amazon SageMaker Build, train, tune, and host your own models Fully managed hosting with Auto Scaling One-click deployment DE PLO Y © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 23. Amazon ECR Trainingdata Modelartifacts Training codeHelper code Model Training (on EC2) Inference code Helpercode Model Hosting (on EC2) GroundTruth Client application Inferencecode Training code Inferenceresponse InferenceEndpoint Inference request Amazon SageMaker © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 24. Amazon ECR 30 50 1010 Model Artifacts InferenceImage Model versions Endpoint configuration InferenceEndpoint Amazon SageMaker Auto Scaling A/B Testing Low latency &high throughput Bring your ownmodel One-click model deployment © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 25. Fully managed hosting with AutoScaling One-click deployment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training BU I LD TR A I N & TU N E DE PLO Y End-to-end encryption with AWS KMS End-to-end VPC support Compliance and auditcapabilities Metadata and experiment management capabilities Pay as yougo Amazon SageMaker Build, train, tune, and host your own models Hyperparameter Tuning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 26. Amazon SageMaker isthe best place to runTensorFlow in the cloud AW S - O P T I M I Z E D T E N S O R F L O W • Fully-managed training and hosting • Near-linear scaling across 100s of GPU • 7 5 % lower inference costs with Amazon Elastic Inference • 3x faster network throughput with EC2 P3 ST O C K T E N S O R F L O W 6 5 % 9 0 % Sc a l i n g e f f i c ie n c y w i t h 2 5 6 G P U s © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 27. Amazon SageMaker Demo ©2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 28. Additional features ofAmazonSageMaker © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 29. Training 9 0 % Trainingdrives research and development Inference (Prediction) 1 0 % © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 30. Predictions drive complexity andcostin production Inference (Prediction) 9 0 % Training 1 0 % © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 31. Lower inference costsMatch capacityto demand Available between 1 to 32 TFLOPS per accelerator Integrated with Amazon EC2 and Amazon SageMaker KEY FEATURES Support for TensorFlow, Apache MXNet, andONNX with PyTorch coming soon Single and mixed-precision operations Amazon Elastic Inference Reduce deep learning inference costs up to 7 5 % © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 32. Neo Amazon SageMaker Neo Trainonce, run anywhere © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 33. K E YF E A T U R E S Open-source device runtime and compiler, 1/10th the size of originalframeworks Amazon SageMaker Neo Train once, run anywhere with 2xperformance © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 34. Automatic model tuning– improve model quality Neural networks Number of layers Hidden layer width Learning rate Embedding dimensions Dropout … Decision trees Tree depth Max leaf nodes Gamma Eta AWS Lambda Alpha … … “Hyperparameters” (algorithm parameters that significantly affect model quality) © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 35. Successful models requirehigh-quality data © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 36. Amazon SageMaker GroundTruth Label machine learning training data easily and accurately © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 37. Subscribe in a singleclick Available in Amazon SageMaker KEY FEATURES Automatic labeling via machine learning IP protection Automated billing and metering Browse or search AWS Marketplace S E L L ER S Broad selection of paid,free and open-source algorithms and models Data protection Discoverable on your AWS bill BU Y E R S AWS Marketplace for Machine Learning ML algorithms and models available instantly © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 38. A fully autonomous1/18th-scale race car designed to help you learn about reinforcement learning through autonomousdriving • Build machine learning models in Amazon SageMaker • Train, test, and iterate on the track using the AWS DeepRacer 3D racing simulator • Compete in the world’s first global autonomous racing league, to race for prizes and a chance to advance to win the coveted AWS DeepRacer Cup Amazon SageMaker RL Get started with Reinforcement learning using AWS DeepRacer © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 39. Amazon SageMaker RL Getstarted with Reinforcement learning using AWS DeepRacer © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 40. © 2019, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. 2D & 3D physics environments and OpenGym support Support Amazon Sumerian, AWS RoboMaker and the open source Robotics Operating System (ROS)project Fully managed Example notebooks and tutorials K E Y F E A T U R E S Amazon SageMaker RL Reinforcement learning for every developer and data scientist
- 41. Key Advantages ofAmazon SageMaker © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 42. One-click model training & deployment 10x betteralgorithm performance Trainonce run anywhere A M A Z O N SAG E M A K E R 70% cost reduction for data labeling using GroundTruth 2x performance increases from model optimization withNeo 75% cost reduction for inference with Elastic Inference RE D UC E CO S TS INC RE A S E P ER FO R MA N CE EA S E- OF - US E © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 43. Useful Amazon SageMakerresources Questions © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.