Tooling for Machine Learning: AWS Products, Open Source Tools, and DevOps Practices
•Download as PPTX, PDF•
6 likes•761 views
This document provides an overview of machine learning tooling on AWS, including data pipelines, modeling and training, and deployment. It discusses AWS products for streaming and batch data ingestion, machine learning services like Amazon Machine Learning, Amazon SageMaker, and AWS Deep Learning AMIs. It also provides best practices for notebooks, model maintenance, and ML lifecycle management using tools like MLFlow and KubeFlow. The document concludes that while AWS provides a strong foundation, operations require additional layers for successful and reproducible machine learning.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Tooling for Machine Learning: AWS Products, Open Source Tools, and DevOps Practices
Similar to Tooling for Machine Learning: AWS Products, Open Source Tools, and DevOps Practices (20)
More from SQUADEX
Recently uploaded
Recently uploaded (20)
Tooling for Machine Learning: AWS Products, Open Source Tools, and DevOps Practices
- 1. Tooling for Machine Learning: AWS Products, Open Source Tools, and DevOps Practices
- 3. Stepan Pushkarev Platform Engineer Rinat Gareev ML Engineer Iskandar Sitdikov ML Engineer
- 5. Overview ● Data Pipelines ● Modeling & Training ● Deployment
- 6. Data Ingestion ● Streaming - as much as possible even if you are not realtime ○ 99% of world's data has a timestamp ○ Fast Data is better that Big Data ○ Rich API for time series data (late arrivals, replays, etc) ○ Streams - advanced durable file system ○ Monitoring and operations friendly ○ Windowing aggregations and stateful transformations ○ Enrichments right in the stream ○ Realtime!
- 9. Batch AirFlow - DAG runner Pros: - Nice UI - Easy to start, plugins ecosystem Cons: - Celery - Challenging for Operations Tip: - Always use Docker or Kube Operators AWS Step Functions Pros - Managed by AWS - Event driven Cons: - Less sophisticated flows - JSON programming - Managed by AWS
- 10. Batch - System Layer - Versioning - Contracts with input / output schemas and transformation rules (DSL) - HTTP, S3, Postgres etc. clients to easily retrieve / persist / upload data - Unified naming and folder structures based on service / resource / timestamp / extension - Transactions, state persistence - Cleanup - Metrics, Logging & Tracing - Trick - allows switching to streaming any time!
- 11. Glue Catalogue - Must have Glue ETL Jobs - Your ETL “System Layer” for Spark
- 12. Overview ● ML Process ● Data Pipelines ● Modeling & Training ● Deployment
- 13. Machine Learning on AWS Amazon Machine Learning Amazon SageMaker AWS Deep Learning AMIs * application services (Comprehend, Lex, Polly, Transcribe, Translate, Rekognition) are out of scope
- 14. Machine Learning on AWS – AML Amazon Machine Learning Amazon SageMaker AWS Deep Learning AMIs Provide data, no code* required *optionally – JSON Programming Efforts ` Flexibility
- 15. Amazon Machine Learning – Overview Good for lazy & quick baselines Inputs: CSV on S3, SQL over Redshift or RDS Mysql Includes: - Data visualization - Data transformation utils (with inferred default ‘recipes’), Feature Selection - Feature crosses - Prediction targets: numeric – linear regression categorical – (multinomial) logistic regression - Evaluation
- 16. Amazon Machine Learning – Limits No hyperparameter tuning (Cf. SageMaker Linear Learner) No built-in cross-validation (requires additional scripting) System limits: 100 KB per observation, max 10000 output features, max 100 classes, etc
- 17. Machine Learning on AWS – SageMaker Amazon Machine Learning Amazon SageMaker AWS Deep Learning AMIs A platform that consists of: ● model implementations ● model tuner ● notebook hosting ● model hosting Programming Efforts ` Flexibility
- 18. SageMaker – Algorithms – Overview Classification/Regression ● Linear Learner ● XGBoost ● Factorization Machine ● K-Nearest Neighbors Image processing ● Image Classification ● Object Detection Text processing ● BlazingText ● Topic Modeling – LDA ● Topic Modeling – NTM Clustering ● K-Means Time Series ● DeepAR Forecasting Dimensionality Reduction ● Principal Component Analysis Encoder-Decoder ● Seq2Seq Anomaly Detection ● Random Cut Forest Bring Your Own
- 19. SageMaker – BlazingText Very fast & convenient FastText implementation Use case: 1. Given few labelled texts for a ML problem and much more unlabelled (domain-specific) 2. Train word embeddings on unlabelled data 3. Use these embeddings to initialize input layer of your neural network Also: BlazingText provides supervised mode for text classification
- 20. SageMaker – Image Classification Implementation of ‘Deep Residual Learning for Image Recognition’ aka ResNet. Two modes: ● Full training – starts from scratch ● Transfer learning – starts from a model trained on ImageNet ○ different options available – 18-200 layers. Other handy features: image resizing, augmentation. Not good: nasty checkpointing, no early stopping.
- 21. TensorFlow on SageMaker You provide a python* script with functions: ● model_fn ● train_input_fn ● eval_input_fn ● serving_input_fn SageMaker does the rest (including TensorBoard in training mode). * 😟 Lack of Python 3 support (so far) Similar to TF Estimator API
- 22. SageMaker – Custom Algorithms Bring Your Own Docker Image Training Inference SageMaker provides: ● JSON with configurations of data & cluster & hyperparameters ● mount data files (or pipes) ● model artifacts SageMaker expects: ● train entry point ● model artifacts after completion ● serve entry point ● responds to /invocations and /ping on port 8080 SageMaker does not provide: ● access to the container, e.g., no TensorBoard without tricks 😕
- 23. SageMaker – Automatic Model Tuning Implementation of Bayesian optimization for hyperparameter value search Good: ● easy-to-use, like Sklearn GridSearch ● supports BYO/custom models ● faster experimentation ● no mess with provisioning instances ● no pricing overhead Not good: ● unlike Sklearn – no cross-validation ● unlike Sklearn – no fitting of an entire pipeline (e.g., including feature extractors and other transformers) ● support of spot instances won’t hurt
- 24. SageMaker – Algorithms – Pros and Cons Not good: ‘S3-centered’, complicates fitting feature extraction into pipeline Good: Unified IO system for training & inference, + Pipe mode Model agnostic Hyperparameter Tuning Distributed training Flexible pricing for training – pay by the second All above is available for your custom algorithm
- 25. Machine Learning on AWS – AWS DL AMIs Amazon Machine Learning Amazon SageMaker AWS Deep Learning AMIs Programming Efforts ` Flexibility Freedom! * https://en.wikipedia.org/wiki/File:Raft-slab.jpg (CC BY-SA 3.0)
- 26. Avoid headache with EC2 instance setup: installation of Nvidia drivers, python, your favourite DL framework, compatibility warnings & errors, configuration, optimization, etc. Use cases: ● to use different data sources rather than S3 ● to implement more sophisticated data pipelines ● to implement more complex training workflows ● to run long training on Spot Instances AWS Deep Learning AMIs
- 27. SageMaker – Jupyter Notebooks AWS value-add: ● Better authentication ● Ready-to-go environments ● Easy IAM setup ● Quick start with example notebooks ● Stop & Restore when necessary ○ Lifecycle configurations
- 28. Notebook Best Practices / versioning - Git (cli, plugins, extensions) - Post-save hook to save .py and .html files alongside with notebook.
- 29. Notebook Best Practices / reproducing Steps: 1. Configure variables 2. Install dependencies 3. List dependencies 4. Define constants 5. Load data 6. Load schema - During instance boot - Within notebook itself (hacky way)
- 30. Notebook Best Practices / share & collaboration Instance Proxy Kernel Instance Kernel Notebook storage Browser Multiuser instance Kernel Kernel Browser Notebook storage SageMaker, JupyterHub Colab, Zeppelin
- 31. ● ML Process ● Data Pipelines ● Modeling & Training ● Deployment Overview
- 32. SageMaker – Deployment Pros: - One click - Single row & batch - Traffic split, A/B - Autoscale group Cons - EC2 instance per model - No Contracts - No shadow, replay tests - No model metrics - No model versioning Alternative: - Do it yourself on EKS/ECS - Opensource: hydrosphere.io
- 33. Deployment Best Practice /predict input: output: JVM DL4j / TF / Other GPU CPU model v2 [ .... ] gRPC HTTP server sidecar serving requests training data stats: - min, max - range - clusters - quantiles - autoencoder
- 34. ● Bad training data ● Bad serving data ● Training/serving data skew ● Misconfiguration ● Performance ● Concept Drift ● Adversarial input ● Upstream error Model Maintenance
- 35. ● Data contracts ● Data Profiling ● Model Performance monitoring ● Concept drift monitoring ● Smart subsampling ● Active retraining and active learning Model Maintenance - Best practices
- 36. Online Monitoring / Profiling infrastructure - Learn the latent space online in a stream of production inputs - Compare “online” snapshots with offline - Index encodings for analysis / audit / exploration
- 37. Stratified sampling from the distribution learnt
- 38. ML lifecycle orchestration - Jenkins :) - AWS CodePipeline - StepFunctions - KubeFlow
- 40. MLFlow - experiments tracking and reproducibility
- 41. KubeFlow = argo + kube configs + ml tutorial
- 42. Takeaways - Well architected Data Platform is a key enabler for successful ML project - AWS ML Ecosystem is a great foundation of your enterprise wide ML platform but you should know your options and extra layer to be built - ML Reproducibility and ML lifecycle management is a wild space, no off the shelf solution
- 43. squadex.com 125 University Avenue Suite 290, Palo Alto California, 94301 Questions, details? We would be happy to answer!