Organizations are using machine learning (ML) to address a host of business challenges, from product recommendations to demand forecasting. Until recently, developing these ML models took much time and effort, and it required expertise. In this session, we introduce Amazon SageMaker, a fully managed ML service that enables developers and data scientists to develop and deploy deep learning models quickly and easily. We walk through the features and benefits of Amazon SageMaker and discuss the uniquely designed ML algorithms that allow for optimized model training, getting you to production fast.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sunil Mallya
Sr. Solutions Architect, ML Solutions Lab, Amazon Web Services
@sunilmallya
BDA301
Work with ML in Amazon SageMaker

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solving Some Of The Hardest Problems In Computer Science
Learning Language Perception Problem
Solving
Reasoning

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning at Amazon: A long heritage
Personalized
Recommendations
Fulfillment automation
& Inventory Management
Drones Voice driven
Interactions
Inventing entirely new
Customer experiences

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tens of thousands of customers running Machine Learning on AWS

5. ML @ AWS
OUR MISSION
Put machine learning in the
hands of every developer and
data scientist

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FRAMEWORKS
KERAS
PLATFORMS
APPLICATION SERVICES
R E K O G N I T I O N R E K O G N I T I O N
V I D E O
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X
Amazon SageMaker Amazon Mechanical Turk
The AWS Machine Learning Stack

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k
s
I n t e r f a c e
s
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k
s
I n t e r f a c e
s
NVIDIA
Tesla V100 GPUs
(14x faster than P2)
P3
Machine Learning
AMIs
5,120 Tensor cores
128GB of memory
1 Petaflop of compute
NVLink 2.0
I N F R A S T R U C T U R E ( G P U )
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k
s
I n t e r f a c e
s
NVIDIA
Tesla V100 GPUs
(14x faster than P2)
P3
Machine Learning
AMIs
5,120 Tensor cores
128GB of memory
1 Petaflop of compute
NVLink 2.0
I N F R A S T R U C T U R E ( G P U )
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner
Intel Xeon
3.0 GHz Skylake CPU
(25% better perf/price than C4)
Machine Learning
AMIs
72 vCPUs
144 GB of memory
AVX 512
Nitro Hypervisor
I N F R A S T R U C T U R E ( C P U )
C5

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning Process - Review

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Discovery: The Analysts
Re-training
• Help formulate the right
questions
• Domain Knowledge

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Integration: The Data Architecture
Retraining
• Build the data platform:
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift
Spectrum

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Visualization &
Analysis
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
• Setup and manage Notebook
Environments
• Setup and manage Training
Clusters
• Write Data Connectors
• Scale ML algorithms to large
datasets
• Distribute ML training
algorithm to multiple
machines
• Secure Model artifacts
Why We built Amazon SageMaker: The Model Training Undifferentiated Heavy Lifting

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BUI LD
Where should you spend your time?
Design training experiments

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BUI LD T RAIN
Where should you spend your time?
Run scalable training

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BUI LD T RAIN D EPLOY
Where should you spend your time?
T UN E
Deploy and operate

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
T RAIN D EPLOY
BUI LD
Build, train, tune, and host your own models
T UN E
Amazon SageMaker

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
Hyperparameter
Tuning
D EPLOY
BUI LD T R AI N & T UN E
Build, train, tune, and host your own models
Amazon SageMaker

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI LD T R AI N & T UN E D EPLOY
Build, train, tune, and host your own models
Hyperparameter
optimization
Amazon SageMaker

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI LD T R AI N & T UN E D EPLOY
Build, train, tune, and host your own models
End-to-end encryption with KMS
End-to-end VPC support
Compliance and audit capabilities
Metadata and experiment management capabilities
Pay as you go
Amazon SageMaker
Hyperparameter
optimization

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker Training Demo

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Built-In Algorithms
Deep Learning
Frameworks
MXNet & Gluon
Tensorflow
Pytorch
Chainer
Custom/ Docker
Training (single or
distributed cluster)
Data
Amazon SageMaker: Algorithm and Framework Support

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Remove undifferentiated
heavy lifting of ML
Iterate, iterate,
iterate!
Training speed Inference speed
Speed in all phases
Amazon SageMaker

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Enabling experimentation speed
Tr a i n w i t h
l o c a l n o t e b o o k s
Train on notebook
instances
PetaFLOP
training on p3.16xl
Go distributed
with one line of code
Same containers
Amazon SageMaker Local Mode

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hyperparameter Optimizer
Amazon SageMaker Automatic Model Tuning
Neural Networks
Number of layers
Hidden layer width
Learning rate
Embedding
dimensions
Dropout
…
Decision Trees
Tree depth
Max leaf nodes
Gamma
Eta
Lambda
Alpha
…
…
“Hyperparameters”
(algorithm parameters that significantly affect model quality)

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hyperparameter Optimization
Amazon SageMaker Automatic Model Tuning
Training JobHyperparameter
Tuning Job
Tuning algorithm
Objective
metrics
Training Job
Training Job
Training Job
Clients
(console, notebook, IDEs, CLI)
model name
model1
model2
…
objective
metric
08
0.75
…
eta
0.07
0.09
…
max_depth
6
5
…
…

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
from sagemaker.tuner import IntegerParameter, CategoricalParameter, ContinuousParameter, HyperparameterTuner
hyperparameter_ranges = {
'batch_size': CategoricalParameter([16, 32, 64, 128]),
'learning_rate': ContinuousParameter(0.0009, 0.01),
'epochs': IntegerParameter(10, 100)}
objective_metric_name = 'Validation-accuracy'
metric_definitions = [{'Name': 'Validation-accuracy',
'Regex': 'Validation-accuracy=([0-9.]+)’}]
tuner = HyperparameterTuner(estimator,
objective_metric_name,
hyperparameter_ranges,
metric_definitions,
max_jobs=20,
max_parallel_jobs=4)
tuner.fit(dataset_location, job_name=job_name)
Amazon SageMaker Automatic Model Tuning

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customer Success With Amazon SageMaker

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker: Launch Customer
“With Amazon SageMaker, we can accelerate our Artificial
Intelligence initiatives at scale by building and deploying our
algorithms on the platform. We will create novel large-scale machine
learning and AI algorithms and deploy them on this platform to solve
complex problems that can power prosperity for our customers.
"
- Ashok Srivastava, Chief Data Officer, Intuit

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Model Hosting
(SageMaker)
Near real-time fraud detection in AWS using SageMaker
Calculate
Features
Reader
Cleanser
Processor
Data
Lookup
Training
Feature Store Model Training
(SageMaker)
Model
Client Service
Amazon
EMR
Amazon
SageMaker
Amazon
SageMaker

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Key Differentiators for SageMaker

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1
I
Notebook Instances
Zero Setup For Exploratory Data Analysis
Authoring &
Notebooks
ETL Access to AWS
Database services
Access to S3 Data
Lake
• Recommendations/Personalization
• Fraud Detection
• Forecasting
• Image Classification
• Churn Prediction
• Marketing Email/Campaign Targeting
• Log processing and anomaly detection
• Speech to Text
• More…
“Just add data”
VPC

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2
I
Algorithms
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (SM builds the Container)
SM Estimators in
Apache Spark Bring Your Own Algorithm (You build the Container)
Amazon SageMaker: 10x better algorithms

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Managed Distributed Training with Flexibility
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (SM builds the Container)
Bring Your Own Algorithm (You build the Container)
3
I
ML Training Service
Fetch Training data
Save Model Artifacts
Fully
managed –
VPC–
Amazon ECR
Save Inference Image
SM Estimators in
Apache Spark
CPU GPU Hyperparameter Tuning

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
Create a Model
ModelName: prod
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
Create weighted
ProductionVariants
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
A/B Testing Models
Create an
EndpointConfiguration from
one or many
ProductionVariant(s)
EndpointConfiguration
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
One-Click!
EndpointConfiguration
Inference Endpoint
Amazon Provided Algorithms
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4
I
ML Hosting Service
✓ Auto-Scaling Inference APIs
✓ A/B Testing (more to come)
✓ Low Latency & High
Throughput
✓ Bring Your Own Model
✓ Python SDK
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker keeps getting better!
Recent launches include:
• New deep learning frameworks (Chainer and PyTorch)
• More security and compliance features
• Expansion to regions around the world - Tokyo, Frankfurt, Sydney
• Automatic model tuning
• Local mode w/ GPU in SageMaker notebooks
• Algorithm Enhancements: DeepAR, BlazingText, Linear Learner
• Tensorflow Pipe-Mode
• Batch Transform

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker
Notebooks
Training
Algorithm
SageMaker
Training
Amazon ECR
Code Commit
Code Pipeline
SageMaker
Hosting
Coco dataset
AWS
Lambda
API
Gateway
SageMaker Sample End-to-End Architecture: Style Transfer
Build
Train
Deploy
static website hosted on S3
Inference requests
Amazon S3
Amazon
Cloudfront
Web assets on
Cloudfront
End-to-end encryption with KMS
disable internet
access
End-to-end VPC support
private link

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Lake Storage
Amazon S3
Security
Access Control
Encryption
VPC
KMS
Auditing
Compliance
Roles
Fine Grained Access Controls
Compute
Powerful GPU & CPU Instances
AWS Lambda
Analytics
Amazon Athena
Amazon EMR
Amazon Redshift & Redshift Spectrum
Broadest & Easiest to Use ML Platform with the
Most Customers

45. Submit Session Feedback
1. Tap on the Agenda icon
2. Select the session you
attended
3. Tap on Session Evaluation to
submit your feedback

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You!
@sunilmallya
Useful SageMaker Resources
https://github.com/awslabs/amazon-sagemaker-examples
https://github.com/aws-samples/aws-ml-vision-end2end
https://github.com/juliensimon
https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html
https://github.com/aws/sagemaker-spark