BDA301 Working with Machine Learning in Amazon SageMaker: Algorithms, Models, and Inferences

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alex Smola, Amazon Web Services
special thanks to Edo Liberty, Craig Wiley, Tom Faulhaber, Owen
Thomas, Vineet Khare, Bing Xu, Ramesh Nallapati,
BDA301
Machine Learning in Amazon
SageMaker:
Algorithms, Models, and Inference

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon’s fast, scalable algorithms
Distributed Apache MXNet and TensorFlow
Bring your own algorithm
Hyperparameter optimization
Building HostingTraining
Amazon SageMaker Components

... or ...
Apache Spark via
Amazon EMR and
the SageMaker
Spark SDK
Build
Hosted notebook
instances on
SageMaker
... or ...
SageMaker‘s
Console for a point
and click
experience
... or ...
your own device
(EC2, laptop, etc.)

Zero setup
Training
Streaming data +
distributed
computing
Docker / Amazon
ECS
Deploy trained models
• Amazon SageMaker
• AWS Greengrass,
• AWS DeepLens
• Locally

One-step
deployment
Low latency
High throughput
High reliability
A/B testing Use your own
model
Hosting

• XGBoost
• Factorization
Machines
• Linear methods
• Autoregressive
models
• K-means
clustering
• PCA
• Word2Vec
embeddings for
clustering and pre-
processing
Image classification
with convolutional
neural networks
(plenty of models in
MXNet ModelZoo)
• Spectral LDA
• Neural Topic Models
• Seq2Seq for
translation and
similar problems
Built-in Algorithms

… explore and
refine models in a
single notebook
instance
TensorFlow & Apache MXNet Containers
… deploy to
production
Sample your
data…
Use the same code
to train on the full
dataset in a cluster
of GPU
instances…

Bring Your Own Algorithm
... add algorithm
code to a Docker
container...
Choose your own framework
... publish to
Amazon ECS
Amazon ECS

Hyperparameter Optimization
Run a large set of training
jobs with varying
hyperparameters...
... and search the
hyperparameter space for
improved accuracy

Platform Services
Application Services
Frameworks & Interfaces
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Torch Keras Gluon
AWS Deep Learning AMIs
Infrastructure
Amazon Machine Learning Stack
EC2
GPUs
EC2
CPUs
IoT Edge

Amazon EC2 Instances for Deep Learning
• Amazon EC2 P3 Instances
The fastest GPU instances in the cloud
• Up to 8 NVIDIA Tesla V100 GPUs
• 1 PetaFLOP of computational performance (14x faster than P2)
• 300 GB/sec GPU-to-GPU communication (NVLink) (9x faster than P2)
• 16-GB GPU memory with 900 GB/sec peak GPU memory bandwidth
• Amazon EC2 C5 Instances
Cost effective CPUs, e.g., for models using INT8
• Powered by 3.0 GHz Intel Xeon (Skylake) platinum processors
• 72 vCPUs and 155-GB RAM (25% price/performance improvement vs. C4)
• Nitro Hypervisor for larger instance sizes

Architecture and Design Choices
Streaming, GPU/CPU
Distributed Learning with Shared State

Streaming
State

Streaming
Data size
Memory
Data size
Time/Cost

Incremental Training
2
3
1
2

Incremental Training
3
1
2

Distributed
GPU State
GPU State
GPU State

GPU/CPU
GPU State

Parameter Server – Distributed (k,v) Store
GPU
GPU
GPU Local
state
Shared
state
Local
state
Local
state

Cost vs. Time vs. Accuracy
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Best alternative

Cost vs. Time vs. Accuracy
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Best alternative
Amazon SageMaker

Production Readiness
Data/Model size
Investment Reasonable
investment level
Unusable data
Wasted opportunity

Production Readiness
Data/Model size
Investment Reasonable
Investment level
No unusable data
No wasted opportunity

Infinitely Scalable ML Algorithms

Linear Learner
Regression:
Estimate a real valued function
Binary classification:
Predict a 0/1 class

Linear Learner
Train
Fit thresholds
and select
Select model with best validation performance
More than 8x speed-up over naive parallel training!

Linear Learner
Regression (mean squared error)
SageMaker Other
1.02 1.06
1.09 1.02
0.332 0.183
0.086 0.129
83.3 84.5
Classification (F1 Score)
SageMaker Other
0.980 0.981
0.870 0.930
0.997 0.997
0.978 0.964
0.914 0.859
0.470 0.472
0.903 0.908
0.508 0.508
30-GB datasets for web-spam and web-URL classification
0
0.2
0.4
0.6
0.8
1
1.2
0 5 10 15 20 25 30
CostinDollars
Billable time in Minutes
sagemaker-url sagemaker-spam other-url other-spam

Factorization Machines
Log_loss F1 Score Seconds
SageMaker 0.494 0.277 820
Other (10 Iter) 0.516 0.190 650
Other (20 Iter) 0.507 0.254 1300
Other (50 Iter) 0.481 0.313 3250
Click Prediction 1-TB advertising dataset,
m4.4xlarge machines, perfect scaling
$-
$20.00
$40.00
$60.00
$80.00
$100.00
$120.00
$140.00
$160.00
$180.00
$200.00
1 2 3 4 5 6 7 8
Costindollars
Billable time in hours
10
machines
20
machines
30
machines
4050

K-Means Clustering

K-Means Clustering
Method Accurate? Passes Efficient
tuning
Comments
Lloyds [1] Yes* 5–10 No
K-Means ++ [2] Yes k+5 to k+10 No scikit-learn
K-Means|| [3] Yes 7–12 No spark.ml
Online [4] No 1 No
Streaming [5,6] No 1 No Impractical
Webscale [7] No 1 No Spark streaming
Coresets [8] No 1 Yes Impractical
Amazon SageMaker Yes 1 Yes
[1] Lloyd, IEEE TIT, 1982
[2] Arthur et. al. ACM-SIAM, 2007
[3] Bahmani et. al., VLDB, 2012
[4] Liberty et. al., 2015
[5] Shindler et. al, NIPS, 2011
[6] Guha et. al, IEEE Trans. Knowl. Data Eng. 2003
[7] Sculley, WWW, 2010
[8] Feldman et. al.

0
1
2
3
4
5
6
7
8
10 100 500
BillableTimeinMinutes Number of clusters
sagemaker other
K-Means Clustering
k SageMaker Other
Text
1.2 GB
10 1.18E3 1.18E3
100 1.00E3 9.77E2
500 9.18.E2 9.03E2
Images
9 GB
10 3.29E2 3.28E2
100 2.72E2 2.71E2
500 2.17E2 Failed
Videos
27 GB
10 2.19E2 2.18E2
100 2.03E2 2.02E2
500 1.86E2 1.85E2
Advertising
127 GB
10 1.72E7 Failed
100 1.30E7 Failed
500 1.03E7 Failed
Synthetic
1,100 GB
10 3.81E7 Failed
100 3.51E7 Failed
500 2.81E7 Failed
Running time vs. Number of clusters
~10x faster!

Principal Component Analysis (PCA)

Principal Component Analysis (PCA)
More than 10x faster
at a fraction the cost!
0.00
20.00
40.00
60.00
80.00
100.00
120.00
8 10 20
Mb/Sec/Machine
Number of machines
other sagemaker-deterministic sagemaker-randomized
Cost vs. time Throughput and scalability
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 5 10 15 20 25 30 35 40 45
CostinDollars
Billable time in minutes
other sagemaker-deterministic sagemaker-randomized

Neural Topic Modeling
• Perplexity vs. number of topics
Encoder: feedforward net
Input term counts vector
Document
posterior
Sampled document
representation
Decoder:
Softmax
Output term counts vector
0
2000
4000
6000
8000
10000
12000
0 50 100 150 200
Perplexity
Number of topics
NTM Other

Time Series Forecasting
Mean absolute
percentage error
P90 Loss
DeepAR R DeepAR R
traffic
Hourly occupancy rate of 963
bay area freeways
0.14 0.27 0.13 0.24
electricity
Electricity use of 370
homes over time
0.07 0.11 0.08 0.09
pageviews
Page view hits of
websites
10k 0.32 0.32 0.44 0.31
180k 0.32 0.34 0.29 NA
1 hour on p2.xlarge, $1
Input
Network

Boosted Decision Trees
Throughput vs. number of machinesXGBoost is one of the most
commonly used
implementations of boosted
decision trees in the world.
It is now available in Amazon
SageMaker!
0
200
400
600
800
1000
1200
1400
0 10 20 30 40 50 60 70
ThroughputinMB/Sec
Number of machines (C4.8xLarge)

Using Amazon SageMaker

Input data
From the Command Line
profile=<your_profile>
arn_role=<your_arn_role>
training_image=382416733822.dkr.ecr.us-east-1.amazonaws.com/kmeans:1
training_job_name=clustering_text_documents_`date '+%Y_%m_%d_%H_%M_%S'`
aws --profile $profile
--region us-east-1
sagemaker create-training-job
--training-job-name $training_job_name
--algorithm-specification TrainingImage=$training_image,TrainingInputMode=File
--hyper-parameters k=10,feature_dim=1024,mini_batch_size=1000
--role-arn $arn_role
--input-data-config '{"ChannelName": "train", "DataSource":
{"S3DataSource":{"S3DataType": "S3Prefix", "S3Uri": "s3://kmeans_demo/train",
"S3DataDistributionType": "ShardedByS3Key"}}, "CompressionType": "None",
"RecordWrapperType": "None"}'
--output-data-config S3OutputPath=s3://training_output/$training_job_name
--resource-config InstanceCount=2,InstanceType=ml.c4.8xlarge,VolumeSizeInGB=50
--stopping-condition MaxRuntimeInSeconds=3600
Hardware
Algorithm

From Amazon EMR
Start training
Parameters
Hardware
Apply model

From Amazon SageMaker Notebooks
Parameters
Hardware
Start training
Host model

ML compute by the
second starting
at $0.0464/hr
ML storage by the
second
at $0.14
per GB-month
Data processed in
notebooks and hosting
at $0.016 per GB
Free trial to get
started quickly
Pay-as-You-Go (Inexpensive, Too)

Amazon SageMaker – Try It Free
aws.amazon.com/sagemaker

Please complete the session
survey in the summit mobile app.

Submit Session Feedback
1. Tap the Schedule icon. 2. Select the session
you attended.
3. Tap Session
Evaluation to submit your
feedback.

Thank you!

BDA301 Working with Machine Learning in Amazon SageMaker: Algorithms, Models, and Inferences

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to BDA301 Working with Machine Learning in Amazon SageMaker: Algorithms, Models, and Inferences

Similar to BDA301 Working with Machine Learning in Amazon SageMaker: Algorithms, Models, and Inferences (20)

More from Amazon Web Services

More from Amazon Web Services (20)

BDA301 Working with Machine Learning in Amazon SageMaker: Algorithms, Models, and Inferences