Algorithms and frameworks form a fundamental part of machine learning (ML). These critical components enable developers and data scientists to easily and quickly build ML models with well-defined interfaces for a range of use cases. The most commonly used algorithms and frameworks, built-in with Amazon SageMaker, make ML easier to address these use cases. In this session, we discuss the built-in algorithms and frameworks and how you can leverage them for your ML models. We also discuss the flexibility of bringing your own algorithm into Amazon SageMaker depending on your needs.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kumar Venkateswar
Sr. Product Manager, Amazon SageMaker, Amazon Web Services
BDA304
Machine Learning with Amazon
SageMaker − Algorithms and Frameworks

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solving Some of the Hardest Problems in Computer Science
Learning Language Perception Problem
Solving
Reasoning

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning at Amazon: A Long Heritage
Personalized
Recommendations
Fulfillment automation
& Inventory Management
Drones Voice-driven
Interactions
Inventing Entirely New
Customer Experiences

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tens of Thousands of Customers Running Machine Learning on AWS

5. ML @ AWS
OUR MISSION
Put machine learning in the
hands of every developer and
data scientist

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FRAMEWORKS
KERAS
PLATFORMS
APPLICATION SERVICES
A M A Z O N
R E K O G N I T I O N
A M A Z O N R E K O G N I T I O N
V I D E O
A M A Z O N
P O L L Y
A M A Z O N
T R A N S C R I B E
A M A Z O N
T R A N S L A T E
A M A Z O N
C O M P R E H E N D
A M A Z O N
L E X
Amazon SageMaker Amazon Mechanical Turk
The AWS Machine Learning Stack

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k s I n t e r f a c e s
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k s I n t e r f a c e s
NVIDIA
Tesla V100 GPUs
(14x faster than P2)
P3
Machine Learning
AMIs
5,120 Tensor cores
128GB of memory
1 Petaflop of compute
NVLink 2.0
I N F R A S T R U C T U R E ( G P U )
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k s I n t e r f a c e s
NVIDIA
Tesla V100 GPUs
(14x faster than P2)
P3
Machine Learning
AMIs
5,120 Tensor cores
128 GB of memory
1 petaflop of compute
NVLink 2.0
I N F R A S T R U C T U R E ( G P U )
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner
Intel Xeon
3.0 GHz Skylake CPU
(25% better perf/price than C4)
Machine Learning
AMIs
72 vCPUs
144 GB of memory
AVX 512
Nitro Hypervisor
I N F R A S T R U C T U R E ( C P U )
C5

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
I
Algorithms
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (SM builds the Container)
SM Estimators in
Apache Spark Bring Your Own Algorithm (You build the Container)
Amazon SageMaker: 10x Better Algorithms

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning Process − Review

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Visualization &
Analysis
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
• Set up and manage notebook
environments
• Set up and manage training
clusters
• Write data connectors
• Scale ML algorithms to large
datasets
• Distribute ML training
algorithm to multiple
machines
• Secure model artifacts
Why We built Amazon SageMaker: The Model Training Undifferentiated Heavy Lifting

14. • XGBoost
• Factorization
Machines
• Linear methods
• Autoregressive
models
• K-Means Clustering
• PCA
Image classification
with convolutional
neural networks
• Spectral LDA
• Neural Topic Models
• Seq2Seq for
translation and similar
problems
• BlazingText
Built-in Algorithms

15. Streaming datasets,
for cheaper training
Train faster, in a
single pass
Greater reliability on
extremely large
datasets
Choice of several ML
algorithms
Algorithms Designed for Huge Datasets

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Single
Machine
Distributed, with
Strong Machines

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming
GPU State

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming
Data Size
Memory
Data Size
Time/Cost

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed
GPU State
GPU State
GPU State

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shared State
GPU
GPU
GPU Local
State
Shared
State
Local
State
Local
State

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Best Alternative
Amazon SageMaker

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker Algorithms and
Frameworks Demo

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Infinitely Scalable Algorithms

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Linear Learner
Regression (mean squared error)
SageMaker Other
1.02 1.06
1.09 1.02
0.332 0.183
0.086 0.129
83.3 84.5
Classification (F1 Score)
SageMaker Other
0.980 0.981
0.870 0.930
0.997 0.997
0.978 0.964
0.914 0.859
0.470 0.472
0.903 0.908
0.508 0.508
30 GB datasets for web-spam and web-url classification
0
0.2
0.4
0.6
0.8
1
1.2
0 5 10 15 20 25 30
CostinDollars
Billable time in Minutes
sagemaker-url sagemaker-spam other-url other-spam

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Factorization Machines
Log_loss F1 Score Seconds
Amazon
SageMaker
0.494 0.277 820
Other (10 Iter) 0.516 0.190 650
Other (20 Iter) 0.507 0.254 1300
Other (50 Iter) 0.481 0.313 3250
Click Prediction 1 TB advertising dataset,
m4.4xlarge machines, perfect scaling.
$-
$20.00
$40.00
$60.00
$80.00
$100.00
$120.00
$140.00
$160.00
$180.00
$200.00
1 2 3 4 5 6 7 8
CostinDollars Billable Time in Hours
10
machines
20
machines
30
machines
4050

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
0
1
2
3
4
5
6
7
8
10 100 500
BillableTimeinMinutes Number of Clusters
sagemaker other
K-Means Clustering
k SageMaker Other
Text
1.2GB
10 1.18E3 1.18E3
100 1.00E3 9.77E2
500 9.18.E2 9.03E2
Images
9GB
10 3.29E2 3.28E2
100 2.72E2 2.71E2
500 2.17E2 Failed
Videos
27GB
10 2.19E2 2.18E2
100 2.03E2 2.02E2
500 1.86E2 1.85E2
Advertising
127GB
10 1.72E7 Failed
100 1.30E7 Failed
500 1.03E7 Failed
Synthetic
1100GB
10 3.81E7 Failed
100 3.51E7 Failed
500 2.81E7 Failed
Running Time vs. Number of Clusters
~10x Faster!

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Principal Component Analysis (PCA)
0.00
20.00
40.00
60.00
80.00
100.00
120.00
8 10 20
Mb/Sec/Machine
Number of Machines
other sagemaker-deterministic sagemaker-randomized
Cost vs. Time Throughput and Scalability
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 10 20 30 40 50
CostinDollars
Billable time in Minutes
other sagemaker-deterministic sagemaker-randomized
More than 10x faster
at a fraction the cost!

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Neural Topic Modeling
Perplexity vs. Number of Topics
(~200K documents, ~100K vocabulary)
Encoder: feedforward net
Input term counts vector
Document
Posterior
Sampled Document
Representation
Decoder:
Softmax
Output term counts vector
0
2000
4000
6000
8000
10000
12000
0 50 100 150 200Perplexity
Number of Topics
NTM Other

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DeepAR − Forecasting
Mean absolute
percentage error
P90 Loss
DeepAR R DeepAR R
traffic
Hourly occupancy rate of 963
bay area freeways
0.14 0.27 0.13 0.24
electricity
Electricity use of 370 homes
over time
0.07 0.11 0.08 0.09
pageviews
Page view hits of
websites
10k 0.32 0.32 0.44 0.31
180k 0.32 0.34 0.29 NA
One hour on p2.xlarge, $1
Input
Network

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
More Great Algorithms

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spectral LDA
Training Time vs. Number of Topics
0
50
100
150
200
250
0 20 40 60 80 100TrainingTimeinMinutes
Number of Topics
lda-data-a lda-data-b other-data-a other-data-b

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Boosted Decision Trees
Throughput vs. Number of MachinesXGBoost is one of the most
commonly used
implementations of boosted
decision trees in the world.
It is now available in Amazon
SageMaker!
0
200
400
600
800
1000
1200
1400
0 10 20 30 40 50 60 70
ThroughputinMB/Sec
Number of Machines (C4.8xLarge)

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sequence to Sequence
English−German Translation
0
5
10
15
20
25
0 5 10 15 20 25
BLEUScore
Billable Time in Hours
P2.16x P2.8x P2.x
Best known result!
Based on Sockeye and Apache
incubated MXNet, Multi-GPU,
and can be used for neural
machine translation.
Supports both RNN / CNN
as encoder / decoder.

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Image Classification
Implementation in MXNet of
ResNet.
Other networks such as
DenseNet and Inception will be
added in the future.
Transfer learning: Begin with a
model already trained on
ImageNet!
0
0.5
1
1.5
2
2.5
3
3.5
0 1 2 3 4 5
Speedup
Number of Machine (P2)
Speedup with Horizontal Scaling

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BlazingText – Word2Vec

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Random Cut Forest

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Built-in Algorithms
Deep Learning
Frameworks
MXNet & Gluon
Tensorflow & Keras
PyTorch
Chainer
Custom / Docker
Training (single or
distributed cluster)
Data
Amazon SageMaker: Algorithm and Framework Support

38. … explore and refine
models in a single
notebook instance
TensorFlow, Apache MXNet, PyTorch, Chainer Containers
… deploy to
production
Sample your data… Use the same code
to train on the full
dataset in a cluster
of GPU instances …

39. Bring Your Own Algorithm
... add algorithm
code to a Docker
container ...
Choose your own framework
... publish to Amazon
ECR
Amazon ECS

40. Custom Docker
• Amazon SageMaker runs “docker run image train” for training
and “docker run image serve” for inference
• tini initializer gets confused by this 
• The exec form of ENTRYPOINT is recommended
• Include only CUDA Toolkit, don’t include drivers
• /opt/ml used for providing data, hyperparameters, etc.
• Inference web server must respond to /invocations and /ping
on port 8080

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customer Success with Amazon SageMaker

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker Keeps Getting Better!
Recent launches include:
• New deep learning frameworks (Chainer and PyTorch)
• More security and compliance features
• Expansion around the world − Tokyo, Seoul, Frankfurt, Sydney
• Automatic model tuning
• Local mode w/ GPU in Amazon SageMaker notebooks
• Algorithm Enhancements: DeepAR, BlazingText, Linear Learner
• TensorFlow Pipe-Mode
• Batch Transform

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Lake Storage
Amazon S3
Security
Access Control
Encryption
VPC
AWS KMS
Auditing
Compliance
Roles
Fine-Grained Access Controls
Compute
Powerful GPU & CPU Instances
AWS Lambda
Analytics
Amazon Athena
Amazon EMR
Amazon Redshift & Redshift Spectrum
Broadest & Easiest to Use ML Platform with the
Most Customers

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Submit Session Feedback
1. Tap the Schedule icon.
2. Select the session you
attended.
3. Tap Session Evaluation to
submit your feedback.

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Useful Amazon SageMaker Resources
https://github.com/awslabs/amazon-sagemaker-examples
https://github.com/aws-samples/aws-ml-vision-end2end
https://github.com/juliensimon
https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html
https://github.com/aws/sagemaker-spark
Thank you!