Amazon SageMaker is a fully managed platform that enables developers and data scientists to build, train, and deploy machine learning (ML) models in production applications easily and at scale. In this chalk talk, we dive deep into training an ML model based on the TensorFlow framework. We discuss the specifics of training a model through Amazon SageMaker by taking an algorithm and running it on a training cluster in an auto-scaling group. This session showcases the scalability of training that is possible with Amazon SageMaker, which reduces the time and cost of training runs.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Keith Steward, Ph.D.
AI Specialist Solutions Architect, AWS
SRV336
Train Machine Learning Models using
Amazon SageMaker with TensorFlow

2. Questions We’ll Answer in This Session:
1. Amazon SageMaker: What is it? How does it work?
2. Why run TensorFlow on SageMaker?
3. How to train a TensorFlow model using SageMaker? (Demo)
4. How to host a trained TensorFlow model on SageMaker to provide
scalable inferencing service? (Demo)
5. How to get started?

3. 1. Amazon SageMaker:
What is it?
How does it work?

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process Is Complex
Re-training

5. The Amazon Machine Learning Stack
FRAMEWORKS & INTERFACES
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Chainer Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker
Amazon
Rekognition
Amazon
Transcribe
Amazon
Translate
Amazon Polly
Amazon
Comprehend
Amazon Lex
AWS
DeepLens
EDUCATION
PLATFORM SERVICES
APPLICATION SERVICES
Amazon Mechanical Turk

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Amazon SageMaker is a fully managed platform that enables
developers and data scientists to quickly and easily build, train, and
deploy machine learning models at any scale.
Amazon SageMaker removes all the barriers that typically slow down
developers who want to use machine learning.

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
One-click training for
ML, DL, and custom
algorithms
Easier training with
hyperparameter
optimization
Highly optimized
machine learning
algorithms
Deployment
without engineering
effort
Fully managed
hosting at scale
BuildPre-built notebook
instances
Deploy
Train
Amazon SageMaker

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Amazon SageMaker
Client application
Training code

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Trainingdata
Training code Helper code
Client application
Training code
Amazon SageMaker

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Client application
Inference code
Training code
Amazon SageMaker

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
Client application
Inference code
Training code
Amazon SageMaker

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
GroundTruth
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

14. ML Hosting Service
Amazon ECR
30 50
10 10
Model Artifacts
Inference Image
Model versions
One-Click!
EndpointConfiguration
Inference Endpoint
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker – with Split Testing
Versions of the
same inference
code saved in
inference
containers. Prod
is the primary
one, 50% of the
traffic must be
served there!
ModelName: prod
Create a ModelCreate versions of a Model
Create weighted
ProductionVariants
Create
EndpointConfiguration from
one or more
ProductionVariant(s)
Create an Endpoint from one
EndpointConfiguration

15. 2. Why run TensorFlow on SageMaker?

16. Remove undifferentiated
heavy lifting of ML
Iterate, iterate,
iterate!
Training speed Inference speed
Speed & agility in all phases
Amazon SageMaker

17. Fully managed
hosting with
auto-scaling
One-click
deployment
Pre-built
notebooks for
common
problems
Built-in, high
performance
algorithms
One-click
training
BUILD T R AI N & T UN E D EPLOY
Build, train, tune, and host your own models
Automated
hyperparameter
optimization
Amazon SageMaker

18. Enabling experimentation speed
Tr a i n w i t h
l o c a l n o t e b o o k s
Train on notebook
instances
PetaFLOP
training on p3.16xl
Go distributed
with one line of code
Same containers
Amazon SageMaker Training Service

19. Automatic Model Tuning
Adjusting algorithm parameters to arrive at the best model
Neural Networks
Number of layers
Hidden layer width
Learning rate
Embedding
dimensions
Dropout
…
Decision Trees
Tree depth
Max leaf nodes
Gamma
Eta
Lambda
Alpha
…
…
“Hyperparameters”
(algorithm parameters that significantly affect model quality)
Tuning Strategy: Customized Bayesian Optimization

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker Model Hosting Service
• Deploys your model for inferencing as an API endpoint
• Requires previously trained model
• Created via web console, or via one line of code
• In Python: sagemaker.deploy() method
• Manages the infrastructure on your behalf:
• Amazon EC2 instances
• Docker containers
• Auto-scaling option

21. 3. How to Train a TensorFlow model using
SageMaker? (Demo)

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Training Iris DNN Model on SageMaker
Using
TensorFlow Estimator (tf.estimator)
Demo:

23. 4. How to host a trained TensorFlow model on
SageMaker to provide scalable inferencing
service? (Demo)

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
{ Return to Demo:
Hosting Iris DNN Model on SageMaker
for Inferencing Requests
}

25. Summary
1. Amazon SageMaker supports the complete TensorFlow ML/DL
life-cycle with speed and agility
• Jupyter notebooks provided
• Built-in cloud-scale ML algorithms
• Fully managed and scalable training service (incl. automated model
tuning)
• Model hosting with auto-scaling
• Full lifecycle can be scripted for CI/CD
2. TensorFlow code can be brought to SageMaker either as scripts
or within Docker containers.
3. We demonstrated a complete example of TensorFlow coding,
training, and model hosting.

26. 5. How to get started?

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Getting Started Running TensorFlow on Amazon
SageMaker:
• Try Amazon SageMaker for free
using the Free Tier
(https://amzn.to/2JfdiZ0)
• Read the abundant documentation
online:
(https://amzn.to/2KRD06y)
• Try the many TensorFlow sample
notebooks included with SageMaker
(code also available on GitHub:
http://bit.ly/2KXLPMc)
• Amazon ML Blog Posts about
SageMaker
(https://amzn.to/2KRxAZn)
• Consult SageMaker TensorFlow
Container repo on GitHub
(http://bit.ly/2KWpDSw)

28. Please Submit Session Feedback:
1. Tap on the Agenda icon
2. Select the session you attended
3. Tap on Session Evaluation to submit
your feedback
Thank You !
Questions ?