Amazon SageMaker is a fully managed platform for data scientists and developers to build, train and deploy machine learning models in production applications. In this workshop, you will learn how to integrate Amazon SageMaker with other AWS services in order to meet enterprise requirements. Using Amazon S3, Amazon Glue, Amazon KMS, Amazon SageMaker, Amazon CodeStar, Amazon ECR, IAM; we will walkthrough the machine learning lifecycle in an integrated AWS environment and discuss best practices. Attendees must have some familiarities with AWS products as well as a good understanding of machine learning theory. The dataset for the workshop will be provided.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Integrating Deep Learning Into Your Enterprise
Yash Pant, Enterprise Solutions Architect
Nick Brandaleone, Solutions Architect
Anjana Kandalam, Solutions Architect
Ro Mullier, Solutions Architect

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Recommendations & Ranking
Personalized ranking,
page generation,
search, similarity, ratings

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Uncovering Images that Resonate with Customers

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
The Amazon machine learning stack
PLATFORM SERVICES
APPLICATION SERVICES
FRAMEWORKS & INTERFACES
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Torch Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker AWS DeepLens
Rekognition Transcribe Translate Polly Comprehend Lex
Amazon Mechanical Turk Amazon ML

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Direct Connect
Kinesis
Snowball
Snowmobile
IoT
Database Migration
Streams, Firehose,
Analytics
Upload
RDS MySQL, PostgreSQL, MariaDB
Oracle, SQL Server
Aurora MySQL
PostgreSQL
Storage, DB & Analytics
S3
EC2
DynamoDB,
Neptune
EMR
Elasticsearch
ElastiCache
Glacier
Redshift
Athena
QuickSight
Lambda
Cloud Search
Spectrum

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training
• Build the data platform:
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift
Spectrum

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training
• Builds the ML Models:
• AWS Deep Learning
AMI
• SparkML on Amazon
EMR
• Amazon SageMaker

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training
• Build Smart Apps
• AWS Lambda
• Amazon S3
• API Gateway
• IoT
• Kinesis
• ECS/ECR
• Mobile Hub
• AWS KMS
• EC2
• Amazon SageMaker

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Enterprise Data is Everywhere!!
And many many more!
How do we know what data is where?
How do we catalog this data?
How do we keep up to date with?
How do we combine data across data sources?
How do we analyze and create models?

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Data Big Picture

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Training Models
Supports both
CPU/GPU Training
Run common libraries –
Scikit-learn, R, Tensorflow,
Keras, MXNet, Gluon,
PyTorch, etc.
Connect multiple EC2
instances to create ML/DL
training clusters
Preprocess large
amounts of data
efficiently for
model training
Take advantage of
Spark ML to build
models
Dataset Training Model
Amazon S3
Amazon EMRAmazon EC2
Amazon EMR
Amazon EC2

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Enterprise Training Considerations
What happens when you have many data scientists?
- How to manage EMR/Spark environments at scale?
- How do you provide DL clusters to many scientists at scale?
- How to work across different lines of business?
- How to control costs?
- How do you make sure everyone operates securely?
Use an ML Platform! (Like Amazon SageMaker)

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Moving to Production
Tensorflow
save_path = saver.save(sess, ”my_model")
saver.restore(sess, "/my_model")
MXNet
mx.model.save("my_model")
Model = mx.model.load("my_model")
XGBoost
bst.save_model(’my_model')
bst2 = xgb.Booster(model_file=’my_model')
Serialized
flat file(s)
& Others
Amazon ECS
AWS IoT &
Greengrass
AWS EC2/F1
AWS Lambda
AWS Batch
Amazon
SageMaker

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon S3
AWS CodeBuildAWS CodePipelineAWS CodeCommit
App developer
Amazon
ECR
Amazon
ECS
Data scientist
Pulls Docker
image from ECR
Downloads
model from S3
DevOps Engineer
End User
Deployment to ECS
Checks in code
Outputs model
Execute Build

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lambda based prediction
Amazon S3
AWS CloudFormation
AWS CodeBuildAWS CodePipelneAWS CodeCommit
AWS Lambda
1.Check in code
2.Execute Build
3. Update function5. Predict
4. Load Model
DevOps Engineer
App Developer

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
ML Platform – Amazon SageMaker
Fully managed service that enables developers and data scientists to quickly
and easily build, train, and deploy machine learning models at any scale

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (SM builds the Container)
Bring Your Own Algorithm (You build the Container)
I
ML Training Service
Fetch Training data
Save Model Artifacts
Fully
managed –
Secured–
Amazon ECR
Save Inference Image
IM Estimators in
Apache Spark
CPU GPU HPO
Amazon SageMaker - Training

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
MinInstanceCount: 5
MaxInstanceCount: 20
ModelName: prod
VariantName: prodPrimary
VariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!
One-Click!
EndpointConfiguration
Inference Endpoint
Amazon Provided Algorithms
Amazon SageMaker
BYOA
Amazon SageMaker - Hosting

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Summary
Data is everywhere
- We need a plan to manage and maintain it
- Tools like S3, Glue, EMR/Spark help
Use an ML Platform such as SageMaker to enable ML at scale
- Options for data management, model training, and hosting