Learning Objectives: - Get an introduction to Amazon SageMaker - Learn how to integrate Amazon SageMaker and other AWS Services within an Enterprise environment - View a walkthrough of the machine learning lifecycle to cover best practices in the ML process

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Integrating Amazon SageMaker
Dan R. Mbanga – Global Lead BD Mgr, Amazon AI Platforms
into your enterprise

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank You

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solving Some Of The Hardest Problems In Computer Science
Learning Language Perception Problem
Solving
Reasoning

4. Machine Learning at Amazon: A long heritage
Personalized
Recommendations
Fulfillment automation
& Inventory Management
Drones Voice driven
Interactions
Inventing entirely new
Customer experiences

5. Tens of thousands of customers running Machine Learning on AWS

6. ML @ AWS
OUR MISSION
Put machine learning in the
hands of every developer and
data scientist

7. FRAMEWORKS
KERAS
PLATFORMS
APPLICATION SERVICES
R E K O G N I T I O N R E K O G N I T I O N
V I D E O
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X
Amazon SageMaker Amazon Mechanical Turk
The AWS Machine Learning Stack

8. K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k
s
I n t e r f a c e
s
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner

9. K E R A S
F R A M E W O R K S A N D I N T E R F A C E S
F r a m e w o r k
s
I n t e r f a c e
s
NVIDIA
Tesla V100 GPUs
(14x faster than P2)
P3
Machine Learning
AMIs
5,120 Tensor cores
128GB of memory
1 Petaflop of compute
NVLink 2.0
I N F R A S T R U C T U R E
Complete Control over Frameworks & Infrastructure
For the data scientist, AI researcher, or advanced ML practitioner

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Let’s Review the ML Process

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Discovery: The Analysts
Re-training
• Help formulate the right
questions
• Domain Knowledge

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Integration: The Data Architecture
Retraining
• Build the data platform:
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift
Spectrum

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
• Setup and manage Notebook
Environments
• Setup and manage Training
Clusters
• Write Data Connectors
• Scale ML algorithms to large
datasets
• Distribute ML training
algorithm to multiple
machines
• Secure Model artifacts
Why We built Amazon SageMaker: The Model Training Undifferentiated Heavy Lifting

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Business Problem –
Model Deployment
Monitoring &
Debugging
– Predictions
• Setup and manage Model
Inference Clusters
• Manage and Scale Model
Inference APIs
• Monitor and Debug Model
Predictions
• Models versioning and
performance tracking
• Automate New Model
version promotion to
production (A/B testing)
Why We built Amazon SageMaker: The Model Deployment Undifferentiated Heavy Lifting

16. BUI LD
Where should you spend your time?
Design training experiments

17. BUI LD T R AI N
Where should you spend your time?
Run scalable training

18. BUI LD T R AI N
Where should you spend your time?
Tune Hyperparameters
T UN E

19. BUI LD T R AI N D EPLOY
Where should you spend your time?
T UN E
Deploy and operate

20. FRAMEWORKS
KERAS
PLATFORMS
APPLICATION SERVICES
R E K O G N I T I O N R E K O G N I T I O N
V I D E O
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X
Amazon SageMaker Amazon Mechanical Turk
The AWS Machine Learning Stack

21. Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
T R AI N D EPLOY
BUI LD
Build, train, tune, and host your own models
T UN E
Amazon SageMaker

22. Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
Hyperparameter
optimization
D EPLOY
BUI LD T R AI N & T UN E
Build, train, tune, and host your own models
Amazon SageMaker

23. Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI LD T R AI N & T UN E D EPLOY
Build, train, tune, and host your own models
Hyperparameter
optimization
Amazon SageMaker

24. Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI LD T R AI N & T UN E D EPLOY
Build, train, tune, and host your own models
End-to-end encryption with KMS
End-to-end VPC support
Compliance and audit capabilities
Metadata and experiment management capabilities
Pay as you go
Amazon SageMaker
Hyperparameter
optimization

25. Remove undifferentiated
heavy lifting of ML
Iterate, iterate,
iterate!
Training speed Inference speed
Speed in all phases
Amazon SageMaker

26. Enabling experimentation speed
Tr a i n w i t h
l o c a l n o t e b o o k s
Train on notebook
instances
PetaFLOP
training on p3.16xl
Go distributed
with one line of code
Same containers
Amazon SageMaker Local Mode

27. Customer Success With Amazon SageMaker

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker: Launch Customer
“With Amazon SageMaker, we can accelerate our Artificial
Intelligence initiatives at scale by building and deploying our
algorithms on the platform. We will create novel large-scale machine
learning and AI algorithms and deploy them on this platform to solve
complex problems that can power prosperity for our customers.
"
- Ashok Srivastava, Chief Data Officer, Intuit

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key benefits of SageMaker at Intuit
Ad-hoc setup and management of
notebook environments
Limited choices for model
deployment
Competing for compute resources
across teams
Easy data exploration
in SageMaker notebooks
Building around virtualization for
flexibility
Auto-scalable model hosting
environment
From To

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Model Hosting
(SageMaker)
Near real -time frau d d etec tion in AWS u sin g S ageMaker
Calculate
Features
Reader
Cleanser
Processor
Data
Lookup
Training
Feature Store Model Training
(SageMaker)
Model
Client Service
Amazon
EMR
Amazon
SageMaker
Amazon
SageMaker

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1 2 3 4
I I I I
Notebook Instances Algorithms ML Training Service ML Hosting Service
Amazon SageMaker

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1
I
Notebook Instances
Zero Setup For Exploratory Data Analysis
Authoring &
Notebooks
ETL Access to AWS
Database services
Access to S3 Data
Lake
• Recommendations/Personalization
• Fraud Detection
• Forecasting
• Image Classification
• Churn Prediction
• Marketing Email/Campaign Targeting
• Log processing and anomaly detection
• Speech to Text
• More…
“Just add data”
VPC

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Streaming datasets, for
cheaper training
Train faster, in a single
pass
Greater reliability on
extremely large
datasets
Choice of several ML
algorithms
Amazon SageMaker: 10x better algorithms
2
I
Algorithms

34. Algorithms for
“infinite scale”
Distributed by
default
Train on a
data stream
Checkpoint
for re-training
Single pass
training
Not memory
bound
Rethinking Algorithms Design for infinite scale & streaming data
Amazon SageMaker

35. Rethinking Algorithms Design for large scale & streaming data
AlgorithmData Model
Amazon SageMaker

36. Rethinking Algorithms Design for large scale & streaming data
AlgorithmData Model
N EW D AT A
PREDICTION
Amazon SageMaker

37. Rethinking Algorithms Design for large scale & streaming data
Convolutional
neural network
Images Model
NE W DATA
OBJECTS, LOCATIONS, FACES
Amazon SageMaker

38. Rethinking Algorithms Design for large scale & streaming data
K-Means
clustering
Inventory Model
NE W DATA
PREDICTED SALES ACTIVITY
Amazon SageMaker

39. Rethinking Algorithms Design for large scale & streaming data
Logistic
Regression
Purchases Model
NE W DATA
FRAUD RISK
Amazon SageMaker

40. K-Means
XGBoost
Linear Learner
PCA
Seq2Seq
DeepAR
Bring your
own algorithm
Factorization Machines
Image Classification
LDA NTM
BlazingText
Algorithms
as-a-service
+ open source
Amazon
SageMaker
Rethinking Algorithms Design for large scale & streaming data
Amazon SageMaker

41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2
I
Algorithms
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (SM builds the Container)
SM Estimators in
Apache Spark Bring Your Own Algorithm (You build the Container)
Amazon SageMaker: 10x better algorithms

42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Managed Distributed Training with Flexibility
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (SM builds the Container)
Bring Your Own Algorithm (You build the Container)
3
I
ML Training Service
Fetch Training data
Save Model Artifacts
Fully
managed –
VPC–
Amazon ECR
Save Inference Image
SM Estimators in
Apache Spark
CPU GPU Hyperparameter Tuning

43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!

44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
Create a Model
ModelName: prod
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
Create versions of a Model
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
Create weighted
ProductionVariants
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
Create an
EndpointConfiguration from
one or many
ProductionVariant(s)EndpointConfiguration
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50

48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there! Create an Endpoint from one
EndpointConfiguration
EndpointConfiguration
Inference Endpoint
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must be
served there!
One-Click!
EndpointConfiguration
Inference Endpoint
Amazon Provided Algorithms
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
InstanceType: c3.4xlarge
InitialInstanceCount: 3
maxInstanceCount: 10
ModelName: prod
VariantName: primary
InitialVariantWeight: 50

50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
✓ Auto-Scaling Inference APIs
✓ A/B Testing (more to come)
✓ Low Latency & High
Throughput
✓ Bring Your Own Model
✓ Python SDK
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SageMaker
Notebooks
Training
Algorithm
SageMaker
Training
Amazon ECR
Code Commit
Code Pipeline
SageMaker
Hosting
Coco dataset
AWS
Lambda
API
Gateway
SageMaker Sample End-to-End Architecture: Style Transfer
Build
Train
Deploy
static website hosted on S3
Inference requests
Amazon S3
Amazon
Cloudfront
Web assets on
Cloudfront

52. Data Lake Storage
Amazon S3
Security
Access Control
Encryption
VPC
KMS
Auditing
Compliance
Roles
Fine Grained Access Controls
Compute
Powerful GPU & CPU Instances
AWS Lambda
Analytics
Amazon Athena
Amazon EMR
Amazon Redshift & Redshift Spectrum
Broadest & Easiest to Use ML Platform with the
Most Customers

53. ml.aws
Dan R. Mbanga – dmmbanga@amazon.com
Global Lead BD Mgr, Amazon AI Platforms

54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
!GO BUILD!
End-to-End Managed ML Platform