Learn how to get started with Amazon SageMaker—our fully-managed service that spans the entire machine learning (ML) workflow—so you can build, train, and deploy models quickly. Use Amazon SageMaker to label and prepare your data, choose an algorithm, train, tune, and optimize it for deployment, make predictions, and take action. Get your models to production faster with Amazon SageMaker SDKs, builder tools, and APIs tailored to your programming language or platform. Also, discover how Amazon SageMaker Ground Truth can aid in the adoption of ML technology for your organization.

1. P U B L I C S E C T O R
S U M M I T

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon SageMaker
Build, Train, and Deploy Machine Learning Models Quickly & Easily, at scale
Kumar Venkateswar
Principal Product Manager
Amazon SageMaker, AWS
2 9 6 6 9 6
Balaji Iyer
Business Development Manager, AI and ML
Worldwide Public Sector, AWS

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Customer-focused
90%+ of our ML roadmap is
defined by customers
Pace of innovation
200+ new ML launches
and major feature updates last year
Breadth and depth
A wide range of AI and ML services
Multi-framework
Support for the most
popular frameworks
Security and analytics
Deep set of security with robust encryption and
analytics
Embedded R&D
Customer-centric approach
Our Approach to Machine Learning

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E L E XR E K O G N I T I O N
V I D E O
Vision Speech Chatbots
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s G R E E N G R A S S
E L A S T I C
I N F E R E N C E
Language Forecasting Recommendations
T E X T R A C T
New
C O M P R E H E N D &
C O M P R E H E N D
M E D I C A L New
NewNew
F O R E C A S T P E R S O N A L I Z E
A M A Z O N
S A G E M A K E R
G R O U N D T R U T H
New
N O T E B O O K S
A W S M A R K E T P L A C E
New
A L G O R I T H M S
R E I N F O R C E M E N T
L E A R N I N G
New
T R A I N I N G
O P T I M I Z A T I O N
( N E O ) New
D E P L O Y M E N T
H O S T I N G
New
RL Coach

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale
1
2
3

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
1
2
3
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Ongoing enhancements to Amazon SageMaker
MXNet 1.3 container | CloudTrail integration for audit logs | TensorFlow 1.7 Containers | Automatic Model Tuning—Add/Delete tags | Jupyter Notebooks IP Filtering
Region expansion to SFO | Image Classification Multi-label Support | TensorFlow and MXNet Containers—Open Sourcing and Local Mode | PyTorch pre-built container
Region expansion to PDT | Batch customer VPC | PCI DSS Compliance | XGBoost Instance Weights | NTM—vocab, metrics, and subsampling
Anomaly Detection (Random Cut Forest) Algorithm | Deep AR algorithm | SageMaker region expansion to ICN | Hyperparameter tuning job cloning on the console
Autoscaling console | PyTorch 1.0 container | Customer VPC support for training and hosting | PrivateLink support for SageMaker inferencing APIs
Horovod support in TensorFlow Container | Variable sizes for notebook EBS volumes |nbexample support in SageMaker notebook instances | Tag-based access control
Automatic Model Tuning—early stopping | IP Insights algorithm | Chainer 4.1 Container | Region expansion to SIN Built-in Algorithms Pipe Mode Support
TensorFlow 1.8 Container | Region expansion to FRA | Training job cloning in console | Algorithm Pipe mode enhancements | Pipe mode support for text, recordIO, and images
TensorFlow 1.5, MXNet 1.0, and CUDA 9 Support | DeepAR Algorithm Enhancements | Linear Learner Multi-class Classification | TensorFlow 1.10 Container
Region expansion to YUL | BlazingText Algorithm | Batch KMS | k-nearest neighbors | Object detection |Chainer pre-built container | Apache Airflow integration
Region expansion to BOM | GDPR compliance | BlazingText Enhancements | TensorFlow 1.9 Container | Notebook bootstrap script
Amazon SageMaker Hosting custom header attribute | Metrics Support in Training Jobs | Object2vec | TensorFlow container enhancements | CloudFormation support
PrivateLink support for SageMaker Control Plane | MXNet 1.2 Container | HIPAA compliance | Ground Truth | Python SDK Marketplace support
Git integration for SageMaker notebooks | Pipe mode support for TensorFlow | ml.p3.2xlarge notebook instances | Internet-free notebook instances
Semantic segmentation algorithm | SageMaker Reinforcement Learning support | Linear Learner Improvements | SageMaker Batch Transform
Region expansion to NRT | High Performance I/O streaming in PIPE Mode | Pause/resume for active learning algorithms | Pre -built scikit-learn container
Step Functions for SageMaker | KMS support for training and hosting | Incremental learning algorithm enhancements | TensorFlow 1.11 container | NTM feature release
Deep Learning Compiler | ONNX Support for Frameworks and Algorithms |Full instance type support | Pipe mode CSV support | Reg ion expansion to LHR
Incremental training platform support | Login anomaly detection algorithm | Serial inference pipeline | Experiment Management | Region expansion to SYD
MXNet container enhancements | Automatic Model Tuning | Automatic Model Tuning—incremental tuning | Spark MLeap 1P container
TensorFlow 1.6 and MXNet 1.1 Containers | Region expansion to SIN | Mead Notebook PrivateLink Support | Linear Learnersupport

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Collect and
prepare training
data
Choose and
optimize your
ML algorithm
Train and
tune models
Set up and
manage
environments
for training
Deploy model
in production
Scale and manage
the production
environment
1
2
3
1
2
3
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Successful models require high-quality data

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Successful models require high-quality data

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Ground Truth
Build highly accurate training datasets and reduce data labeling costs
by up to 70% using machine learning

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Ground Truth
How it works

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Ground Truth
How it works

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Ground Truth
How it works

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Ground Truth
How it works

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Ground Truth
How it works

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
K E Y F E A T U R E S
Automatic labeling via
machine learning
Ready-made and
custom workflows
Label
management
Private and public
human workforce
Amazon SageMaker Ground Truth
Label machine learning training data easily and accurately

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Collect and
prepare training
data
Choose and
optimize your
ML algorithm
Train and
tune model
Set up and
manage
environments
for training
Deploy model
in production
Scale and manage
the production
environment
Amazon SageMaker
Ground Truth
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Marketplace for Machine Learning
Over 150 algorithms and models that
can be deployed directly to Amazon SageMaker

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Marketplace for Machine Learning
ML algorithms and models available instantly
Subscribe in a
single click
Available in
Amazon SageMaker
KEY FEATURES
Automatic labeling via machine learning
IP protection
Automated billing and metering
Browse or search
AWS Marketplace
S E L L E R S
Broad selection of paid, free, and
open-source algorithms and models
Data protection
Discoverable on your AWS bill
B U Y E R S

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Over 200 algorithms and models
Natural Language
Processing
Grammar & Parsing Text OCR Computer Vision
Named Entity
Recognition
Video Classification
Speech Recognition Text-to-Speech Speaker Identification Text Classification 3D Images Anomaly Detection
Text Generation Object Detection Regression Text Clustering
Handwriting
Recognition
Ranking
A V A I L A B L E A L G O R I T H M & M O D E L S
S E L E C T E D V E N D O R S

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Collect and
prepare training
data
Choose and
optimize your
ML algorithm
Train and
tune models
Set up and
manage
environments
for training
Deploy model
in production
Scale and manage
the production
environment
Amazon EC2 P3dn
Instances
Amazon SageMaker
Ground Truth
Amazon Elastic
Inference
AWS Marketplace for
Machine Learning
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Optimization is extremely complex

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Neo
Train once, run anywhere with 2x the performance

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Neo: Train once, run anywhere
Neo

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker Neo
Train once, run anywhere with 2x the performance
K E Y F E A T U R E S
Open-source device runtime and compiler,
1/10th the size of original frameworks

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Machine Learning is a highly collaborative
process every step of the way

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
SageMaker Workflows
Experiment Management
Organize, track and evaluate
model training experiments
with SageMaker Search

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
SageMaker Workflows
Experiment Management Automation
Organize, track, and evaluate
model training experiments
with SageMaker Search
Use AWS Step Functions to
automate end-to-end
workflows
Integrate with Apache Airflow

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
SageMaker Workflows
Experiment Management Automation Collaboration
Organize, track, and evaluate
model training experiments
with SageMaker Search
Use AWS Step Functions to
automate end-to-end
workflows
Integrate with Apache Airflow
Link GitHub, AWS
CodeCommit and self-hosted
Git repositories to notebooks
Clone public and private
repositories
Secure information with IAM,
LDAP and AWS Secrets
Manager

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Sowhat’snextfor
machinelearning?
Howdoyouteachmachinelearningmodelstomakedecisions
whenthereisnotrainingdata?

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Introducing Reinforcement learning (RL)
Reinforcement learning
(RL)
Supervised learning
(ASR, computer vision)
Unsupervised learning
(Anomaly detection,
identifying text topics)
Amount of labeled training data required
Complexityofdecisions

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
What is a RL environment?
Representation
of the real world
Programmed
to represent real-
world conditions
Enables interaction
with user or a
computer program
Dynamic and updates
itself based on the
interactions and
programmed behavior

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
RLmodelslearnhowtomakedecisions
toaccomplishtasks

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
This makes RL applicable in many domains
and not just gaming
Robotics Industrial
control
HVAC Autonomous
vehicles
NLP Operations Finance Resource
allocation
Advertising
Online content
delivery

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Reinforcement Learning
Achieve outcomes, not decisions
Robotics Industrial controls Natural language
systems
Games

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How does RL work?

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How does RL work?
Extremely complex Expensive
Effectively out of reach

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker RL
New machine learning capabilities in Amazon SageMaker to
build, train, and deploy with reinforcement learning

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon SageMaker RL
Reinforcement learning for every developer and data scientist
2D & 3D physics
environments and
OpenGym support
Support Amazon Sumerian, AWS
RoboMaker and the open source
Robotics Operating System
(ROS) project
Fully
managed
Example notebooks
and tutorials
K E Y F E A T U R E S

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Predictions drive
complexity and cost in
production
Inference
(Prediction)
90%
Training
10%

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
The challenges of inference in production
One size does not fit allLow utilization and high costs
How do we optimize resources and reduce costs?

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Elastic Inference
Reduce Deep Learning inference costs by up to 75%

49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Elastic Inference
Lower inference costs Match capacity to
demand
Available between
1 to 32 TFLOPS per
accelerator
KEY FEATURES
Integrated with
Amazon EC2 and
Amazon SageMaker
Support for TensorFlow,
Apache MXNet, and ONNX
with PyTorch coming soon
Single and
mixed-precision
operations
Reduce deep learning inference costs up to 75%

50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How does Elastic Inference work with SageMaker?
SageMaker Notebooks
 Prototype deployments with
Notebooks in local mode
SageMaker Hosted Endpoints
 Scale endpoints with low-cost
Elastic Inference Acceleration

51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Model Support
Amazon EI enabled TensorFlow
Serving and Apache MXNet
ONNX
Amazon EI
enabled
TensorFlow
Serving
Amazon EI
enabled Apache
MXNet
Applied using
Apache MXNet
 Auto discovery of accelerators
 IAM-based authentication
 Available via: the AWS Deep
Learning AMIs, for download
via S3 and automatically
though SageMaker

52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Recap: Amazon SageMaker
Collect and
prepare training
data
Train and
tune models
Set up and
manage
environments
for training
Deploy models
in production
Scale and manage
the production
environment
Amazon EC2 P3
Instances
Amazon SageMaker RL
Amazon SageMaker
Ground Truth
Amazon Elastic
Inference
AWS Marketplace for
Machine Learning Amazon SageMaker Neo

53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E L E XR E K O G N I T I O N
V I D E O
Vision Speech Chatbots
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s G R E E N G R A S S
E L A S T I C
I N F E R E N C E
Language Forecasting Recommendations
T E X T R A C T
New
C O M P R E H E N D &
C O M P R E H E N D
M E D I C A L New
NewNew
F O R E C A S T P E R S O N A L I Z E
A M A Z O N
S A G E M A K E R
G R O U N D T R U T H
New
N O T E B O O K S
A W S M A R K E T P L A C E
New
A L G O R I T H M S
R E I N F O R C E M E N T
L E A R N I N G
New
T R A I N I N G
O P T I M I Z A T I O N
( N E O ) New
D E P L O Y M E N T
H O S T I N G
New
RL Coach

54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Resources
- SageMaker Product Page
- SageMaker Console
- Ground Truth Product Page
- Neo Product Page
- SageMaker RL Documentation
- SageMaker 10-Minute Tutorial
- SageMaker Related Blogs
- Ground Truth Webinar (Dec 2018)

55. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Kumar Venkateswar
vekumar@amazon.com
Balaji Iyer
balaiyer@amazon.com

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T