This document outlines an agenda for a machine learning workshop. The agenda includes an introduction to machine learning at AWS, an overview of the machine learning process, and details about the workshop layout and outcomes. It describes a business use case for predicting car evaluations based on attributes like price and safety features. It provides details about the features, sample data, and the machine learning approach using Amazon SageMaker's XGBoost algorithm. It also outlines the workshop execution plan, which is divided into three parts: data preparation and ingestion, machine learning process and pipeline, and machine learning inference.

1. S U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine Learning:
From inception to inference
Aashmeet Kalra
Solutions Architect
Amazon Web Services

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Introduction to machine learning at AWS
• Machine learning process
• Workshop layout and outcomes
• Business use case
• Data ingestion
• Machine learning process
• Inference

4. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. Some of our machine learning customers…

6. AI SERVICES
AMAZON
POLLY
AMAZON
TRANSCRIBE
AMAZON
TRANSLATE
AMAZON
COMPREHEND
AMAZON
LEX
AMAZON
REKOGNITION
VIDEO
AMAZON
FORECAST
AMAZON
PERSONALISE
AMAZON
REKOGNITION
IMAGE
Vision Speech Language Chatbots Forecasting
AMAZON
TEXTRACT
Recommendations
(App developers with
little knowledge of ML)
AMAZON
SAGEMAKER
BUILD TRAIN DEPLOY
Pre-built algorithms and notebooks
Data labelling (AMAZON GROUND TRUTH)
One-click model training and tuning
Optimisation (AMAZON NEO)
One-click deployment and hostingML SERVICES
Reinforcement learning
Algorithms and models
(AWS MARKETPLACE FOR MACHINE LEARNING)
AWS
DEEP RACER
AWS
DeepLens
(ML developers
and data scientists)
Optimised
Frameworks
ML FRAMEWORKS AND
INFRASTRUCTURE
Interfaces Infrastructure
AMAZON
EC2 P3
& P3dn
AMAZON
EC2 C5
FPGAs AWS IOT
GREENGRASS
AMAZON
ELASTIC
INFERENCE
(ML researchers and academics)
AMAZON
INFERENTIA
The Amazon ML stack: Broadest and deepest set of capabilities

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Case study - Reducing the cost of sentiment analysis
“Using Amazon SageMaker and Amazon EC2 C5
instances, it's easier than ever for us to deliver
high-quality sentiment analysis.”
Jonathan Dodson, Vice President of Engineering, Zignal Labs
Zignal Labs performs nuanced sentiment analysis on billions of
stories per month using AWS. Zignal Labs offers solutions that
analyse the entire digital media landscape to deliver instant
insights for the company’s Fortune 1000 customers. The company
built a sentiment analysis pipeline that uses Amazon SageMaker
for machine learning capabilities and Amazon EC2 C5 instances
with Intel Xeon Scalable (Skylake) processors for faster model
training and evaluation.
90%Reduction of operations cost

8. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Glossary
Notebooks
Features
Deep Learning
Model
Parameters
Training
Hyperparameters
Inference
Devops
Pipeline

10. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine learning process
Data Visualisation &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are
Business
Goals
met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training

12. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Human Error
Getting ML out
of the lab
Black Box
Science Project
Time-to-Market
Black Box
Input
Output
Quality Gates
Automated Test
CollaborationTransparency & MonitoringAutomation
DevOps Practices
Data Science Pain Points
Deploy
Test
Build
IT
R&D
?!
Machine Learning Operations ( ML/Ops)

14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Lab prerequisites
• You will use your own AWS Account to build out the labs today
• You must have a sign-in with administrative privileges
• There are resources required by this workshop that are eligible for the
AWS free tier if your account is less than 12 months old
• It is your responsibility to delete any AWS resources after today
to prevent ongoing costs!

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Workshop learnings and outcomes
• Explore a dataset using AWS Analytics services
• Transform and process the data for machine learning
• Develop and train a machine learning model and optimise/tune the
parameters
• Build a pipeline for the machine learning process
• Deploy the trained Machine Learning model as a microservice and run
inferences from it

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Workshop execution plan
• A lecture-style presentation to help enable you on your journey
• The workshop will be divided into 3 parts:
• Data preparation and ingestion - 20 mins
• Machine learning process and pipeline - 60 mins
Lunch/Break - 5 mins
• Machine learning inference - 45 mins

18. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Business problem - Car evaluation
• Our goal is to predict the acceptability of a specific car based on
different attributes
• The outcome of the inference will be evaluation of the car based on
the core features:
• unacc - Unacceptable
• Acc - Acceptable
• good - Good condition
• vgood - Very Good condition

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine Learning approach
At the core, it is a classification problem, and we will train a machine
learning model using Amazon SageMaker’s built-in XGBoost algorithm

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Features
Car Evaluation Database was derived from a simple hierarchical decision
model
The model evaluates cars according to the following concept structure:
CAR car acceptability
. PRICE overall price
. . buying buying price
. . maint price of the maintenance
. TECH technical characteristics
. . COMFORT comfort
. . . doors number of doors
. . . persons capacity in terms of persons to carry
. . . lug_boot the size of luggage boot
. . safety estimated safety of the car

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Attribute values
Attributes:
buying: vhigh, high, med, low
maint: vhigh, high, med, low
doors: 2, 3, 4, 5more
persons: 2, 4, more
lug_boot: small, med, big
safety: low, med, high

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Sample data
vhigh,vhigh,2,2,small,low,unacc
vhigh,vhigh,2,2,small,med,unacc
vhigh,vhigh,2,2,small,high,unacc
vhigh,vhigh,2,2,med,low,unacc
vhigh,vhigh,2,2,med,med,unacc
vhigh,vhigh,2,2,med,high,unacc
vhigh,vhigh,2,2,big,low,unacc
vhigh,vhigh,2,2,big,med,unacc
vhigh,vhigh,2,2,big,high,unacc
vhigh,vhigh,2,4,small,low,unacc
vhigh,vhigh,2,4,small,med,unacc
vhigh,vhigh,2,4,small,high,unacc
vhigh,vhigh,2,4,med,low,unacc
vhigh,vhigh,2,4,med,med,unacc

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Workshop URL
https://rebrand.ly/MLWorkshopSydSummit

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Environment set up
• Set up Jupyter Notebook
• Start a notebook instance and download the notebook
• Prepare files and roles
• Set up S3 bucket
• Create a GitHub OAuth Token
• Create your token at GitHub's Token Settings, making sure to select scopes of repo and
admin:repo_hook. After clicking Generate Token, make sure to save your OAuth Token in a
secure location. The token will not be shown again.

27. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine learning process - Ingestion
Business problem –
ML problem framing
Data visualisation
and
analysis
Data collection
Data integration
Data preparation and
cleaning
Feature engineering
Model training and
parameter tuning
Model evaluation
Are
business
goals
met?
Model deployment
Monitoring and
debugging
– Predictions
YesNo
Dataaugmentation
Feature
augmentation
Re-training

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Ingestion process

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Execution
• Open Jupyter Notebook ‘Lab 1 –
Car Evaluation Data Engineering’
• Access the Jupyter notebook to
run the steps required to
transform the dataset
• The detailed instructions are in
the Jupyter notebook

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we have done so far
• Set up S3 bucket
• Upload files to S3
• Executed SparkML pre-processing and post-processing scripts and
applied them for processing training data using AWS Glue
• Serialise and capture these artifacts produced by AWS Glue to Amazon
S3 using MLeap, a common serialisation format and execution engine
for machine learning pipelines
• Training Data is ready in the S3 bucket

32. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine learning process
Data visualisation
and
analysis
Business problem –
ML problem framing Data collection
Data integration
Data preparation and
cleaning
Feature engineering
Model training and
parameter tuning
Model evaluation
Are
business
goals
met?
Model deployment
Monitoring and
debugging
– Predictions
YesNo
Dataaugmentation
Feature
augmentation
Re-training

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale
1
2
3

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale
1
2
3

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
1
2
3
Amazon SageMaker:
Build, train, and deploy ML models at scale

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
1
2
3
RL Coach
Amazon SageMaker:
Build, train, and deploy ML models at scale

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
1
2
3
Amazon SageMaker:
Build, train, and deploy ML models at scale

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
1
2
3
Amazon SageMaker:
Build, train, and deploy ML models at scale

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
1
2
3
Amazon SageMaker:
Build, train, and deploy ML models at scale

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Execution
• Open Jupyter Notebook ‘Lab 2 – Car Evaluation Machine Learning’
• Access the Jupyter notebook to run the steps required to train and
deploy the model
• The instructions are in the Jupyter notebook

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Sagemaker inference Pipeline
• Amazon SageMaker Inference Pipelines enable the definition and
deployment of any combination of pretrained Amazon SageMaker
built-in algorithms and your own custom algorithms packaged in
Docker containers.
• These inference pipelines are fully managed and can combine pre-
processing, predictions, and post-processing as part of a data science
process. Pipeline model invocations are handled as a sequence of HTTP
requests.

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we have done so far ...

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we have done so far..
• Run Amazon SageMaker XGBoost Training Job
• Create SageMaker Endpoint with pipeline

45. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine learning pipeline

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Execution - Refer instructions in the repository
• Click on the Launch Stack button below to launch the AWS
CloudFormation Stack to set up the Amazon SageMaker pipeline
• Follow instructions from the repository
• Validate the AWS CodePipeline generated

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we have done so far

49. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hyperparameter optimisation

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XgBoost hyper parameters

52. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine learning process - inference
Data visualisation
and analysis
Business problem –
ML problem framing Data collection
Data integration
Data preparation and
cleaning
Feature engineering
Model training and
parameter tuning
Model evaluation
Are
business
goals
met?
Model deployment
Monitoring and
debugging
– Predictions
YesNo
Dataaugmentation
Feature
augmentation
Re-training

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Single page application for inference

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Execution
• Refer the instructions in Module
Inference of the repository
• Deploy the Single Page
Application
• Add the car parameter inputs
• Derive inference

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Performance and monitoring
• When building your deployments, you may find you need to monitor or
debug your endpoint, and the new inference pipelines change how the
logs appear in Amazon CloudWatch.
• You can now see logs and metrics for each of your containers within a
single endpoint. To see these logs, return to the AWS Management
Console, and go to Services, Amazon SageMaker, Inference, and then
Endpoints. Locate your pipeline-xgboost endpoint in the list, and select
it by the name to see the endpoint details.

57. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Inference
(Prediction)
90%
Training
10%

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The challenges of inference in production

60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Match capacity
to demand
Available between
1 to 32 TFLOPS
Key features
Integrated with Amazon EC2,
Amazon SageMaker, and
Amazon DL AMIs
Support for TensorFlow,
Apache MXNet, and ONNX
with PyTorch coming soon
Single and
mixed-precision
operations
Lower inference costs
Amazon Elastic inference
Reduce deep learning inference costs up to 75%

61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
High throughput
Low latency
Hundreds of TOPS
Multiple
data types
Multiple
ML frameworks
INT8, FP16,
mixed precision
TensorFlow, MXNet,
PyTorch, Caffe2, ONNX
Amazon EC2 instances
Amazon SageMaker
Amazon Elastic Inference
AWS Inferentia: Machine Learning inference chip

62. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

63. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

64. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dream big
• Enhance customer experience using emerging technologies
• Provides data driven guidance to making business decisions
Start small
• Select a simple use-case with major business impact for quick-wins
• Amazon SageMaker removes the heavy lifting of machine learning
Build fast
• Links to tutorials and labs by tapping badge at exit
• Download demos by tapping badge at exit

65. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

66. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aashmeet Kalra
aashmee@amazon.com