Amazon SageMaker is a fully managed platform that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale. Amazon SageMaker takes away the heavy lifting of machine learning, thus removing the typical barriers associated with machine learning. In this session, we'll dive deep into the technical details of each of the modules of Amazon SageMaker to showcase the capabilities of the platform. We also discuss the practical deployments of Amazon SageMaker through real-world customer examples.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build, Train, and Deploy ML Models
with Amazon SageMaker
Andrea Olgiati
Principal Engineer
AWS ML
A I M 4 0 4
Michael Chi
Director of Engineering
The National Football League / Next Gen Stats

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Review Amazon SageMaker
• Build, train, and deploy
• Algorithms, frameworks, bring your own, and automatic model tuning
Realtime deployment at scale
• Creating and updating endpoints
• Reduced risk deployments
• Autoscaling
Customer story
• NFL Next Gen Stats
• How the NFL delivers with Amazon SageMaker

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots &
Contact Centers
M L S E R V I C E S A M A Z O N
S A G E M A K E R
A M A Z O N E C 2
C 5 I n s t a n c e s
A M A Z O N E C 2
P 3 I n s t a n c e s
F P G A s
Frameworks Interfaces
Infrastructure
The Amazon Machine Learning Stack

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1
2
3

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning Made Simple
One-click
model training
& deployment
10x
better algorithm
performance
Predictive insights
to improve business
decision making
M L S E R V I C E S
A M A Z O N
S A G E M A K E R

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker simplifies Machine Learning
Build Train Deploy

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker modules
Build Train Deploy
• Notebook instances
• Call APIs from your
device

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker modules
Build Train Deploy
• Managed
• Distributed
• High performance
I/O

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker modules
Build Train Deploy
• Realtime endpoints
• Batch transform
• AWS Greengrass
• AWS DeepLens

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker features
SageMaker
algorithms
Frameworks
Bring
your own
Automatic
model tuning

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker features
SageMaker
algorithms
Frameworks
Bring
your own
Automatic
model tuning
• Designed for speed and
scale
• Supervised,
unsupervised, computer
vision, and NLP

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker features
SageMaker
algorithms
Frameworks
Bring
your own
Automatic
model tuning
• 20 lines of Python
• Open sourced
• Local mode for testing

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker features
SageMaker
algorithms
Frameworks
Bring
your own
Automatic
model tuning
• Publish to a container
registry
• R, Java, Julia, etc.

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker features
SageMaker
algorithms
Frameworks
Bring
your own
Automatic
model tuning
• Efficient meta-model
hyperparameter tuning
• Works with algorithms,
frameworks, and BYO

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating endpoints
Easy deployment to
production REST API
Scalable, high
throughput, and high
reliability

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating endpoints
Model aws sagemaker create-model
--model-name model1
--primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”,
“ModelDataUrl”: “s3://bkt/model1.tar.gz”}’
--execution-role-arn arn:aws:iam::123:role/me

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating endpoints
Model
Endpoint
configuration
aws sagemaker create-model
--model-name model1
--primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”,
“ModelDataUrl”: “s3://bkt/model1.tar.gz”}’
--execution-role-arn arn:aws:iam::123:role/me
aws sagemaker create-endpoint-config
--endpoint-config-name model1-config
--production-variants ‘{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 1,
”ModelName”: “model1”,
”VariantName”: “AllTraffic”}’

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating endpoints
Model
Endpoint
configuration
Endpoint
aws sagemaker create-model
--model-name model1
--primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”,
“ModelDataUrl”: “s3://bkt/model1.tar.gz”}’
--execution-role-arn arn:aws:iam::123:role/me
aws sagemaker create-endpoint-config
--endpoint-config-name model1-config
--production-variants ‘{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 1,
”ModelName”: “model1”,
”VariantName”: “AllTraffic”}’
aws sagemaker create-endpoint
--endpoint-name my-endpoint
--endpoint-config-name model1-config

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Updating endpoints
Blue-green
deployments means
no scheduled
downtime
Deploy one or more
models behind the
same endpoint

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Updating endpoints
New model aws sagemaker create-model
--model-name model2
--primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”,
“ModelDataUrl”: “s3://bkt/model2.tar.gz”}’
--execution-role-arn arn:aws:iam::123:role/me

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Updating endpoints
New model
New endpoint
configuration
aws sagemaker create-model
--model-name model2
--primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”,
“ModelDataUrl”: “s3://bkt/model2.tar.gz”}’
--execution-role-arn arn:aws:iam::123:role/me
aws sagemaker create-endpoint-config
--endpoint-config-name model2-config
--production-variants ‘{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 1,
”ModelName”: “model2”,
”VariantName”: “AllTraffic”}’

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Updating endpoints
New model
New endpoint
configuration
Same
endpoint
aws sagemaker create-model
--model-name model2
--primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”,
“ModelDataUrl”: “s3://bkt/model2.tar.gz”}
--execution-role-arn arn:aws:iam::123:role/me
aws sagemaker create-endpoint-config
--endpoint-config-name model2-config
--production-variants ‘{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 1,
”ModelName”: “model2”,
”VariantName”: “AllTraffic”}’
aws sagemaker update-endpoint
--endpoint-name my-endpoint
--endpoint-config-name model2-config

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reduced risk deployments
Incrementally retrain
models with new data
Try new models and
improved algorithms

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reduced risk deployments
Two model
endpoint
configuration
aws sagemaker create-endpoint-config
--endpoint-config-name both-models-config
--production-variants ‘[{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 95,
”ModelName”: “model1”,
”VariantName”: “model1-traffic”},
{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 5,
”ModelName”: “model2”,
”VariantName”: “model2-traffic”}]’

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reduced risk deployments
Two model
endpoint
configuration
Same
endpoint
aws sagemaker create-endpoint-config
--endpoint-config-name both-models-config
--production-variants ‘[{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 95,
”ModelName”: “model1”,
”VariantName”: “model1-traffic”},
{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 5,
”ModelName”: “model2”,
”VariantName”: “model2-traffic”}]’
aws sagemaker update-endpoint
--endpoint-name my-endpoint
--endpoint-config-name both-models-config

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reduced risk deployments
Two model
endpoint
configuration
Same
endpoint
Swap
aws sagemaker create-endpoint-config
--endpoint-config-name both-models-config
--production-variants ‘[{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 95,
”ModelName”: “model1”,
”VariantName”: “model1-traffic”},
{“InitialInstanceCount”: 2,
“InstanceType”: “ml.m4.xlarge”,
”InitialVariantWeight”: 5,
”ModelName”: “model2”,
”VariantName”: “model2-traffic”}]’
aws sagemaker update-endpoint
--endpoint-name my-endpoint
--endpoint-config-name both-models-config
aws sagemaker update-endpoint-weights-and-capacities
--endpoint-name my-endpoint
--desired-weights-and-capacities ‘{“VariantName”: ”model1”,
“DesiredWeight”: 5}’

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automatically scaling endpoints
SageMaker console settings:
• Min and max instances
• Target invocations per
instance
• Scaling cool downs

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why automatic scaling?

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automatic scaling in action

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling criteria
Algorithms have
different memory,
CPU, or GPU
requirements
Autoscale based on
endpoint instance’s
CloudWatch Metrics

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating an automatic scaling policy
Variant aws application-autoscaling register-scalable-target
--service-namespace sagemaker
--resource-id endpoint/my-endpoint/variant/model2
--scalable-dimension sagemaker:variant:DesiredInstanceCount
--min-capacity 2
--max-capacity 5

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating an automatic scaling policy
Variant
Policy
aws application-autoscaling register-scalable-target
--service-namespace sagemaker
--resource-id endpoint/my-endpoint/variant/model2
--scalable-dimension sagemaker:variant:DesiredInstanceCount
--min-capacity 2
--max-capacity 5
aws application-autoscaling put-scaling-policy
--policy-name model2-scaling
--service-namespace sagemaker
--resource-id endpoint/my-endpoint/variant/model2
--scalable-dimension sagemaker:variant:DesiredInstanceCount
--policy-type TargetTrackingScaling
--target-tracking-scaling-policy-configuration
‘{"TargetValue": 50,
"CustomizedMetricSpecification":
{"MetricName": "CPUUtilization",
"Namespace": "/aws/sagemaker/Endpoints",
"Dimensions":
[{"Name": "EndpointName", "Value": "my-endpoint"},
{"Name": "VariantName","Value": ”model2"}],
"Statistic": "Average",
"Unit": "Percent”}}’

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating an automatic scaling policy
Variant
Policy
aws application-autoscaling register-scalable-target
--service-namespace sagemaker
--resource-id endpoint/my-endpoint/variant/model2
--scalable-dimension sagemaker:variant:DesiredInstanceCount
--min-capacity 2
--max-capacity 5
aws application-autoscaling put-scaling-policy
--policy-name model2-scaling
--service-namespace sagemaker
--resource-id endpoint/my-endpoint/variant/model2
--scalable-dimension sagemaker:variant:DesiredInstanceCount
--policy-type TargetTrackingScaling
--target-tracking-scaling-policy-configuration
‘{"TargetValue": 50,
"CustomizedMetricSpecification":
{"MetricName": "CPUUtilization",
"Namespace": "/aws/sagemaker/Endpoints",
"Dimensions":
[{"Name": "EndpointName", "Value": "my-endpoint"},
{"Name": "VariantName","Value": ”model2"}],
"Statistic": "Average",
"Unit": "Percent”}}’

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale by utilization

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is the National Football League?

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Next Gen
Stats?
• The NFL’s player tracking
initiative
• Capture, process and
distribute data in real-time
• 1 TB of data per season
RFID Chip

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The Next Gen Stats
team
• ‘Lean’ team consisting of
• Engineers
• Operations
• Product/Analysts
• PMs
• Strong at software
development
• Limited data science
experience

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we deliver
• Entire platform runs on AWS
• DC/OS cluster on Amazon Elastic Compute Cloud (Amazon EC2)
• Amazon API Gateway

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we deliver – with Amazon SageMaker
• Entire platform runs on AWS
• DC/OS cluster on EC2
• API Gateway

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we create stats
Derived Metrics
// pseudo code for Who Covered Who
if (defenderLinedUpAgainstReceiver()) {
return isClosestToReceiverAtPass() &&
isClosestToReceiverAtCatchOrIncompletion();
} else {
if (defenderClosesInOnReceiver()) {
return isClosestToReceiverAtPass() &&
isClosestToReceiverAtCatchOrIncompletion();
} else {
return false;
}
}
Tuple<Double, Double> passerCoordinate =
new Tuple<>(passInfo.getPasserX(), passInfo.getPasserY());
Tuple<Double, Double> receiverCoordinate =
new Tuple<>(passInfo.getTargetX(), passInfo.getTargetY());
if (passInfo.hasPasserCoordinate() &&
passInfo.hasTargetCoordinate()) {
Double airYards;
if (LEFT.equals(play.getPlayDirection())) {
airYards = play.getAbsoluteYardlineNumber() –
receiverCoordinate.getT1();
} else {
airYards = receiverCoordinate.getT1() –
play.getAbsoluteYardlineNumber();
}
Double airYardsToSticks = airYards - play.getYardsToGo();
passInfo.setAirDistance(useDistanceFormula(passerCoordinate,
receiverCoordinate));
passInfo.setAirYards(airYards);
passInfo.setAirYardsToSticks(airYardsToSticks);
}
Rules Based

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we create stats
Machine Learned
xgb = sagemaker.estimator.Estimator(training_img,
role,
base_job_name='ecp-modeling',
train_instance_count=1,
train_instance_type='ml.m4.xlarge',
output_path='s3://{}/{}/output'.format(bucket, prefix),
sagemaker_session=sess)
xgb.set_hyperparameters(
eval_metric='error',
objective='binary:logistic',
num_round=200,
rate_drop=0.2,
silent=0
)
hyperparameter_ranges = {
'eta': ContinuousParameter(0.1, 0.5),
'alpha': ContinuousParameter(0, 2),
'min_child_weight': ContinuousParameter(0, 120),
'subsample': ContinuousParameter(0.5, 1),
'max_depth': IntegerParameter(1, 10)
}
tuner = HyperparameterTuner(xgb,
'validation:error',
hyperparameter_ranges,
objective_type='Minimize',
max_jobs=30,
max_parallel_jobs=3,
base_tuning_job_name='ecp-hpo-
tuning')
tuner.fit({'train': s3_input_train, 'validation':
s3_input_validation})

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Completion Probability – Quantifying the difficulty of
a pass based on numerous features
• Receiver separation
• Receiver lineup
• Time to throw
• QB separation from rusher
• Air yards/distance
• Scramble yards
• Target Separation from CB
• Double coverage

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Completion Probability – with Amazon SageMaker
• SageMaker XGBoost
• Trained with ~35k plays
• 11 key features
• Iterative Process
print("Deploy Model")
endpoint_name = 'ecp-tuned-test-params'
try:
xgb_predictor = tuner.deploy(
initial_instance_count=1,
instance_type='ml.m4.xlarge’,
endpoint_name=endpoint_name
)
print("Building a new predictor")
except:
xgb_predictor =
sagemaker.predictor.RealTimePredictor(endpoint=endpoi
nt_name, sagemaker_session=sess)
print("Using an existing endpoint")
xgb_predictor.content_type = 'text/csv'
xgb_predictor.serializer = csv_serializer
xgb_predictor.deserializer = None

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Early Lessons
• Feature selection
• Inefficient model training
• Consistent process

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Advantages
• Lowers barrier to entry
• Common approach for
development teams
• Consistent deployment

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Conclusion
• Amazon SageMaker is a versatile platform to build, train,
and deploy machine learning models at scale.
• Customers like the NFL are benefiting from integrating
Amazon SageMaker into their delivery pipelines.
• Explore Amazon SageMaker (free tier eligible*) and build
models of your own.
*https://aws.amazon.com/free/

52. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Michael Chi, NFL
Andrea Olgiati, AWS

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.