Downloaded 28 times
Uploaded byJulien SIMON
Build, train and deploy Machine Learning models on Amazon SageMaker (May 2019)
1) British Airways uses Amazon SageMaker to build machine learning models to predict aircraft maintenance needs from flight data collected from over 290 aircraft. 2) The presentation provides an overview of British Airways' use of Amazon SageMaker, including how they process and analyze flight data, build models to detect issues, and plan to expand their use of SageMaker for additional predictive maintenance. 3) Amazon SageMaker is highlighted as a tool that allows British Airways to easily build, train, deploy and manage machine learning models for predictive maintenance without having to manage any underlying infrastructure.
More Related Content
Similar to Build, train and deploy Machine Learning models on Amazon SageMaker (May 2019)
Build, train and deploy Machine Learning models on Amazon SageMaker (May 2019)
- 1. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Build, train and deploy Machine Learning models on Amazon SageMaker Julien Simon Global Evangelist, AI & Machine Learning @julsimon Nils Mohr Flight Data Programmer Analyst British Airways
- 2. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon SageMaker 1 2 3
- 3. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Build, train and deploy models using SageMaker Business Problem ML problem framing Data collection Data integration Data preparation and cleaning Data visualization and analysis Feature engineering Model training and parameter tuning Model evaluation Monitoring and debugging Model deployment Predictions Are business goals met? YESNO Dataaugmentation Feature augmentation Re-training Neo Elastic inference Ground Truth
- 4. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Aircraft Information Intelligence British Airways Predictive Maintenance Platform Nils Mohr Flight Data Programmer Analyst British Airways S U M M I T
- 5. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T British Airways • 290+ aircraft • 46 million+ passengers per year • 200+ destinations in 75+ countries S U M M I T
- 6. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 7. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T What is flight data?
- 8. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Timeseries data
- 9. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Timeseries data
- 10. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Report data
- 11. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Our dataset
- 12. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Reasons for the development
- 13. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 14. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T High level architecture British Airways Storage in Amazon S3 AWS Cloud On prem software/tools Transfer into AWS
- 15. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Our AWS Architecture AWS Cloud Amazon VPC Storage in Amazon S3 Data optimization and cleaning on Amazon Fargate Config and Metadata store Amazon SQS Metadata generation on Amazon Fargate Amazon SQS Problem evaluation on AWS Lambda Amazon SQS
- 16. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Traditional Event detection AWS Cloud Amazon VPC Amazon SQS Config and Metadata store Lambda function checks config and sends SQS messages Lambda function runs custom/bespoke script and creates an alert if necessary Amazon SQS
- 17. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Event detection with Amazon SageMaker AWS Cloud Amazon VPC Amazon SQS Config and Metadata store Lambda function checks config and creates endpoints Amazon SageMaker models trained on specific features Amazon SageMaker endpoint Lambda function creates dataset for inference Amazon SQS Amazon SQS Lambda function evaluates data and create alerts if needed
- 18. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Lessons learned Understand the problem Ensure data quality Ask questions Apply the easiest solution
- 19. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 20. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Where do we go from here? Aircraft Engineers Create a training job in Amazon SageMaker Resulting model
- 21. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Where do we go from here? Aircraft Engineers
- 22. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 23. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Model options Training code Factorization Machines Linear Learner Principal Component Analysis K-Means Clustering XGBoost And more Built-in Algorithms (17) No ML coding required No infrastructure work required Distributed training Pipe mode Bring Your Own Container Full control, run anything! R, C++, etc. No infrastructure work required Built-in Frameworks Bring your own code: script mode Open source containers No infrastructure work required Distributed training Pipe mode
- 24. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Built-in Deep Learning frameworks: just add your code • Built-in containers for training and prediction. • Code available on Github, e.g. https://github.com/aws/sagemaker-tensorflow-containers • Build them, run them on your own machine, customize them, etc. • Script mode: use the same code as on your laptop No infrastructure work required: simply define instance type and instance count Distributed training out of the box: zero setup Pipe mode: stream infinitely large datasets directly from Amazon S3
- 25. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS: The platform of choice to run TensorFlow 85% of all TensorFlow workloads in the cloud runs on AWS
- 26. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Training ResNet-50 with the ImageNet dataset using our optimized build of Tensorflow 1.11 on a c5.18xlarge instance type is 11x faster than training on the stock binaries. Optimizing Tensorflow for Amazon EC2 instances C5 instances (Intel Skylake) 65% 90% P3 instances (NVIDIA V100)
- 27. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Demo: Keras+Tensorflow Script mode Automatic model tuning Elastic inference https://gitlab.com/juliensimon/dlnotebooks/tree/master/keras/04-fashion- mnist-sagemaker-advanced
- 28. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T ApacheMXNet:DeepLearningforenterprisedevelopers • Gluon CV Gluon NLP ONNX 2x faster Java Scala
- 29. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Demo: Gluon CV State of the art models in just a few lines of code https://gluon-cv.mxnet.io/
- 30. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Getting started http://aws.amazon.com/free https://aws.amazon.com/sagemaker https://github.com/aws/sagemaker-python-sdk https://github.com/awslabs/amazon-sagemaker-examples https://medium.com/@julsimon https://gitlab.com/juliensimon/dlnotebooks
- 31. Thank you! S UM M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Julien Simon Global Evangelist, AI and Machine Learning @julsimon
- 32. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.