Downloaded 21 times
![© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Example: MNISTwithaFullyConnected network
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)](https://image.slidesharecdn.com/aim401rdlapplicationsusingtensorflow-techsessionuploadedversion-181130205704/75/AWS-re-Invent-2018-AIM401-Deep-Learning-using-Tensorflow-12-2048.jpg)
![© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MatrixAnalytics
• The Colorado-based startup uses the Deep
Learning AMI,TensorFlow and GPU
instances to track disease progression for
patients diagnosed with pulmonary nodules.
• Their tools are able to outperform previous
methods in their ability to diagnose cancer
from a CT scan.
• The software automates follow-up care in
order to monitor changes to the patient’s
condition.
« Using the convenience of the [Deep Learning] AMI on AWS gives us
the opportunity to offer up different business models, which allows
us to become excellent technology partners as the market evolves at
an ever-increasing pace. »
Dr. Aki Alzubaidi, Founder, Matrix Analytics](https://image.slidesharecdn.com/aim401rdlapplicationsusingtensorflow-techsessionuploadedversion-181130205704/75/AWS-re-Invent-2018-AIM401-Deep-Learning-using-Tensorflow-15-2048.jpg)
![© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming TFRecord fileswithPipeMode
from sagemaker_tensorflow import PipeModeDataset
features = { 'data': tf.FixedLenFeature([], tf.string),
'labels': tf.FixedLenFeature([], tf.int64), }
def parse(record):
parsed = tf.parse_single_example(record, features)
return ({ 'data': tf.decode_raw(parsed['data'], tf.float64) }, parsed['labels’])
def train_input_fn(training_dir, hyperparameters):
ds = PipeModeDataset(channel='training', record_format='TFRecord’)
ds = ds.repeat(20)
ds = ds.prefetch(10)
ds = ds.map(parse, num_parallel_calls=10)
ds = ds.batch(64)
return ds](https://image.slidesharecdn.com/aim401rdlapplicationsusingtensorflow-techsessionuploadedversion-181130205704/75/AWS-re-Invent-2018-AIM401-Deep-Learning-using-Tensorflow-23-2048.jpg)
Uploaded byJulien SIMON
AWS re:Invent 2018 - AIM401 - Deep Learning using Tensorflow
The document provides an overview of Amazon SageMaker and TensorFlow. It discusses how TensorFlow can be used on Amazon SageMaker for deep learning applications. Key points include: - Amazon SageMaker is a fully managed service that allows users to build, train and deploy machine learning models. - TensorFlow is an open-source machine learning framework that can be used with Amazon SageMaker. - Amazon SageMaker supports TensorFlow out of the box with optimized containers for training and inference. This allows users to easily build and deploy TensorFlow models on SageMaker.
More Related Content
Similar to AWS re:Invent 2018 - AIM401 - Deep Learning using Tensorflow
AWS re:Invent 2018 - AIM401 - Deep Learning using Tensorflow
- 2. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Deep LearningApplications UsingTensorFlow Julien Simon PrincipalTechnical Evangelist, AI & Machine Learning AmazonWeb Services @julsimon A I M 4 0 1 - R
- 3. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Agenda • A quick overview of Amazon SageMaker • A quick overview ofTensorFlow • TensorFlow on Amazon SageMaker • Demo • Resources
- 4. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
- 5. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. AmazonSageMaker Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Easily build, train, and deploy Machine Learning models
- 6. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. AmazonSageMaker Notebook instances K-Means Clustering Principal Component Analysis Neural Topic Modelling Factorization Machines Linear Learner XGBoost Latent Dirichlet Allocation Image Classification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch Caffe2, CNTK, Torch FRAMEWORKS Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Built-in, high- performance algorithms Build
- 7. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. AmazonSageMaker Notebook instances Built-in, high- performance algorithms One-click training Automatic Model Tuning Build Train Deploy model in production Scale and manage the production environment
- 8. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. AmazonSageMaker Fully managed hosting with auto-scaling One-click deployment Notebook instances Built-in, high- performance algorithms One-click training Automatic Model Tuning Build Train Deploy
- 9. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. SelectedAmazonSageMaker customers
- 10. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
- 11. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. TensorFlow • Open source software library for Machine Learning • MainAPI in Python, experimental support for other languages • Built-in support for many network architectures: FC, CNN, LSTM, etc. • Support for symbolic execution, as well as imperative execution since v1.7 (aka eager execution) • Complemented by the Keras high-levelAPI
- 12. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Example: MNISTwithaFullyConnected network import tensorflow as tf mnist = tf.keras.datasets.mnist (x_train, y_train),(x_test, y_test) = mnist.load_data() x_train, x_test = x_train / 255.0, x_test / 255.0 model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(), tf.keras.layers.Dense(512, activation=tf.nn.relu), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10, activation=tf.nn.softmax) ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.fit(x_train, y_train, epochs=5) model.evaluate(x_test, y_test)
- 13. “Of 388 projects,80 percent using TensorFlow and other frameworks are running exclusively on AWS. 88% using only TensorFlow are running exclusively on AWS.” Nucleus Research report, December 2017 https://aws.amazon.com/tensorflow AWS is the place of choice for TensorFlow workloads
- 14. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Selectedcustomers runningTensorFlowonAWS
- 15. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. MatrixAnalytics • The Colorado-based startup uses the Deep Learning AMI,TensorFlow and GPU instances to track disease progression for patients diagnosed with pulmonary nodules. • Their tools are able to outperform previous methods in their ability to diagnose cancer from a CT scan. • The software automates follow-up care in order to monitor changes to the patient’s condition. « Using the convenience of the [Deep Learning] AMI on AWS gives us the opportunity to offer up different business models, which allows us to become excellent technology partners as the market evolves at an ever-increasing pace. » Dr. Aki Alzubaidi, Founder, Matrix Analytics
- 16. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
- 17. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. TensorFlow:afirst-classcitizen • Built-inTensorFlow containers for training and prediction. • Code available on Github: https://github.com/aws/sagemaker-tensorflow-containers • Build it, run it on your own machine, customize it, etc. • Supported versions: 1.4.1, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.11.0 • Advanced features • Optimized both for GPUs and CPUs (Intel MKL-DNN library). • Distributed training. • Pipe mode. • TensorBoard. • Keras • Automatic ModelTuning
- 18. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Training aTensorFlow model • Simply add your own code. • Python 2.7 and Python 3. • Must implement model_fn() or keras_model_fn() or estimator_fn() to select a model • Must implement train_input_fn() to preprocess and load training data. • Must implement eval_input_fn() to preprocess and load evaluation data. • Optional: implement serving_input_fn() if the model will be deployed from sagemaker.tensorflow import TensorFlow tf_estimator = TensorFlow(entry_point='tf-train.py', role='SageMakerRole’, training_steps=10000, evaluation_steps=100, train_instance_count=1, train_instance_type='ml.p3.2xlarge’) tf_estimator.fit('s3://bucket/path/to/training/data')
- 19. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. OptimizingTensorFlow training onC5 https://aws.amazon.com/blogs/machine-learning/faster-training-with- optimized-tensorflow-1-6-on-amazon-ec2-c5-and-p3-instances/ (March 2018) Training a ResNet-50 benchmark with the synthetic ImageNet dataset using our optimized build ofTensorFlow 1.11 on a c5.18xlarge instance type is 11X faster than training on the stock binaries. https://aws.amazon.com/about-aws/whats-new/2018/10/chainer4- 4_theano_1-0-2_launch_deep_learning_ami/ (October 2018)
- 20. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Training aTensorFlow model inlocalmode • You can train on the notebook instance itself, aka local mode. • This is particularly useful while experimenting: you can save time and money by not firing up training instances. from sagemaker.tensorflow import TensorFlow tf_estimator = TensorFlow(entry_point='tf-train.py', role='SageMakerRole’, training_steps=10000, evaluation_steps=100, train_instance_type=‘local’) tf_estimator.fit('s3://bucket/path/to/training/data')
- 21. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Training aTensorFlow model on multipleinstances • Aka DistributedTraining • Amazon SageMaker takes care of all infrastructure setup. • Only change required in your script: use global steps in model_fn(). from sagemaker.tensorflow import TensorFlow tf_estimator = TensorFlow(entry_point='tf-train.py', role='SageMakerRole’, training_steps=10000, evaluation_steps=100, train_instance_count=4, train_instance_type='ml.p3.2xlarge’) tf_estimator.fit('s3://bucket/path/to/training/data')
- 22. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Training on infinitelylarge datasetswithPipeMode • By default,Amazon SageMaker copies the data set to all training instances. • This is the best option when the data set fits in memory. • For larger data sets, Pipe Mode lets you stream data from Amazon S3. • Training starts faster. • You can train on infinitely large data sets. from sagemaker.tensorflow import TensorFlow tf_estimator = TensorFlow(entry_point='tf-train.py', role='SageMakerRole’, training_steps=10000, evaluation_steps=100, train_instance_count=1, train_instance_type='ml.p3.2xlarge’, input_mode=‘Pipe’) tf_estimator.fit('s3://bucket/path/to/training/data')
- 23. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Streaming TFRecord fileswithPipeMode from sagemaker_tensorflow import PipeModeDataset features = { 'data': tf.FixedLenFeature([], tf.string), 'labels': tf.FixedLenFeature([], tf.int64), } def parse(record): parsed = tf.parse_single_example(record, features) return ({ 'data': tf.decode_raw(parsed['data'], tf.float64) }, parsed['labels’]) def train_input_fn(training_dir, hyperparameters): ds = PipeModeDataset(channel='training', record_format='TFRecord’) ds = ds.repeat(20) ds = ds.prefetch(10) ds = ds.map(parse, num_parallel_calls=10) ds = ds.batch(64) return ds
- 24. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Visualizingtraining withTensorBoard • TensorBoard is a suite of visualization tools: graph, metrics, etc. • When enabled, it will run on the notebook instance. • You can access it at https://NOTEBOOK_INSTANCE/proxy/6006/ tf_estimator.fit(inputs, run_tensorboard_locally=True)
- 25. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Deploying aTensorFlowmodel toan HTTPSendpoint from sagemaker.tensorflow import TensorFlow tf_estimator = TensorFlow(entry_point='tf-train.py', ..., train_instance_count=1, train_instance_type='ml.c4.xlarge’) tf_estimator.fit(inputs) predictor = tf_estimator.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge') Model trained on Amazon SageMaker from sagemaker.tensorflow import TensorFlowModel tf_model = TensorFlowModel(model_data='s3://mybucket/model.tar.gz’, …, entry_point='entry.py', name='model_name’) predictor = tf_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge') Model trained elsewhere
- 26. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Using Keras onAmazonSageMaker • Keras is a popular API running on top ofTF,Theano and Apache MXNet. • The tf.keras API is natively supported in Amazon SageMaker • To use Keras itself (keras.*), you need to build a custom container. • This is not difficult! • Write a Dockerfile. • Build the container. • Push it to Amazon ECR. • Use it with sagemaker.estimator.Estimator. • Full instructions and demo in this AWS Innovate talk: https://www.youtube.com/watch?v=c8Nhwr9VmfM
- 27. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. UsingTensorFlowwithAWS DeepLens • AWS DeepLens can runTensorFlow models. • Inception • MobileNet • NasNet • ResNet • VGG • Train or fine-tune your model on Amazon SageMaker. • Deploy to DeepLens through AWS Greengrass. HD video camera Custom-designed Deep Learning engine Micro-SD Mini-HDMI USB USB Reset Audio out Power
- 28. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
- 29. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
- 30. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Resources https://ml.aws https://tensorflow.org/ https://keras.io/ https://aws.amazon.com/sagemaker https://github.com/awslabs/amazon-sagemaker-examples https://github.com/aws/sagemaker-python-sdk https://medium.com/@julsimon
- 31. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Breakout repeats Thursday, November 29th Deep Learning Applications usingTensorflow , featuring Siemens Financial Services 3:15 PM - 4:15 PM |Venetian, Level 2,VenetianTheatre Friday, November 30th Deep Learning Applications usingTensorflow, featuring Advanced Microgrid Systems 10:45AM – 11:45AM |Venetian, Level 5, Palazzo O
- 32. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
- 33. Thank you! © 2018,Amazon Web Services, Inc. or its affiliates. All rights reserved. Julien Simon PrincipalTechnical Evangelist, AI & Machine Learning AmazonWeb Services @julsimon
Editor's Notes
- #6 Amazon SageMaker removes the complexity that holds back developer success with each of these steps. Amazon SageMaker includes modules that can be used together or independently to build, train, and deploy your machine learning models.
- #7 SageMaker makes it easy to build ML models and get them ready for training by providing everything you need to quickly connect to your training data, and to select and optimize the best algorithm and framework for your application. Amazon SageMaker includes hosted Jupyter notebooks that make it is easy to explore and visualize your training data stored in Amazon S3. You can connect directly to data in S3, or use AWS Glue to move data from Amazon RDS, Amazon DynamoDB, and Amazon Redshift into S3 for analysis in your notebook. To help you select your algorithm, Amazon SageMaker includes the 10 most common machine learning algorithms which have been pre-installed and optimized to deliver up to 10 times the performance you’ll find running these algorithms anywhere else. Amazon SageMaker also comes pre-configured to run TensorFlow and Apache MXNet, two of the most popular open source frameworks, or you have the option of using your own framework.
- #8 You can begin training your model with a single click in the Amazon SageMaker console. The service manages all of the underlying infrastructure for you and can easily scale to train models at petabyte scale. To make the training process even faster and easier, Amazon SageMaker can automatically tune your model to achieve the highest possible accuracy.
- #9 Once your model is trained and tuned, SageMaker makes it easy to deploy in production so you can start generating predictions on new data (a process called inference). Amazon SageMaker deploys your model on an auto-scaling cluster of Amazon EC2 instances that are spread across multiple availability zones to deliver both high performance and high availability. It also includes built-in A/B testing capabilities to help you test your model and experiment with different versions to achieve the best results. For maximum versatility, we designed Amazon SageMaker in three modules – Build, Train, and Deploy – that can be used together or independently as part of any existing ML workflow you might already have in place.
- #10 https://www.businesswire.com/news/home/20180404006122/en/Tens-Thousands-Customers-Flocking-AWS-Machine-Learning Edmunds.com is a car-shopping website that offers detailed, constantly updated information about vehicles to 20 million monthly visitors. “We have a strategic initiative to put machine learning into the hands of all our engineers,” said Stephen Felisan, Chief Information Officer at Edmunds.com. “Amazon SageMaker is key to helping us achieve this goal, making it easier for engineers to build, train, and deploy machine learning models and algorithms at scale. We are excited to see how we can use Amazon SageMaker to innovate new solutions across the organization for our customers.” The Move, Inc. network, which includes Realtor.com, Doorsteps, and Moving.com, provides real estate information, tools, and professional expertise across a family of websites and mobile experiences for consumers and real estate professionals. “We believe that Amazon SageMaker is a transformative addition to the realtor.com toolset as we support consumers along their homeownership journey," said Vineet Singh, Chief Data Officer and Senior Vice President at Move, Inc. "Machine learning workflows that have historically taken a long time, like training and optimizing models, can be done with greater efficiency and by a broader set of developers, empowering our data scientists and analysts to focus on creating the richest experience for our users." Dow Jones is a publishing and financial information firm that publishes the world's most trusted business news and financial information in a variety of media. It delivers breaking news, exclusive insights, expert commentary and personal finance strategies. “As Dow Jones continues to focus on integrating machine learning into our products and services, AWS has been a great resource,” said Ramin Beheshti, Group Chief Product and Technology Officer. “Leading up to our recent Machine Learning Hackathon, the AWS team provided training to participants on Amazon SageMaker and Amazon Rekognition, and offered day-of support to all the teams. The result was that our teams developed some great ideas for how we can apply machine learning, many of which we we’ll continue to develop on AWS. The event was a huge success, and an example of what a great relationship can look like.” Every day Grammarly’s algorithms help millions of people communicate more effectively by offering writing assistance on multiple platforms across devices. Through a combination of natural language processing and advanced machine learning technologies, Grammarly is tackling critical communication and business challenges. “Amazon SageMaker makes it possible for us to develop our TensorFlow models in a distributed training environment,” said Stanislav Levental, Technical Lead at Grammarly. “Our workflows also integrate with Amazon EMR for pre-processing, so we can get our data from Amazon Simple Storage Service (Amazon S3), filtered with Amazon EMR and Spark from a Jupyter notebook, and then train in Amazon SageMaker with the same notebook. Amazon SageMaker is also flexible for our different production requirements. We can run inferences on Amazon SageMaker itself, or if we need just the model, we download it from Amazon S3 and run inferences of our mobile device implementations for iOS and Android customers.” Cookpad is Japan’s largest recipe sharing service, with about 60 million monthly users in Japan and about 90 million monthly users globally. “With the increasing demand for easier use of Cookpad’s recipe service, our data scientists will be building more machine learning models in order to optimize the user experience,” said Mr. Yoichiro Someya, Research Engineer at Cookpad. “Attempting to minimize the number of training job iterations for best performance, we recognized a significant challenge in the deployment of machine learning inference endpoints, which was slowing down our development processes. To automate the machine learning model deployment such that data scientists could deploy models by themselves, we used Amazon SageMaker inference APIs and proved that Amazon SageMaker would eliminate the need for application engineers to deploy machine learning models. We anticipate automating this process with Amazon SageMaker in production.”
- #13 Image source: https://commons.wikimedia.org/wiki/File:MnistExamples.png
- #15 https://aws.amazon.com/tensorflow/
- #16 https://aws.amazon.com/blogs/machine-learning/matrix-analytics-uses-deep-learning-on-aws-to-boost-early-cancer-detection/
- #18 *** UPDATE: added version 1.11.0
- #19 *** UPDATE: added Python 3 and serving_input_fn()
- #28 Personal picture
- #29 Updated