Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amazon SageMaker 紹介 & ハンズオン(2018/07/03 実施)

5,295 views

Published on

2018/07/03 に実施した Amazon SageMaker ハンズオンの資料です.
https://pages.awscloud.com/AmazonSageMaker20180703-jp.html

Published in: Technology
  • Be the first to comment

Amazon SageMaker 紹介 & ハンズオン(2018/07/03 実施)

  1. 1. 2018.07.03 Amazon SageMaker
  2. 2. 7/3/2018 Amazon SageMaker 16:00-17:00 Amazon SageMaker - - Amazon SageMaker 3 - Amazon SageMaker - Build-in Algorithm - Native Framework (TF/MXNet/Chainer) - Bring your own Algorithm - Amazon SageMaker - Amazon SageMaker - Greengrass 17:30-19:00 Amazon SageMaker - Build-in Algorithm - Chainer 17:00- Q&A and Break 19:00-19:30 Q&A | Close
  3. 3. • • • ⎼ Amazon Athena ⎼ AWS Glue ⎼ Amazon SageMaker
  4. 4. Agenda SageMaker SageMaker
  5. 5. • • 1 1 • 1 • •
  6. 6. SageMaker
  7. 7. SageMaker 5
  8. 8. SageMaker • • • • • • • • •
  9. 9. Jupyter Notebook • • • 4 ml.t2, ml.m4, ml.p2, ml.p3 • VPC ENI VPC
  10. 10. CreateTrainingJob API Docker • • 2 • m4, m5, c4, c5, p2, p3 • S3 https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html
  11. 11. : CloudWatch Logs CloudWatch Logs
  12. 12. : Estimater hyperparameters SageMaker https://github.com/awslabs/amazon-sagemaker-examples/tree/master/hyperparameter_tuning https://github.com/aws/sagemaker-python-sdk#sagemaker-automatic-model-tuning
  13. 13. / : SageMaker Tensorflow / MXNet / Chainer github • SageMaker pull / • instance_type ’local’ https://github.com /aw s/sagem aker-python-sdk#local-m ode https://github.com /aw s/sagem aker-tensorflow -containers https://github.com /aw s/sagem aker-m xnet-containers https://github.com /aw s/sagem aker-chainer-container
  14. 14. CreateEndpoint API Docker • AB • • https://docs.aw s.am azon.com /sagem aker/latest/dg/API_CreateEndpointConfig.htm l https://docs.aw s.am azon.com /sagem aker/latest/dg/API_CreateEndpoint.htm l https://docs.aw s.am azon.com /ja_jp/sagem aker/latest/dg/API_UpdateEndpoint.htm l https://docs.aw s.am azon.com /ja_jp/sagem aker/latest/dg/API_UpdateEndpointW eightsAndCapacities.htm l
  15. 15. : AWS Greengrass SageMaker / . .- . / - ./ .- . • SageMaker S3 • DeepLens
  16. 16. SageMaker
  17. 17. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
  18. 18. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
  19. 19. create-endpoint create-notebook-instance create-training-job delete-endpoint delete-notebook-instance describe-endpoint describe-notebook-instance … estimator = TensorFlow(…) estimator.set_hyperparameters(…) estimator.fit(…) predictor = estimator.deploy(…) Predictor.predict(…)
  20. 20. SageMaker 1. SageMaker 2. Tensorflow/MXNet/Chainer 3.
  21. 21. SageMaker 1. SageMaker → 2. Tensorflow/MXNet/ Chainer → 3. →
  22. 22. SageMaker
  23. 23. • Linear Learner • Factorization Machines • XGBoost • Image Classification • seq2seq • K-means • PCA • LDA • Neural Topic Model • DeepAR Forecasting • BlazingText (word2vec) • Random Cut Forest New! New! https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html New!
  24. 24. Image Classification ResNet • ResNet CNN • ILSVRC 2015 1 • ImageNet • use_pretrained_model 1 0 https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning.ipynb https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/image-classification.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html
  25. 25. SageMaker S3
  26. 26. estimator.fit() SageMaker S3
  27. 27. estimator.deploy() SageMaker S3
  28. 28. https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html
  29. 29. Chainer / PyTorch / Tensorflow / MXNet
  30. 30. Chainer • __main__ • model_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/chainer.html https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/chainer_mnist/chainer_mnist_single_machine.py
  31. 31. PyTorch • __main__ • model_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/pytorch.html https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/pytorch/README.rst
  32. 32. Tensorflow • model_fn • estimator_fn tensorflow.estimator • keras_model_fn tf.keras • train_input_fn • eval_input_fn • serving_input_fn • input_fn • output_fn https://docs.aws.amazon.com/sagemaker/latest/dg/tf-training-inference-code-template.html
  33. 33. MXNet • train • save • model_fn: • transform_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet-training-inference-code-template.html
  34. 34. * S3 SageMaker S3
  35. 35. estimator.fit() . AWS SageMaker S3
  36. 36. estimator.deploy() AWS SageMaker S3
  37. 37. ECR push SageMaker S3
  38. 38. S3ECR SageMaker estimator.fit()
  39. 39. S3ECR SageMaker estimator.deploy()
  40. 40. SageMaker 3 1: • SageMaker 2: GPU • AWS
  41. 41. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK
  42. 42. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK SageMaker SDK AWS SDK
  43. 43. ML • SageMaker 1 ML • • 0.14 USD/GB/ • • 0.016 USD/GB https://aws.amazon.com/jp/sagemaker/pricing/
  44. 44. 3 SageMaker Example Notebooks • https://github.com/awslabs/amazon-sagemaker-examples SageMaker SDK • https://github.com/aws/sagemaker-python-sdk (Doc : https://readthedocs.org/projects/sagemaker/) SageMaker • https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/whatis.html
  45. 45. https://github.com/awslabs/amazon-sagemaker-examples/tree/master/introduction_to_amazon_algorithms https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/algos.html • • • • • • New! • • • • • •
  46. 46. Tensorflow • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras/tensorflow_abalone_age_predictor_using_keras.ipynb • Tensorflow Tensorboard • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_resnet_cifar10_with_tensorboard • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_distributed_mnist/tensorflow_distributed_mnist.ipynb • Keras + Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras • MXNet • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk
  47. 47. • scikit-learn SageMaker SageMaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/scikit_bring_your_ow n/scikit_bring_your_ow n.ipynb • SageMaker R • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/install_r_kernel/exam ple_r_notebook.ipynb • SageMaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/handling_km s_encrypted_data/handling_km s_encrypted_data.ipynb • XGBoost SageMaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/xgboost_bring_your_ow n_m odel/xgboost_bring_your_ow n_m odel.ipynb • Tensorflow Sagemaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/tensorflow _iris_byom /tensorflow _BYO M _iris.ipynb • Redshift MXNet R • https://github.com /aw slabs/am azon-sagem aker-exam ples/tree/m aster/advanced_functionality
  48. 48. Amazon Web Services Japan, K. K. Amazon SageMaker
  49. 49. SageMaker
  50. 50. • • ml.t2.medium • IAM
  51. 51. IAM S3 S3
  52. 52. • VPC, •
  53. 53. Jupyter Notebook • InService Jupyter Notebook • New → Terminal
  54. 54. Terminal cd SageMaker/ wget http://bit.ly/sagemaker-notebooks -O sagemaker-notebooks.zip unzip sagemaker-notebooks.zip O( )
  55. 55. Jupyter Notebook sagemaker-notebooks
  56. 56. • MNIST • • Amazon SageMaker 1 XGBoost • Chainer MLP ( ) Amazon SageMaker

×