Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amazon SageMaker 紹介 & ハンズオン(2018/07/25 実施)

2018/07/03 に実施した Amazon SageMaker ハンズオンの資料です.
https://pages.awscloud.com/AmazonSageMaker20180725-jp.html

  • Login to see the comments

Amazon SageMaker 紹介 & ハンズオン(2018/07/25 実施)

  1. 1. 2018.07.25 Amazon SageMaker
  2. 2. 2018/7/25 Amazon SageMaker + • • • • • • • • • • •
  3. 3. • • • ⎼ Amazon Athena ⎼ AWS Glue ⎼ Amazon SageMaker
  4. 4. Agenda SageMaker SageMaker
  5. 5. • • 1 1 • 1 • •
  6. 6. SageMaker
  7. 7. SageMaker 5
  8. 8. SageMaker • • • • • • • • •
  9. 9. Jupyter Notebook • • • 4 ml.t2, ml.m4, ml.p2, ml.p3 • VPC ENI VPC
  10. 10. CreateTrainingJob API Docker • • 2 • m4, m5, c4, c5, p2, p3 • S3 https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html
  11. 11. : CloudWatch Logs CloudWatch Logs
  12. 12. : Estimater hyperparameters SageMaker https://github.com/awslabs/amazon-sagemaker-examples/tree/master/hyperparameter_tuning https://github.com/aws/sagemaker-python-sdk#sagemaker-automatic-model-tuning
  13. 13. / : SageMaker Tensorflow / MXNet / Chainer github • SageMaker pull / • instance_type ’local’ https://github.com/aws/sagemaker-python-sdk#local-mode https://github.com/aws/sagemaker-tensorflow-containers https://github.com/aws/sagemaker-mxnet-containers https://github.com/aws/sagemaker-chainer-container
  14. 14. CreateEndpoint API Docker • AB • • https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateEndpointConfig.html https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateEndpoint.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/API_UpdateEndpoint.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/API_UpdateEndpointWeightsAndCapacities.html
  15. 15. : Batch Transform Job • CreateTransformJob API • • S3 S3
  16. 16. : AWS Greengrass SageMaker / . .- . / - ./ .- . • SageMaker S3 • DeepLens
  17. 17. SageMaker
  18. 18. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
  19. 19. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
  20. 20. create-endpoint create-notebook-instance create-training-job delete-endpoint delete-notebook-instance describe-endpoint describe-notebook-instance … estimator = TensorFlow(…) estimator.set_hyperparameters(…) estimator.fit(…) predictor = estimator.deploy(…) Predictor.predict(…)
  21. 21. SageMaker 1. SageMaker 2. Tensorflow/Chainer/PyTorch/MXNet 3.
  22. 22. SageMaker 1. SageMaker → 2. Tensorflow/Chainer/PyTorch/MXNet → 3. →
  23. 23. SageMaker
  24. 24. • Linear Learner • Factorization Machines • XGBoost • Image Classification • seq2seq • K-means • k-NN • PCA • LDA • Neural Topic Model • DeepAR Forecasting • BlazingText (word2vec) • Random Cut Forest • Object Detection New! https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html New!
  25. 25. Image Classification ResNet • ResNet CNN • ILSVRC 2015 1 • ImageNet • use_pretrained_model 1 0 https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning.ipynb https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/image-classification.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html
  26. 26. Object Detection SSD • • VGG or ResNet • ImageNet • use_pretrained_model 1 0 https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/object_detection_pascalvoc_coco/object_detection_image_json_format.ipynb https://arxiv.org/pdf/1512.02325.pdf https://docs.aws.amazon.com/sagemaker/latest/dg/object-detection.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html
  27. 27. SageMaker S3
  28. 28. estimator.fit() SageMaker S3
  29. 29. estimator.deploy() SageMaker S3
  30. 30. https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html
  31. 31. Tensorflow / PyTorch / Chainer / MXNet
  32. 32. Chainer • __main__ • model_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/chainer.html https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/chainer_mnist/chainer_mnist_single_machine.py
  33. 33. PyTorch • __main__ • model_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/pytorch.html https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/pytorch/README.rst
  34. 34. Tensorflow • model_fn • estimator_fn tensorflow.estimator • keras_model_fn tf.keras • train_input_fn • eval_input_fn • serving_input_fn • input_fn • output_fn https://docs.aws.amazon.com/sagemaker/latest/dg/tf-training-inference-code-template.html
  35. 35. MXNet • train • save • model_fn: • transform_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet-training-inference-code-template.html
  36. 36. * S3 SageMaker S3
  37. 37. estimator.fit() . AWS SageMaker S3
  38. 38. estimator.deploy() AWS SageMaker S3
  39. 39. ECR push SageMaker S3
  40. 40. S3ECR SageMaker estimator.fit()
  41. 41. S3ECR SageMaker estimator.deploy()
  42. 42. SageMaker 3 1: • SageMaker 2: GPU • AWS
  43. 43. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK
  44. 44. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK SageMaker SDK AWS SDK
  45. 45. ML • SageMaker 1 ML • • 0.14 USD/GB/ • • 0.016 USD/GB https://aws.amazon.com/jp/sagemaker/pricing/
  46. 46. 3 SageMaker Example Notebooks • https://github.com/awslabs/amazon-sagemaker-examples SageMaker SDK • https://github.com/aws/sagemaker-python-sdk (Doc : https://readthedocs.org/projects/sagemaker/) SageMaker • https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/whatis.html
  47. 47. https://github.com/awslabs/amazon-sagemaker-examples/tree/master/introduction_to_amazon_algorithms https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/algos.html • • • • • • New! • • • • • •
  48. 48. Tensorflow • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras/tensorflow_abalone_age_predictor_using_keras.ipynb • Tensorflow Tensorboard • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_resnet_cifar10_with_tensorboard • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_distributed_mnist/tensorflow_distributed_mnist.ipynb • Keras + Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras • MXNet • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk
  49. 49. • scikit-learn SageMaker SageMaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb • SageMaker R • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/install_r_kernel/example_r_notebook.ipynb • SageMaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/handling_kms_encrypted_data/handling_kms_encrypted_data.ipynb • XGBoost SageMaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/xgboost_bring_your_own_model/xgboost_bring_your_own_model.ipynb • Tensorflow Sagemaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/tensorflow_iris_byom/tensorflow_BYOM_iris.ipynb • Redshift MXNet R • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/advanced_functionality
  50. 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Web Services Japan, K. K. Amazon SageMaker
  51. 51. (SAMEJIMA, Masaki) SageMaker AWS Amazon SageMaker
  52. 52. SageMaker https://aws.amazon.com/jp/console/ (Chrome, Firefox IE, Safari ) SageMaker
  53. 53. SageMaker
  54. 54. • • ml.t2.medium • IAM
  55. 55. IAM S3 S3
  56. 56. • VPC, •
  57. 57. Jupyter Notebook • InService Jupyter Notebook • New → Terminal
  58. 58. Terminal cd SageMaker/ wget http://bit.ly/sagemaker-notebooks -O sagemaker-notebooks.zip unzip sagemaker-notebooks.zip O( )
  59. 59. Jupyter Notebook sagemaker-notebooks
  60. 60. • • Random Cut Forest • XGBoost MNIST • Chainer on SageMaker • Chainer MLP MNIST
  61. 61. : Random Cut Forest Random Forest (1) (2) …… ≤ 1 > 1 ≤ 0.2 > 0.2
  62. 62. : XGboost …… ≤ 1 > 1 ≤ 0.2 > 0.2 ≤ 0.8 > 1 ≤ 0.6 > 0.6

×