Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2018512 AWS上での機械学習システムの構築とSageMaker

1,801 views

Published on

【クラウド上の最新ディープラーニング開発環境を体験しよう!】簡単に人工知能アプリを作る為のAmazon SageMakerクラウド・ハンズオン(https://lab-kadokawa50.peatix.com/ )で使用した資料です。
AWS上での機械学習システムの構築とSageMakerの概要を説明しています。

Published in: Internet
  • Be the first to comment

2018512 AWS上での機械学習システムの構築とSageMaker

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1,8 2 / /5 0./ , ,2 A
  2. 2. / ) • • CW EUI T S • C F G • ( ? m • SH A • ) a F Aeb a • / O CP • ) r A h • o
  3. 3. • 2018 5 12 AWS (http://aws.amazon.com) • AWS AWS • • AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Sources: Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares 0 1 2 3 4 5 6 7 8 9 1990 2000 2010 2020 DataVolume Year Generated Data
  6. 6. . • • & . . • • . . . • • M L
  7. 7. 0 Search & Discovery Fulfilment & Logistics Existing Products New Initiatives
  8. 8. / x a • l • l • & • • • • • I • • Q o • • x • I • • IA • m n e 8
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Kinesis Database Migration Service Oracle, Netezza Amazon S3 Direct Connect Snowball Interoperate with everything and many more… Redshift EMR Athena AI and Machine Learning as-a-service QuickSight and many more…
  11. 11. A S W L 6 5 W % M 6 S A S 5 8S 0 S 5 5 2 % 1 2 3
  12. 12. 12 Services • • I • Platforms • M • M P • Frameworks • M L • • A M
  13. 13. Services Amazon Rekognition Image Platform Frameworks TensorFlow Caffe Torch Theano CNTK Keras CPU IoT MobileInfrastructure GPU MXNet AWS Deep Learning AMI (Amazon Linux / Ubuntu / Windows) Amazon Polly Amazon Lex Amazon Comprehend Amazon Translate Amazon Transcribe Amazon Rekognition Video Amazon Machine Learning Apache Spark Amazon EMR Amazon Kinesis AWS Batch Amazon ECS Amazon SageMaker
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 14
  15. 15. Batch / Real time
  16. 16. Batch / Real time
  17. 17. Batch / Real time AWS AWS ETL Amazon Mechanical Turk
  18. 18. Batch / Real time AWS AWS ETL Amazon Mechanical TurkAWS ML
  19. 19. Batch / Real time AWS AWS ETL Amazon Mechanical TurkAWS ML AWS
  20. 20. Batch / Real time EMR Athena QuickSight S3 Glue Redshift SageMaker Kinesis
  21. 21. Amazon Kinesis AWS Glue Amazon S3 Amazon Glacier Amazon ECS Amazon EC2 Lambda function Amazon DynamoDB Amazon RDS Amazon Redshift Amazon Athena Amazon ES Amazon EMR Amazon QuickSight Amazon Kinesis Analytics Amazon Kinesis Streams Amazon SageMaker Amazon Machine Learning AWS Batch AWS IoT AWS Greengrass Lambda function Ingest Store Compute Database Analyze Intelligence The Edge
  22. 22. Redshift EMR (Yarn/Spark) Aggregator Model Development &Training / Amazon Athena Realtime Event Processor Agg. Cached Data Aggregator Realtime Search Kinesis Analytics Decision on the Cloud Feedback Aggregator Quicksight Kinesis Streams Kinesis Firehose Amazon S3 AWS Greengrass Lambda function AWS Glue AWS IOT Amazon SageMaker
  23. 23. Amazon SageMaker
  24. 24. Amazon SageMaker ML/DL/ 1
  25. 25. Amazon SageMaker ML/DL/ 1
  26. 26. : d cu • n r o • L : B a o • • B M o • M g k
  27. 27. • IA e a M S P
  28. 28. Amazon SageMaker
  29. 29. S M A 29
  30. 30. P • • N B • r D A g • B g • I • A k M • a S / J g • I e • / / g g
  31. 31. . 3 A AX W N M XS M XS • 1 23 2 • T A N M M XS T 31
  32. 32. . 3 A A a T S W TX • 1 23 2 • A T W TX e N e A M 32
  33. 33. K A . 3 D 1. 3 33 SageMaker S3
  34. 34. ) E AM g 34 SageMaker S3 S 2 ) f AD W ai ) . ( ) ai D K e ai S
  35. 35. ) Dd 35 SageMaker S3 KSD 3 3 D ) W ge M E D ) . ( ) KS iAaA
  36. 36. 36
  37. 37. 37 SageMaker SDK Tensorflow Estimator S3
  38. 38. 38
  39. 39. 3
  40. 40. ( ) ( A $$ $ A$ ($ • o m r • iw z nG Sgj • 52 Ib P U I • s e e 59 IaP O A $$ $ A$ ($AB :$ • I iw % % ($. $ • I t 0 3 % % $. • 1/cW p %%%h T % %%) Amazon S3
  41. 41. • • • • 1 5TB • HTTPS PUT/GET • • • AWS 100 s3:// / / ( ) S3 PUT GET
  42. 42. • N - 3 • N - 0 32 CEGA I • D 0 1 2 13
  43. 43. (8 A: ) 1 > 03 ( 12 DCE GI • 1 3: > 1 > DCE GI 1 > 3 3: > DCE GI 0 20 40 60 80 100 120 140 K80 P100 V100 Mixed/FP16 Perf (TFLOPS) 0 2 4 6 8 10 12 14 16 K80 P100 V100 FP32 Perf (TFLOPS) 0 1 2 3 4 5 6 7 8 K80 P100 V100 FP64 Perf (TFLOPS) 0 1000 2000 3000 4000 5000 6000 K80 P100 V100 Resnet-50 8 GPU (Images/sec) 1 0 2 .
  44. 44. 3 2 2 A U S L IM C E A D a P 89 AW 45 https://aws.amazon.com/jp/amazon-ai/amis/ https://aws.amazon.com/jp/blogs/ai/announcing-new-aws-deep-learning-ami-for-amazon-ec2-p3-instances/#more-2093
  45. 45. 46 CUDA9, nvidia-docker AMI nvidia-docker NVIDIA GPU Cloud Docker Docker
  46. 46. - - ID • • • - - - • ID <E DH ID - - >C <E F E
  47. 47. / / / CA K E E G / / / , / .
  48. 48. // / / / 50 • IN Xe MSG g AMPT • / / / GM LAg A F • T S hXe c G U f G a • W Trained models and Lambdas Extracted IntelligenceInferences and take actions locally on device AWS Cloud for training
  49. 49. A GD C AA )A C AA AA A C - AC )A C )A C A BB C A AC BB C A AC BB A A C C A DB CCG 2BA C ( 2 ) D D BAG C A 3 )A (C C DD AC DA
  50. 50. g S WM L B D W f A eb a d
  51. 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  52. 52. MG K F S A Q 54 .1 . . 1 . . 33
  53. 53. . . A . . A . . 1 M 55
  54. 54. A G K M F G . 3 S 56 . 3 3 3 3 1
  55. 55. S R VQ c M C A C M 1 aG CF e C G K 57 3 1 1 .3 1
  56. 56. B B G RQa FA B C E A D M K S D 58 3 . 3 . 23 .
  57. 57. D C DA B 2 E 2 DA S 59 . .
  58. 58. G M R K F B EA S D M F 60 . . 3 . . . 2 .
  59. 59. R Q R AC F SAC T VA a SA K G K 61 2 2 2 2.3 2
  60. 60. I DSGTM Pec DW UC A Q Aa 62 Camera Device 3 Greengrass Core . . . 3 3 . . .
  61. 61. A C U D G 3 P 63 Camera Device .. Greengrass Core
  62. 62. G I . MA 3 64 .
  63. 63. D 3 A . . G A 65 Greengrass Core . .
  64. 64. AG I A S Q 3 66 3 . . 3
  65. 65. G C M U A G C P S Q 67 Camera Device 4 . 3 . 4 HTML/Javascript/model
  66. 66. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 68
  67. 67. • W DS p e k D aA r Dg D • O p g aA r g • M s D aA 69

×