Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20190306 AWS Black Belt Online Seminar Amazon EC2スポットインスタンス

473 views

Published on

AWS公式オンラインセミナー: https://amzn.to/JPWebinar
過去資料: https://amzn.to/JPArchive

Published in: Technology
  • Be the first to comment

20190306 AWS Black Belt Online Seminar Amazon EC2スポットインスタンス

  1. 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Webinar https://amzn.to/JPWebinar https://amzn.to/JPArchive EC2 2019-03-06 Amazon EC2 [AWS Black Belt Online Seminar]
  2. 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ( ) / EC2 EC2 EC2 AWS • Amazon EC2 Auto Scaling • AWS Support
  3. 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Black Belt Online Seminar • • ① 吹き出しをクリック ② 質問を入力 ③ Sendをクリック Twitter #awsblackbelt
  4. 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • 2019 3 6 AWS (http://aws.amazon.com/) • AWS AWS • • AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.
  5. 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • ( ) ( ) • • ( ) • ( ) • CloudWatch ( ) • • ( ) : • ( ) • ( ) • ( ) • ( ) ( )
  6. 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  7. 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - Amazon EC2
  8. 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2購入オプションをどう組み合わせるか?
  9. 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon EC2 • 9 • - HPC/ • 1,000 • • API • •
  10. 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  11. 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - 110 • ( ) • • AWS • • 2017 8 26 vCPU 110 ( ) • • ( )
  12. 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - 110 • ( ) • • AWS • • 2017 8 26 vCPU 110 ( ) • • ( )
  13. 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  14. 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • • ( ) • •
  15. 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS OSS, AWS CloudFormation AWS OpsWorks Amazon EMR Amazon ECS AWS Data Pipeline AWS BatchAuto Scaling AWS Thinkbox Deadline
  16. 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  17. 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - 2017 11
  18. 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (1)
  19. 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (2)
  20. 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (3)
  21. 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (4)
  22. 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (1) • •
  23. 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (2) • • •
  24. 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (1) •
  25. 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (2) • • •
  26. 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • • AWS 2 • Amazon EC2 • ( )
  27. 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  28. 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • 2 • • • CloudWatch Events
  29. 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - • • • http://169.254.169.254/latest/meta-data/ • https://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/ec2- instance-metadata.html • • http://169.254.169.254/latest/meta-data/spot/instance-action • • 5 • https://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/spot- interruptions.html
  30. 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark { "version": "0", "id": "12345678-1234-1234-1234-123456789012", "detail-type": "EC2 Spot Instance Interruption Warning", "source": "aws.ec2", "account": "123456789012", "time": "yyyy-mm-ddThh:mm:ssZ", "region": "us-east-2", "resources": ["arn:aws:ec2:us-east-2:123456789012:instance/i-1234567890abcdef0"], "detail": { "instance-id": "i-1234567890abcdef0", "instance-action": "action" } }
  31. 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - CloudWatch Events
  32. 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark - CloudWatch Events
  33. 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  34. 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS CLI (aws ec2 ...) (EC2) run-instances (request-spot- instances) one-time (sync) / persistent (sync) request-spot- fleet / vCPU ( ) request (async) / maintain (async) EC2 create-fleet / vCPU ( ) instant (sync) / request (async) / maintain (async) Auto Scaling (New!)
  35. 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark m4.large Spot ASG Min: 1 Max: 10 m5.large Spot ASG Min: 1 Max: 10 c4.xlarge O-D ASG Min: 1 Max: 10 Availability Zone 1 Availability Zone 2 Availability Zone 3 - ASG ASG - ASG ASG: Auto Scaling Group O-D: On Demand Instance
  36. 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark m4.large Spot Instances m5.large Spot Instances c4.xlarge On-Demand instances Availability Zone 1 Availability Zone 2 Availability Zone3 ASG Single ASG ASG: Auto Scaling Group O-D: On Demand Instance RI: Reserved Instance
  37. 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  38. 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS CLI (aws ec2 ...) (EC2) run-instances (request-spot- instances) one-time (sync) / persistent (sync) request-spot- fleet / vCPU ( ) request (async) / maintain (async) EC2 create-fleet / vCPU ( ) instant (sync) / request (async) / maintain (async) Auto Scaling (New!)
  39. 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  40. 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (1)
  41. 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (2) • Load balancing: • Flexible workloads:
  42. 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (3) • vCPU •
  43. 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (4) • vCPU • •
  44. 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (5) • EC2 •
  45. 45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (6) • • •
  46. 46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2 Auto Scaling Group (1) • EC2 EC2 Auto Scaling
  47. 47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2 Auto Scaling Group (2)
  48. 48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2 Auto Scaling Group (3) • •
  49. 49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2 Auto Scaling Group (4) • • 1. • 2. • 3. • • 12 • 2 • 30:70 • • 2 +3 • 7
  50. 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark < > EC2 Auto Scaling Spot Fleet (1) EC2 Auto Scaling Spot Fleet AZ [1] 1. Total target capacity 2. "Optional On-demand portion 3. [2] 1. Desired Capacity 2. "On-Demand base" 3. OD:Spot Ratio ELB Launch Template $latest, $default [1] https://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/spot-fleet.html#on-demand-in-spot [2] https://docs.aws.amazon.com/ja_jp/autoscaling/ec2/userguide/AutoScalingGroup.html#asg-purchase-options
  51. 51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • (Termination Policy) • (Termination Protection) • • (Entering Standby) • (Suspending Processes) • (Rebalancing activities) < > EC2 Auto Scaling Spot Fleet (2) - • • • ( ) ( )
  52. 52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark -
  53. 53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark -
  54. 54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • - Amazon EC2 - Amazon EC2 | AWS https://aws.amazon.com/jp/ec2/spot/pricing/
  55. 55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (1) EC2
  56. 56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark (2) AWS CLI: describe-spot-price-history $ aws ec2 describe-spot-price-history --instance-types m3.medium { "SpotPriceHistory": [ { "Timestamp": "2019-03-04T21:36:38.000Z", "AvailabilityZone": "ap-northeast-1a", "SpotPrice": "0.109600", "ProductDescription": "SUSE Linux", "InstanceType": "m3.medium" }, { "Timestamp": "2019-03-04T21:36:38.000Z", "AvailabilityZone": "ap-northeast-1c", "SpotPrice": "0.109600", "ProductDescription": "SUSE Linux", "InstanceType": "m3.medium" }, :
  57. 57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  58. 58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ( ) On Demand Auto- Scaling group Output S3 bucket Availability Zone A Availability Zone B Spot Fleet Upload object into input S3 bucket Job SQS Queue Auto Scaling group and Spot fleet scale up based on queue depth and scale down based on CPU utilization CW metrics Workers will check for jobs in the queue Workers will update Job status (start time, SLA end time, etc) in DynamoDB Uploads to S3 will trigger a Lamda function to put jobs in SQS and DynamoDB EFSEC2 instance worker fleet Cost-effective Batch Processing with Amazon EC2 Spot | AWS Compute Blog https://aws.amazon.com/blogs/compute/cost-effective- batch-processing-with-amazon-ec2-spot/
  59. 59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Batch ( ) JobQueue submit-job list-jobs describe-jobs cancel-jobs terminate-jobs Compute Environment ECR/Docker Registry S3/EFS/NFS Autoscaling ECS Docker Image JobDefinitio n job AWS Batch
  60. 60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ビッグデータ領域のリファレンスアーキテクチャ Master Node (OD or RI) S3 EMRFS Master Fleet Task Node (Spot) Task Node (Spot) Task Node (Spot) Task Fleet Instance Store EBS Volume Instance Store EBS Volume Instance Store EBS Volume Core Fleet HDFS Instance Fleet Core Node (OD or RI) Core Node (OD or RI) Core Node (OD or RI) Amazon EMR
  61. 61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark コンテナを活用したステートレスウェブアプリケーション Stateless Web Server (Spot) Stateless Web Server (Spot) Session State Data Stateless Web Server (On-Demand) Elastic Load Balancing Amazon EC2 Auto Scaling ECS container 1 Stateless Web Server (On-Demand) ECS container 2 ECS container 1 ECS container 2 https://github.com/awslabs/ec2-spot-labs/tree/master/workshops/ec2-spot-fleet-web-app
  62. 62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark < > ECS • ECS • ECS (/etc/ecs/ecs.config) • ECS AMI
  63. 63. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark < > ECS ( ) • • Draining • ( )SNS • →
  64. 64. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark < > ECS • CloudFormation • • • Draining • SNS E
  65. 65. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  66. 66. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 4+1
  67. 67. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 4+1 - • ( ) • • AWS • • 2017 8 26 vCPU 110 ( ) • • ( )
  68. 68. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 4+1 -
  69. 69. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
  70. 70. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Auto Scaling 2019 - AWS Auto Scaling 2019 | http://blog.father.gedow.net/2019/02/05/aws-auto-scaling-2019/ 2019
  71. 71. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2 Spot Instances Workshops https://ec2spotworkshops.com/
  72. 72. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ( ) • ( ) ( ) • • ( ) • ( ) • CloudWatch ( ) • • ( ) : • ( ) • ( ) • ( ) • ( ) ( )
  73. 73. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ( ) • 9 • • • • CloudWatch Events • • : • • • • ( )
  74. 74. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Q&A AWS Japan Blog https://aws.amazon.com/jp/blogs/news/
  75. 75. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS AWS https://amzn.to/JPArchive
  76. 76. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark [ ] AWS Innovate https://amzn.to/AWSInnovateJP
  77. 77. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Well-Architected 個別技術相談会 • •
  78. 78. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Webinar https://amzn.to/JPWebinar https://amzn.to/JPArchive

×