Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014

7,327 views

Published on

Amazon EC2 instances give customers a variety of high-bandwidth networking choices. In this session, we discuss how to choose among Amazon EC2 networking technologies and examine how to get the best performance out of Amazon EC2 enhanced networking and cluster networking. We also share best practices and useful tips for success.

Published in: Technology
  • Wrote Cloud Design Fundamentals book with alot of AWS coverage, http://www.amazon.com/Cloud-Design-Fundamentals-Multilayered-Engineers/dp/1508470979/ref=sr_1_1/179-9314079-3914054?s=books&ie=UTF8&qid=1430589752&sr=1-1
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014

  1. 1. November 12, 2014 | Las Vegas, NV Becky Weiss, Principal Software Engineer, Amazon EC2 Networking
  2. 2. Elastic network interface Subnet A us-east-1a 10.0.1.0/24 10.0.1.100 Subnet A2 us-east-1a 10.0.2.0/24 10.0.1.101 10.0.2.50 10.0.2.51 Subnet C us-east-1c 10.0.3.0/24 10.0.3.99 Instance 1 Instance 2 Instance 3 Instance 4
  3. 3. elastic network interface Subnet A us-east-1a 10.0.1.0/24 10.0.1.100 Subnet A2 us-east-1a 10.0.2.0/24 10.0.1.101 10.0.2.50 10.0.2.51 Subnet C us-east-1c 10.0.3.0/24 10.0.3.99 Instance 1 Instance 2 Instance 3 Instance 4 Placement group
  4. 4. Subnet A is in us-east-1a
  5. 5. C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count 2 --------------------------------------------------------------------------------- | RunInstances | +----------------------------------------+--------------------------------------+ | OwnerId | 123456789012 | | ReservationId | r-9f5404b5 | +----------------------------------------+--------------------------------------+ | Instances | |+-----------------------------------+-----------------------------------------+| || AmiLaunchIndex | 0 || || Architecture | x86_64 || || ClientToken | None || || EbsOptimized | False || || Hypervisor | xen || || ImageId | ami-b66ed3de ||
  6. 6. C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count 2 --------------------------------------------------------------------------------- | RunInstances | +----------------------------------------+--------------------------------------+ | OwnerId | 123456789012 | | ReservationId | r-9f5404b5 | +----------------------------------------+--------------------------------------+ | Instances | |+-----------------------------------+-----------------------------------------+| || AmiLaunchIndex | 0 || || Architecture | x86_64 || || ClientToken | None || || EbsOptimized | False || || Hypervisor | xen || || ImageId | ami-b66ed3de || AMI: More about this choice later…
  7. 7. C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count 2 --------------------------------------------------------------------------------- | RunInstances | +----------------------------------------+--------------------------------------+ | OwnerId | 123456789012 | | ReservationId | r-9f5404b5 | +----------------------------------------+--------------------------------------+ | Instances | |+-----------------------------------+-----------------------------------------+| || AmiLaunchIndex | 0 || || Architecture | x86_64 || || ClientToken | None || || EbsOptimized | False || || Hypervisor | xen || || ImageId | ami-b66ed3de || Big instance type: c3.8xlarge
  8. 8. Avg: 0.167msec
  9. 9. NetworkingTestPlacementGroup available cluster
  10. 10. C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count 2 --placement GroupName=NetworkingTestPlacementGroup --------------------------------------------------------------------------------- | RunInstances | +----------------------------------------+--------------------------------------+ | OwnerId | 123456789012 | | ReservationId | r-13374839 | +----------------------------------------+--------------------------------------+ | Instances | |+-----------------------------------+-----------------------------------------+| || AmiLaunchIndex | 0 || || Architecture | x86_64 || || ClientToken | None || || EbsOptimized | False || || Hypervisor | xen || || ImageId | ami-b66ed3de ||
  11. 11. Avg: .099msec
  12. 12. Instance 1 Instance 2 ...........
  13. 13. Virtualization layer eth0 eth1 Instance Virtual NICs Physical NIC
  14. 14. Virtualization layer eth0 Instance Physical NIC VF Driver eth1 VF
  15. 15. [ec2-user@ip-10-0-3-70 ~]$ ethtool -i eth0 driver: vif version: firmware-version: bus-info: vif-0 … [ec2-user@ip-10-0-3-70 ~]$ ethtool -i eth0 driver: ixgbevf version: 2.14.2+amzn firmware-version: N/A bus-info: 0000:00:03.0 …
  16. 16. amzn-ami-hvm-2012.03.1.x86_64-ebs hvm
  17. 17. --attribute sriovNetSupport InstanceId i-37c5d1d9 Not yet!
  18. 18. [ec2-user@ip-10-0-3-125 ~]$ sudo yum update OS update
  19. 19. reboot-instances Reboot (OS update) (Not shown here: analogous steps for other Linux distros)
  20. 20. Add to Windows driver store
  21. 21. stop-instances Stop the instance
  22. 22. stop-instances --sriov-net-support simple Enable SRIOV Cannot be undone
  23. 23. start-instances Start
  24. 24. start-instances --attribute sriovNetSupport InstanceId i-37c5d1d9 Value simple We’re on
  25. 25.  modinfo ixgbevf  aws ec2 register-image --name MyEnhancedNetworkingImage --image-location … --sriov-net-support-simple
  26. 26. i2.8xlarge Storage-optimized instance
  27. 27. require 'mongo‘ 'randomdb' until Time SECONDS_TO_RUN KEY_MAX :key Time if :times_accessed :key else :key :value :times_accessed end Time end Spin in tight loop: Read a random document Then write it back
  28. 28. def add_write_statistic :sample_count :sum :minimum :minimum :maximum :maximum end Aggregating statistics for CloudWatch
  29. 29. require 'aws-sdk' AWS CloudWatch Client if Time :namespace 'NetworkingTest/MongoDemo', :metric_data => [{:metric_name => 'WriteTime', :dimensions => [{:name => 'RunId', :value => MY_RUN_ID}], :statistic_values => write_stats}], :unit => 'Seconds' Time :sample_count :sum end CloudWatch PutMetricData: Writing a custom metric
  30. 30. # ec2-run-instances ami-b66ed3de --instance-type c3.large --subnet subnet-c03cfb99 -- group sg-72caf017 --placement-group NetworkingTestPlacementGroup --monitor --user-data- file my_startup_script.sh --iam-profile NetworkingTestIAMRole --instance-count 10 RESERVATION r-d13d6f37 123456789012 INSTANCE i-fb6d5352 ami-b66ed3de ip-10-0-1-113.ec2.internal pending NetworkingTestSSHKey 0 c3.large 2014-10-30T13:26:33+0000 us-east-1a monitoring-pending 10.0.1.113 vpc-ca28afaf subnet-c03cfb99 ebs NetworkingTestPlacementGroup hvm xen sg-72caf017 defaultfalse arn:aws:iam::123456789012:instance- profile/NetworkingTestIAMRole NIC eni-b560caed subnet-c03cfb99 vpc-ca28afaf 123456789012 in-use 10.0.1.113 true NICATTACHMENT eni-attach-fb6ddf9d 0 attaching 2014-10-30T06:26:33-0800 true GROUP sg-72caf017 default ...
  31. 31. # ec2-run-instances ami-b66ed3de --instance-type c3.large --subnet subnet-c03cfb99 -- group sg-72caf017 --placement-group NetworkingTestPlacementGroup --monitor --user-data- file my_startup_script.sh --iam-profile NetworkingTestIAMRole --instance-count 10 RESERVATION r-d13d6f37 123456789012 INSTANCE i-fb6d5352 ami-b66ed3de ip-10-0-1-113.ec2.internal pending NetworkingTestSSHKey 0 c3.large 2014-10-30T13:26:33+0000 us-east-1a monitoring-pending 10.0.1.113 vpc-ca28afaf subnet-c03cfb99 ebs NetworkingTestPlacementGroup hvm xen sg-72caf017 defaultfalse arn:aws:iam::123456789012:instance- profile/NetworkingTestIAMRole NIC eni-b560caed subnet-c03cfb99 vpc-ca28afaf 123456789012 in-use 10.0.1.113 true NICATTACHMENT eni-attach-fb6ddf9d 0 attaching 2014-10-30T06:26:33-0800 true GROUP sg-72caf017 default ... CloudWatch detailed monitoring: 1-minute metrics
  32. 32. # ec2-run-instances ami-b66ed3de --instance-type c3.large --subnet subnet-c03cfb99 -- group sg-72caf017 --placement-group NetworkingTestPlacementGroup --monitor --user-data- file my_startup_script.sh --iam-profile NetworkingTestIAMRole --instance-count 10 RESERVATION r-d13d6f37 123456789012 INSTANCE i-fb6d5352 ami-b66ed3de ip-10-0-1-113.ec2.internal pending NetworkingTestSSHKey 0 c3.large 2014-10-30T13:26:33+0000 us-east-1a monitoring-pending 10.0.1.113 vpc-ca28afaf subnet-c03cfb99 ebs NetworkingTestPlacementGroup hvm xen sg-72caf017 defaultfalse arn:aws:iam::123456789012:instance- profile/NetworkingTestIAMRole NIC eni-b560caed subnet-c03cfb99 vpc-ca28afaf 123456789012 in-use 10.0.1.113 true NICATTACHMENT eni-attach-fb6ddf9d 0 attaching 2014-10-30T06:26:33-0800 true GROUP sg-72caf017 default ... Startup script file
  33. 33. # cat startup_script.sh Download client test script from S3 Then gogogo!
  34. 34. # ec2-run-instances ami-b66ed3de --instance-type c3.large --subnet subnet-c03cfb99 -- group sg-72caf017 --placement-group NetworkingTestPlacementGroup --monitor --user-data- file my_startup_script.sh --iam-profile NetworkingTestIAMRole --instance-count 10 RESERVATION r-d13d6f37 123456789012 INSTANCE i-fb6d5352 ami-b66ed3de ip-10-0-1-113.ec2.internal pending NetworkingTestSSHKey 0 c3.large 2014-10-30T13:26:33+0000 us-east-1a monitoring-pending 10.0.1.113 vpc-ca28afaf subnet-c03cfb99 ebs NetworkingTestPlacementGroup hvm xen sg-72caf017 defaultfalse arn:aws:iam::123456789012:instance- profile/NetworkingTestIAMRole NIC eni-b560caed subnet-c03cfb99 vpc-ca28afaf 123456789012 in-use 10.0.1.113 true NICATTACHMENT eni-attach-fb6ddf9d 0 attaching 2014-10-30T06:26:33-0800 true GROUP sg-72caf017 default ... Security best practice: Launch instances with IAM roles if they need to access any AWS resources
  35. 35. # aws iam list-role-policies --role-name NetworkingTestIAMRole { "PolicyNames": [ "NetworkingTestIAMRole-CloudWatchPolicy", "NetworkingTestIAMRole-S3Policy" ] }
  36. 36. # aws iam get-role-policy --role-name NetworkingTestIAMRole --policy-name NetworkingTestIAMRole-S3Policy Allow retrieving objects from a particular S3 bucket
  37. 37. # aws iam get-role-policy --role-name NetworkingTestIAMRole --policy-name NetworkingTestIAMRole- CloudWatchPolicy Allow CloudWatch PutMetricData
  38. 38. Label WriteTime 389483.0 2014-10-29T02:30:00Z Seconds 390189.0 2014-10-29T02:33:00Z Seconds 392373.0 2014-10-29T02:34:00Z Seconds 392387.0 2014-10-29T02:32:00Z Seconds 377256.0 2014-10-29T02:31:00Z Seconds SampleCount statistic: How many of these WriteTime statistics were written across all instances during each minute?
  39. 39. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 “WriteTime” SampleCount statistic by number of client instances TPS, regular TPS, enhanced
  40. 40. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 DiskWriteBytes 1-minute Sum statistic by number of client instances Regular Enhanced
  41. 41. Placement group
  42. 42. Instance Virtualization layer VF driver
  43. 43. http://bit.ly/awsevals

×