Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Colm MacCárthaigh – AWS Principal Engineer
Octob...
Elastic Load Balancing
Security Scalability Availability
Security
Threat modeling
SSL/TLS SSL security policies
• Same-day mitigation for POODLE
• Same-day mitigation for LogJam
• Same-day mitigation for ...
SSL/TLS management
SSL/TLS cipher suites
• Always prefer perfect forward secrecy
• Prefer AES over 3DES over RC4
• Prefer GCM over CBC + HMAC...
SSL/TLS cipher suites
• Legacy clients can cause compatibility issues
• Old firmware in embedded systems
• TVs, controller...
Elastic Load Balancing
2015-05-13T23:39:43.945958Z my-loadbalancer
192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048
0.001...
Elastic Load Balancing
2015-05-13T23:39:43.945958Z my-loadbalancer
192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048
0.001...
Elastic Load Balancing
2015-05-13T23:39:43.945958Z my-loadbalancer
192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048
0.001...
ELB and security compartmentalization
Public subnet
Private subnet
ELB and security compartmentalization
Public subnet
Private subnet
ELB and security compartmentalization
Public subnet
Private subnet
ELB and security compartmentalization
Public subnet
Private subnet
ELB and security compartmentalization
Public subnet
Private subnet
ELB and security compartmentalization
Public subnet
Private subnet
Threat modeling
Scalability
Scalability
L = λW
Scalability
W = L / λ
Scalability
Latency = Load /
Throughput
Scalability
Memory Latency
Scalability
Caching and cache misses
Scalability
Processing time
Count
Scalability
GET / HTTP/1.1
GET /monthly_report/ HTTP/1.1
Scalability
Processing time
Count
Scalability
Wait time
Count
Scalability
Scalability
Wait time
Count
Weighed Round Robin
Single server
Scalability
Scalability
Wait time
Count
Weighed Round Robin
Single server
Least Connections
Scalability
Wait time
Count
Beware of
blackholing
traffic
Weighed Round Robin
Single server
Least Connections
ELB’s own scaling is a mix of preemptive, based
on the instance capacity you add, and reactive,
based on the load you rece...
CloudWatch and Auto Scaling
All load balancer metrics can be used for Auto Scaling.
Allow you to scale dynamically based o...
13 CloudWatch metrics provided for each load
balancer.
Provide detailed insight into the health of the load
balancer and a...
HealthyHostCount
The count of the number of healthy instances
in each Availability Zone.
Most common cause of unhealthy ho...
Latency
Measures the time elapsed in seconds after the request leaves the load
balancer until the response is received.
Te...
SurgeQueue and spillovers
Count of the number of requests that could not be sent to back-end
instances.
Queue up to 1,024 ...
• timestamp
• elb name
• client:port
• backend:port
• request_processing_time
• backend_processing_time
• response_process...
Global scalability
ELB integrates with Amazon Route 53 latency–based routing and geo-
based routing
Useful for application...
Availability
Seamless and graceful replacement
of instances with
no downtime
Health checks
ELB
EC2
instance
EC2
instance
EC2
instance
Health checks
Support for TCP and HTTP health checks
Customize frequency and failure thresholds
Must return a 2xx response...
Idle timeouts allow for connections to be
closed by the load balancer when no
longer in use.
Length of time that an idle connection should be kept open
For both client and back-end connections
Defaults to 60 seconds...
15s
3s
3s
ELB
15s
EC2
instances
Amazon S3
Amazon RDS
Amazon SWF
3s
9s
Idle timeouts
Multiple Availability Zones
VPC
EC2
instanceELB
ELB
EC2
instance
us-west-1aus-west-1b
Amazon
Route 53
Protected by Amazon Route 53 health checks
All load balancers scaled to handle loss
of single Availability Zone.
Amazon Ro...
Health checkers and edge locations
perform the same volume of activity,
whether endpoints are healthy or
unhealthy.
Consta...
Multiple Availability Zones
VPC
EC2
instanceELB
ELB
EC2
instance
us-west-1aus-west-1b
Amazon
Route 53
Always associate two
or more subnets in
different zones with
the load balancer
Using multiple Availability Zones
does bring a few challenges
Requestcount
Time
Traffic imbalances
DNS caching and spreading
DNS TTLs are generally honored
But sometimes there simply are not enough DNS servers
to spread l...
DNS caching by clients and ISPs can often cause clients to target
a specific IP address or stop resolving at all.
Register...
Multiple Availability Zones
VPC
EC2
instanceELB
ELB
EC2
instance
us-west-1aus-west-1b
Amazon
Route 53
Multiple Availability Zones
EC2
instanceELB
ELB
us-west-1aus-west-1b
Amazon
Route 53
VPC
Requestcount
Time
Traffic imbalances
Cross-zone enabled
Load balancer absorbs impact of DNS caching
Eliminates imbalances in back-end instance utilization
Requests distributed ev...
Integrated with AWS CloudFormation, AWS Opsworks,
AWS Elastic Beanstalk, Amazon EC2 Container Service,
Amazon API Gateway,...
Remember to complete
your evaluations!
Thank you!
Upcoming SlideShare
Loading in …5
×

(CMP401) Elastic Load Balancing Deep Dive and Best Practices

3,652 views

Published on

Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances for fault tolerance and load distribution. In this session, we go into detail about Elastic Load Balancing's configuration and day-to-day management, as well as its use in conjunction with Auto Scaling. We explain how to make decisions about the service and share best practices and useful tips for success.

Published in: Technology

(CMP401) Elastic Load Balancing Deep Dive and Best Practices

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Colm MacCárthaigh – AWS Principal Engineer October 2015 CMP401 Elastic Load Balancing Deep Dive and Best Practices
  2. 2. Elastic Load Balancing Security Scalability Availability
  3. 3. Security
  4. 4. Threat modeling
  5. 5. SSL/TLS SSL security policies • Same-day mitigation for POODLE • Same-day mitigation for LogJam • Same-day mitigation for Heartbleed • RC4 removed in advance of ratings and compliance changes
  6. 6. SSL/TLS management
  7. 7. SSL/TLS cipher suites • Always prefer perfect forward secrecy • Prefer AES over 3DES over RC4 • Prefer GCM over CBC + HMAC • Compare against billions of connections from real-world clients
  8. 8. SSL/TLS cipher suites • Legacy clients can cause compatibility issues • Old firmware in embedded systems • TVs, controllers, web scrapers… • ELB defaults strike a balance • Access log gap analysis • We recommend ELBSecurityPolicy-2015-05
  9. 9. Elastic Load Balancing 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1” "curl/7.38.0" DHE-RSA-AES128-SHA TLSv1.2 S3
  10. 10. Elastic Load Balancing 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1” "curl/7.38.0" DHE-RSA-AES128-SHA TLSv1.2
  11. 11. Elastic Load Balancing 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1” "curl/7.38.0" DHE-RSA-AES128-SHA TLSv1.2
  12. 12. ELB and security compartmentalization Public subnet Private subnet
  13. 13. ELB and security compartmentalization Public subnet Private subnet
  14. 14. ELB and security compartmentalization Public subnet Private subnet
  15. 15. ELB and security compartmentalization Public subnet Private subnet
  16. 16. ELB and security compartmentalization Public subnet Private subnet
  17. 17. ELB and security compartmentalization Public subnet Private subnet
  18. 18. Threat modeling
  19. 19. Scalability
  20. 20. Scalability L = λW
  21. 21. Scalability W = L / λ
  22. 22. Scalability Latency = Load / Throughput
  23. 23. Scalability Memory Latency
  24. 24. Scalability Caching and cache misses
  25. 25. Scalability Processing time Count
  26. 26. Scalability GET / HTTP/1.1 GET /monthly_report/ HTTP/1.1
  27. 27. Scalability Processing time Count
  28. 28. Scalability Wait time Count
  29. 29. Scalability
  30. 30. Scalability Wait time Count Weighed Round Robin Single server
  31. 31. Scalability
  32. 32. Scalability Wait time Count Weighed Round Robin Single server Least Connections
  33. 33. Scalability Wait time Count Beware of blackholing traffic Weighed Round Robin Single server Least Connections
  34. 34. ELB’s own scaling is a mix of preemptive, based on the instance capacity you add, and reactive, based on the load you receive.
  35. 35. CloudWatch and Auto Scaling All load balancer metrics can be used for Auto Scaling. Allow you to scale dynamically based on the load balancers' view of the application. Important to consider all metrics when using Auto Scaling; may not be aware of resource contention on another metric. You may be at peak multiple times a day.
  36. 36. 13 CloudWatch metrics provided for each load balancer. Provide detailed insight into the health of the load balancer and application stack. CloudWatch alarms can be configured to notify or take action, in case any metric goes outside of the acceptable range. All metrics provided at 1-minute granularity. Amazon CloudWatch metrics
  37. 37. HealthyHostCount The count of the number of healthy instances in each Availability Zone. Most common cause of unhealthy hosts are health check exceeding the allocated timeout. Test by making repeated requests to the back- end instance from another Amazon EC2 instance. View at the zonal dimension.
  38. 38. Latency Measures the time elapsed in seconds after the request leaves the load balancer until the response is received. Test by sending requests to the back-end instance from another instance. Using minimum, average, and maximum, CloudWatch stats provide upper and lower bounds for latency. Debug individual requests using access logs.
  39. 39. SurgeQueue and spillovers Count of the number of requests that could not be sent to back-end instances. Queue up to 1,024 requests per load balancer node, after which 503 errors will be returned. Often caused by not being able to open connections to the back-end instance. Normally a sign of an underscaled application.
  40. 40. • timestamp • elb name • client:port • backend:port • request_processing_time • backend_processing_time • response_processing_time • elb_status_code • backend_state_code • received_bytes • sent_bytes • “request” • “User-Agent” • Ciphersuite • SSL/TLS protocol version Access logs 2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1” "curl/7.38.0" DHE-RSA- AES128-SHA TLSv1.2
  41. 41. Global scalability ELB integrates with Amazon Route 53 latency–based routing and geo- based routing Useful for applications where latency is critical Online advertising bidding Trading 53
  42. 42. Availability
  43. 43. Seamless and graceful replacement of instances with no downtime
  44. 44. Health checks ELB EC2 instance EC2 instance EC2 instance
  45. 45. Health checks Support for TCP and HTTP health checks Customize frequency and failure thresholds Must return a 2xx response Think hard about health check “depth”
  46. 46. Idle timeouts allow for connections to be closed by the load balancer when no longer in use.
  47. 47. Length of time that an idle connection should be kept open For both client and back-end connections Defaults to 60 seconds but can be set between 1 and 3,600 seconds Timeouts should decrease as you go up the stack Idle timeouts
  48. 48. 15s 3s 3s ELB 15s EC2 instances Amazon S3 Amazon RDS Amazon SWF 3s 9s Idle timeouts
  49. 49. Multiple Availability Zones VPC EC2 instanceELB ELB EC2 instance us-west-1aus-west-1b Amazon Route 53
  50. 50. Protected by Amazon Route 53 health checks All load balancers scaled to handle loss of single Availability Zone. Amazon Route 53 health checks shift traffic away from the failed Availability Zone. Completed within 150 seconds. No other external or control plane dependencies.
  51. 51. Health checkers and edge locations perform the same volume of activity, whether endpoints are healthy or unhealthy. Constant work time System activity Time to react When nothing is failing, the volume of API calls is zero. When failure occurs, the volume of API calls spikes. time System activity Time to react Work on failure
  52. 52. Multiple Availability Zones VPC EC2 instanceELB ELB EC2 instance us-west-1aus-west-1b Amazon Route 53
  53. 53. Always associate two or more subnets in different zones with the load balancer
  54. 54. Using multiple Availability Zones does bring a few challenges
  55. 55. Requestcount Time Traffic imbalances
  56. 56. DNS caching and spreading DNS TTLs are generally honored But sometimes there simply are not enough DNS servers to spread load around fairly Mobile networks typically have a dozen or so top-level resolvers Enterprise networks may have as few as one
  57. 57. DNS caching by clients and ISPs can often cause clients to target a specific IP address or stop resolving at all. Register a wildcard CNAME or ALIAS within Amazon Route 53. // Create a wildcard CNAME or ALIAS in Route 53. *.example.com ALIAS … elb-12345.us-east-1.elb.amazon.com *.example.com CNAME elb-12345.us-east-1.elb.amazon.com // prepend random content for each lookup made by the application. PROMPT> dig +short 25a8ade5-6557-4a54-a60e-8f51f3b195d1.example.com 192.0.2.1 192.0.2.2 DNS optimization
  58. 58. Multiple Availability Zones VPC EC2 instanceELB ELB EC2 instance us-west-1aus-west-1b Amazon Route 53
  59. 59. Multiple Availability Zones EC2 instanceELB ELB us-west-1aus-west-1b Amazon Route 53 VPC
  60. 60. Requestcount Time Traffic imbalances Cross-zone enabled
  61. 61. Load balancer absorbs impact of DNS caching Eliminates imbalances in back-end instance utilization Requests distributed evenly across multiple Availability Zones Check connection limits before enabling No additional bandwidth charge for cross-zone traffic Cross-zone load balancing
  62. 62. Integrated with AWS CloudFormation, AWS Opsworks, AWS Elastic Beanstalk, Amazon EC2 Container Service, Amazon API Gateway, Asgard Load balancers are a common gateway for blue/green deployments Load balancers can be managed programmatically for immutable deployments ELB and DevOps
  63. 63. Remember to complete your evaluations!
  64. 64. Thank you!

×