Application Load Balancer and the integration with AutoScaling and ECS - Pop-up Loft TLV 2017
•Download as PPTX, PDF•
4 likes•7,371 views
- Elastic Load Balancing automatically distributes application traffic across multiple EC2 instances to improve availability and scalability. - The Application Load Balancer provides advanced request routing features like path-based routing and integration with containers. It also offers improved security, performance, and monitoring capabilities compared to the Classic Load Balancer. - Key components of Application Load Balancing include listeners, target groups, targets, rules, health checks, and metrics in CloudWatch. These components work together to route traffic, monitor instances, and scale capacity as needed.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Application Load Balancer and the integration with AutoScaling and ECS - Pop-up Loft TLV 2017
Similar to Application Load Balancer and the integration with AutoScaling and ECS - Pop-up Loft TLV 2017 (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
Application Load Balancer and the integration with AutoScaling and ECS - Pop-up Loft TLV 2017
- 1. Iftach Ragoler, Head of Engineering, Elastic Load Balancing 03/23/2017 Elastic Load Balancing Deep Dive
- 2. EC2 Instance
- 3. Load Balancer used to route incoming requests to multiple EC2 instances. ELB EC2 Instance EC2 Instance EC2 Instance
- 4. Elastic Load Balancing automatically distributes incoming application traffic across multiple applications, microservices and containers hosted on Amazon EC2 instances.
- 5. SecureElastic Integrated Cost Effective
- 6. Elastic Load Balancing provides high availability by utilizing multiple Availability Zones
- 8. Layer 7 (application)Layer 4 (network) Supports TCP and SSL Incoming client connection bound to server connection No header modification Proxy Protocol prepends source and destination IP and ports to request Supports HTTP and HTTPS Connection terminated at the load balancer and pooled to the server Headers may be modified X-Forwarded-For header contains client IP address
- 9. Application Load Balancer Advanced request routing with support for microservices and container-based applications.
- 10. Classic Application Protocol TCP, SSL, HTTP, HTTPS HTTP, HTTPS Platforms EC2-Classic, EC2-VPC EC2-VPC Health checks ✔ Improved CloudWatch metrics ✔ Improved Path-based routing ✔ Container support ✔ WebSockets & HTTP/2 ✔
- 11. New, feature rich, layer 7 load balancing platform Fully-managed, scalable and highly available load balancing platform Content-based routing allows requests to be routed to different applications behind a single load balancer Application Load Balancer
- 12. Application Load Balancer allows for multiple applications to be hosted behind a single load balancer
- 13. EC2 instances registered behind a Classic Load Balancer ELB EC2 Instance EC2 Instance EC2 Instance
- 14. Running two separate applications with Classic Load Balancer requires multiple load balancers ELB EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance ELB EC2 Instance orders.example.com images.example.com
- 15. ELB /orders example.com EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance /images Application Load Balancer allows for multiple applications to be hosted behind a single load balancer
- 16. Multiple applications behind a single load balancer provides a significant cost saving
- 17. Consider blast radius and isolation when grouping applications behind a single load balancer
- 18. Application Load Balancer provides native support for microservice and container-based architectures
- 19. Instances can be registered with multiple ports, allowing for requests to be routed to multiple containers on a single instance ECS will automatically register tasks with the load balancer using a dynamic port mapping Can also be used with other container technologies Application Load Balancer
- 20. ELB /orders example.com EC2 Instance EC2 Instance EC2 Instance /images Application Load Balancer allows containers to be registered with the load balancerECS Container ECS Container ECS Container
- 21. Microservice and container-based architectures provide further cost savings by improving resource utilization
- 22. New API version provided for creating, configuring and managing Application Load Balancers Follows latest AWS best practices for resource identifiers and API design Provides several new resource types, including target groups, targets and rules Application Load Balancer
- 24. Define the protocol and port on which the load balancer listens for incoming connections Each load balancer needs at least one listener to accept incoming traffic, and can support up to 10 listeners Routing rules are defined on listeners Listeners
- 25. Load Balancer Listener Listener Target Group #1 Health Check Health Check Health Check Target Group #2 Target Group #3
- 26. Logical grouping of targets behind a load balancer Target groups can be exist independently from the load balancer, and be associated with a load balancer when needed Regional construct that can be associated with AutoScaling group Target Groups
- 27. Load Balancer Target Group #1 Health Check Health Check Health Check EC2 EC2 EC2 EC2 EC2 EC2 ECS ECS ECS Listener Listener Target Group #2 Target Group #3
- 28. Logical load balancing target, which can be an EC2 instances, micro-service or container-based application EC2 instances can be registered with the same target group using multiple ports A single target can be registered with multiple target groups Targets
- 29. Load Balancer Target Group #1 Health Check Health Check Health Check EC2 EC2 EC2 EC2 EC2 EC2 ECS ECS ECS Listener Listener Target Group #2 Target Group #3 Rule (default) Rule (*/img/*) Rule (default)
- 30. Provide the link between listeners and target groups and consist of conditions and actions When a request meets the condition of the rule, the associated action is taken Today, rules can forward requests to a specified target group Rules
- 31. Conditions can be specified in path pattern format A path pattern is case sensitive, can be up to 128 characters in length, and can contain any of the following characters: • A-Z, a-z, 0-9 • _ - . $ / ~ " ' @ : + • & (using &) • * (matches 0 or more characters) • ? (matches exactly 1 character) Rules (continued)
- 32. Load Balancer Target Group #1 Health Check Health Check Health Check EC2 EC2 EC2 EC2 EC2 EC2 ECS ECS ECS Listener Rule (default) Rule (*/img/*) Listener Rule (default) Target Group #2 Target Group #3
- 33. Today, load balancers support up to 10 rules
- 34. Support for up to 100 rules coming soon to Application Load Balancers
- 35. Use API deletion protection to prevent a load balancer from being erroneously deleted
- 36. Application Load Balancer provides improved performance for Internet applications
- 37. Native support for WebSockets, supporting full-duplex communication channels over a single TCP connection Support for HTTP/2 provides improved page load times from most of today’s browsers Improved performance for real-time and streaming applications Application Load Balancer
- 38. No additional configuration is required to enable WebSockets or HTTP/2
- 39. Classic Load Balancers have offered IPv6 support for some time
- 40. Native support for IPv6 is supported now in Application Load Balancers
- 41. Improvements to application availability and scalability
- 42. EC2 Instance
- 43. Health checks allow for traffic to be shifted away from impaired or failed instances
- 44. ELB EC2 Instance EC2 Instance EC2 Instance Health checks ensure that request traffic is shifted away from a failed instance.
- 45. HTTP and HTTPS health checks Customize the frequency, failure thresholds, and list of successful response codes Detailed reasons for health check failures are now returned via the API and displayed in the Management Console Health Checks
- 46. Application Load Balancer will fail open should all back-ends fail the health check
- 47. Always use multiple Availability Zones
- 48. ELB VPC Customer VPC EC2 InstanceELB ELB EC2 Instance us-west-1aus-west-1b Amazon Route 53
- 49. ELB VPC Customer VPC EC2 InstanceELB ELB us-west-1aus-west-1b Amazon Route 53
- 50. 6 1 Available Zone Risks Availability 2 Available Zones 6 6 100% Extra Capacity 3 Available Zones 3 3 3 50% Extra Capacity
- 51. Using multiple Availability Zones can bring a few challenges
- 52. Imbalanced Instance Capacity ELB VPC Customer VPC EC2 InstanceELB ELB us-west-1aus-west-1b Amazon Route 53 EC2 Instances
- 53. Cross-Zone Load Balancing ELB VPC Customer VPC EC2 InstanceELB ELB us-west-1aus-west-1b Amazon Route 53 EC2 Instances
- 54. Distributes requests evenly across multiple Availability Zones. Absorbs impact of DNS caching and eliminates imbalances in backend instance utilization No additional bandwidth charge for cross-zone traffic. Cross-Zone Load Balancing
- 55. Cross Zone Load Balancing enabled by default on all Application Load Balancers
- 56. Auto Scaling now supports the scaling of applications at the target group level
- 57. Application Load Balancer integrates with Auto Scaling to manage the scaling of each target group independently ELB /orders example.com EC2 Instance EC2 Instance EC2 Instance EC2 Instance /images EC2 Instance
- 58. When using Auto Scaling, keep in mind that your application may be under load during quiet times
- 59. Continued support for advanced application security features
- 60. SSL Negotiation Policies provide selection of ciphers and protocols that adhere to the latest industry best practices Optimized for balance between security and client connectivity, as testing with Amazon.com traffic New: TLSv2, TLSv3 and WinXP policies SSL Offloading
- 61. Application Load Balancer supports security groups to limit access to specified ranges
- 62. Web Application Firewall support to Application Load Balancers
- 63. SSL Negotiation Policies provide selection of ciphers and protocols that adhere to the latest industry best practices Optimized for balance between security and client connectivity, as testing with Amazon.com traffic Website Application Firewall
- 64. Improved load balancer and application monitoring
- 65. CloudWatch metrics provided for each load balancer Provide detailed insight into the health of the load balancer and application stack All metrics provided at the 1-minute granularity Amazon CloudWatch Metrics
- 66. Metrics provided at both the load balancer and target group level CloudWatch alarms can be configured to notify or take action should any metric go outside of the acceptable range Auto Scaling can use these metrics for scaling of the back-end fleet. Amazon CloudWatch Metrics
- 67. HealthyHostCount The count of the number of healthy instances in each Availability Zone Most common cause of unhealthy hosts is health check exceeding the allocated timeout Test by making repeated requests to the backend instance from another EC2 instance View at the zonal dimension
- 68. Latency Measures the elapsed time, in seconds, from when the request leaves the load balancer until the response is received Test by sending requests to the backend instance from another instance Using min, average, and max CloudWatch stats, provide upper and lower bounds for latency Debug individual requests using access logs
- 69. Rejected Connection Count The number of connections that were rejected because the load balancer could not establish a connection with a healthy target in order to route the request This replaces surge queue metrics which are used by the Classic Load Balancer Surge queues often impact client applications, which fast request rejection improves Normally a sign of an under-scaled application
- 70. Target Group Metrics The following metrics are now provided at the target group level, allowing for individual applications to be closely monitored: • RequestCount • HTTPCode_Target_2XX_Count • HTTPCode_Target_3XX_Count • HTTPCode_Target_4XX_Count • HTTPCode_Target_5XX_Count • TargetResponseTime (Latency) • UnHealthyHostCount • HealthyHostCount
- 71. Load balancer request response times are now provided with percentile dimensions Provides visibility into the 90th, 95th, 99th or 99.9th percentile of response times Allows for more meaningful, and aggressive, performance targets for applications CloudWatch Percentiles
- 73. Provide detailed information on each request processed by the load balancer Includes request time, client IP address, latencies, request path, server responses, ciphers and protocols, and user-agents Delivered to an S3 bucket every 5 or 60 minutes Access Logs
- 74. Application Load Balancers insert a unique trace identifier into each request using a custom header: X-Amzn-Trace-ID Trace identifiers are preserved through the request chain to allow for request tracing Trace identifiers are included in access logs and can also be logged by applications themselves Request Tracing
- 75. When should I use Application Load Balancer?
- 76. Classic Application Protocol TCP, SSL, HTTP, HTTPS HTTP, HTTPS Platforms EC2-Classic, EC2-VPC EC2-VPC Health checks ✔ Improved CloudWatch metrics ✔ Improved Path-based routing ✔ Container support ✔ WebSockets & HTTP/2 ✔
- 77. For TCP/SSL or EC2-Classic, use Classic Load Balancer; all other use-cases, use Application Load Balancer
- 78. Thank you!
Editor's Notes
- We’ve all started here, a single instance serving a basic application. It does not take much to realize that this is not an architecture you’d want to take into production. From an availability point of view, you don’t have much hope. From a scalability point of view, you’re down to what a single EC2 instance can support with no plan to add capacity if required.
- Elastic Load Balancing allows you to route application request traffic over 1 to many EC2 instances and ensures that any failed instances does not impact your customers by removing them from service. This is how you want your application to look
- Elastic: scales dynamically as request load increases, we watch all different metrics, throughput CPU, memory, and scale accordingly Secure: support for end-to-end traffic encryption using latest protocols and ciphers, we handle the SSL for you so you can focus on building awesome applications Integrated: Amazon EC2, Auto Scaling, Beanstalk, CloudWatch and Route 53, ECS Cost Effective: cheaper to run an ELB then to try and do yourself with EC2 only pay for what you use - ~$18.50 per month plus a bandwidth charge, ELB becomes cheaper with scale, the larger you get
- EXPLAIN WE WANT AZS FOR FAILURES…. EC2-VPC Architecture for the load balancer. Customers instances in their VPC, spread across two subnets (shown in blue). Load Balancer nodes in a separate VPC, owned by the ELB account. Customer associates subnet with ELB when it is created. ELB takes 2 ENIs from the customers account and attaches them to each load balancer node This is how we give you control using security groups, and how we get very very secure access into your network If public ELB put public IP, if internal, private, which will only be accesible from inside VPC Amazon Route 53 used for DNS and used round robin to direct traffic to each of the load balancer nodes. You get the ELB DNS name from the API, that you can CNAME to or use the R53 alias feature We are HUGE supporters of R53… highly recommend you guys take a look at the health check feature
- Connections: TCP: each connection terminated to LB, but bound to the connection on the back-end; we don’t look at it, just flip it to the backend If you want to to SSL on backend, you can just pass through and do it yourself HTTP: a connection pool is used to the back-end instance. Headers: TCP: the headers are left unchanged and forwarded to the back-end instance HTTP: headers may be inserted depending on the features that are enabled on the load balancer., for example x-forwarded-for Source IP: Since ELB proxies all incoming connection, the back-end instance will see the connection coming from the ELB nodes themselves. TCP: proxy protocol can be used to retrieve the source IP address and port, we append this to the front of the packet HTTP: X-Forwarded-For appended to header contains the source IP address. Algorithms: TCP: round robin is used., the reason for this is no connection pooling, we don’t look at the packet HTTP: least outstanding requests, which is a request-based form of the leastconns algorithm is used, ELB with fewest outstadding requests will get the next request Sticky Sessions: although we always recommend architectures that utilize caching off instance, such as ElastiCache, we do support cookie-based sticky sessions for HTTP listeners.
- Connections: TCP: each connection terminated to LB, but bound to the connection on the back-end; we don’t look at it, just flip it to the backend If you want to to SSL on backend, you can just pass through and do it yourself HTTP: a connection pool is used to the back-end instance. Headers: TCP: the headers are left unchanged and forwarded to the back-end instance HTTP: headers may be inserted depending on the features that are enabled on the load balancer., for example x-forwarded-for Source IP: Since ELB proxies all incoming connection, the back-end instance will see the connection coming from the ELB nodes themselves. TCP: proxy protocol can be used to retrieve the source IP address and port, we append this to the front of the packet HTTP: X-Forwarded-For appended to header contains the source IP address. Algorithms: TCP: round robin is used., the reason for this is no connection pooling, we don’t look at the packet HTTP: least outstanding requests, which is a request-based form of the leastconns algorithm is used, ELB with fewest outstadding requests will get the next request Sticky Sessions: although we always recommend architectures that utilize caching off instance, such as ElastiCache, we do support cookie-based sticky sessions for HTTP listeners.
- We built a special relationship with EC2 to get you your cross zone traffic for free
- Elastic Load Balancing allows you to route application request traffic over 1 to many EC2 instances and ensures that any failed instances does not impact your customers by removing them from service. This is how you want your application to look
- Elastic Load Balancing allows you to route application request traffic over 1 to many EC2 instances and ensures that any failed instances does not impact your customers by removing them from service. This is how you want your application to look
- Elastic Load Balancing allows you to route application request traffic over 1 to many EC2 instances and ensures that any failed instances does not impact your customers by removing them from service. This is how you want your application to look
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- Elastic Load Balancing allows you to route application request traffic over 1 to many EC2 instances and ensures that any failed instances does not impact your customers by removing them from service. This is how you want your application to look
- We built a special relationship with EC2 to get you your cross zone traffic for free
- EXPLAIN WE WANT AZS FOR FAILURES…. EC2-VPC Architecture for the load balancer. Customers instances in their VPC, spread across two subnets (shown in blue). Load Balancer nodes in a separate VPC, owned by the ELB account. Customer associates subnet with ELB when it is created. ELB takes 2 ENIs from the customers account and attaches them to each load balancer node This is how we give you control using security groups, and how we get very very secure access into your network If public ELB put public IP, if internal, private, which will only be accesible from inside VPC Amazon Route 53 used for DNS and used round robin to direct traffic to each of the load balancer nodes. You get the ELB DNS name from the API, that you can CNAME to or use the R53 alias feature We are HUGE supporters of R53… highly recommend you guys take a look at the health check feature
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- EXPLAIN WE WANT AZS FOR FAILURES…. EC2-VPC Architecture for the load balancer. Customers instances in their VPC, spread across two subnets (shown in blue). Load Balancer nodes in a separate VPC, owned by the ELB account. Customer associates subnet with ELB when it is created. ELB takes 2 ENIs from the customers account and attaches them to each load balancer node This is how we give you control using security groups, and how we get very very secure access into your network If public ELB put public IP, if internal, private, which will only be accesible from inside VPC Amazon Route 53 used for DNS and used round robin to direct traffic to each of the load balancer nodes. You get the ELB DNS name from the API, that you can CNAME to or use the R53 alias feature We are HUGE supporters of R53… highly recommend you guys take a look at the health check feature
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- EXPLAIN WE WANT AZS FOR FAILURES…. EC2-VPC Architecture for the load balancer. Customers instances in their VPC, spread across two subnets (shown in blue). Load Balancer nodes in a separate VPC, owned by the ELB account. Customer associates subnet with ELB when it is created. ELB takes 2 ENIs from the customers account and attaches them to each load balancer node This is how we give you control using security groups, and how we get very very secure access into your network If public ELB put public IP, if internal, private, which will only be accesible from inside VPC Amazon Route 53 used for DNS and used round robin to direct traffic to each of the load balancer nodes. You get the ELB DNS name from the API, that you can CNAME to or use the R53 alias feature We are HUGE supporters of R53… highly recommend you guys take a look at the health check feature
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- EXPLAIN WE WANT AZS FOR FAILURES…. EC2-VPC Architecture for the load balancer. Customers instances in their VPC, spread across two subnets (shown in blue). Load Balancer nodes in a separate VPC, owned by the ELB account. Customer associates subnet with ELB when it is created. ELB takes 2 ENIs from the customers account and attaches them to each load balancer node This is how we give you control using security groups, and how we get very very secure access into your network If public ELB put public IP, if internal, private, which will only be accesible from inside VPC Amazon Route 53 used for DNS and used round robin to direct traffic to each of the load balancer nodes. You get the ELB DNS name from the API, that you can CNAME to or use the R53 alias feature We are HUGE supporters of R53… highly recommend you guys take a look at the health check feature
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- EXPLAIN WE WANT AZS FOR FAILURES…. EC2-VPC Architecture for the load balancer. Customers instances in their VPC, spread across two subnets (shown in blue). Load Balancer nodes in a separate VPC, owned by the ELB account. Customer associates subnet with ELB when it is created. ELB takes 2 ENIs from the customers account and attaches them to each load balancer node This is how we give you control using security groups, and how we get very very secure access into your network If public ELB put public IP, if internal, private, which will only be accesible from inside VPC Amazon Route 53 used for DNS and used round robin to direct traffic to each of the load balancer nodes. You get the ELB DNS name from the API, that you can CNAME to or use the R53 alias feature We are HUGE supporters of R53… highly recommend you guys take a look at the health check feature
- We built a special relationship with EC2 to get you your cross zone traffic for free
- We’ve all started here, a single instance serving a basic application. It does not take much to realize that this is not an architecture you’d want to take into production. From an availability point of view, you don’t have much hope. From a scalability point of view, you’re down to what a single EC2 instance can support with no plan to add capacity if required.
- Describe instance health has to be called One of our awesome features to help you maintain a good experience for your customers, and proactively notify you of potential issues Mitigating failures is a hugely important feature of ELB One of the machines starts having issues Health check is reaching out to backend at an interval set by you the customer Anything but a 200 is not healthy, and we will fail away from that instance in event of the failure You get notified, fix the issue, then the backend is marked as healthy, and ELB starts routing traffic again
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- Please please make sure you are using multiple Azs for all of your applications! Dotted line is Azs R53 on top left, does the load balancing between the Azs using roud robin Works very well as long as clients are resolving DNS
- We believe so strongly in multiple AZ that we will use multi AZ even if you don’t We will always use 2 Azs And in order for this to work we need 2 subnets from you Even you don’t have instances in that second AZ, that’s fine
- Here you can see the one AZ is running hot since it only has 1 instance, the other AZ has 3 instances Some customers might run like this, however, usually it’s either a deployment, or you may have problem with some instances Ideally you want to allocate traffic evenly across ALL instances
- Cross Zone LB can solve this problem If client does not obey DNS, we will absorb the balance, bad client might be hitting me in one AZ, but we will scale up and still distribute across all AZs
- We built a special relationship with EC2 to get you your cross zone traffic for free
- Elastic Load Balancing allows you to route application request traffic over 1 to many EC2 instances and ensures that any failed instances does not impact your customers by removing them from service. This is how you want your application to look
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- TCP health checks are shallow, they really just require power to your network card, typically we see customers using these since they could not get HTTP to work with their application, good chance TCP could be working, but application not working How deep should your check go? The should go deep enough to remove an individual node, this can be a whole additional talk If you have a health check that goes too deep, and there is a shared dependency across all nodes, for example your health check actually makes a DB call, and the DB goes down, and execute an actual full customer path, all nodes will be removed from the ELB, and we’ll start throwing 503s, so make sure you remove your shared dependencies from your health checks and monitor those separately
- CW Metrics Amazed at times when customers do not know how many metrics we actually make available With ELB we give you 1 min by default all the time
- CW Metrics Amazed at times when customers do not know how many metrics we actually make available With ELB we give you 1 min by default all the time
- HealthHost and Unhealth host count, sum should always be backend instances behidn the load balancer This comes back to the health check we discussed earlier The most common reason for unhealthy hosts is timeouts! Looks to them health check succeeds, but it took to long to respond Check from another EC2 instance and see why failing
- This is the other very interesting metric This measures the time after we sent the first byte to when we receive the first byte of response from the backend Very good indication of how your app is doing
- Surge queue is a que in the LB where we will queue requests if you don’t have enough backend capacity We will queue them as best we can, we can hold 1024 requests but then we’ll start dropping Amazon has dropped all surge queues, you can see clients timing out on surge queus and sets problems If you see this metric, usually indication of under scaled
- You can auto scale on all CW metrics, surge queue might be late, but latency could be a great one You may be at peak multiple times a day! Important to consider all possible bottlenecks, you may be scaling on CPU, but need to watch IO, memory, etc. different traffic patterns might use different resources Also many people are under scale at the troughs, they remove too much capacity at the trough of their curve
- CW Metrics Amazed at times when customers do not know how many metrics we actually make available With ELB we give you 1 min by default all the time
- CW Metrics Amazed at times when customers do not know how many metrics we actually make available With ELB we give you 1 min by default all the time
- Access logs very useful do dive further, and no which event is driving high latency, requests every single request going through LB Example of customer with very high latencies that were able to diagnose the issue using Access Logs. Integrated with other log providers like Splunk to give near time traffic analysis So if you saw a latency spike you can see THE request that caused the problem
- Access logs very useful do dive further, and no which event is driving high latency, requests every single request going through LB Example of customer with very high latencies that were able to diagnose the issue using Access Logs. Integrated with other log providers like Splunk to give near time traffic analysis So if you saw a latency spike you can see THE request that caused the problem
- Connections: TCP: each connection terminated to LB, but bound to the connection on the back-end; we don’t look at it, just flip it to the backend If you want to to SSL on backend, you can just pass through and do it yourself HTTP: a connection pool is used to the back-end instance. Headers: TCP: the headers are left unchanged and forwarded to the back-end instance HTTP: headers may be inserted depending on the features that are enabled on the load balancer., for example x-forwarded-for Source IP: Since ELB proxies all incoming connection, the back-end instance will see the connection coming from the ELB nodes themselves. TCP: proxy protocol can be used to retrieve the source IP address and port, we append this to the front of the packet HTTP: X-Forwarded-For appended to header contains the source IP address. Algorithms: TCP: round robin is used., the reason for this is no connection pooling, we don’t look at the packet HTTP: least outstanding requests, which is a request-based form of the leastconns algorithm is used, ELB with fewest outstadding requests will get the next request Sticky Sessions: although we always recommend architectures that utilize caching off instance, such as ElastiCache, we do support cookie-based sticky sessions for HTTP listeners.