Capital One began moving to AWS just two years ago. Every day, the amount of traffic we serve from the cloud continues to grow. With development teams having the freedom to choose their own technology stacks, many teams have quickly started moving applications to Docker. In this session, learn how Capital One uses a combination of the Elastic Load Balancing service along with Application Load Balancer features to increase deployment speed and reliability.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Jeff Storey, Engineering Tools Lead, Capital One
Kit Ewbank, Lead Cloud Engineer, Capital One
Peter Dalbhanjan, Solutions Architect, AWS
12/01/2016
From Amazon EC2 to Amazon ECS
How Capital One Uses Application Load
Balancer Features to Serve Traffic at Scale
NET203

2. What to Expect from the Session
• Classic Load Balancer / Application Load Balancer
features
• Load balancers at Capital One
• Capital One’s container journey

3. ELB/ALB features

4. Elastic Load Balancing automatically distributes
incoming application traffic across multiple
Amazon EC2 instances.

5. EC2
Instance
Single instance

6. Classic Load Balancer
ELB
EC2
Instance
EC2
Instance
EC2
Instance
HTTP/HTTPS
TCP/SSL

7. ELB
EC2
Instance
EC2
Instance
EC2
Instance
example.com/app2
example.com/app1
app1
app2
app1/
app2
Application Load Balancer

8. • Health checks
• Idle timeouts
• Cross-zone
• Path-based routing
• Integration with AWS services
• Security
• Monitoring
• Logging
Features – at a glance

9. Support for TCP and HTTP health checks.
Customize the frequency and failure
thresholds.
Must return a 2xx response.
Consider the depth and accuracy of your
health checks.
Health Checks

10. Length of time that an idle connection should be kept open.
For both client and back-end connections.
Defaults to 60 seconds but can be set between 1 and 3,600
seconds.
Timeouts should decrease as you go
up the stack.
Idle Timeouts

11. 15s
3s
3s
ELB
15s
EC2
Instances
Amazon S3
Amazon RDS
Amazon SWF
3s
9s
Idle Timeouts

12. Load balancer absorbs impact of DNS caching.
Eliminates imbalances in back-end instance utilization.
Requests distributed evenly across multiple
Availability Zones.
Check connection limits before enabling.
No additional bandwidth charge for
cross-zone traffic.
Cross-Zone Load Balancing

13. • Health Checks
• Idle Timeouts
• Cross-Zone
• Path-based routing
• Integration with AWS Services
• Security
• Monitoring
• Logging
Features – At a Glance

14. Load balancing at Capital One

15. Why are load balancers so important to Capital
One?
• REST APIs everywhere
• Numerous API implementation technologies
• Load balancing as a service

16. Load balancers at Capital One
• Load balancers everywhere
• ELBs in front of Auto Scaling groups
• ELBs in front of Amazon ECS services
• ALBs in front of Auto Scaling groups
• ALBs in front of Amazon ECS services

17. Classic Load Balancer
ELB
EC2 in ASG
Apache mod_proxy
EC2 in ASGELB
/credit-card
/mortgage

18. What we manage for a “simple” proxy
• Classic LB
• Auto Scaling group
• EC2 instances
• Provisioning
• Logging/monitoring/security
• Apache proxy
• Versions/patching
• Configuration/rules

19. Application Load Balancer
ELB
EC2 in ASG
/credit-card
/mortgage

20. What we manage for an ALB proxy
• ALB target groups

21. Target groups
What are target groups?
• A set of targets
• Targets dynamically register with target groups
• Different routes for each target group

22. Target Group
Path-based routing
Load Balancer
ListenerRules
Target Target
Health
Check Target Group
ListenerRules
Target Target
Health
Check
Port 80 Port 8080
/credit-card /mortgage

23. Capital One’s container journey

24. Container orchestration – the problem
Container Instance
Container
Container
Container
?

25. ECS containers with Classic ELBs
• Auto Scaling with EC2 instances and ELBs just works
• For ECS, not so much

26. Fixed port mapping
Container Instance
Service 1
Container Port 80
Host Port 8080
Service 2
Container Port 80
Host Port 8081
Container Instance
Service 1
Container Port 80
Host Port 8080
Service 2
Container Port 80
Host Port 8081
Load Balancer
Service 1
Port 80
Load Balancer
Service 2
Port 80

27. Container Instance
nginx + registrator + consul + consul-template
Container
registratorContainer
Container
Consul Cluster
nginx
consul-template

28. Container Instance
nginx + registrator + confd + DynamoDB
Container
registratorContainer
Container
confd
DynamoDB
Lambda TTL
function
nginx

29. Container Instance
Container orchestration with ALB
Container
Container
Target Group
Container
Target Group

30. Auto Scaling
• Use Amazon CloudWatch alarms to trigger Auto Scaling
• Container host Auto Scaling
• Service-level Auto Scaling

31. ALB health checking
• Health checks defined for each target group
• Layer 7 health checks only
• Customizable range of “healthy” status codes – not just
200

32. Deployments

33. Capital One deployments
AWS Region
Container
Container
Container
Container
Container
Container
AWS Region
Container
Container
Container
Container
Container
Container
Amazon
Route 53

34. Built-in ALB deployments with ECS
Container Container Container
Container Container Container
X X X
Target Group

35. ALB deployments with ECS
Container
Container
Target Group
Container
Container
Target Group

36. Operating ALBs at scale

37. Transport Security
• Layer 7
• HTTPS security
• Integration with Amazon Certificate Manager (ACM)
• TLS terminated at ALB
• May be re-wrapped to target

38. Logging
• ALB access logs
type | timestamp | elb | client:port | target:port |
request_processing_time | target_processing_time |
response_processing_time | elb_status_code |
target_status_code | received_bytes | sent_bytes | "request"
| "user_agent" | ssl_cipher | ssl_protocol | target_group_arn

39. Monitoring
• CloudWatch metrics provided for each target group
• Includes most metrics that were available in ELBs
• New notable metrics:
• ActiveConnectionCount
• NewConnectionCount
• ProcessedBytes

40. Cost
• Cost model differs for Classic LB vs. ALB
• Classic: Base price + Data transfer charges
• ALB: Base price + Load Balancer Capacity Unit (LCU)
charges
• LCU – Combination of new connections + active connections
+ bandwidth
• Base price 10% less for ALB

41. Product feedback

42. ALB desired features
• Blue/green deployments with weighted routing built into
ECS
• Host-based routing vs. path-based routing
• Support for two-way TLS

43. Conclusion
• Capital One teams continue to move to Application Load
Balancers for newly developed services
• New features drastically simplify ECS integration
• Like every AWS service, we expect it to only get better
as it matures

44. Thank you!

45. Remember to complete
your evaluations!