The document discusses container scheduling with Amazon ECS. It provides an overview of ECS's core scheduling components including scheduling engines, placement engines, and extensions. It also discusses how ECS supports massive scale, placement strategies, task scheduling patterns, and examples of using placement constraints and priorities.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Dive into Container Scheduling with
Amazon ECS
A n t h o n y S u a r e z — G M , A m a z o n E C S & E C R
S h u b h a R a o — S e n i o r P r o d u c t M a n a g e r , A m a z o n E C S
N o v e m b e r 2 9 , 2 0 1 7
C O N 4 0 4
AWS re:INVENT

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Getting started with containers is easy!

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From one container…

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
…to hundreds…

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
…to thousands…

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECS manages clusters at scale!
ALB ALB
AZ 1 AZ 2
user/scheduler
Scheduler
Cluster State Service
Event Stream
Placement Engine
Scheduling Engine

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Supporting massive scale
450+%
growth
Hundreds of millions
of containers started each week
Millions
of container instances

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Core components
Scheduling engine Placement engine Extensions

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling engines

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Schedulers
DaemonBatchServices Events

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open-source schedulers?
• An open-source project for container scheduling
• These OSS schedulers will power Amazon ECS
• Actively working on a daemon scheduler
• Follow along, comment, or contribute!
http://blox.github.io
https://github.com/blox

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Placement engine

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task placement engine
Name Example

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task placement selection
Cluster Constraints
Custom Constraints
Placement Strategies
Apply Filter
Satisfy CPU, memory, and port requirements
Filter for location, instance type, AMI, or custom
attribute constraints
Identify instances that meet spread or binpack
placement strategy
Select final container instances for placement

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Supported placement strategies
Binpacking Spread Affinity Distinct instance

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task placement chaining
Zone A Zone B
Spread tasks across
zones and binpack
within each zone

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task placement: Easy to get started

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task placement: Easy to get started

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: Spread placement
us-east-1d us-east-1a us-east-1c
g2.2xlarge t2.small
t2.medium t2.small
t2.micro t2.medium
g2.2xlarge t2.small
g2.2xlarge t2.medium
t2.micro t2.small
aws ecs run-task --cluster ecs-demo --task-definition myapp --count 9 --placement-strategy
type=“spread” ,field=“attribute:ecs.availability-zone”

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: Spread & binpack
us-east-1d us-east-1a us-east-1c
g2.2xlarge t2.small
t2.medium t2.small
t2.micro t2.medium
g2.2xlarge t2.small
g2.2xlarge t2.medium
t2.micro t2.small
aws ecs run-task --cluster ecs-demo --task-definition myapp --count 9 --placement-strategy
type=“spread” ,field=“attribute:ecs.availability-zone” type=“binpack” ,field=“memory”

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: Multiple services & strategies
us-east-1d us-east-1a us-east-1c
t2.medium t2.small t2.medium t2.small t2.micro t2.small
aws ecs create-service --service-name srvc-spread --cluster ecs-demo --task-definition myapp-spread
--desired-count 6 --placement-strategy type=“spread” ,field=“attribute:ecs.availability-zone”
aws ecs create-service --service-name srvc-binpk --cluster ecs-demo --task-definition myapp binpk
--desired-count 5 --placement-strategy type=“binpack” ,field=“memory”

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Extensions

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster query language

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS event stream
Cluster state
Task state
Cluster
State
Service
Custom scheduler
Monitoring tools
ECS API

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Placement of containers

26. ECS placement constraints example

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Matt Callanan
Engineering Manager/Tech Lead
“Cloud Acceleration Team”
Expedia
Brisbane, Australia
mcallanan@expedia.com
linkedin.com/in/matthewcallanan
@mcallana

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Avoid relocating tasks to instances about
to be drained during cluster update
• Tasks can get rescheduled to another old instance in the Auto Scaling group that is about
to be replaced—so tasks can get bumped from instance to instance until all instances are
replaced
• Deploy services with a placement constraint on the task definition
• This means that a service won’t be placed on an instance that has an attribute named
“state” with value “pre-drain”
• At cluster replacement time, we will stand up all new clusters, place old clusters into the
“pre-drain” state, and terminate the old instances in batches
• Relocated tasks will only be placed on new instances, avoiding the default “thundering
herd” scenario

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Cluster update—Phase 1: Expand
AA
Blue Auto Scaling Group
Blue CFN Stack

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 1: Expand
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Disable Auto Scaling Processes

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 1: Expand
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Disable Auto Scaling Processes
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 1: Expand
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Disable Auto Scaling Processes
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processesstate = ‘pre-drain’

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
AA
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Blue Auto Scaling Group
Blue CFN Stack
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
Disable Auto Scaling Processes
state = ‘pre-drain’
A
A

42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Green Auto Scaling Group
Green CFN Stack
Disable Auto Scaling Processes
A
A

43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster update—Phase 2: Relocate tasks
( B a t c h e s o f 3 i n s t a n c e s )
Old Instance
New Instance
Draining Instance
Active Task
Relocated Task
“DRAINING” Task
Green Auto Scaling Group
Green CFN Stack
A
A

44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task definition placement constraint

45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task definition placement constraint
•
•
•
•
•

46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling patterns

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling based on…
1. Service metrics
2. Periodic
3. Triggers
4. Dependencies
5. Priorities

48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling based on service metrics
Users
Service CPU, memory utilization alarm
Update service,
increase task count

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduled tasks
Run task or
create service
CloudWatch
events
DevOps

50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling triggered by AWS services
Load
balancing
Lambda
Trigger
Lambda
function
Trigger
CloudWatch alarm
from LB metrics
+1 Desired count
in service
Users

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling triggered by AWS services
2
Run ECS task
1
Enqueue job
Poll
queue
Amazon
SQS

52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling dependencies
Simple job dependency modeling
Job A Job CJob BAWS Batch
ECS

53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling dependencies
“Job-A”
“Job-C”
C:0
…
C:1
C:2
C:3
C:9999
D:0
D:1
D:2
D:3
D:9999
“Job-D”“Job-B”
B:0
…
B:1
B:9
B:2
“Job-E”
Heavy
network I/O
CPU-intensive Large
memory
Setup Cleanup
…
AWS Batch
ECS

54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling priorities
Job 1
Job 5Job 4
Time-critical
High
Low Job 2 Job 3
Job 6
Priority-based job scheduling
AWS Batch
ECS

55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Simplifying container deployment

56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CI/CD for long-running services
AWS
CodePipeline
Developers Git repository ECR ECS

57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Rolling deployments
Max: 200%
minHealthy: 100%
Update service
ECS
Max: 100%
minHealthy: 75%
Service A
Task def’n: 2
Desired count: 4

58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Canary blue-green deployments
More at:
Task
100%
0%

59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
alpha.internal
ECS service discovery with Route 53
App: IP3,
IP4
Frontend:
IP1, IP2
beta.internal
App: IP3,
IP4
Frontend:
IP1, IP2
Users
ECS
Route 53
More at:

60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling compute for containers

61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling compute capacity
1. Spot instances
2. Auto Scaling groups—scale up
3. Auto Scaling group—scale down, draining

62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Compute with EC2 Spot Instances
Spot Instance
compute container 1
Spot Instance
compute container 3
Spot Instance
compute container 5
Spot Instance
compute container 2
Spot Instance
compute container 4
Spot Instance
compute container 6

63. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Compute auto scale up
Event: Per cluster CPU, memory
reservation, or usage
New services
ECS
ECS cluster
CloudWatch
Developers
CloudWatch event

64. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Draining
Compute auto scale down
ECS
ECS cluster
CloudWatch
Event: Per cluster CPU, memory
reservation, or usage

65. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introducing AWS Fargate
Enable developers
to focus 100% on
their applications

66. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MANAGED BY AWS
ELASTIC
INTEGRATED
With AWS Fargate, instance management
goes away
ECS
EC2AWS Fargate

67. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
More on AWS Fargate
CON214 – Introducing AWS Fargate
Today, 4 p.m. Aria, Level 1, Pinyon 2
CON201 – Containers on AWS – State of the Union
Tomorrow, 12:15 p.m. Aria, Level 1, Pinyon 5
CON333 – Deep Dive into AWS Fargate
Tomorrow, 1 p.m. Aria, Level 1, Pinyon 2

68. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!