Docker containers have become a de facto industry standard for a reason : they are self-contained, predictable and easy to deploy. Managing containers at scale, however, can be a daunting task. Luckily, AWS offers several services enabling scalability for containers. Managed solutions like ECS, EKS and Fargate stand right next to the end-to-end "build-it-yourself" option of using EC2. In this presentation, we will make sense of the available options, their features, limitations and when to choose which.

1. 12 April 2018
Managing
Containers on
AWS
Prepared for:
Montreal AWS Meetup
trinimbus.com

2. TABLE OF
CONTENTS
TriNimbus 2
Quick Review of Containers 03
Containers at Scale 08
AWS Solutions 12
ECR 13
Non-Managed Cluster in EC2 16
ECS 18
EKS 29
Fargate 37
Choosing the Right Solution 45
Questions 50

3. Quick Review of
Containers
3

4. 12 April 2018 TriNimbus 4
What are Containers ?
A container is a packaged
filesystem including all files
required to run a given application.
It guarantees the same behaviour
in all environments without
requiring a virtualization layer.
Source

5. 12 April 2018 TriNimbus 5
Container Technologies
Although there are other container
technologies in the ecosystem,
none is as widely adopted as
Docker.

6. 12 April 2018 TriNimbus 6
Deploying Containers
Deploying Docker containers is
facilitated by the use of a Registry. A
registry is a service similar in spirit to
a package manager and allows for
the storage, versioning and
distribution of Docker containers.
hub

7. 12 April 2018 TriNimbus 7
Running Containers
pjcliche@devbox:~/Code$ docker run nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
2a72cbf407d6: Pull complete
e19f9e910af9: Pull complete
2f3d26a87e79: Pull complete
Digest:
sha256:e36d7f5dabf1429d84135bb8a8086908e1150f1a178c75719a9e0e53ebb90353
Status: Downloaded newer image for nginx:latest

8. Containers at Scale
8

9. 12 April 2018 TriNimbus 9
Single-Host Clusters
Host
Single-host considerations include
(but are not limited to) :
- Container health and
interdependency
- Container interconnectivity
- Shared storage
- Container scheduling and scaling
- Resource management
- Log management
- Service discovery
- Ingress management

10. 12 April 2018 TriNimbus 10
Multi-Host Clusters
Host Host
Multi-host clusters add a layer of
complexity to the single-host
considerations by requiring
coordination among resources.
Additionally, the following become
important :
- Container placement
- Host interconnectivity
Host Host

11. 12 April 2018 TriNimbus 11
Orchestration Frameworks
Because of the different layers of
complexity involved, managing the
state of a container cluster is best
handled by orchestration frameworks.
Orchestration frameworks handle the
different aspects of coordinating the
deployment and operation of
container clusters at both the host
and container layers.

12. AWS Solutions
12

13. Elastic Container
Registry - ECR
13

14. - AWS-managed container registry
- IAM-based authorization
- Resource-level permissions
12 April 2018 TriNimbus 14
What is ECR ?

15. - Usage is free
- Pay only for image storage and
outgoing data transfers (going
outside of AWS)
12 April 2018 TriNimbus 15
Pricing

16. Non-Managed Cluster
in EC2
16

17. 12 April 2018 TriNimbus 17
Overview
The most flexible yet
high-maintenance solution.
Hosts are deployed onto EC2, which
can then be leveraged to autoscale
the host layer of the cluster.
EC2 EC2
EC2 EC2

18. Elastic Container
Service - ECS
18

19. 12 April 2018 TriNimbus 19
Overview
AWS-proprietary managed
orchestration framework
Generally available since 2015
Mature and deeply integrated
Worker nodes are provided by user
through EC2
Containers are orchestrated through
Tasks and Services

20. 12 April 2018 TriNimbus 20
Topology
Host Host Host Host
ECS
EC2 / VPC

21. 12 April 2018 TriNimbus 21
Hosts and Provisioning
- AWS provides baseline ECS
AMIs for :
- AWS Linux
- Ubuntu
- CoreOS
- Windows
- Using ECS is free, pay only for
EC2 usage
- Hosts are EC2-based and not
provisioned by ECS
- Only requirement for hosts is to
run the ECS Container Agent
- The lightweight requirements
allow for the use of custom-built
and specialized worker nodes

22. 12 April 2018 TriNimbus 22
Tasks
Tasks are logical groupings of
containers that will always be
deployed together (on the same
instance)
- Contains configuration for every
container in the group (image,
exposed ports, mounted volumes)
- Can be scheduled through a
cron-like interface and triggered
by CloudWatch Events or
manually

23. 12 April 2018 TriNimbus 23
Short-lived Task Scheduling
Host Host Host Host
ECS
EC2 / VPC
Queue
SQS/Kinesis
CloudWatch
Event

24. 12 April 2018 TriNimbus 24
Services
Services are schedulers managing
long-lived tasks
- Handles auto-scaling by
integrating with CloudWatch
Events
- Automatically registers
dynamically allocated container
ports in *LBs
- Can be set to leverage Route 53
DNS Service Discovery

25. 12 April 2018 TriNimbus 25
*LB Service Registration
Host Host Host Host
ECS
EC2 / VPC
*LB
/foo
/bar
Updates Target Groups
Service A

26. 12 April 2018 TriNimbus 26
Route 53 Service Registration
Host Host Host Host
ECS
EC2 / VPC
/baz
Updates A/SRV Record
Service B

27. 12 April 2018 TriNimbus 27
Task Placement
When using the RunTask and
CreateService interfaces, task
placement strategies and
constraints can be specified
- Strategies determine how
instances will be chosen for task
deployment
- Binpack : Highest density
- Random
- Spread : Round-robin
- Constraints limit task
deployments to specific instances
- Distinct : One per instance
- MemberOf : Specific instances
- Strategies and constraints can be
multi-layered and combined

28. 12 April 2018 TriNimbus 28
Task Placement Algorithm
When Amazon ECS places tasks, it uses the following process to select
container instances:
1) Filter on resources (CPU, memory, port)
2) Filter on constraints
3) Filter on strategies
4) Select instance

29. Elastic Container
Service for
Kubernetes - EKS
29

30. 12 April 2018 TriNimbus 30
What is EKS ?
AWS-managed Kubernetes (k8s) cluster
Abstracts control plane
Integrates with some AWS services
Previewing in us-west-2
Containers are orchestrated through Pods,
Controllers and Services

31. 12 April 2018 TriNimbus 31
A Look at the Kubernetes Model
Source

32. EKS Layers
12 April 2018 TriNimbus 32
EKS
EC2 / VPC

33. 12 April 2018 TriNimbus 33
Hosts and Provisioning
- Hosts (k8s worker nodes) are
EC2-based and not provisioned
by EKS
- Nodes must be provisioned with
kubelet, a container runtime
interface and kube-proxy
- AWS provides a CloudFormation
template to deploy instances to an
EKS cluster
- Pricing is still unclear

34. 12 April 2018 TriNimbus 34
Masters
- Security is HIPAA and
PCI-compliant
- Network policies are baked in
- Calico
- Tigera
- Behind the scenes, EKS deploys
three masters across three AZs
and exposes a single API
endpoint
- Masters autoscale as required by
workload
- Version upgrades are automated
- Patches are automatically
applied
- Minors are scheduled
- Supports up to latest - 2

35. 12 April 2018 TriNimbus 35
AWS Integrations
- RBAC ⇔ IAM
RBAC authentication is enabled
and integrated with IAM using
Heptio Authenticator
Source

36. 12 April 2018 TriNimbus 36
AWS Integrations
- In-VPC communication with
masters through PrivateLink
- ENIs leveraged for pod
networking
- Native pod access to the VPC
network and ENIs achieved
through AWS-built open-source
CNI module
- ELBs leveraged for ingress
- ALB/NLB incoming
- CloudWatch Logs and CloudTrail
leveraged for logging and auditing

37. Fargate
37

38. 12 April 2018 TriNimbus 38
What is Fargate ?
AWS-provided serverless container
orchestration facility
General availability for ECS in us-east-1
Full release by EOY
Supplements ECS and EKS by
abstracting worker nodes
Similar in behaviour to Lambda,
incompatible with long-lived service pattern

39. 12 April 2018 TriNimbus 39
ECS Topology
Host Host Host Host
ECS
EC2 / VPC

40. 12 April 2018 TriNimbus 40
ECS on Fargate Topology
Host Host Host Host
ECS
Fargate

41. EKS Layers
12 April 2018 TriNimbus 41
EKS
EC2 / VPC

42. EKS on Fargate Layers
12 April 2018 TriNimbus 42
EKS
Fargate

43. Fargate Topology
12 April 2018 TriNimbus 43
- Fargate tasks are instantiated with
VPC-attached ENI for security
- 10GB allocated storage per task
- An additional 4GB for shared
volumes
- 4GB max size of image
Task Task
Fargate
VPC
ENI ENI

44. 12 April 2018 TriNimbus 44
Usage
ECS on Fargate
LaunchType of RunTask/Service set
to FARGATE
- Not compatible with task
placement constraints
- Not compatible with container
links
- Not compatible host-based
volume mounting
EKS on Fargate
Still not released - TBD

45. Choosing the Right
Solution
45

46. 12 April 2018 TriNimbus 46
Managed/Not
- Managed takes a considerable
operational load off your team
- Managed abstracts
masters/orchestrators, lock you
in predefined behaviour
- Not Managed is fully extensible
and customisable

47. 12 April 2018 TriNimbus 47
ECS/EKS
- ECS is AWS-locked, EKS is
closer to agnosticity
- ECS is integrated end-to-end
with AWS, EKS has been
retrofitted into it (will get better)
- ECS is conceptually simple
- ECS is developed by AWS and
only marginally extensible, EKS
leverages k8s which is
open-source and has a massive
plugin library
- ECS is limited to AWS, EKS can
be extended to manage on-prem
as well through k8s federation

48. 12 April 2018 TriNimbus 48
Serverful/Serverless
- Serverful gives you full access
to node capabilities, allows for
specialized nodes (GPU, Spot,
etc)
- Serverless removes your
operational load entirely
- Serverless limits you to
pre-defined instance
configuration
- Serverless is not compatible
with the service model
- Serverless is “infinitely” scalable

49. Choice tree
12 April 2018 TriNimbus 49
Credits Mency Woo

50. Questions?
TriNimbus.com
pierre@trinimbus.com
Pierre Cliche
12 April 2018 TriNimbus 50