A short description of ecs works and what are the limitations of this service as a whole. The challenges one can face when using ECS

1. AWS ECS
Lessons Learned
Humayun Jamal
DevOps Practitioner
PropertyFinder Group

2. Hello!
Humayun Jamal
DevOps Practitioner @ PropertyFinder Group
Twitter: @humayunj1
LinkedIn: humayun.jamal
Mail: humayunjamal@hotmail.com

3. What is Elastic Container
Service ?
● A Docker container scheduling service like
Kubernetes/Mesos etc but with very less
features/customisation.
● No need to know in depth of scheduling services to
understand and start using ECS.
● Does not follow or map exactly all Docker features or
latest version.
● Have certain annoying limitations, for e.g memory
reservations and non flexible resource management.

4. How can we use it ?
Ec2 ASG
ECS
optimised
AMI
ECS
TasksECS
Service
ECS
Tasks
● ECS Task definition (json) ~ docker-compose file.
● ECS Tasks runs on ECS Cluster (EC2 hosts).
● ECS Service controls ECS Tasks distribution, state and auto
scaling.
Task
def

5. Example: Hosting a Web App
ELB : 80 APP : 8080
● Classic ELBs are not a good fit for ECS service hosting a web application.
● WHY? ELBs only support hard coded port mappings , ECS will only allow one
container to use that specific port (8080) per ECS host.
● Solution? Use Application ELB

6. Using Application ELB
ELB : 80 APP
● Application ELB’s use dynamic port mappings to work with ECS.
● Any number of web containers spread across ECS hosts can launch on
any dynamic port which will be automatically detected by the App ELB

7. ECS AutoScaling Feature
● Two layers of auto scaling (Service Level & Host Level) .
● Scaling metrics for Service (Memory & CPU reservation) .
● Scaling metrics for Host (Normal EC2 based metrics or Custom ).
● Service scaling based on custom metrics is kind of tricky and require more in depth
knowledge of the behaviour of your application.
● Based on application behaviour and response , the memory/cpu reservation should be
carefully configured, so one does not end up running too many or too few containers on
each host.

8. Deployment
● Only one deployment strategy is currently available based on
minimum/maximum healthy percentage.
● As new task version get in line for deployment the scheduler
creates new tasks based on the healthy percentage and then
drains the current running task and eventually detach from ELB
and then stop it. The new task version then takes over
● This deployment strategy is a let down in other cases but if your
app versions are always forward/backward compatible (utopia)
then you are all set :)

9. Images?
● ECR is local AWS docker registry service.
● The speed of downloading the image was suppose to be faster
but I did some benchmarking against dockerhub and there is not
a lot of difference.
● ECS acts really weird sometimes specially when being used
locally , hidden errors/bugs that just disappears if you go out for
a smoke break :|

10. Logging?
● ECS has a basic dashboard with very limited error
reporting.
● Just throws some error codes in your face then it is up to
you to log in the host ,tail agent logs and try to figure out
what went wrong. (or via ELK).
● Application logging can be done via shipper to a central
location.

11. Monitoring
● ECS only comes with two metrics , memory/cpu reservation.
● Monitoring has to be done using your own method/tools.
● I would recommend SysDig (have not used myself, typical).
● Docker container monitoring is always the tricky part.

12. Automation
● Very poor integration or support from Cloudformation (last checked Jan
2017)
● Terraform can produce a complete ECS stack with both auto scaling
(myth ? )
● Persistent storage ? (may be EFS ).
● Task definitions should be generated via templating. I use jenkins to
create image and produce an updated task definition.

13. Agent
● Lots of bugs were fixed in the latest update.
● No need of automated cleanups but still require a huge volume just to be
on the safe side.
● Behind in docker versions but at least stable version is supported.

14. Verdict!!
◇ Not many people are talking about it
on blogs/youtube. Not much support
or use cases being shared in the
community.
◇ Good for some very specific use cases
and scenarios but not up to the mark
of Kubernetes/Mesos etc.
◇ Deployment strategy is a big
drawback , can not use it for a live
complex application.
◇ Memory Reservation is a good metric
to plan and control the scaling of
environment.
◇ CPU reservation just does not work ,
no metrics really explain how it is
suppose to function.

15. Questions?