The document describes a presentation on getting started with Docker and Amazon ECS. It discusses how Realtor.com migrated their application deployment process from using AMIs to using containers with Amazon ECS. They initially used a "build/burn AMI" process, then migrated to AWS Elastic Beanstalk, and finally moved to using Amazon ECS directly by defining tasks with CloudFormation templates. This improved their deployment speed and ability to deploy multiple versions simultaneously.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
G e tt in g S t arte d w i t h D o c ke r a n d A m azon E C S
N a t h a n P e c k
D e v e l o p e r A d v o c a t e
A W S
N o v e m b e r 2 0 1 7
C O N 2 0 2
J e a n D o m i n g u e z
D i r e c t o r C l o u d S e r v i c e s
R e a l t o r . c o m

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introduction to Containers and
Docker

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application environment components
Runtime Engine Code
Dependencies Configuration

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local Laptop Staging / QA Production On-Prem
Different environments

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local Laptop Staging / QA Production On-Prem
It worked on my machine, why not in
prod?
v6.0.0 v7.0.0 v4.0.0 v7.0.0

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Docker to the rescue
Runtime Engine
Code
Dependencies

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Docker container image
bootfs
kernel
Base image
Image
Image
Container
References
parent
image
Read only image that is used as a
template to launch a container.
Start from base images that have
your dependencies, add your
custom code.
Docker file for easy, reproducable
builds.

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local Laptop Staging / QA Production On-Prem
Four environments, same container

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Virtual machine versus Docker
Server (Host)
Host OS
Hypervisor
App 2
Guest OS Guest OS Guest OS
Bins/Libs Bins/Libs Bins/Libs
App 1
App 2
App 3
VM
Server (Host)
Host OS
Docker
Bins/Libs Bins/Libs Bins/Libs
App 1 App 2 App 3
Container

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container & Docker Benefits
Portable application artifact that runs reliably everywhere
Run different applications or application versions with different
dependencies simultaneously
Better resource utilization by running multiple lightweight containers per
host

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introduction to
Amazon Elastic Container Service

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Running a single host is easy
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A few hosts?
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Many hosts!
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS
Server
Guest
OS

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECS
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet LOAD
BALANCER
LOAD
BALANCER

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster of hosts on Amazon EC2
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet LOAD
BALANCER
LOAD
BALANCER
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lightweight agent on each host
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet LOAD
BALANCER
LOAD
BALANCER
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API for launching containers on the cluster
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet LOAD
BALANCER
LOAD
BALANCER
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container task is placed on a host
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet LOAD
BALANCER
LOAD
BALANCER

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Traffic is sent to your host
EC2 INSTANCES
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
Internet LOAD
BALANCER
LOAD
BALANCER

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Designed for many use cases
Long running applications
One time jobs
Multiple schedulers

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Designed for use with other AWS Services
Elastic Load Balancing
Amazon Elastic Block Store
Amazon Virtual Private Cloud
Amazon CloudWatch
AWS Identity and Access Management
AWS CloudTrail

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Designed for massive scale
Fully managed control plane
Scale to thousands of hosts
Control and monitoring

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introducing Realtor.com

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Jean Dominguez
Director Cloud Services

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
is a le ading online re al e state de stination

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key part of News Corp

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Largest global digital RE network
Proptiger.com
Housing.com
makaan

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Transformation
• Realtor.com: 20 years of
operation
• 2015: Transform to compete
• Put the customer first
• Iterate on customer feedback
• Improve product velocity and
quality
• Rewrite Core Products
• DevOps / Agile / Cloud
• Keep it simple
• Design Services
• Cloud First (AWS)
• Utilize AWS Platforms
• Immutable Infrastructure

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Phase I – Build/Burn AMI (Bakery)
Base AMI Build Instance Burn AMI
Runtime
Engine
Code Dependencies
Launch Config
ASG instances
ELB / ALB • Start with a Base AMI
• Build Instance with
Runtime Engine, Code,
and Dependencies
• Burn (Bake) a new AMI
• Update Launch Config
with new AMI
• Profit

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Phase I – Build/Burn AMI (Bakery)
AMI Problems:
• Large artifacts.
• High degree of duplication from
one service to the next
• A single team can do several
builds and deployments a day.
• Versioning and management
became difficult
• AMIs only need to be built once or
twice a month.
Container benefits:
• Smaller artifacts.
• Less duplication from one service
to the next.
• Registry simplifies container
versioning and management.
• Faster deployments.
• Better portability.
• Cost savings through better
resource management.

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Conway’s West
“Organizations which design systems ... are constrained to produce designs
which are copies of the communication structures of these organizations.”
Amazon ECS

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Phase II – AWS Elastic Beanstalk
• Elastic Beanstalk is an
Orchestration Platform
as a Service that provides
an additional layer of
abstraction to simplify
the deployment of code
and creation of
resources.
• Supports docker images.
• Manages ECS, task
definition, ASG, ELB, and
more.
AMI
Runtime
Engine
Code Dependenci
es
EB-CLI or
Console AWS Elastic
Beanstalk
Amazon ECS
Logging
Alerting

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Phase II – Elastic Beanstalk
• Easy to use. Low learning curve.
• Many teams were already familiar with Elastic Beanstalk.
• Teams shared delivery patterns.
• Deployments were still “slow” compared to Build/Burn.
• For each new task, Elastic Beanstalk launched a new instance.
• Elastic Beanstalk offered very little in the way of managing
application density.

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Phase III - ECS Without Amazon Beanstalk
AWS
CloudFormation
template
stack
Application
Load Balancer
Amazon ECS
Auto Scaling
Base AMI
AWS Lambda
Functions
CloudWatch
Alarms
CloudWatch
Events

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Phase IV - Standardization
• All new work on containerized applications is done using ECS as the
orchestration layer.
• Guild focused on a standard delivery pipeline for containerized
applications using ECS.
• We continue to evolve and improve our utilization of the platform,
and share those improvements across the organization

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Evolve
• Don’t try to build the perfect platform or process on day one.
• Elastic Beanstalk provides the least friction to getting started with
containers on AWS.
• Create a minimum viable process and improve iteratively.
• Get feedback from others, whether customers or colleagues.
• Find the path that works best for you, based upon your business
needs.

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We are Hiring!
careers.move.com/jobs

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU!