This document discusses moving from a monolithic architecture to microservices using Amazon ECS. It begins with an overview of microservices and the challenges of running them at scale. It then demonstrates how to build and deploy the first containerized microservice on ECS, exploring tasks, services, scheduling, networking and automation. Finally, it shares how Instacart uses ECS for their polyglot architecture with continuous delivery pipelines and blue/green deployments.

2. WIFI: awsDevDay | PASS: CodeHappy
U P N E X T :
Building and Scaling Your
First Containerized
Microservice on AWS

3. T H A N K S T O O U R F R I E N D S A T :

4. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pierre Steckmeyer
August 1, 2017
From Monolith to Microservices
Building and scaling your first containerized
microservice on AWS

5. What to Expect from the Session
• Review microservices architecture and how it differs
from monolithic and service-oriented architectures
• Examine the challenges in running microservices at
scale
• Demonstrate how to run microservices on AWS using
Amazon ECS
• Explore a real-world customer use case with Instacart

6. Microservices Architecture

7. What are microservices?
“A software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using
language-agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices

8. Monolithic vs. SOA vs. Microservices
SOA
Coarse-
grained
Microservices
Fine-grained
Monolithic
Single Unit

9. Monolithic Architecture
webserver
.package
Order UI
Order Service
Inventory
Service
Shipping
Service

10. Monolithic Architecture – Scaling

11. Monolith Development Lifecycle
developers
releasetestbuild
delivery pipelineapp

12. User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Microservices Architecture
Order UI

13. Order UI User UI UI
Order
Service
Service
Shipping
Service
Order UI
Order UI
User UI UIShipping
UI
Order
ServiceOrder
Service
Service
Service
Service
Service
User
Service
Shipping
Service
Microservices Architecture – Scaling

14. Microservices Development Lifecycle
developers delivery pipelinemicroservices
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline

15. Characteristics of Microservice Architectures
Do one
thing well
Independent
Decentralized
Black box
Polyglot
You build it, you run it

16. Challenges in running
microservices

17. Microservice Challenge #1 – Resource Management
Managing a large fleet by hand is impossible:
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
AZ 1 AZ 2
AZ 3

18. Microservices Challenge #2 – Monitoring
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• How do you know if an individual service is healthy?
• How do you measure the performance of an individual
service?
• How do you troubleshoot and debug an individual
service?

19. Microservices Challenge #3: Service Discovery
Each microservice scales up and down independently of
one another:
• How does Service A know the URLs for all instances of
Service B?
• How do you allow services to scale independently while
still using load balancers?
• How does a new instance of a service announce itself to
other services?

20. Microservices Challenge #4: Deployment
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• Each service will be developed, tested, and deployed on
its own timeline – How do you manage this across large
numbers of services?
• Services are polyglot – different languages, frameworks
– how do you efficiently deploy them?
• How do you decide which hosts to deploy a service on?

21. Introducing Amazon ECS
• Fully managed elastic service – You don’t need
to run anything, and the service scales as your
microservices architecture grows
• Shared state optimistic scheduling
• Fully ACID compliant resource and state
management
• Integration with CloudWatch service for
monitoring and logging
• Integration with Code* services for continuous
integration and delivery (CI/CD)

22. Amazon ECS Architecture

23. Amazon ECS
EC2 INSTANCES
LOAD
BALANCER ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
LOAD
BALANCER
Internet

24. The Twelve-Factor App
1. Codebase
2. Dependencies
3. Config
4. Backing Services
5. Build, Release Run
6. Processes
7. Port Binding
8. Concurrency
9. Disposability
10. Dev/Prod parity
11. Logs
12. Admin Processes
https://12factor.net/

25. Example Microservice Architecture on ECS
Amazon
ECR
Amazon
RDS
Application Load
Balancer
ECS Cluster
ECS Cluster
IAM
Amazon API
Gateway*
Amazon
Route 53
Amazon CloudWatch

26. Automatic Service Scaling
Publish metrics
Auto Scaling ECS service
Availability
Zone A
Availability
Zone B
TASK A
Add/Remove ECS
tasks
TASK C
TASK BScaling Policies
Amazon
CloudWatch
Amazon ECS
Application
Load Balancer

27. IAM Roles For Tasks
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
Amazon
DynamoDB
Amazon
S3

28. Secrets Management
•prod.app1.db-pass
•general.license-code
•prod.app2.user-name
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
EC2 System Manager –
Parameter Store

29. Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and
recorded for a period of two weeks
Available metrics: CPUReservation, MemoryReservation,
CPUUtilization, MemoryUtilization
Available dimensions: ClusterName, ServiceName

30. Monitoring with Amazon CloudWatch

31. Service Discovery with Route 53 and Application
Load Balancers
Application Load
Balancer
i-aaa i-bbb i-ccc
i-aaa i-bbb i-ccc
oAuth Target Group
8080 8081
Portal Target Group
Weather Target Group
8000 8001 8002
8080
Amazon
Route 53
ECS Cluster
mydomain.com
mydomain.com mydomain.com/weather
mydomain.com/auth

32. Consuming Events for Service Discovery
app1-tst  10.1.0.11
db1-tst  10.1.0.14
app2  10.1.0.16
db2  10.1.0.18
my-app  10.1.0.20
websrv1 10.1.0.1
websrv2 10.1.0.2
websrv3 10.1.0.4
app-dev1 10.1.0.9
app-dev2 10.1.0.5
app-dev3 10.1.0.8
db-dev 10.1.0.19

33. Deploying Containers on ECS – Choose a Scheduler
Batch Jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-Running Apps
ECS service scheduler
Health management
Scale-up and scale-down
AZ aware
Grouped containers

34. Scheduling Containers: Long-Running App
Deploy using the least space: minimumHealthyPercent =
50%, maximumPercent = 100%

35. Scheduling Containers: Long-Running App
Deploy quickly without reducing service capacity:
minimumHealthyPercent = 100%, maximumPercent = 200%

36. Scheduling Containers: Long-Running App
Blue-Green Deployments
• Define two ECS services
• Each service is associated w/ load
balancer
• Both load balancers in Route 53
record set with weighted routing
policy, 100% primary, 0% secondary
• Deploy to blue or green service and
switch weights
TaskTask
Route 53
record set
with
weighted
routing
policy
0%
100%

37. Task Placement

38. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge
t2.small
t2.small t2.medium
us-east-1aus-east-1d
Placement: Targeting Instance Type & Zone

39. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Spread across Zone and Binpack

40. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Affinity and Anti-Affinity

41. Running a Service

42. t2.medium t2.small t2.small
us-east-1aus-east-1d
t2.medium t2.micro t2.small
us-east-1c
Placement: Multiple Services on a Cluster

43. t2.medium g2.2xlarge t2.micro t2.small
t2.small t2.small g2.2xlarge t2.small
t2.small t2.small
g2.2xlarge t2.small
Placement: Services – Distinct Instances

44. Continuous Delivery to ECS with Jenkins
4. Push image to
Docker registry
2. Build image
from sources
3. Run test on image
1. Code push
triggers build
5. Update service
6. Pull image

45. Continuous Delivery to ECS with Jenkins
Easy Deployment
Developers – Merge into master, done!
Jenkins Build Steps
Trigger via webhooks, monitoring, Lambda
Build Docker image via build and publish plugin
Push Docker image into registry
Register updated job with ECS API

46. Continuous Deployment
AWS
CodeCommit AWS
CodePipeline
AWS
CodeBuild
Amazon
ECR
Amazon
ECS
instance
Spot
Instance
AWS
CloudFormati
on
1. Commit
Code
2. Trigger
Pipeline
3.Build
Artifact
5.Update
Stack
6. Update Service
4. Push Image

47. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How Instacart Uses ECS

48. Instacart
Started in 2012
Same-day delivery from local stores
Small engineering/infrastructure team
“Every minute counts”

49. The Stack
Primarily Ruby/Rails/JS, Python, & R
Few primary domains, each with own SLA
Domains comprised of many small-ish services
Monorepo
Custom PaaS

50. Instacart’s PaaS
Coordinated through RDS database
Same tooling local & cloud
Jobs take desired state from DB, and
turn it into cloud reality

51. Why not EC2?
Relatively slow
Deployments/rollbacks not atomic
Constrained to EC2 instance sizes

52. From EC2 to ECS
Transparent to developers: container vs. instance
Each container gets an IP address & DNS entry
SSH access, FS access

53. Deployment
Atomic through AWS CloudFormation
Rollbacks are instant (& atomic)
Custom “GC” policies per application type
Applications sized ”correctly”
Continuous delivery possible

54. Deployment Pipelines
Webhooks on Jenkins servers
Deploys pipelined to 10% of users, then to 100%
Trust your metrics: 5xx & latency are king

55. Blue/Green Deployments
ALB registration/deregistration for blue/green
Keep one deploy running for instant rollback
Separate ports for separate deploys

56. Discovery
Connected via RabbitMQ
Service discovery through
queues
Instant (de)registration
Back endFront end

57. Thank You!

58. Don’t Forget Evaluations!