Introduction to Amazon EC2 Container Service and setting up build pipeline with ECS and Jenkins. Presented by our DevOps engineer at a meetup conducted in our WhiteHedge office premises.

1. Introduction
to
Amazon
EC2
Container
Service and
setting
up
build
pipeline
with
ECS
and
Jenkins
Swapnil Dahiphale
DevOps
Engineer,
WhiteHedge
Technologies

2. Agenda
• Containers
• Orchestration
• Why
ECS?
• What
is
ECS?
• Key
Components
• Typical
user
workflow
• Build
pipeline
with
Jenkins
and
ECS

3. Containers
“automates
the
deployment
of
any
application
as
a
lightweight,
portable,
self-­‐sufficient container
that
will
run
virtuallyanywhere”

4. What
are containers?
OS
virtualization
Process isolation
Automation
Images
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

5. Why
use Docker?
With Docker we can solve many problems
• “it works on my machine”
• reducing build & deploy time
• Infrastructure
configuration –automation!
• Libs
dependency hell
• Cost
control
and granularity

6. A
container pipeline

7. A
container pipeline
IT Operations
Base
Image
PatchesUtilities

8. A
container pipeline
IT Operations
Base
Image
PatchesUtilities
Ruby
Redis
Logger

9. A
container pipeline
IT Operations Developer
Base
Image
PatchesUtilities
Ruby
Redis
Logger
App

10. A
container pipeline
Base
Image
PatchesUtilities
IT Operations Developer
Ruby
Redis
Logger
App

11. Orchestration!

12. Orchestration
is
a
mean
to
Automate
Manual
Process
Orchestration

13. Why
Orchestration?
• Microservice
Architectures
• Fault
tolerance
/
High
Availability
• Scalability
• Resource
Utilization

14. Orchestration
Tools
• Kubernetes
• Docker
Swarm
• Amazon EC2 Container Service

15. Why
EC2
Container
Service ?

18. What
is
ECS?
Building Block Service

19. What
is
ECS?
Easily
Manage
Clusters
for
Any Scale
• Nothing
to run
• Complete state
• Control
and monitoring
• Scale
ECS
List*
and
Describe*
APIactions

20. What
is
ECS?
Flexible
Container Placement
• Applications
• Batch jobs
• Multiple schedulers

21. What
is
ECS?
Designed
for
use
with
other
AWS services
• Virtual
Private Cloud
• Elastic
Load Balancing
• Elastic
Block Store
• IAM
• CloudTrail

22. What
is
ECS?
Performance at scale

23. What
is
ECS?
Secure
Your containers,
Your instances

24. What
is
ECS?
Extensible
• ComprehensiveAPIs
• Open
sourceagent
• Customschedulers

25. Key
Components
Tasks
Containers
Clusters
Container
Instances

26. Key
Components:
Container
Instances
Amazon
EC2
instances
Docker
daemon
Amazon
ECS
agent

27. Key
Components:
Clusters
Regional
Resource
pool
Grouping
of
Container
Instances
Start
empty,
dynamically
scalable

28. Key
Components:
Tasks
Unit
of
work
Grouping
of
related
containers
Run
on
Container
Instances

29. Key
components:
task definitions
{
"environment": [],
"name": "simple-demo",
"image": "my-
demo", "cpu": 10,
"memory": 500,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"mountPoints": [
{
"sourceVolume": "my-vol",
"containerPath":
"/var/www/my-
vol"
}
],
"entryPoint": [
"/usr/sbin/apache2"
, "-D",
"FOREGROUND"
],
"essential": true
},
{
"name": "busybox",
"image":
"busybox",
"cpu": 10,
"memory":
500,
"volumesFrom"
: [
{
"sourceContainer": "simple-demo"
}
],
"entryPoint"
: [
"sh",
"-c"
],
"command": [
"/bin/sh -c "while true; do
/bin/date > /var/www/my-vol/date; sleep 1;
done""
],
"essential": false
}

30. {
"environment": [],
"name": "simple-demo",
"image": "amazon/amazon-ecs-
sample", "cpu": 10,
"memory": 500,
"portMappings":
[
{
"containerPort": 80,
"hostPort": 80
}
],
"mountPoints": [
{
"sourceVolume": "my-vol",
"containerPath":
"/var/www/my-
vol"
}
],
"entryPoint": [
"/usr/sbin/apache2
", "-D",
"FOREGROUND"
],
"essential": true
},
Key
components:
task definitions
Essential to our task
10 CPU units (1024 is full CPU),
500 megabytes of memory
Expose port 80 in container
to port 80 on host
Create and mount volumes

31. {
Key
components:
task definitions
"name": "busybox",
"image": "busybox",
"cpu": 10,
"memory":
500,
"volumesFrom": [
{
"sourceContainer": "simple-­‐demo"
}
],
"entryPoint": [
"sh",
"-­‐c"
],
"command": [
"/bin/sh
-­‐c
"while
true; do
/bin/date
>
/var/www/my-­‐vol/date;
sleep
1; done""
],
"essential": false
}
[
{
"image": "tutum/wordpress-
stackable", "name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db"
],
"entryPoint": [
"/bin/sh",
"-c"
],
"environment": [
…
],
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
]
},
]
From Docker Hub
Mount volume from other container
Command to exec

32. Typical
user workflow
I want to run aservice

33. Typical
user workflow
Run Instances Amazon
EC2
Use custom AMI with
Docker support and
ECS Agent. Instances
will register with
default cluster.

34. Typical
user workflow
Create Task Definition
Declare resource
requirements for
containers
Shared Data Volume
Node.jsApp
Time of day
App

35. Typical
user workflow
Create Service
Declare resource
requirements for
service
Shared Data Volume
Node.jsApp
Time of day
App
Elastic
Load
Balancing
X 5

36. Typical
user workflow
Describe Service

37. Demo

38. Build
Pipeline
with
ECS
and
Jenkins

39. Continuous
delivery
with Jenkins
Build image
Push image
Update service

40. Q&A

41. Thank You