Introduction to Amazon EC2 Container Service and setting up build pipeline with ECS and Jenkins

1. Introduction to Amazon EC2 Container
Service and setting up build pipeline with
ECS and Jenkins
Swapnil Dahiphale
DevOps Engineer, WhiteHedge Technologies

2. Agenda
• Containers
• Orchestration
• Why ECS?
• What is ECS?
• Key Components
• Typical user workflow
• Build pipeline with Jenkins and ECS

3. Containers
“automates the deployment of any
application as a lightweight,
portable, self-sufficient container
that will run virtually anywhere”

4. What are containers?
OS virtualization
Process isolation
Automation
Images
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

5. Why use Docker?
With Docker we can solve many problems
• “it works on my machine”
• reducing build & deploy time
• Infrastructure configuration –automation!
• Libs dependency hell
• Cost control and granularity

6. A container pipeline
IT Operations
Base
Image
PatchesUtilities

7. A container pipeline
IT Operations
Base
Image
PatchesUtilities
Ruby
Redis
Logger

8. A container pipeline
IT Operations Developer
Base
Image
PatchesUtilities
Ruby
Redis
Logger
App

9. A container pipeline
Base
Image
PatchesUtilities
IT Operations Developer
Ruby
Redis
Logger
App

10. Guest OS
Server
Bins/Libs Bins/Libs
App2App1
$ docker run myimage

12. Orchestration!

13. Orchestration is a mean to Automate Manual Process
Orchestration

14. Why Orchestration?
• Microservice Architectures
• Fault tolerance / High Availability
• Scalability
• Resource Utilization

15. Orchestration Tools
• Kubernetes
• Docker Swarm
• Amazon EC2 Container Service

16. Why EC2 Container Service ?

19. What is ECS?
Building Block Service

20. What is ECS?
Easily Manage Clusters for Any Scale
• Nothing to run
• Complete state
• Control and monitoring
• Scale
ECS List* and Describe* APIactions

21. What is ECS?
Flexible Container Placement
• Applications
• Batch jobs
• Multiple schedulers

22. What is ECS?
Designed for use with other AWS services
• Virtual Private Cloud
• Elastic Load Balancing
• Elastic Block Store
• IAM
• CloudTrail

23. What is ECS?
Performance at scale

24. What is ECS?
Secure
Your containers,
Your instances

25. What is ECS?
Extensible
• ComprehensiveAPIs
• Open source agent
• Custom schedulers

26. Key Components
Tasks
Containers
Clusters
Container Instances

27. Key Components: Container Instances
Amazon EC2 instances
Docker daemon
Amazon ECS agent

28. Key Components: Clusters
Regional
Resource pool
Grouping of Container Instances
Start empty, dynamically scalable

29. Key Components: Tasks
Unit of work
Grouping of related containers
Run on Container Instances

30. Key components: task definitions
{
"environment": [],
"name": "simple-demo",
"image": "my-
demo", "cpu": 10,
"memory": 500,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"mountPoints": [
{
"sourceVolume": "my-vol",
"containerPath":
"/var/www/my-
vol"
}
],
"entryPoint": [
"/usr/sbin/apache2"
, "-D",
"FOREGROUND"
],
"essential": true
},
{
"name": "busybox",
"image":
"busybox",
"cpu": 10,
"memory":
500,
"volumesFrom"
: [
{
"sourceContainer": "simple-demo"
}
],
"entryPoint"
: [
"sh",
"-c"
],
"command": [
"/bin/sh -c "while true; do
/bin/date > /var/www/my-vol/date; sleep 1;
done""
],
"essential": false
}

31. {
"environment": [],
"name": "simple-demo",
"image": "amazon/amazon-ecs-
sample", "cpu": 10,
"memory": 500,
"portMappings":
[
{
"containerPort": 80,
"hostPort": 80
}
],
"mountPoints": [
{
"sourceVolume": "my-vol",
"containerPath":
"/var/www/my-
vol"
}
],
"entryPoint": [
"/usr/sbin/apache2
", "-D",
"FOREGROUND"
],
"essential": true
},
Key components: task definitions
Essential to our task
10 CPU units (1024 is full CPU),
500 megabytes of memory
Expose port 80 in container
to port 80 on host
Create and mount volumes

32. {
Key components: task definitions
"name": "busybox",
"image": "busybox",
"cpu": 10,
"memory": 500,
"volumesFrom": [
{
"sourceContainer": "simple-demo"
}
],
"entryPoint": [
"sh",
"-c"
],
"command":[
"/bin/sh -c "while true; do
/bin/date > /var/www/my-vol/date; sleep 1; done""
],
"essential": false
}
[
{
"image": "tutum/wordpress-
stackable", "name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db"
],
"entryPoint": [
"/bin/sh",
"-c"
],
"environment": [
…
],
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
]
},
]
From Docker Hub
Mount volume from other container
Command to exec

33. Typical user workflow
I want to run aservice

34. Typical user workflow
Run Instances Amazon
EC2
Use custom AMI with
Docker support and
ECS Agent. Instances
will register with
defaultcluster.

35. Typical user workflow
Create TaskDefinition
Declare resource
requirements for
containers
Shared DataVolume
Node.jsApp
Time ofday
App

36. Typical user workflow
Create Service
Declare resource
requirements for
service
Shared DataVolume
Node.jsApp
Time ofday
App
Elastic
Load
Balancing
X 3

37. Typical user workflow
Describe Service

38. Demo

39. Build Pipeline with ECS and Jenkins

40. Continuous delivery with Jenkins
Build image
Push image
Update service

41. Q&A

42. Thank You
github.com/swapnildahiphale
Swapnil Dahiphale
@Swapnil2233
sdahiphale@whitehedge.com