SlideShare a Scribd company logo
1 of 19
UFO
Tung Nguyen
Jan 2017
About Me
I’ve done a little bit of everything. Both dev and ops.
@tongueroo - Twitter.
https://medium.com/@tongueroo - Blog.
tongueroo.com - Personal Site.
lono - CloudFormation template generator.
jack - Manage ElasticBeanstalk environments.
thor_template - used this to generate this tool.
ufo - ship docker images to ecs. What this talk is about.
These tools are in Ruby.
More: http://tongueroo.com/projects/
Other Tools
Why?
Wanted get my hands dirty with ECS.
Wanted more control over the ECS task definition.
Wanted to automated the building of the docker image as
part of building task definition.
Ending up using it on my side projects.
ECS Introduction - Terms
Task definition - blueprint for docker container.
Task - Actual running container.
Service - Long running tasks.
Cluster - Logic group of EC2 Container instances.
Container Instance - EC2 instance that is part of an ECS
Cluster.
ECS Introduction - Terms
Steps for Deploying on ECS
1.Build and push docker image
2.Build and register task definition
3.Update the ECS service (create the service if needed)
Google Search: “deploying to ecs”
silinternational/ecs-deploy - pulls down existing task
definition.
CircleCI deploy.sh example - nice and simple easy to build
from.
Empire - heroku like opensource SaaS from Remind
engineering. A little more than I need but cool project.
Programmer Virtue #1: Laziness ->
Research
Wanted more control over the task definition template.
Would be nice if the task definition could be reused for
common and similar processes: web, clock, worker.
More Control
Task Definition Similarities
$ cat ufo/output/hi-web.json
{
"family": "hi-web",
"containerDefinitions": [
{
"name": "web",
"image": "tongueroo/hi:ufo-2016-12-04T17-54-
23-2d1634d",
"cpu": 128,
"memoryReservation": 256,
"portMappings": [
{
"containerPort": "3000",
"protocol": "tcp"
}
],
"command": [
"bin/web"
],
"environment": [
{
"name": "ADMIN_PASSWORD",
"value": "secret"
}
],
"essential": true
$ cat ufo/output/hi-worker.json
{
"family": "hi-worker",
"containerDefinitions": [
{
"name": "worker",
"image": "tongueroo/hi:ufo-2016-12-04T17-54-
23-2d1634d",
"cpu": 128,
"memoryReservation": 256,
"command": [
"bin/worker"
],
"environment": [
{
"name": "ADMIN_PASSWORD",
"value": "secret"
}
],
"essential": true
}
]
}$
UFO Template DSL
UFO will use a ERB template to generate the task
definition for all the processes you need: web, clock,
worker, etc.
Allows sharing of code and settings.
Let’s see what it looks like.
UFO Task Definition Template DSL
$ cat ufo/templates/main.json.erb
{
"family": "<%= @family %>",
"containerDefinitions": [
{
"name": "<%= @name %>",
"image": "<%= @image %>",
"cpu": <%= @cpu %>,
<% if @memory %>
"memory": <%= @memory %>,
<% end %>
<% if @memory_reservation %>
"memoryReservation": <%=
@memory_reservation %>,
<% end %>
<% if @container_port %>
"portMappings": [
{
"containerPort": "<%=
@container_port %>",
"protocol": "tcp"
}
],
<% end %>
"command": <%= @command.to_json %>,
<% if @environment %>
$ cat ufo/task_definitions.rb
common = {
image: helper.full_image_name,
cpu: 128,
memory_reservation: 256,
environment: env_file(".env")
}
task_definition "hi-web" do
source "main"
variables(common.dup.deep_merge(
family: task_definition_name,
name: "web",
container_port: helper.dockerfile_port,
command: ["bin/web"]
))
end
task_definition "hi-worker" do
source "main"
variables(common.dup.deep_merge(
family: task_definition_name,
name: "worker",
command: ["bin/worker"]
))
end
UFO Task Definition Output
$ cat ufo/output/hi-web.json
{
"family": "hi-web",
"containerDefinitions": [
{
"name": "web",
"image": "tongueroo/hi:ufo-2016-12-04T17-54-
23-2d1634d",
"cpu": 128,
"memoryReservation": 256,
"portMappings": [
{
"containerPort": "3000",
"protocol": "tcp"
}
],
"command": [
"bin/web"
],
"environment": [
{
"name": "ADMIN_PASSWORD",
"value": "secret"
}
],
"essential": true
$ cat ufo/output/hi-worker.json
{
"family": "hi-worker",
"containerDefinitions": [
{
"name": "worker",
"image": "tongueroo/hi:ufo-2016-12-04T17-54-
23-2d1634d",
"cpu": 128,
"memoryReservation": 256,
"command": [
"bin/worker"
],
"environment": [
{
"name": "ADMIN_PASSWORD",
"value": "secret"
}
],
"essential": true
}
]
}$
ufo ship: whole point of this tool
NOTE: The ECS Cluster and a Container Instance must
already exist. I created Container Instances with the lono
tool.
1. Build and push docker image
2. Build and register task definition
3. Update the ECS service (create the
service automatically if needed)
UFO Demo
ufo ship demo
What Happened Review Summary
ufo init - generates the skeleton ufo files.
Mainly ufo/task_definitons.rb and
ufo/templates/main.json.erb - most configurations and
changes go in here.
ufo/settings.yml - is nice to know about a useful setting in
here.
bin/deploy - wrapper script.
In ufo/task_definitions.rb there are some helper methods.
helper.full_image_name - this is the generated docker
image name.
helper.dockerfile_port - parsed from the Dockerfile.
env_file - useful helper method to translate a env list to
the json formatted list for an ECS Task definition.
Review tasks_definitions.rb Helpers
ufo docker build - only build the docker image
ufo tasks build - only build the task definitions
ufo scale hi-web 1 - scale up and down tasks
Misc ufo commands
THE END
Hacker Noon Blog Post - Ufo—Easily Build Docker Images
and Ship Containers to AWS ECS
Setting up CI with UFO, ECS, and CircleCI
GitHub - https://github.com/tongueroo/ufo
Slideshare - http://www.slideshare.net/tongueroo/ufo-
ship-for-aws-ecs-70885296
tongueroo@gmail.com - feel free to contact me

More Related Content

What's hot

Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
ke4qqq
 
wwc start-launched
wwc start-launchedwwc start-launched
wwc start-launched
Mat Schaffer
 
Nikita Tuk: Handling background processes in iOS: problems & solutions
Nikita Tuk: Handling background processes in iOS: problems & solutionsNikita Tuk: Handling background processes in iOS: problems & solutions
Nikita Tuk: Handling background processes in iOS: problems & solutions
mdevtalk
 
Deployment with Fabric
Deployment with FabricDeployment with Fabric
Deployment with Fabric
andymccurdy
 
PyCon AU 2010 - Getting Started With Apache/mod_wsgi.
PyCon AU 2010 - Getting Started With Apache/mod_wsgi.PyCon AU 2010 - Getting Started With Apache/mod_wsgi.
PyCon AU 2010 - Getting Started With Apache/mod_wsgi.
Graham Dumpleton
 

What's hot (20)

Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
 
Packer, where DevOps begins
Packer, where DevOps beginsPacker, where DevOps begins
Packer, where DevOps begins
 
wwc start-launched
wwc start-launchedwwc start-launched
wwc start-launched
 
Puppet at janrain
Puppet at janrainPuppet at janrain
Puppet at janrain
 
RingoJS
RingoJSRingoJS
RingoJS
 
Automation with Packer and TerraForm
Automation with Packer and TerraFormAutomation with Packer and TerraForm
Automation with Packer and TerraForm
 
Go-Couchbase Golang Paris 2015/12/17
Go-Couchbase Golang Paris 2015/12/17Go-Couchbase Golang Paris 2015/12/17
Go-Couchbase Golang Paris 2015/12/17
 
Apache Cassandra and Go
Apache Cassandra and GoApache Cassandra and Go
Apache Cassandra and Go
 
Elasticsearch (R)Evolution — You Know, for Search… by Philipp Krenn at Big Da...
Elasticsearch (R)Evolution — You Know, for Search… by Philipp Krenn at Big Da...Elasticsearch (R)Evolution — You Know, for Search… by Philipp Krenn at Big Da...
Elasticsearch (R)Evolution — You Know, for Search… by Philipp Krenn at Big Da...
 
Nikita Tuk: Handling background processes in iOS: problems & solutions
Nikita Tuk: Handling background processes in iOS: problems & solutionsNikita Tuk: Handling background processes in iOS: problems & solutions
Nikita Tuk: Handling background processes in iOS: problems & solutions
 
AnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and TricksAnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and Tricks
 
Ansible fest Presentation slides
Ansible fest Presentation slidesAnsible fest Presentation slides
Ansible fest Presentation slides
 
Deployment with Fabric
Deployment with FabricDeployment with Fabric
Deployment with Fabric
 
Ansible for beginners ...?
Ansible for beginners ...?Ansible for beginners ...?
Ansible for beginners ...?
 
Weird things we've seen with OpenStack Neutron
Weird things we've seen with OpenStack NeutronWeird things we've seen with OpenStack Neutron
Weird things we've seen with OpenStack Neutron
 
A Introduction of Packer
A Introduction of PackerA Introduction of Packer
A Introduction of Packer
 
Data integration with embulk
Data integration with embulkData integration with embulk
Data integration with embulk
 
Functional Hostnames and Why they are Bad
Functional Hostnames and Why they are BadFunctional Hostnames and Why they are Bad
Functional Hostnames and Why they are Bad
 
Building Docker images with Puppet
Building Docker images with PuppetBuilding Docker images with Puppet
Building Docker images with Puppet
 
PyCon AU 2010 - Getting Started With Apache/mod_wsgi.
PyCon AU 2010 - Getting Started With Apache/mod_wsgi.PyCon AU 2010 - Getting Started With Apache/mod_wsgi.
PyCon AU 2010 - Getting Started With Apache/mod_wsgi.
 

Viewers also liked

Project Blue Book
Project Blue BookProject Blue Book
Project Blue Book
Kukuasu
 
Ghosts,Demons,Mermaids and Aliens...
Ghosts,Demons,Mermaids and Aliens...Ghosts,Demons,Mermaids and Aliens...
Ghosts,Demons,Mermaids and Aliens...
Guneet Singh
 
Presentation1ghost facts
Presentation1ghost factsPresentation1ghost facts
Presentation1ghost facts
hudanasreen
 
Music presentation.ppt
Music presentation.pptMusic presentation.ppt
Music presentation.ppt
Toni
 

Viewers also liked (20)

UfO's presentation
UfO's presentationUfO's presentation
UfO's presentation
 
Ufo
UfoUfo
Ufo
 
Ufo
UfoUfo
Ufo
 
UFO: Sebuah Analisa dan Implikasi
UFO: Sebuah Analisa dan ImplikasiUFO: Sebuah Analisa dan Implikasi
UFO: Sebuah Analisa dan Implikasi
 
UFO & Aliens
UFO & AliensUFO & Aliens
UFO & Aliens
 
Center for the New Age - UFO Tours
Center for the New Age - UFO ToursCenter for the New Age - UFO Tours
Center for the New Age - UFO Tours
 
Project Blue Book
Project Blue BookProject Blue Book
Project Blue Book
 
J. Allen Hynek - The UFO Experience - A Scientific Inquiry (1972)
J. Allen Hynek - The UFO Experience - A Scientific Inquiry (1972)J. Allen Hynek - The UFO Experience - A Scientific Inquiry (1972)
J. Allen Hynek - The UFO Experience - A Scientific Inquiry (1972)
 
UFOs: Abduction by the marketing hype cycle
UFOs: Abduction by the marketing hype cycleUFOs: Abduction by the marketing hype cycle
UFOs: Abduction by the marketing hype cycle
 
Ufo-fact or fiction
Ufo-fact or fictionUfo-fact or fiction
Ufo-fact or fiction
 
Ghosts,Demons,Mermaids and Aliens...
Ghosts,Demons,Mermaids and Aliens...Ghosts,Demons,Mermaids and Aliens...
Ghosts,Demons,Mermaids and Aliens...
 
The History Of Classical Music (1600 2000)
The History Of Classical Music (1600 2000)The History Of Classical Music (1600 2000)
The History Of Classical Music (1600 2000)
 
English ppt ufo
English ppt ufoEnglish ppt ufo
English ppt ufo
 
Presentation1ghost facts
Presentation1ghost factsPresentation1ghost facts
Presentation1ghost facts
 
Ghost
GhostGhost
Ghost
 
Music presentation.ppt
Music presentation.pptMusic presentation.ppt
Music presentation.ppt
 
Wi max
Wi maxWi max
Wi max
 
Study: The Future of VR, AR and Self-Driving Cars
Study: The Future of VR, AR and Self-Driving CarsStudy: The Future of VR, AR and Self-Driving Cars
Study: The Future of VR, AR and Self-Driving Cars
 
UX, ethnography and possibilities: for Libraries, Museums and Archives
UX, ethnography and possibilities: for Libraries, Museums and ArchivesUX, ethnography and possibilities: for Libraries, Museums and Archives
UX, ethnography and possibilities: for Libraries, Museums and Archives
 
Hype vs. Reality: The AI Explainer
Hype vs. Reality: The AI ExplainerHype vs. Reality: The AI Explainer
Hype vs. Reality: The AI Explainer
 

Similar to Ufo Ship for AWS ECS

Puppetpreso
PuppetpresoPuppetpreso
Puppetpreso
ke4qqq
 
Docker Online Meetup #3: Docker in Production
Docker Online Meetup #3: Docker in ProductionDocker Online Meetup #3: Docker in Production
Docker Online Meetup #3: Docker in Production
Docker, Inc.
 
Cloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep DiveCloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep Dive
Kazuto Kusama
 

Similar to Ufo Ship for AWS ECS (20)

Amazon Web Services and Docker: from developing to production
Amazon Web Services and Docker: from developing to productionAmazon Web Services and Docker: from developing to production
Amazon Web Services and Docker: from developing to production
 
Ansible with oci
Ansible with ociAnsible with oci
Ansible with oci
 
Puppetpreso
PuppetpresoPuppetpreso
Puppetpreso
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
 
Discovering OpenBSD on AWS
Discovering OpenBSD on AWSDiscovering OpenBSD on AWS
Discovering OpenBSD on AWS
 
An intro to Docker, Terraform, and Amazon ECS
An intro to Docker, Terraform, and Amazon ECSAn intro to Docker, Terraform, and Amazon ECS
An intro to Docker, Terraform, and Amazon ECS
 
Containers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific TrioContainers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific Trio
 
Kubernetes for the PHP developer
Kubernetes for the PHP developerKubernetes for the PHP developer
Kubernetes for the PHP developer
 
Novices guide to docker
Novices guide to dockerNovices guide to docker
Novices guide to docker
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013
 
Docker 101
Docker 101 Docker 101
Docker 101
 
Docker Online Meetup #3: Docker in Production
Docker Online Meetup #3: Docker in ProductionDocker Online Meetup #3: Docker in Production
Docker Online Meetup #3: Docker in Production
 
Docker, c'est bonheur !
Docker, c'est bonheur !Docker, c'est bonheur !
Docker, c'est bonheur !
 
Ansible inside
Ansible insideAnsible inside
Ansible inside
 
Build Your Own CaaS (Container as a Service)
Build Your Own CaaS (Container as a Service)Build Your Own CaaS (Container as a Service)
Build Your Own CaaS (Container as a Service)
 
Fullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-endFullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-end
 
Learn you some Ansible for great good!
Learn you some Ansible for great good!Learn you some Ansible for great good!
Learn you some Ansible for great good!
 
Cloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep DiveCloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep Dive
 
Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.
 
ABCs of docker
ABCs of dockerABCs of docker
ABCs of docker
 

More from Tung Nguyen

More from Tung Nguyen (11)

re:Invent 2019 Highly Available ECS Spot Architecture: Save 50%-90%
re:Invent 2019 Highly Available ECS Spot Architecture: Save 50%-90%re:Invent 2019 Highly Available ECS Spot Architecture: Save 50%-90%
re:Invent 2019 Highly Available ECS Spot Architecture: Save 50%-90%
 
Jets: The Ruby Serverless Framework Nashville Tennessee 2019 August
Jets: The Ruby Serverless Framework Nashville Tennessee 2019 AugustJets: The Ruby Serverless Framework Nashville Tennessee 2019 August
Jets: The Ruby Serverless Framework Nashville Tennessee 2019 August
 
Jets: The Ruby Serverless Framework Balkan Ruby Bulgaria 2019 May
Jets: The Ruby Serverless Framework Balkan Ruby Bulgaria 2019 MayJets: The Ruby Serverless Framework Balkan Ruby Bulgaria 2019 May
Jets: The Ruby Serverless Framework Balkan Ruby Bulgaria 2019 May
 
Jets: The Ruby Serverless Framework Ruby Kaigi Japan 2019 April
Jets: The Ruby Serverless Framework Ruby Kaigi Japan 2019 AprilJets: The Ruby Serverless Framework Ruby Kaigi Japan 2019 April
Jets: The Ruby Serverless Framework Ruby Kaigi Japan 2019 April
 
Getting Started with ECS: An Easy Way to Run Docker Containers - AWS Summit A...
Getting Started with ECS: An Easy Way to Run Docker Containers - AWS Summit A...Getting Started with ECS: An Easy Way to Run Docker Containers - AWS Summit A...
Getting Started with ECS: An Easy Way to Run Docker Containers - AWS Summit A...
 
Ruby Conference Belarus 2019 Apr Jets Ruby Serverless Framework
Ruby Conference Belarus 2019 Apr Jets Ruby Serverless FrameworkRuby Conference Belarus 2019 Apr Jets Ruby Serverless Framework
Ruby Conference Belarus 2019 Apr Jets Ruby Serverless Framework
 
AWS Summit Santa Slara 2019 Mar ECS
AWS Summit Santa Slara 2019 Mar ECSAWS Summit Santa Slara 2019 Mar ECS
AWS Summit Santa Slara 2019 Mar ECS
 
Serverless Cron Jobs with Ruby on Jets
Serverless Cron Jobs with Ruby on JetsServerless Cron Jobs with Ruby on Jets
Serverless Cron Jobs with Ruby on Jets
 
Jets: A Ruby Serverless Framework
Jets: A Ruby Serverless FrameworkJets: A Ruby Serverless Framework
Jets: A Ruby Serverless Framework
 
Ruby Support for AWS Lambda at Native Speed with Jets
Ruby Support for AWS Lambda at Native Speed with JetsRuby Support for AWS Lambda at Native Speed with Jets
Ruby Support for AWS Lambda at Native Speed with Jets
 
Getting Started with Amazon ECS: Run Docker Containers on AWS
Getting Started with Amazon ECS: Run Docker Containers on AWSGetting Started with Amazon ECS: Run Docker Containers on AWS
Getting Started with Amazon ECS: Run Docker Containers on AWS
 

Recently uploaded

Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
Kamal Acharya
 
Microkernel in Operating System | Operating System
Microkernel in Operating System | Operating SystemMicrokernel in Operating System | Operating System
Microkernel in Operating System | Operating System
Sampad Kar
 
Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..
MaherOthman7
 
Seizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networksSeizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networks
IJECEIAES
 
1893-part-1-2016 for Earthquake load design
1893-part-1-2016 for Earthquake load design1893-part-1-2016 for Earthquake load design
1893-part-1-2016 for Earthquake load design
AshishSingh1301
 

Recently uploaded (20)

Augmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxAugmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptx
 
SLIDESHARE PPT-DECISION MAKING METHODS.pptx
SLIDESHARE PPT-DECISION MAKING METHODS.pptxSLIDESHARE PPT-DECISION MAKING METHODS.pptx
SLIDESHARE PPT-DECISION MAKING METHODS.pptx
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AI
 
Worksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxWorksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptx
 
handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailing
 
Lab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docxLab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docx
 
Microkernel in Operating System | Operating System
Microkernel in Operating System | Operating SystemMicrokernel in Operating System | Operating System
Microkernel in Operating System | Operating System
 
Interfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfInterfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdf
 
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
 
"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.
 
Filters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsFilters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility Applications
 
Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..
 
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)
 
Introduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoIntroduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of Arduino
 
Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...
 
Seizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networksSeizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networks
 
1893-part-1-2016 for Earthquake load design
1893-part-1-2016 for Earthquake load design1893-part-1-2016 for Earthquake load design
1893-part-1-2016 for Earthquake load design
 
Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...
 

Ufo Ship for AWS ECS

  • 2. About Me I’ve done a little bit of everything. Both dev and ops. @tongueroo - Twitter. https://medium.com/@tongueroo - Blog. tongueroo.com - Personal Site.
  • 3. lono - CloudFormation template generator. jack - Manage ElasticBeanstalk environments. thor_template - used this to generate this tool. ufo - ship docker images to ecs. What this talk is about. These tools are in Ruby. More: http://tongueroo.com/projects/ Other Tools
  • 4. Why? Wanted get my hands dirty with ECS. Wanted more control over the ECS task definition. Wanted to automated the building of the docker image as part of building task definition. Ending up using it on my side projects.
  • 5. ECS Introduction - Terms Task definition - blueprint for docker container. Task - Actual running container. Service - Long running tasks. Cluster - Logic group of EC2 Container instances. Container Instance - EC2 instance that is part of an ECS Cluster.
  • 7. Steps for Deploying on ECS 1.Build and push docker image 2.Build and register task definition 3.Update the ECS service (create the service if needed)
  • 8. Google Search: “deploying to ecs” silinternational/ecs-deploy - pulls down existing task definition. CircleCI deploy.sh example - nice and simple easy to build from. Empire - heroku like opensource SaaS from Remind engineering. A little more than I need but cool project. Programmer Virtue #1: Laziness -> Research
  • 9. Wanted more control over the task definition template. Would be nice if the task definition could be reused for common and similar processes: web, clock, worker. More Control
  • 10. Task Definition Similarities $ cat ufo/output/hi-web.json { "family": "hi-web", "containerDefinitions": [ { "name": "web", "image": "tongueroo/hi:ufo-2016-12-04T17-54- 23-2d1634d", "cpu": 128, "memoryReservation": 256, "portMappings": [ { "containerPort": "3000", "protocol": "tcp" } ], "command": [ "bin/web" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true $ cat ufo/output/hi-worker.json { "family": "hi-worker", "containerDefinitions": [ { "name": "worker", "image": "tongueroo/hi:ufo-2016-12-04T17-54- 23-2d1634d", "cpu": 128, "memoryReservation": 256, "command": [ "bin/worker" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ] }$
  • 11. UFO Template DSL UFO will use a ERB template to generate the task definition for all the processes you need: web, clock, worker, etc. Allows sharing of code and settings. Let’s see what it looks like.
  • 12. UFO Task Definition Template DSL $ cat ufo/templates/main.json.erb { "family": "<%= @family %>", "containerDefinitions": [ { "name": "<%= @name %>", "image": "<%= @image %>", "cpu": <%= @cpu %>, <% if @memory %> "memory": <%= @memory %>, <% end %> <% if @memory_reservation %> "memoryReservation": <%= @memory_reservation %>, <% end %> <% if @container_port %> "portMappings": [ { "containerPort": "<%= @container_port %>", "protocol": "tcp" } ], <% end %> "command": <%= @command.to_json %>, <% if @environment %> $ cat ufo/task_definitions.rb common = { image: helper.full_image_name, cpu: 128, memory_reservation: 256, environment: env_file(".env") } task_definition "hi-web" do source "main" variables(common.dup.deep_merge( family: task_definition_name, name: "web", container_port: helper.dockerfile_port, command: ["bin/web"] )) end task_definition "hi-worker" do source "main" variables(common.dup.deep_merge( family: task_definition_name, name: "worker", command: ["bin/worker"] )) end
  • 13. UFO Task Definition Output $ cat ufo/output/hi-web.json { "family": "hi-web", "containerDefinitions": [ { "name": "web", "image": "tongueroo/hi:ufo-2016-12-04T17-54- 23-2d1634d", "cpu": 128, "memoryReservation": 256, "portMappings": [ { "containerPort": "3000", "protocol": "tcp" } ], "command": [ "bin/web" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true $ cat ufo/output/hi-worker.json { "family": "hi-worker", "containerDefinitions": [ { "name": "worker", "image": "tongueroo/hi:ufo-2016-12-04T17-54- 23-2d1634d", "cpu": 128, "memoryReservation": 256, "command": [ "bin/worker" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ] }$
  • 14. ufo ship: whole point of this tool NOTE: The ECS Cluster and a Container Instance must already exist. I created Container Instances with the lono tool. 1. Build and push docker image 2. Build and register task definition 3. Update the ECS service (create the service automatically if needed)
  • 16. What Happened Review Summary ufo init - generates the skeleton ufo files. Mainly ufo/task_definitons.rb and ufo/templates/main.json.erb - most configurations and changes go in here. ufo/settings.yml - is nice to know about a useful setting in here. bin/deploy - wrapper script.
  • 17. In ufo/task_definitions.rb there are some helper methods. helper.full_image_name - this is the generated docker image name. helper.dockerfile_port - parsed from the Dockerfile. env_file - useful helper method to translate a env list to the json formatted list for an ECS Task definition. Review tasks_definitions.rb Helpers
  • 18. ufo docker build - only build the docker image ufo tasks build - only build the task definitions ufo scale hi-web 1 - scale up and down tasks Misc ufo commands
  • 19. THE END Hacker Noon Blog Post - Ufo—Easily Build Docker Images and Ship Containers to AWS ECS Setting up CI with UFO, ECS, and CircleCI GitHub - https://github.com/tongueroo/ufo Slideshare - http://www.slideshare.net/tongueroo/ufo- ship-for-aws-ecs-70885296 tongueroo@gmail.com - feel free to contact me

Editor's Notes

  1. Time Machine. Infrastructure is Code Philosophy, Culture, Mindset Devs should be able to spin up the cluster themselves w/o any ops involvement. Everyone wants their own internal PasS, their own heroku. That's still the ultimate goal.