Fosdem chef-101-app-deploy

Deploying Apps with Chef
Speaker:

Joshua Timberman Sr. Technical Evangelist
‣ joshua@opscode.com
‣ @jtimberman
‣ www.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 1
Sunday, February 6, 2011

Chef 101
Speaker:

Joshua Timberman Sr. Technical Evangelist
‣ joshua@opscode.com
‣ @jtimberman
‣ www.opscode.com

(They make computers?)


(Security training/certiﬁcation)


Developers?
System administrators?
“Business” people?

http://www.ﬂickr.com/photos/timyates/2854357446/sizes/l/


At a High Level...

‣ A library for configuration management
‣ A configuration management system
‣ A systems integration platform
‣ An API for your entire Infrastructure

http://www.flickr.com/photos/asten/2159525309/sizes/l/


Principles

Idempotent
Data-driven
Sane defaults
Hackability
TMTOWTDI


Multiple applications of
an operation do not
change the result


We start with APIs, you
supply data


option :json_attribs,
:short => "-j JSON_ATTRIBS",
:long => "--json-attributes JSON_ATTRIBS",
:description => "Load attributes from a
JSON file or URL",
:proc => nil

option :node_name,
:short => "-N NODE_NAME",
:long => "--node-name NODE_NAME",
:description => "The node name for this
client",
Defaults are sane, but
:proc => nil

easily changed

Open source and
community


TIMTOWTDI is a Perl
motto

Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.ﬂickr.com/photos/lidarose/225156612 27

A Tour of Chef


Chef Client runs on your
systems


Clients talk to a Chef
Server


Clients authenticate
with RSA keys

Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.ﬂickr.com/photos/debbcollins/3401944550/ 31

The Opscode Platform
is a Chef Server


Command-line API
utility, Knife

http://www.ﬂickr.com/photos/myklroventine/3474391066/

We call each system you
conﬁgure a Node
Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.ﬂickr.com/photos/peterrosbjerg/3913766224/ 34

Nodes have Attributes

Kernel info!
{
"kernel": {
"machine": "x86_64",
"name": "Darwin",
"os": "Darwin",
"version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;
root:xnu-1504.7.4~1/RELEASE_I386",
"release": "10.4.0"
},
"platform_version": "10.6.4",
"platform": "mac_os_x",
"platform_build": "10F569",
"domain": "local",
Platform info!
"os": "darwin",
"current_user": "jtimberman",
"ohai_time": 1278602661.60043,
"os_version": "10.4.0",
"uptime": "18 days 17 hours 49 minutes 18 seconds",
"ipaddress": "10.13.37.116",
"hostname": "cider",
"fqdn": "cider.local",
Hostname and IP!
"uptime_seconds": 1619358
}


Attributes are
Searchable
$ knife search node ‘platform:mac_os_x’
search(:node, ‘platform:mac_os_x’)


Nodes have a Run List
What Roles or Recipes to apply
in Order


Nodes have a Run List
{
"run_list": [
"role[production]",
"role[webserver]"
]
}


Nodes have Roles


Roles have a Run List
What Roles or Recipes to apply
in Order


name "webserver"
description "Systems that serve HTTP traffic"

run_list(
"recipe[apache2]",
"recipe[apache2::mod_ssl]"
)

default_attributes(
"apache" => {
"listen_ports" => [ "80", "443" ]
}
)


Roles are Searchable

$ knife search role ‘listen_ports:80’
search(:role, ‘listen_ports:80’)


Chef manages
Resources on Nodes


Resources...
Declare a description of the state a part of the node should be in

‣ Have a type package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
‣ Have a name end

template "/etc/apache2/apache2.conf" do
‣ Have parameters source "apache2.conf.erb"
owner "root"
‣ Take action to put the group "root"
mode 0644
resource in the action :create
declared state end


Resources take action
through Providers


Providers...
Know how to actually perform the actions speciﬁed by a resource.

Apt, Yum, Rubygems,
Multiple providers
per resource type.
Portage, Macports,
FreeBSD Ports, etc.


Resources

Platform

Provider

Chef::Platform
:ubuntu => {
:default => {
:package => Chef::Provider::Package::Apt,
:service => Chef::Provider::Service::Debian,
:cron => Chef::Provider::Cron,
:mdadm => Chef::Provider::Mdadm
}
},


Recipes are lists of
Resources


Recipes...
Apply resources in the order they are speciﬁed

1
package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
‣ Evaluates resources in end
the order they appear
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
‣ Adds each resource to owner "root"
the Resource Collection group "root"
mode 0644
action :create
2
end


Recipes are just Ruby!
extra_packages = case node[:platform]
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end

extra_packages.each do |pkg|
package pkg do
action :install
end
end


Cookbooks are
packages for Recipes


Cookbooks are
shareable!

cookbooks.opscode.com

Data bags store
arbitrary data


A user data bag item...
% knife data bag show users jtimberman
{
"comment": "Joshua Timberman",
"groups": "sysadmin",
"ssh_keys": "ssh-rsa SUPERSEKRATS jtimberman@cider",
"files": {
".zshrc": {
"mode": "0644",
"source": "dot-zshrc"
},
".vimrc": {
"mode": "0644",
"source": "dot-vimrc"
}
},
"id": "jtimberman",
"uid": 7004,
"shell": "/usr/bin/zsh",
"openid": "http://jtimberman.myopenid.com/"
}

Copyright © 2010 Opscode, Inc - All Rights Reserved


Data Bags are
Searchable
$ knife search users ‘shell:/bin/bash’
search(:users, ‘/bin/bash’)


bash_users = search(:users, 'shell:/bin/bash')

bash_users.each do |u|
user u['id'] do
uid u['id']
shell "/usr/bin/zsh"
comment u['comment']
supports :manage_home => true

Data bags make recipes home "/home/#{u['id']}"
end

awesome-r (that’s
directory "/home/#{u['id']}/.ssh" do
owner u['id']
group u['id']
mode 0700

totally a word)
end

template "/home/#{u['id']}/.ssh/authorized_keys" do
source "authorized_keys.erb"
owner u['id']
group u['id']
mode 0600
variables :ssh_keys => u['ssh_keys']
end
end

I can has applications?

Application Deployment



tar -x -C /app -f app.tar
rsync ~/dev/app www:/app
cap deploy


Server Conﬁguration



Web Servers
Load Balancers
Database Servers



% vi /etc/mysql/my.cnf
#!/bin/bash
Capﬁle


Capistrano anyone?
def install_package(pkg)
if pkg.kind_of?(Array)
run("apt-get -y install #{pkg.join(' ')}")
else
run("apt-get -y install #{pkg}")
end
end

packages = [
"build-essential",
"ruby",
"ruby1.8-dev",
"libopenssl-ruby",
"rake",
"irb",
"zlib1g-dev",
"libssl-dev",
"git-core"
]
logger.info("Installing baseline packages: #{packages.join(' ')}")
install_package(packages)


Application Repository
‣ Source
‣ CI / Build

Chef Repository
cider:~/dev/rails-quick-start (ruby-1.9.2-p0)
master ✔ % ls -l
total 16
-rw-r--r-- 1 jtimberman staff 3521 Nov 5 13:09 README.md
-rw-r--r-- 1 jtimberman staff 2171 Nov 5 13:09 Rakefile
drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 certificates/
drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 config/
drwxr-xr-x 26 jtimberman staff 884 Nov 12 08:16 cookbooks/
drwxr-xr-x 4 jtimberman staff 136 Nov 5 13:25 data_bags/
drwxr-xr-x 9 jtimberman staff 306 Nov 12 08:16 roles/


Chef Repository

Roles
Cookbooks
Application Information
‣ Data Bag!


Application Information

Data Bag
JSON
Predeﬁned structure


Walkthrough


knife ec2 server create 'role[production]' 'role[base]'
'role[radiant_database_master]' 'role[radiant]'
'role[radiant_run_migrations]' 'recipe[radiant::db_bootstrap]'
-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu
-G default -i ami-a403f7cd -f m1.small


'role[radiant_database_master]'

'role[radiant]' 'role[radiant_run_migrations]'
'recipe[radiant::db_bootstrap]'

'role[radiant]'

'role[radiant_load_balancer]'


{
"id": "radiant",
"server_roles": [
"radiant"
],
"type": {
"radiant": [
"rails",
"unicorn"
]
},
"database_master_role": [
"radiant_database_master"
],
"repository": "git://github.com/radiant/radiant.git",
"revision": {
"production": "0.9.1"
},


Server Roles

base.rb
production.rb
radiant.rb
radiant_database_master.rb
radiant_load_balancer.rb
radiant_run_migrations.rb


Base Role

name "base"
description "Base role applied to all nodes."
run_list(
"recipe[apt]",
"recipe[git]",
"recipe[build-essential]",
"recipe[ruby]"
)


Production Role

name "production"
description "Nodes in the production
environment."
default_attributes(
"app_environment" => "production"
)


Radiant Role

name "radiant"
description "radiant front end application
server."
run_list(
"recipe[mysql::client]",
"recipe[application]"
)


Application Recipe

search(:apps) do |app|
(app["server_roles"] & node.run_list.roles).each do |app_role|
app["type"][app_role].each do |thing|
node.run_state[:current_app] = app
include_recipe "application::#{thing}"
end
end
end

node.run_state.delete(:current_app)


Application Rails Recipe

app['gems'].each do |gem,ver|
gem_package gem do
action :install
version ver if ver && ver.length > 0
end
end

deploy_revision app['id'] do
revision app['revision'][node.app_environment]
repository app['repository']
user app['owner']
group app['group']
deploy_to app['deploy_to']
environment 'RAILS_ENV' => node.app_environment
action app['force'][node.app_environment] ? :force_deploy : :deploy

...

Radiant Database Master Role

name "radiant_database_master"
description "Database master for the radiant
application."
run_list(
"recipe[database::master]"
)


Database Master Recipe
search(:apps) do |app|
(app['database_master_role'] & node.run_list.roles).each do |
dbm_role|
app['databases'].each do |env,db|
if env =~ /#{node[:app_environment]}/
root_pw = node["mysql"]["server_root_password"]
mysql_database "create #{db['database']}" do
host "localhost"
username "root"
password root_pw
database db['database']
action [:create_db]
end
end
end
end
end


Radiant Load Balancer Role

name "radiant_load_balancer"
description "radiant load balancer"
run_list(
"recipe[haproxy::app_lb]"
)
override_attributes(
"haproxy" => {
"app_server_role" => "radiant"
}
)


Haproxy App Load Balancer Recipe

pool_members = search("node", "role:#{node['haproxy']
['app_server_role']} AND app_environment:#{node['app_environment']}")
|| []

template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members
notifies :restart, resources(:service => "haproxy")
end


Control Deployment
knife data bag edit apps radiant

"force": {
"production": false
},

"force": {
"production": true
},


Control Migrations
data bag item has migrate setting
need an attribute set as well

"migrate": {
"production": true
}

name "radiant_run_migrations"
description "Run db:migrate on demand for radiant"
override_attributes(
"apps" => {
"radiant" => {
"production" => {
"run_migrations" => true
}
}
}
)

Add the role and migrations will be run
knife node run list add NODE ‘role[radiant_run_migrations]’

Migrations will run, and the role
is removed by Chef
automatically.
ruby_block "remove_run_migrations" do
block do
if node.role?("#{app['id']}_run_migrations")
Chef::Log.info("Migrations were run, removing role[#{app
['id']}_run_migrations]")
node.run_list.remove("role[#{app['id']}_run_migrations]")
end
end
end

Your Application
Your application is different than Radiant.
But not a unique snowﬂake,
right?
Mostly, you will just need to modify the data and create application
speciﬁc roles...
But wait, I’m using Rails 3!


Use the Gems data
Use bundler or bundler08 in the gems hash of the application data

"gems": { "gems": {
"bundler": "1.0.9" "bundler08": "0.8.5"
}, },

before_migrate do
if app['gems'].has_key?('bundler')
execute "bundle install" do
ignore_failure true
cwd release_path
end
elsif app['gems'].has_key?('bundler08')
execute "gem bundle" do
ignore_failure true
cwd release_path
end


How does it scale?


Quick FAQ


Testing


Reporting


vs [Other tool]


These slides will be
posted


Resources/Questions

www.opscode.com/chef
IRC and Mailing lists
‣ irc.freenode.net #chef
‣ lists.opscode.com

Twitter:
‣ @opscode, #opschef
‣ @jtimberman

Questions?


Fosdem chef-101-app-deploy

Recommended

Recommended

More Related Content

Similar to Fosdem chef-101-app-deploy

Similar to Fosdem chef-101-app-deploy (20)

Recently uploaded

Recently uploaded (20)

Fosdem chef-101-app-deploy