More Related Content Similar to Fosdem chef-101-app-deploy (20) Fosdem chef-101-app-deploy1. Deploying Apps with Chef
Speaker:
Joshua Timberman Sr. Technical Evangelist
‣ joshua@opscode.com
‣ @jtimberman
‣ www.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 1
Sunday, February 6, 2011
2. Chef 101
Speaker:
Joshua Timberman Sr. Technical Evangelist
‣ joshua@opscode.com
‣ @jtimberman
‣ www.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 2
Sunday, February 6, 2011
3. Copyright © 2010 Opscode, Inc - All Rights Reserved 3
Sunday, February 6, 2011
4. Copyright © 2010 Opscode, Inc - All Rights Reserved 4
Sunday, February 6, 2011
7. Copyright © 2010 Opscode, Inc - All Rights Reserved 7
Sunday, February 6, 2011
8. Copyright © 2010 Opscode, Inc - All Rights Reserved 8
Sunday, February 6, 2011
9. Copyright © 2010 Opscode, Inc - All Rights Reserved 9
Sunday, February 6, 2011
10. Copyright © 2010 Opscode, Inc - All Rights Reserved 10
Sunday, February 6, 2011
11. Copyright © 2010 Opscode, Inc - All Rights Reserved 11
Sunday, February 6, 2011
12. Copyright © 2010 Opscode, Inc - All Rights Reserved 12
Sunday, February 6, 2011
13. Copyright © 2010 Opscode, Inc - All Rights Reserved 13
Sunday, February 6, 2011
14. Copyright © 2010 Opscode, Inc - All Rights Reserved 14
Sunday, February 6, 2011
15. Copyright © 2010 Opscode, Inc - All Rights Reserved 15
Sunday, February 6, 2011
16. Copyright © 2010 Opscode, Inc - All Rights Reserved 16
Sunday, February 6, 2011
17. Copyright © 2010 Opscode, Inc - All Rights Reserved 17
Sunday, February 6, 2011
18. Developers?
System administrators?
“Business” people?
http://www.flickr.com/photos/timyates/2854357446/sizes/l/
Copyright © 2010 Opscode, Inc - All Rights Reserved 18
Sunday, February 6, 2011
19. Copyright © 2010 Opscode, Inc - All Rights Reserved 19
Sunday, February 6, 2011
20. At a High Level...
‣ A library for configuration management
‣ A configuration management system
‣ A systems integration platform
‣ An API for your entire Infrastructure
http://www.flickr.com/photos/asten/2159525309/sizes/l/
Sunday, February 6, 2011
21. Principles
Idempotent
Data-driven
Sane defaults
Hackability
TMTOWTDI
Copyright © 2010 Opscode, Inc - All Rights Reserved 21
Sunday, February 6, 2011
22. Multiple applications of
an operation do not
change the result
Copyright © 2010 Opscode, Inc - All Rights Reserved 22
Sunday, February 6, 2011
23. We start with APIs, you
supply data
Copyright © 2010 Opscode, Inc - All Rights Reserved 23
Sunday, February 6, 2011
24. option :json_attribs,
:short => "-j JSON_ATTRIBS",
:long => "--json-attributes JSON_ATTRIBS",
:description => "Load attributes from a
JSON file or URL",
:proc => nil
option :node_name,
:short => "-N NODE_NAME",
:long => "--node-name NODE_NAME",
:description => "The node name for this
client",
Defaults are sane, but
:proc => nil
easily changed
Copyright © 2010 Opscode, Inc - All Rights Reserved 24
Sunday, February 6, 2011
25. Open source and
community
Copyright © 2010 Opscode, Inc - All Rights Reserved 25
Sunday, February 6, 2011
26. Copyright © 2010 Opscode, Inc - All Rights Reserved 26
Sunday, February 6, 2011
27. TIMTOWTDI is a Perl
motto
Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/lidarose/225156612 27
Sunday, February 6, 2011
28. A Tour of Chef
Copyright © 2010 Opscode, Inc - All Rights Reserved 28
Sunday, February 6, 2011
29. Chef Client runs on your
systems
Copyright © 2010 Opscode, Inc - All Rights Reserved 29
Sunday, February 6, 2011
30. Clients talk to a Chef
Server
Copyright © 2010 Opscode, Inc - All Rights Reserved 30
Sunday, February 6, 2011
31. Clients authenticate
with RSA keys
Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/debbcollins/3401944550/ 31
Sunday, February 6, 2011
32. The Opscode Platform
is a Chef Server
Copyright © 2010 Opscode, Inc - All Rights Reserved 32
Sunday, February 6, 2011
33. Command-line API
utility, Knife
http://www.flickr.com/photos/myklroventine/3474391066/
Copyright © 2010 Opscode, Inc - All Rights Reserved 33
Sunday, February 6, 2011
34. We call each system you
configure a Node
Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/peterrosbjerg/3913766224/ 34
Sunday, February 6, 2011
35. Nodes have Attributes
Kernel info!
{
"kernel": {
"machine": "x86_64",
"name": "Darwin",
"os": "Darwin",
"version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;
root:xnu-1504.7.4~1/RELEASE_I386",
"release": "10.4.0"
},
"platform_version": "10.6.4",
"platform": "mac_os_x",
"platform_build": "10F569",
"domain": "local",
Platform info!
"os": "darwin",
"current_user": "jtimberman",
"ohai_time": 1278602661.60043,
"os_version": "10.4.0",
"uptime": "18 days 17 hours 49 minutes 18 seconds",
"ipaddress": "10.13.37.116",
"hostname": "cider",
"fqdn": "cider.local",
Hostname and IP!
"uptime_seconds": 1619358
}
Copyright © 2010 Opscode, Inc - All Rights Reserved 35
Sunday, February 6, 2011
36. Attributes are
Searchable
$ knife search node ‘platform:mac_os_x’
search(:node, ‘platform:mac_os_x’)
Copyright © 2010 Opscode, Inc - All Rights Reserved 36
Sunday, February 6, 2011
37. Nodes have a Run List
What Roles or Recipes to apply
in Order
Copyright © 2010 Opscode, Inc - All Rights Reserved 37
Sunday, February 6, 2011
38. Nodes have a Run List
{
"run_list": [
"role[production]",
"role[webserver]"
]
}
Copyright © 2010 Opscode, Inc - All Rights Reserved 38
Sunday, February 6, 2011
39. Nodes have Roles
Copyright © 2010 Opscode, Inc - All Rights Reserved 39
Sunday, February 6, 2011
40. Roles have a Run List
What Roles or Recipes to apply
in Order
Copyright © 2010 Opscode, Inc - All Rights Reserved 40
Sunday, February 6, 2011
41. name "webserver"
description "Systems that serve HTTP traffic"
run_list(
"recipe[apache2]",
"recipe[apache2::mod_ssl]"
)
default_attributes(
"apache" => {
"listen_ports" => [ "80", "443" ]
}
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 41
Sunday, February 6, 2011
42. Roles are Searchable
$ knife search role ‘listen_ports:80’
search(:role, ‘listen_ports:80’)
Copyright © 2010 Opscode, Inc - All Rights Reserved 42
Sunday, February 6, 2011
43. Chef manages
Resources on Nodes
Copyright © 2010 Opscode, Inc - All Rights Reserved 43
Sunday, February 6, 2011
44. Resources...
Declare a description of the state a part of the node should be in
‣ Have a type package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
‣ Have a name end
template "/etc/apache2/apache2.conf" do
‣ Have parameters source "apache2.conf.erb"
owner "root"
‣ Take action to put the group "root"
mode 0644
resource in the action :create
declared state end
Sunday, February 6, 2011
45. Resources take action
through Providers
Copyright © 2010 Opscode, Inc - All Rights Reserved 45
Sunday, February 6, 2011
46. Providers...
Know how to actually perform the actions specified by a resource.
Apt, Yum, Rubygems,
Multiple providers
per resource type.
Portage, Macports,
FreeBSD Ports, etc.
Sunday, February 6, 2011
47. Resources
Platform
Provider
Sunday, February 6, 2011
48. Chef::Platform
:ubuntu => {
:default => {
:package => Chef::Provider::Package::Apt,
:service => Chef::Provider::Service::Debian,
:cron => Chef::Provider::Cron,
:mdadm => Chef::Provider::Mdadm
}
},
Copyright © 2010 Opscode, Inc - All Rights Reserved 48
Sunday, February 6, 2011
49. Recipes are lists of
Resources
Copyright © 2010 Opscode, Inc - All Rights Reserved 49
Sunday, February 6, 2011
50. Recipes...
Apply resources in the order they are specified
1
package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
‣ Evaluates resources in end
the order they appear
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
‣ Adds each resource to owner "root"
the Resource Collection group "root"
mode 0644
action :create
2
end
Sunday, February 6, 2011
51. Recipes are just Ruby!
extra_packages = case node[:platform]
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end
Copyright © 2010 Opscode, Inc - All Rights Reserved 51
Sunday, February 6, 2011
52. Cookbooks are
packages for Recipes
Copyright © 2010 Opscode, Inc - All Rights Reserved 52
Sunday, February 6, 2011
53. Cookbooks are
shareable!
cookbooks.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 53
Sunday, February 6, 2011
54. Data bags store
arbitrary data
Copyright © 2010 Opscode, Inc - All Rights Reserved 54
Sunday, February 6, 2011
55. A user data bag item...
% knife data bag show users jtimberman
{
"comment": "Joshua Timberman",
"groups": "sysadmin",
"ssh_keys": "ssh-rsa SUPERSEKRATS jtimberman@cider",
"files": {
".zshrc": {
"mode": "0644",
"source": "dot-zshrc"
},
".vimrc": {
"mode": "0644",
"source": "dot-vimrc"
}
},
"id": "jtimberman",
"uid": 7004,
"shell": "/usr/bin/zsh",
"openid": "http://jtimberman.myopenid.com/"
}
Copyright © 2010 Opscode, Inc - All Rights Reserved
Sunday, February 6, 2011
56. Data Bags are
Searchable
$ knife search users ‘shell:/bin/bash’
search(:users, ‘/bin/bash’)
Copyright © 2010 Opscode, Inc - All Rights Reserved 56
Sunday, February 6, 2011
57. bash_users = search(:users, 'shell:/bin/bash')
bash_users.each do |u|
user u['id'] do
uid u['id']
shell "/usr/bin/zsh"
comment u['comment']
supports :manage_home => true
Data bags make recipes home "/home/#{u['id']}"
end
awesome-r (that’s
directory "/home/#{u['id']}/.ssh" do
owner u['id']
group u['id']
mode 0700
totally a word)
end
template "/home/#{u['id']}/.ssh/authorized_keys" do
source "authorized_keys.erb"
owner u['id']
group u['id']
mode 0600
variables :ssh_keys => u['ssh_keys']
end
end
Copyright © 2010 Opscode, Inc - All Rights Reserved 57
Sunday, February 6, 2011
58. I can has applications?
Copyright © 2010 Opscode, Inc - All Rights Reserved 58
Sunday, February 6, 2011
59. Copyright © 2010 Opscode, Inc - All Rights Reserved 59
Sunday, February 6, 2011
60. Copyright © 2010 Opscode, Inc - All Rights Reserved 60
Sunday, February 6, 2011
61. Copyright © 2010 Opscode, Inc - All Rights Reserved 61
Sunday, February 6, 2011
62. Copyright © 2010 Opscode, Inc - All Rights Reserved 62
Sunday, February 6, 2011
63. Copyright © 2010 Opscode, Inc - All Rights Reserved 63
Sunday, February 6, 2011
65. Application Deployment
tar -x -C /app -f app.tar
rsync ~/dev/app www:/app
cap deploy
Copyright © 2010 Opscode, Inc - All Rights Reserved 65
Sunday, February 6, 2011
66. Server Configuration
Copyright © 2010 Opscode, Inc - All Rights Reserved 66
Sunday, February 6, 2011
67. Server Configuration
Web Servers
Load Balancers
Database Servers
Copyright © 2010 Opscode, Inc - All Rights Reserved 67
Sunday, February 6, 2011
68. Server Configuration
% vi /etc/mysql/my.cnf
#!/bin/bash
Capfile
Copyright © 2010 Opscode, Inc - All Rights Reserved 68
Sunday, February 6, 2011
69. Capistrano anyone?
def install_package(pkg)
if pkg.kind_of?(Array)
run("apt-get -y install #{pkg.join(' ')}")
else
run("apt-get -y install #{pkg}")
end
end
packages = [
"build-essential",
"ruby",
"ruby1.8-dev",
"libopenssl-ruby",
"rake",
"irb",
"zlib1g-dev",
"libssl-dev",
"git-core"
]
logger.info("Installing baseline packages: #{packages.join(' ')}")
install_package(packages)
Copyright © 2010 Opscode, Inc - All Rights Reserved 69
Sunday, February 6, 2011
70. Application Deployment
Application Repository
‣ Source
‣ CI / Build
Chef Repository
cider:~/dev/rails-quick-start (ruby-1.9.2-p0)
master ✔ % ls -l
total 16
-rw-r--r-- 1 jtimberman staff 3521 Nov 5 13:09 README.md
-rw-r--r-- 1 jtimberman staff 2171 Nov 5 13:09 Rakefile
drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 certificates/
drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 config/
drwxr-xr-x 26 jtimberman staff 884 Nov 12 08:16 cookbooks/
drwxr-xr-x 4 jtimberman staff 136 Nov 5 13:25 data_bags/
drwxr-xr-x 9 jtimberman staff 306 Nov 12 08:16 roles/
Copyright © 2010 Opscode, Inc - All Rights Reserved 70
Sunday, February 6, 2011
71. Chef Repository
Roles
Cookbooks
Application Information
‣ Data Bag!
Copyright © 2010 Opscode, Inc - All Rights Reserved 71
Sunday, February 6, 2011
72. Application Information
Data Bag
JSON
Predefined structure
Copyright © 2010 Opscode, Inc - All Rights Reserved 72
Sunday, February 6, 2011
73. Walkthrough
Copyright © 2010 Opscode, Inc - All Rights Reserved 73
Sunday, February 6, 2011
74. knife ec2 server create 'role[production]' 'role[base]'
'role[radiant_database_master]' 'role[radiant]'
'role[radiant_run_migrations]' 'recipe[radiant::db_bootstrap]'
-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu
-G default -i ami-a403f7cd -f m1.small
Copyright © 2010 Opscode, Inc - All Rights Reserved 74
Sunday, February 6, 2011
75. knife ec2 server create 'role[production]' 'role[base]'
'role[radiant_database_master]'
-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu
-G default -i ami-a403f7cd -f m1.small
knife ec2 server create 'role[production]' 'role[base]'
'role[radiant]' 'role[radiant_run_migrations]'
'recipe[radiant::db_bootstrap]'
-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu
-G default -i ami-a403f7cd -f m1.small
knife ec2 server create 'role[production]' 'role[base]'
'role[radiant]'
-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu
-G default -i ami-a403f7cd -f m1.small
knife ec2 server create 'role[production]' 'role[base]'
'role[radiant_load_balancer]'
-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu
-G default -i ami-a403f7cd -f m1.small
Copyright © 2010 Opscode, Inc - All Rights Reserved 75
Sunday, February 6, 2011
76. {
"id": "radiant",
"server_roles": [
"radiant"
],
"type": {
"radiant": [
"rails",
"unicorn"
]
},
"database_master_role": [
"radiant_database_master"
],
"repository": "git://github.com/radiant/radiant.git",
"revision": {
"production": "0.9.1"
},
Copyright © 2010 Opscode, Inc - All Rights Reserved 76
Sunday, February 6, 2011
77. Server Roles
base.rb
production.rb
radiant.rb
radiant_database_master.rb
radiant_load_balancer.rb
radiant_run_migrations.rb
Copyright © 2010 Opscode, Inc - All Rights Reserved 77
Sunday, February 6, 2011
78. Base Role
name "base"
description "Base role applied to all nodes."
run_list(
"recipe[apt]",
"recipe[git]",
"recipe[build-essential]",
"recipe[ruby]"
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 78
Sunday, February 6, 2011
79. Production Role
name "production"
description "Nodes in the production
environment."
default_attributes(
"app_environment" => "production"
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 79
Sunday, February 6, 2011
80. Radiant Role
name "radiant"
description "radiant front end application
server."
run_list(
"recipe[mysql::client]",
"recipe[application]"
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 80
Sunday, February 6, 2011
81. Application Recipe
search(:apps) do |app|
(app["server_roles"] & node.run_list.roles).each do |app_role|
app["type"][app_role].each do |thing|
node.run_state[:current_app] = app
include_recipe "application::#{thing}"
end
end
end
node.run_state.delete(:current_app)
Copyright © 2010 Opscode, Inc - All Rights Reserved 81
Sunday, February 6, 2011
82. Application Rails Recipe
app['gems'].each do |gem,ver|
gem_package gem do
action :install
version ver if ver && ver.length > 0
end
end
deploy_revision app['id'] do
revision app['revision'][node.app_environment]
repository app['repository']
user app['owner']
group app['group']
deploy_to app['deploy_to']
environment 'RAILS_ENV' => node.app_environment
action app['force'][node.app_environment] ? :force_deploy : :deploy
...
Copyright © 2010 Opscode, Inc - All Rights Reserved 82
Sunday, February 6, 2011
83. Radiant Database Master Role
name "radiant_database_master"
description "Database master for the radiant
application."
run_list(
"recipe[database::master]"
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 83
Sunday, February 6, 2011
84. Database Master Recipe
search(:apps) do |app|
(app['database_master_role'] & node.run_list.roles).each do |
dbm_role|
app['databases'].each do |env,db|
if env =~ /#{node[:app_environment]}/
root_pw = node["mysql"]["server_root_password"]
mysql_database "create #{db['database']}" do
host "localhost"
username "root"
password root_pw
database db['database']
action [:create_db]
end
end
end
end
end
Copyright © 2010 Opscode, Inc - All Rights Reserved 84
Sunday, February 6, 2011
85. Radiant Load Balancer Role
name "radiant_load_balancer"
description "radiant load balancer"
run_list(
"recipe[haproxy::app_lb]"
)
override_attributes(
"haproxy" => {
"app_server_role" => "radiant"
}
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 85
Sunday, February 6, 2011
86. Haproxy App Load Balancer Recipe
pool_members = search("node", "role:#{node['haproxy']
['app_server_role']} AND app_environment:#{node['app_environment']}")
|| []
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members
notifies :restart, resources(:service => "haproxy")
end
Copyright © 2010 Opscode, Inc - All Rights Reserved 86
Sunday, February 6, 2011
87. Control Deployment
knife data bag edit apps radiant
"force": {
"production": false
},
"force": {
"production": true
},
Copyright © 2010 Opscode, Inc - All Rights Reserved 87
Sunday, February 6, 2011
88. Control Migrations
data bag item has migrate setting
need an attribute set as well
"migrate": {
"production": true
}
name "radiant_run_migrations"
description "Run db:migrate on demand for radiant"
override_attributes(
"apps" => {
"radiant" => {
"production" => {
"run_migrations" => true
}
}
}
)
Copyright © 2010 Opscode, Inc - All Rights Reserved 88
Sunday, February 6, 2011
89. Add the role and migrations will be run
knife node run list add NODE ‘role[radiant_run_migrations]’
Migrations will run, and the role
is removed by Chef
automatically.
ruby_block "remove_run_migrations" do
block do
if node.role?("#{app['id']}_run_migrations")
Chef::Log.info("Migrations were run, removing role[#{app
['id']}_run_migrations]")
node.run_list.remove("role[#{app['id']}_run_migrations]")
end
end
end
Copyright © 2010 Opscode, Inc - All Rights Reserved 89
Sunday, February 6, 2011
90. Your Application
Your application is different than Radiant.
But not a unique snowflake,
right?
Mostly, you will just need to modify the data and create application
specific roles...
But wait, I’m using Rails 3!
Copyright © 2010 Opscode, Inc - All Rights Reserved 90
Sunday, February 6, 2011
91. Use the Gems data
Use bundler or bundler08 in the gems hash of the application data
"gems": { "gems": {
"bundler": "1.0.9" "bundler08": "0.8.5"
}, },
before_migrate do
if app['gems'].has_key?('bundler')
execute "bundle install" do
ignore_failure true
cwd release_path
end
elsif app['gems'].has_key?('bundler08')
execute "gem bundle" do
ignore_failure true
cwd release_path
end
Copyright © 2010 Opscode, Inc - All Rights Reserved 91
Sunday, February 6, 2011
92. How does it scale?
Copyright © 2010 Opscode, Inc - All Rights Reserved 92
Sunday, February 6, 2011
93. Quick FAQ
Copyright © 2010 Opscode, Inc - All Rights Reserved 93
Sunday, February 6, 2011
94. Testing
Copyright © 2010 Opscode, Inc - All Rights Reserved 94
Sunday, February 6, 2011
95. Reporting
Copyright © 2010 Opscode, Inc - All Rights Reserved 95
Sunday, February 6, 2011
96. vs [Other tool]
Copyright © 2010 Opscode, Inc - All Rights Reserved 96
Sunday, February 6, 2011
97. These slides will be
posted
Copyright © 2010 Opscode, Inc - All Rights Reserved 97
Sunday, February 6, 2011
98. Resources/Questions
www.opscode.com/chef
IRC and Mailing lists
‣ irc.freenode.net #chef
‣ lists.opscode.com
Twitter:
‣ @opscode, #opschef
‣ @jtimberman
Questions?
Copyright © 2010 Opscode, Inc - All Rights Reserved 98
Sunday, February 6, 2011