Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Automated Infrastructure is
  on the Menu with Chef

            OSCON 2010

Speakers:

Joshua Timberman Technical Evangel...
Agenda

Chef 101
Getting Started
Cooking with Chef




           Copyright © 2010 Opscode, Inc - All Rights Reserved   2
Meta

Rate the tutorial and comment
 ‣ http://bit.ly/chef-oscon2010

Twitter IDs, hashtags
 ‣ #OSCON
 ‣ @opscode, #opschef...
Joshua Timberman
Aaron Peterson



               Copyright © 2010 Opscode, Inc - All Rights Reserved
      http://www.flic...
Developers?
Systems Administrators?



                                                                 http://www.flickr.c...
The Opscode Platform
  is our Chef Server
    http://www.opscode.com




          Copyright © 2010 Opscode, Inc - All Rig...
Copyright © 2010 Opscode, Inc - All Rights Reserved                            7
http://www.brooklynstreetart.com/theBlog/...
At a High Level...

‣ A library for configuration management
‣ A configuration management system
‣ A systems integration pla...
Principles

Idempotent
Data-driven
Sane defaults
Hackability
TMTOWTDI



             Copyright © 2010 Opscode, Inc - All ...
Multiple applications of
 an operation do not
  change the result

         Copyright © 2010 Opscode, Inc - All Rights Res...
We start with APIs, you
     supply data


        Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr....
option :json_attribs,
    :short => "-j JSON_ATTRIBS",
    :long => "--json-attributes JSON_ATTRIBS",
    :description => ...
Open source and
  community


     Copyright © 2010 Opscode, Inc - All Rights Reserved   13
Copyright © 2010 Opscode, Inc - All Rights Reserved   14
Tim Toady is a Perl
      motto
      Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/photos/li...
The world moves pretty fast




Primitives
Enable YOU
Know your systems



                http://www.flickr.com/photos/gi/...
You better be ready!

Sysadmins are programmers
You need a 3GL




                http://www.flickr.com/photos/gi/51861315...
Do I need to know Ruby?

A little
Simple syntax
Complex as you scale




              Copyright © 2010 Opscode, Inc - All...
A Tour of Chef




    Copyright © 2010 Opscode, Inc - All Rights Reserved   19
Chef Client runs on your
        systems


         Copyright © 2010 Opscode, Inc - All Rights Reserved   20
Clients talk to a Chef
       Server


        Copyright © 2010 Opscode, Inc - All Rights Reserved   21
Clients authenticate
   with RSA keys


       Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/...
Chef Server
                    Chef Server

                                         API
                                ...
RESTful API w/ JSON
    responses


       Copyright © 2010 Opscode, Inc - All Rights Reserved   24
Chef can also stand
 alone - Chef Solo


      Copyright © 2010 Opscode, Inc - All Rights Reserved   25
We call each system you
   configure a Node
        Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr....
Nodes have Attributes

                                          Kernel info!
{
  "kernel": {
    "machine": "x86_64",
   ...
Attributes are
         Searchable
$ knife search node ‘platform:mac_os_x’
  search(:node, ‘platform:mac_os_x’)



       ...
Nodes have a Run List
What Roles or Recipes to apply
          in Order


           Copyright © 2010 Opscode, Inc - All R...
Nodes have a Run List
% knife node show web01-prod.example.com -r
{
  "run_list": [
    "role[production]",
    "role[webs...
Nodes have Roles

     Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/photos/laenulfean/374398...
Roles

Describe the node
  ‣   webserver
  ‣   dbserver
  ‣   monitoring
  ‣   ... etc




                   Copyright © ...
Roles have a Run List
What Roles or Recipes to apply
          in Order


           Copyright © 2010 Opscode, Inc - All R...
name "webserver"
description "Systems that serve HTTP traffic"

run_list(
  "role[base]",
                                ...
% knife role show webserver
{
  "name": "webserver",
  "default_attributes": {
    "apache": {
                           ...
Roles are Searchable

$ knife search role ‘max_children:50’
  search(:role, ‘max_children:50’)



              Copyright ...
Chef manages
Resources on Nodes


      Copyright © 2010 Opscode, Inc - All Rights Reserved   37
Resources...
   Declare a description of the state a part of the node should be in


‣ Have a type                 package...
Resources take action
  through Providers


       Copyright © 2010 Opscode, Inc - All Rights Reserved   39
Providers...
    Know how to actually perform the actions specified by a resource.


                                      ...
Resources

Platform

Provider
 http://www.flickr.com/photos/acurbelo/2628837104/sizes/o/
Chef::Platform
:ubuntu    => {
   :default => {
     :package => Chef::Provider::Package::Apt,
     :service => Chef::Prov...
Recipes are lists of
   Resources


      http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
           Copy...
Recipes...
              Apply resources in the order they are specified


                                      package "a...
Order Matters



    Copyright © 2010 Opscode, Inc - All Rights Reserved   45
Recipes are just Ruby!
    extra_packages = case node[:platform]
      when "ubuntu","debian"
        %w{
          ruby1....
Cookbooks are
packages for Recipes


       Copyright © 2010 Opscode, Inc - All Rights Reserved   47
Cookbooks

Distributable
Infrastructure as Code
Version control repository




             Copyright © 2010 Opscode, Inc ...
Common Cookbook Components

Recipes
Assets (files/templates)
Attributes
Metadata




            Copyright © 2010 Opscode, ...
Cookbook assets

Files
  ‣ Static assets
  ‣ Downloaded via cookbook_file
  ‣ File specificity

Templates
  ‣ Dynamic assets...
Cookbooks

Attributes
 ‣ Node attributes
 ‣ default, normal, override


    default[:apache][:listen_ports] = [ "80","443"...
Cookbooks
  Metadata
cookbooks/django/metadata.rb
maintainer         "Opscode, Inc."
maintainer_email   "cookbooks@opscode...
Cookbooks are
  shareable!




 cookbooks.opscode.com
      Copyright © 2010 Opscode, Inc - All Rights Reserved   53
Data bags store
 arbitrary data


    Copyright © 2010 Opscode, Inc - All Rights Reserved   54
A user data bag item...
% knife data bag show users jtimberman
{
  "comment": "Joshua Timberman",
  "groups": "sysadmin",
...
Data Bags are
        Searchable
$ knife search users ‘shell:/bin/bash’
     search(:users, ‘/bin/bash’)



              ...
bash_users = search(:users, 'shell:/bin/bash')

   bash_users.each do |u|
     user u['id'] do
       uid u['id']
       s...
http://www.flickr.com/photos/38299630@N05/3635356091/
   Copyright © 2010 Opscode, Inc - All Rights Reserved   58
Getting Started




    Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/photos/rowens27/3163470...
Debian                                                                  Mac OS X
                                 SuSE
   ...
Ruby



Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/photos/thisisbossi/3526698689/   61
Today’s Examples

Opscode Platform
Mac OS X 10.6.4
Ubuntu 10.04
RubyGems




             Copyright © 2010 Opscode, Inc - ...
The Opscode Platform
  is our Chef Server
    http://www.opscode.com




          Copyright © 2010 Opscode, Inc - All Rig...
RubyGems Installation
cd /tmp
wget http://production.cf.rubygems.org/
rubygems/rubygems-1.3.7.tgz -O- | tar zxf -
cd rubyg...
apt.opscode.com



     Copyright © 2010 Opscode, Inc - All Rights Reserved   65
ELFF Yum Repo



    Copyright © 2010 Opscode, Inc - All Rights Reserved   66
Create Chef Repository
% git clone git://github.com/opscode/chef-repo.git
% cd chef-repo
% ls -la
drwxr-xr-x 13 jtimberman...
Copyright © 2010 Opscode, Inc - All Rights Reserved   68
Setup User Environment

 cp USERNAME.pem ~/chef-repo/.chef
 cp ORG-validator.pem ~/chef-repo/.chef
 cp knife.rb ~/chef-rep...
Configure Knife
% cat .chef/knife.rb
current_dir = File.dirname(__FILE__)
log_level                :info
log_location      ...
Explore Knife’s sub-
    commands


       Copyright © 2010 Opscode, Inc - All Rights Reserved   71
Knife Sub-commands
knife NOUN verb NOUN (options)

 knife       client list
 knife       node show cider.local
 knife     ...
Configure Chef on
           workstation
% knife configure client /etc/chef
INFO: Creating client configuration
INFO: Writi...
Chef::Config
log_level        :info
log_location     STDOUT
chef_server_url 'https://api.opscode.com/
organizations/oscon'
...
Download getting-
         started cookbook
% knife cookbook site vendor getting-started
INFO: Downloading getting-started...
Copyright © 2010 Opscode, Inc - All Rights Reserved   76
git checkout -b 
chef-vendor-#{name_args[0]}




         Copyright © 2010 Opscode, Inc - All Rights Reserved   77
Upload getting-started
    to Chef Server
% knife cookbook upload getting-started
INFO: Saving getting-started
INFO: Valid...
Apply getting-started
    Recipe to workstation

% knife node run list add cider.local "recipe[getting-started]"
{
  "run_...
Run chef-client!

% sudo chef-client
[Thu, 08 Jul 2010 21:35:49 -0600] INFO:                                  Starting Che...
Inside the getting-
started cookbook


      Copyright © 2010 Opscode, Inc - All Rights Reserved   81
http://www.flickr.com/photos/38299630@N05/3635356091/
   Copyright © 2010 Opscode, Inc - All Rights Reserved   82
Cooking with Chef



      Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/photos/mr_t_in_dc/33...
remote_file
                                         link
cookbook_file
                                                  ...
Resources have
parameters and actions


        Copyright © 2010 Opscode, Inc - All Rights Reserved   85
Resource Parameters and Actions

Most resources have defaults
Defaults are sane
  ‣ http://wiki.opscode.com/display/chef/R...
packages

        package "apache2" do
          action :install
        end




Actions: install, upgrade, remove, purge
...
Package shortcuts

Map to providers
gem_package
dpkg_package
rpm_package
And more!



              Copyright © 2010 Opsco...
services

         service "apache2" do
           action [ :enable, :start ]
         end




Actions: enable, disable, s...
init script capabilities
   service "apache2" do
     supports(
       :status => true,
       :restart => true,
       :r...
files
file "/etc/chef/client.rb" do
  owner "root"
  group "root"
  mode 0644
  action :create
end




  Actions: create, d...
file content!
file "/tmp/example" do
  content "This is a file!"
end

file "/tmp/example2" do
  content IO.read("/etc/hosts...
remote_file
remote_file "/tmp/nginx-0.7.67.tar.gz" do
  source "http://sysoev.ru/nginx/nginx-0.7.67.tar.gz"
  action :creat...
cookbook_file
cookbook_file "/etc/perl/CPAN/Config.pm" do
  source "Config-5.10.1.pm"
  owner "root"
  group "root"
  mode ...
template
template "/etc/apache2/ports.conf" do
  source "ports.conf.erb"
  owner "root"
  group "root"
  mode 0644
end



...
local templates
template "/tmp/config.conf" do
  local true
  source "/tmp/config.conf.erb"
end




          Copyright © ...
templates are ERB
<% node[:apache][:listen_ports].each do |port| -%>
Listen <%= port %>
NameVirtualHost *:<%= port %>

<% ...
Cookbook and Template
    File Specificity


        Copyright © 2010 Opscode, Inc - All Rights Reserved   98
preferences = [
  File.join("host-#{fqdn}", "#{file_name}"),
  File.join("#{platform}-#{version}", "#
{file_name}"),
  Fil...
execute

execute "apt-get update" do
  action :run
end




        Actions: run
        Copyright © 2010 Opscode, Inc - Al...
script
bash "compile_nginx_source" do
  cwd "/tmp"
  code <<-EOH
    tar zxf nginx-0.7.67.tar.gz
    cd nginx-0.7.67 && ./...
ruby_block
ruby_block "save the node" do
  block do
    node.save
  end
end




        Action: create
         Copyright ...
scm: git...
git "/srv/apps/chef" do
  repository "git://github.com/opscode/
chef.git"
  reference "0.9.6"
  action :checko...
...and subversion
subversion "/srv/couchdb" do
  repository "http://svn.apache.org/repos/asf/
couchdb/trunk"
  revision "H...
deploy

http://wiki.opscode.com/display/chef/Deploy+Resource




     Actions: deploy, force_deploy, rollback
            ...
deploy "/srv/radiant" do
  repo "git://github.com/radiant/radiant.git"
  revision "HEAD"
  user "railsdev"
  migrate true
...
Meta-parameter
  madness!


    Copyright © 2010 Opscode, Inc - All Rights Reserved   107
action :nothing



    Copyright © 2010 Opscode, Inc - All Rights Reserved   108
not_if & only_if



     Copyright © 2010 Opscode, Inc - All Rights Reserved   109
execute "runit-hup-init" do
  command "telinit q"
  only_if "grep ^SV /etc/inittab"
  action :nothing
end

execute "rabbit...
Resource notification
template "nginx.conf" do
  path "/etc/nginx/nginx.conf"
  source "nginx.conf.erb"
  owner "root"
  gr...
supports



 Copyright © 2010 Opscode, Inc - All Rights Reserved   112
Anatomy of a Chef Run



        Copyright © 2010 Opscode, Inc - All Rights Reserved   113
Anatomy of a Chef Run

Node discovery
Set the node name
Register with server




              Copyright © 2010 Opscode, I...
Anatomy of a Chef Run

Build node object
  ‣ node.save

Synchronize cookbooks
  ‣ node.save

Converge
  ‣ node.save




  ...
Chef Run Convergence

Compile
Execute




             Copyright © 2010 Opscode, Inc - All Rights Reserved   116
Development workflow
      with Chef


       Copyright © 2010 Opscode, Inc - All Rights Reserved   http://www.flickr.com/ph...
Development workflow

Gather requirements
Write recipes
Commit to repository
Run Chef in testing
Run Chef in production



...
http://www.flickr.com/photos/38299630@N05/3635356091/
   Copyright © 2010 Opscode, Inc - All Rights Reserved   119
Automating the Cloud
     with Chef

      http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/
                  ...
GoTime



Copyright © 2010 Opscode, Inc - All Rights Reserved   121
Add your Cloud
    credentials to knife.rb
          vi ~/chef-repo/.chef/knife.rb


# Cloud credentials
knife[:aws_access...
Download some
              cookbooks
% knife cookbook site vendor nagios -d
INFO: Downloading nagios from the cookbooks s...
Upload Cookbooks!
      knife cookbook upload -a




            These run as root, kids.
Let’s not blindly trust the upst...
Build some roles
% vi roles/monitoring.rb

name "monitoring"
description "Nagios monitoring server"

run_list(
  "role[bas...
Upload Roles

% knife role from file roles/monitoring.rb
WARN: HTTP Request Returned 404 Not Found: Cannot load role      ...
Launch a new
    Monitoring Server

knife ec2 server create ‘role[monitoring]’




                Copyright © 2010 Opscod...
Chef runs on your new
         server
              sudo chef-client

INFO: Starting Chef Run
...
INFO: Chef Run complete ...
Shef is Chef in IRB



      Copyright © 2010 Opscode, Inc - All Rights Reserved   129
Resources/Questions

www.opscode.com
IRC and Mailing lists
  ‣ irc.freenode.net #chef
  ‣ lists.opscode.com

Twitter:
  ‣ ...
Upcoming SlideShare
Loading in …5
×

Automated infrastructure is on the menu

18,328 views

Published on

Presentation from the OSCON 2010 tutorial on Chef

Published in: Technology
  • Hello Mr. Timberman

    Thank you for your follow-up comment.
    I signed up Opscode Open Source Training page (http://opscode.com/training) and downloaded the Chef fundamentals guide.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi Tetsuro!

    You may also be interested in our free Open Training program, details here:

    http://opscode.com/training
    http://info.opscode.com/opentraining
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I'd like to introduce this product to my colleagues..
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Automated infrastructure is on the menu

  1. 1. Automated Infrastructure is on the Menu with Chef OSCON 2010 Speakers: Joshua Timberman Technical Evangelist Aaron Peterson Technical Evangelist Copyright © 2010 Opscode, Inc - All Rights Reserved 1
  2. 2. Agenda Chef 101 Getting Started Cooking with Chef Copyright © 2010 Opscode, Inc - All Rights Reserved 2
  3. 3. Meta Rate the tutorial and comment ‣ http://bit.ly/chef-oscon2010 Twitter IDs, hashtags ‣ #OSCON ‣ @opscode, #opschef ‣ @jtimberman, @metaxis Questions Copyright © 2010 Opscode, Inc - All Rights Reserved 3
  4. 4. Joshua Timberman Aaron Peterson Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/ 4
  5. 5. Developers? Systems Administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/ Copyright © 2010 Opscode, Inc - All Rights Reserved 5
  6. 6. The Opscode Platform is our Chef Server http://www.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 6
  7. 7. Copyright © 2010 Opscode, Inc - All Rights Reserved 7 http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg
  8. 8. At a High Level... ‣ A library for configuration management ‣ A configuration management system ‣ A systems integration platform ‣ An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/
  9. 9. Principles Idempotent Data-driven Sane defaults Hackability TMTOWTDI Copyright © 2010 Opscode, Inc - All Rights Reserved 9
  10. 10. Multiple applications of an operation do not change the result Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/redjar/360111326/ 10
  11. 11. We start with APIs, you supply data Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/ninjanoodles/153893226/ 11
  12. 12. option :json_attribs, :short => "-j JSON_ATTRIBS", :long => "--json-attributes JSON_ATTRIBS", :description => "Load attributes from a JSON file or URL", :proc => nil option :node_name, :short => "-N NODE_NAME", :long => "--node-name NODE_NAME", :description => "The node name for this client", Defaults are sane, but :proc => nil easily changed Copyright © 2010 Opscode, Inc - All Rights Reserved 12
  13. 13. Open source and community Copyright © 2010 Opscode, Inc - All Rights Reserved 13
  14. 14. Copyright © 2010 Opscode, Inc - All Rights Reserved 14
  15. 15. Tim Toady is a Perl motto Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/lidarose/225156612 15
  16. 16. The world moves pretty fast Primitives Enable YOU Know your systems http://www.flickr.com/photos/gi/518613153/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 16
  17. 17. You better be ready! Sysadmins are programmers You need a 3GL http://www.flickr.com/photos/gi/518613153/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 17
  18. 18. Do I need to know Ruby? A little Simple syntax Complex as you scale Copyright © 2010 Opscode, Inc - All Rights Reserved 18
  19. 19. A Tour of Chef Copyright © 2010 Opscode, Inc - All Rights Reserved 19
  20. 20. Chef Client runs on your systems Copyright © 2010 Opscode, Inc - All Rights Reserved 20
  21. 21. Clients talk to a Chef Server Copyright © 2010 Opscode, Inc - All Rights Reserved 21
  22. 22. Clients authenticate with RSA keys Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/debbcollins/3401944550/ 22
  23. 23. Chef Server Chef Server API Client Interaction Search Indexer AMQP SOLR CouchDB Data store Copyright © 2010 Opscode, Inc - All Rights Reserved 23
  24. 24. RESTful API w/ JSON responses Copyright © 2010 Opscode, Inc - All Rights Reserved 24
  25. 25. Chef can also stand alone - Chef Solo Copyright © 2010 Opscode, Inc - All Rights Reserved 25
  26. 26. We call each system you configure a Node Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/peterrosbjerg/3913766224/ 26
  27. 27. Nodes have Attributes Kernel info! { "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", Platform info! "os": "darwin", "current_user": "jtimberman", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "cider", "fqdn": "cider.local", Hostname and IP! "uptime_seconds": 1619358 } Copyright © 2010 Opscode, Inc - All Rights Reserved 27
  28. 28. Attributes are Searchable $ knife search node ‘platform:mac_os_x’ search(:node, ‘platform:mac_os_x’) Copyright © 2010 Opscode, Inc - All Rights Reserved 28
  29. 29. Nodes have a Run List What Roles or Recipes to apply in Order Copyright © 2010 Opscode, Inc - All Rights Reserved 29
  30. 30. Nodes have a Run List % knife node show web01-prod.example.com -r { "run_list": [ "role[production]", "role[webserver]" ] } Copyright © 2010 Opscode, Inc - All Rights Reserved 30
  31. 31. Nodes have Roles Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/laenulfean/374398044/ 31
  32. 32. Roles Describe the node ‣ webserver ‣ dbserver ‣ monitoring ‣ ... etc Copyright © 2010 Opscode, Inc - All Rights Reserved 32
  33. 33. Roles have a Run List What Roles or Recipes to apply in Order Copyright © 2010 Opscode, Inc - All Rights Reserved 33
  34. 34. name "webserver" description "Systems that serve HTTP traffic" run_list( "role[base]", Can include "recipe[apache2]", other roles! "recipe[apache2::mod_ssl]" ) default_attributes( "apache" => { "listen_ports" => [ "80", "443" ] } ) override_attributes( "apache" => { "max_children" => "50" } ) Copyright © 2010 Opscode, Inc - All Rights Reserved 34
  35. 35. % knife role show webserver { "name": "webserver", "default_attributes": { "apache": { Uploading roles to "listen_ports": [ the Chef Server "80", "443" converts Ruby DSL ] } to JSON! }, "json_class": "Chef::Role", "run_list": [ "role[base]", "recipe[apache2]", "recipe[apache2::mod_ssl]" ], "description": "Systems that serve HTTP traffic", "chef_type": "role", "override_attributes": { "apache2": { "max_children": "50" } } } Copyright © 2010 Opscode, Inc - All Rights Reserved 35
  36. 36. Roles are Searchable $ knife search role ‘max_children:50’ search(:role, ‘max_children:50’) Copyright © 2010 Opscode, Inc - All Rights Reserved 36
  37. 37. Chef manages Resources on Nodes Copyright © 2010 Opscode, Inc - All Rights Reserved 37
  38. 38. Resources... Declare a description of the state a part of the node should be in ‣ Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install ‣ Have a name end template "/etc/apache2/apache2.conf" do ‣ Have parameters source "apache2.conf.erb" owner "root" ‣ Take action to put the group "root" mode 0644 resource in the action :create declared state end
  39. 39. Resources take action through Providers Copyright © 2010 Opscode, Inc - All Rights Reserved 39
  40. 40. Providers... Know how to actually perform the actions specified by a resource. Apt, Yum, Rubygems, Multiple providers per resource type. Portage, Macports, FreeBSD Ports, etc. Can be overridden with package "apache2" do provider "Chef::Provider::Package::Dpkg" the provider action :install parameter on a end resource. http://www.flickr.com/photos/affableslinky/562950216/
  41. 41. Resources Platform Provider http://www.flickr.com/photos/acurbelo/2628837104/sizes/o/
  42. 42. Chef::Platform :ubuntu => { :default => { :package => Chef::Provider::Package::Apt, :service => Chef::Provider::Service::Debian, :cron => Chef::Provider::Cron, :mdadm => Chef::Provider::Mdadm } }, Copyright © 2010 Opscode, Inc - All Rights Reserved 42
  43. 43. Recipes are lists of Resources http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 43
  44. 44. Recipes... Apply resources in the order they are specified package "apache2" do version "2.2.11-2ubuntu2.6" action :install 1 ‣ Evaluates resources in [ end 1 the order they appear "package[apache2]", "template[/etc/apache2/apache2.conf]" template "/etc/apache2/apache2.conf" do 2 ] source "apache2.conf.erb" ‣ Adds each resource to owner "root" the Resource Collection group "root" mode 0644 action :create 2 end http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  45. 45. Order Matters Copyright © 2010 Opscode, Inc - All Rights Reserved 45
  46. 46. Recipes are just Ruby! extra_packages = case node[:platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end Copyright © 2010 Opscode, Inc - All Rights Reserved 46
  47. 47. Cookbooks are packages for Recipes Copyright © 2010 Opscode, Inc - All Rights Reserved 47
  48. 48. Cookbooks Distributable Infrastructure as Code Version control repository Copyright © 2010 Opscode, Inc - All Rights Reserved 48
  49. 49. Common Cookbook Components Recipes Assets (files/templates) Attributes Metadata Copyright © 2010 Opscode, Inc - All Rights Reserved 49
  50. 50. Cookbook assets Files ‣ Static assets ‣ Downloaded via cookbook_file ‣ File specificity Templates ‣ Dynamic assets ‣ ERB (erubis) ‣ File specificity Copyright © 2010 Opscode, Inc - All Rights Reserved 50
  51. 51. Cookbooks Attributes ‣ Node attributes ‣ default, normal, override default[:apache][:listen_ports] = [ "80","443" ] default[:apache][:keepalive] = "On" default[:apache][:contact] = "ops@example.com" default[:apache][:timeout] = 300 set[:apache][:log_dir] = "/var/log/apache2" set[:apache][:user] = "www-data" override[:apache][:dir] = "/etc/apache2" Copyright © 2010 Opscode, Inc - All Rights Reserved 51
  52. 52. Cookbooks Metadata cookbooks/django/metadata.rb maintainer "Opscode, Inc." maintainer_email "cookbooks@opscode.com" license "Apache 2.0" description "Installs DJango" long_description IO.read(File.join(File.dirname(__FILE__), 'README.rdoc')) version "0.8.0" recipe "django", "Installs django and apache2 with mod_python" %w{ ubuntu debian }.each do |os| supports os end %w{ apache2 python }.each do |cb| depends cb end Copyright © 2010 Opscode, Inc - All Rights Reserved 52
  53. 53. Cookbooks are shareable! cookbooks.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 53
  54. 54. Data bags store arbitrary data Copyright © 2010 Opscode, Inc - All Rights Reserved 54
  55. 55. A user data bag item... % knife data bag show users jtimberman { "comment": "Joshua Timberman", "groups": "sysadmin", "ssh_keys": "ssh-rsa SUPERSEKRATS jtimberman@cider", "files": { ".zshrc": { "mode": "0644", "source": "dot-zshrc" }, ".vimrc": { "mode": "0644", "source": "dot-vimrc" } }, "id": "jtimberman", "uid": 7004, "shell": "/usr/bin/zsh", "openid": "http://jtimberman.myopenid.com/" } Copyright © 2010 Opscode, Inc - All Rights Reserved
  56. 56. Data Bags are Searchable $ knife search users ‘shell:/bin/bash’ search(:users, ‘/bin/bash’) Copyright © 2010 Opscode, Inc - All Rights Reserved 56
  57. 57. bash_users = search(:users, 'shell:/bin/bash') bash_users.each do |u| user u['id'] do uid u['id'] shell "/usr/bin/zsh" comment u['comment'] supports :manage_home => true Data bags make recipes home "/home/#{u['id']}" end awesome-r (that’s directory "/home/#{u['id']}/.ssh" do owner u['id'] group u['id'] mode 0700 totally a word) end template "/home/#{u['id']}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['id'] mode 0600 variables :ssh_keys => u['ssh_keys'] end end Copyright © 2010 Opscode, Inc - All Rights Reserved 57
  58. 58. http://www.flickr.com/photos/38299630@N05/3635356091/ Copyright © 2010 Opscode, Inc - All Rights Reserved 58
  59. 59. Getting Started Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/rowens27/3163470179/ 59
  60. 60. Debian Mac OS X SuSE CentOS Gentoo Solaris ArchLinux OpenBSD Platforms Windows FreeBSD Ubuntu Red Hat Fedora Scientific Copyright © 2010 Opscode, Inc - All Rights Reserved 60
  61. 61. Ruby Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/thisisbossi/3526698689/ 61
  62. 62. Today’s Examples Opscode Platform Mac OS X 10.6.4 Ubuntu 10.04 RubyGems Copyright © 2010 Opscode, Inc - All Rights Reserved 62
  63. 63. The Opscode Platform is our Chef Server http://www.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 63
  64. 64. RubyGems Installation cd /tmp wget http://production.cf.rubygems.org/ rubygems/rubygems-1.3.7.tgz -O- | tar zxf - cd rubygems-1.3.7 && sudo ruby setup.rb ln -svf /usr/bin/gem1.8 /usr/bin/gem sudo gem install chef Copyright © 2010 Opscode, Inc - All Rights Reserved 64
  65. 65. apt.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 65
  66. 66. ELFF Yum Repo Copyright © 2010 Opscode, Inc - All Rights Reserved 66
  67. 67. Create Chef Repository % git clone git://github.com/opscode/chef-repo.git % cd chef-repo % ls -la drwxr-xr-x 13 jtimberman staff 442 Jul 7 16:48 ./ drwxr-xr-x 3 jtimberman staff 102 Jul 7 15:54 ../ drwxr-xr-x 5 jtimberman staff 170 Jul 7 17:55 .chef/ drwxr-xr-x 12 jtimberman staff 408 Jul 7 16:48 .git/ -rw-r--r-- 1 jtimberman staff 23 Jul 7 16:48 .gitignore -rw-r--r-- 1 jtimberman staff 269 Jul 7 15:54 README -rw-r--r-- 1 jtimberman staff 2171 Jul 7 15:54 Rakefile drwxr-xr-x 3 jtimberman staff 102 Jul 7 15:54 certificates/ drwxr-xr-x 7 jtimberman staff 238 Jul 7 17:03 config/ drwxr-xr-x 3 jtimberman staff 102 Jul 7 15:54 cookbooks/ drwxr-xr-x 3 jtimberman staff 102 Jul 7 15:54 data_bags/ drwxr-xr-x 3 jtimberman staff 102 Jul 7 15:54 roles/ drwxr-xr-x 3 jtimberman staff 102 Jul 7 15:54 site-cookbooks/ Copyright © 2010 Opscode, Inc - All Rights Reserved 67
  68. 68. Copyright © 2010 Opscode, Inc - All Rights Reserved 68
  69. 69. Setup User Environment cp USERNAME.pem ~/chef-repo/.chef cp ORG-validator.pem ~/chef-repo/.chef cp knife.rb ~/chef-repo/.chef Copyright © 2010 Opscode, Inc - All Rights Reserved 69
  70. 70. Configure Knife % cat .chef/knife.rb current_dir = File.dirname(__FILE__) log_level :info log_location STDOUT node_name "oscon" client_key "#{current_dir}/oscon.pem" validation_client_name "oscon-validator" validation_key "#{current_dir}/oscon-validator.pem" chef_server_url "https://api.opscode.com/organizations/oscon" cache_type 'BasicFile' cache_options( :path => "#{ENV['HOME']}/.chef/checksums" ) cookbook_path ["#{current_dir}/../cookbooks"] Per-directory configuration! Copyright © 2010 Opscode, Inc - All Rights Reserved 70
  71. 71. Explore Knife’s sub- commands Copyright © 2010 Opscode, Inc - All Rights Reserved 71
  72. 72. Knife Sub-commands knife NOUN verb NOUN (options) knife client list knife node show cider.local knife role show webserver knife search node “*:*” -i knife --help Copyright © 2010 Opscode, Inc - All Rights Reserved 72
  73. 73. Configure Chef on workstation % knife configure client /etc/chef INFO: Creating client configuration INFO: Writing client.rb INFO: Writing validation.pem % ls -l /etc/chef/ total 24 -rw-r--r-- 1 jtimberman staff 151 Jul 8 21:29 client.rb -rw-r--r--@ 1 jtimberman staff 1679 Jul 8 21:29 validation.pem Copyright © 2010 Opscode, Inc - All Rights Reserved 73
  74. 74. Chef::Config log_level :info log_location STDOUT chef_server_url 'https://api.opscode.com/ organizations/oscon' validation_client_name 'oscon-validator' http://wiki.opscode.com/display/chef/Chef+Configuration+Settings Copyright © 2010 Opscode, Inc - All Rights Reserved 74
  75. 75. Download getting- started cookbook % knife cookbook site vendor getting-started INFO: Downloading getting-started from the cookbooks site at version 0.2.0 ... INFO: Cookbook getting-started version 0.2.0 successfully vendored! Copyright © 2010 Opscode, Inc - All Rights Reserved 75
  76. 76. Copyright © 2010 Opscode, Inc - All Rights Reserved 76
  77. 77. git checkout -b chef-vendor-#{name_args[0]} Copyright © 2010 Opscode, Inc - All Rights Reserved 77
  78. 78. Upload getting-started to Chef Server % knife cookbook upload getting-started INFO: Saving getting-started INFO: Validating ruby files INFO: Validating templates INFO: Syntax OK INFO: Generating Metadata INFO: Uploading files ... INFO: Upload complete! Copyright © 2010 Opscode, Inc - All Rights Reserved 78
  79. 79. Apply getting-started Recipe to workstation % knife node run list add cider.local "recipe[getting-started]" { "run_list": [ "recipe[getting-started]" ] } Copyright © 2010 Opscode, Inc - All Rights Reserved 79
  80. 80. Run chef-client! % sudo chef-client [Thu, 08 Jul 2010 21:35:49 -0600] INFO: Starting Chef Run [Thu, 08 Jul 2010 21:35:55 -0600] INFO: Writing updated content for template[/tmp/chef-getting-started.txt] to /tmp/chef-getting- started.txt [Thu, 08 Jul 2010 21:35:56 -0600] INFO: Chef Run complete in 6.650602 seconds % cat /tmp/chef-getting-started.txt Welcome to Chef! This is Chef version 0.9.6. Running on mac_os_x. Version 10.6.4. Copyright © 2010 Opscode, Inc - All Rights Reserved 80
  81. 81. Inside the getting- started cookbook Copyright © 2010 Opscode, Inc - All Rights Reserved 81
  82. 82. http://www.flickr.com/photos/38299630@N05/3635356091/ Copyright © 2010 Opscode, Inc - All Rights Reserved 82
  83. 83. Cooking with Chef Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/mr_t_in_dc/3305638738/ 83
  84. 84. remote_file link cookbook_file service ruby_block template Chef Resources in execute Depth user bash git package log deploy http_request Copyright © 2010 Opscode, Inc - All Rights Reserved 84
  85. 85. Resources have parameters and actions Copyright © 2010 Opscode, Inc - All Rights Reserved 85
  86. 86. Resource Parameters and Actions Most resources have defaults Defaults are sane ‣ http://wiki.opscode.com/display/chef/Resources Copyright © 2010 Opscode, Inc - All Rights Reserved 86
  87. 87. packages package "apache2" do action :install end Actions: install, upgrade, remove, purge Copyright © 2010 Opscode, Inc - All Rights Reserved 87
  88. 88. Package shortcuts Map to providers gem_package dpkg_package rpm_package And more! Copyright © 2010 Opscode, Inc - All Rights Reserved 88
  89. 89. services service "apache2" do action [ :enable, :start ] end Actions: enable, disable, start, stop, restart, reload Copyright © 2010 Opscode, Inc - All Rights Reserved 89
  90. 90. init script capabilities service "apache2" do supports( :status => true, :restart => true, :reload => true ) action [ :enable, :start ] end Copyright © 2010 Opscode, Inc - All Rights Reserved 90
  91. 91. files file "/etc/chef/client.rb" do owner "root" group "root" mode 0644 action :create end Actions: create, delete, touch Copyright © 2010 Opscode, Inc - All Rights Reserved 91
  92. 92. file content! file "/tmp/example" do content "This is a file!" end file "/tmp/example2" do content IO.read("/etc/hosts") end Content is a string Copyright © 2010 Opscode, Inc - All Rights Reserved 92
  93. 93. remote_file remote_file "/tmp/nginx-0.7.67.tar.gz" do source "http://sysoev.ru/nginx/nginx-0.7.67.tar.gz" action :create_if_missing end Actions: create, create_if_missing Copyright © 2010 Opscode, Inc - All Rights Reserved 93
  94. 94. cookbook_file cookbook_file "/etc/perl/CPAN/Config.pm" do source "Config-5.10.1.pm" owner "root" group "root" mode 0644 end Actions: create, create_if_missing, delete Copyright © 2010 Opscode, Inc - All Rights Reserved 94
  95. 95. template template "/etc/apache2/ports.conf" do source "ports.conf.erb" owner "root" group "root" mode 0644 end Actions: create Copyright © 2010 Opscode, Inc - All Rights Reserved 95
  96. 96. local templates template "/tmp/config.conf" do local true source "/tmp/config.conf.erb" end Copyright © 2010 Opscode, Inc - All Rights Reserved 96
  97. 97. templates are ERB <% node[:apache][:listen_ports].each do |port| -%> Listen <%= port %> NameVirtualHost *:<%= port %> <% end -%> Copyright © 2010 Opscode, Inc - All Rights Reserved 97
  98. 98. Cookbook and Template File Specificity Copyright © 2010 Opscode, Inc - All Rights Reserved 98
  99. 99. preferences = [ File.join("host-#{fqdn}", "#{file_name}"), File.join("#{platform}-#{version}", "# {file_name}"), File.join("#{platform}", "#{file_name}"), File.join("default", "#{file_name}") ] host-node[:fqdn] node[:platform]-node[:version] node[:platform] default files/web1prod.example.com files/ubuntu-9.10 files/ubuntu files/default Copyright © 2010 Opscode, Inc - All Rights Reserved 99
  100. 100. execute execute "apt-get update" do action :run end Actions: run Copyright © 2010 Opscode, Inc - All Rights Reserved 100
  101. 101. script bash "compile_nginx_source" do cwd "/tmp" code <<-EOH tar zxf nginx-0.7.67.tar.gz cd nginx-0.7.67 && ./configure make && make install EOH end Interpreters: bash, ruby, python, perl, csh Copyright © 2010 Opscode, Inc - All Rights Reserved 101
  102. 102. ruby_block ruby_block "save the node" do block do node.save end end Action: create Copyright © 2010 Opscode, Inc - All Rights Reserved 102
  103. 103. scm: git... git "/srv/apps/chef" do repository "git://github.com/opscode/ chef.git" reference "0.9.6" action :checkout end Actions: sync, checkout, export Copyright © 2010 Opscode, Inc - All Rights Reserved 103
  104. 104. ...and subversion subversion "/srv/couchdb" do repository "http://svn.apache.org/repos/asf/ couchdb/trunk" revision "HEAD" action :sync end Actions: sync, checkout, export Copyright © 2010 Opscode, Inc - All Rights Reserved 104
  105. 105. deploy http://wiki.opscode.com/display/chef/Deploy+Resource Actions: deploy, force_deploy, rollback Copyright © 2010 Opscode, Inc - All Rights Reserved 105
  106. 106. deploy "/srv/radiant" do repo "git://github.com/radiant/radiant.git" revision "HEAD" user "railsdev" migrate true migration_command "rake db:migrate" environment "production" restart_command "touch tmp/restart.txt" action :deploy end Copyright © 2010 Opscode, Inc - All Rights Reserved 106
  107. 107. Meta-parameter madness! Copyright © 2010 Opscode, Inc - All Rights Reserved 107
  108. 108. action :nothing Copyright © 2010 Opscode, Inc - All Rights Reserved 108
  109. 109. not_if & only_if Copyright © 2010 Opscode, Inc - All Rights Reserved 109
  110. 110. execute "runit-hup-init" do command "telinit q" only_if "grep ^SV /etc/inittab" action :nothing end execute "rabbitmqctl add_vhost /chef" do not_if "rabbitmqctl list_vhosts| grep /chef" end Enclose in quotes for shell commands or use a do..end or { } style ruby block Copyright © 2010 Opscode, Inc - All Rights Reserved 110
  111. 111. Resource notification template "nginx.conf" do path "/etc/nginx/nginx.conf" source "nginx.conf.erb" owner "root" group "root" mode "0644" notifies :restart, resources(:service => "nginx") end Copyright © 2010 Opscode, Inc - All Rights Reserved 111
  112. 112. supports Copyright © 2010 Opscode, Inc - All Rights Reserved 112
  113. 113. Anatomy of a Chef Run Copyright © 2010 Opscode, Inc - All Rights Reserved 113
  114. 114. Anatomy of a Chef Run Node discovery Set the node name Register with server Copyright © 2010 Opscode, Inc - All Rights Reserved 114
  115. 115. Anatomy of a Chef Run Build node object ‣ node.save Synchronize cookbooks ‣ node.save Converge ‣ node.save Copyright © 2010 Opscode, Inc - All Rights Reserved 115
  116. 116. Chef Run Convergence Compile Execute Copyright © 2010 Opscode, Inc - All Rights Reserved 116
  117. 117. Development workflow with Chef Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/kylemay/1393258810/ 117
  118. 118. Development workflow Gather requirements Write recipes Commit to repository Run Chef in testing Run Chef in production Copyright © 2010 Opscode, Inc - All Rights Reserved 118
  119. 119. http://www.flickr.com/photos/38299630@N05/3635356091/ Copyright © 2010 Opscode, Inc - All Rights Reserved 119
  120. 120. Automating the Cloud with Chef http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/ Copyright © 2010 Opscode, Inc - All Rights Reserved 120
  121. 121. GoTime Copyright © 2010 Opscode, Inc - All Rights Reserved 121
  122. 122. Add your Cloud credentials to knife.rb vi ~/chef-repo/.chef/knife.rb # Cloud credentials knife[:aws_access_key_id] = ENV['AWS_ACCESS_KEY_ID'] knife[:aws_secret_access_key] = ENV['AWS_SECRET_ACCESS_KEY'] Copyright © 2010 Opscode, Inc - All Rights Reserved 122
  123. 123. Download some cookbooks % knife cookbook site vendor nagios -d INFO: Downloading nagios from the cookbooks site at version 0.3.3 ... INFO: Cookbook nagios version 0.3.3 successfully vendored! INFO: Downloading apache2 from the cookbooks site at version 0.12.0 INFO: Cookbook apache2 version 0.12.0 successfully vendored! Uses the “vendor branch” pattern, so you can make changes and track the upstream Copyright © 2010 Opscode, Inc - All Rights Reserved 123
  124. 124. Upload Cookbooks! knife cookbook upload -a These run as root, kids. Let’s not blindly trust the upstream too much! Copyright © 2010 Opscode, Inc - All Rights Reserved 124
  125. 125. Build some roles % vi roles/monitoring.rb name "monitoring" description "Nagios monitoring server" run_list( "role[base]”, “recipe[nagios::server]" ) override_attributes( "apache" => { "allowed_openids" => "http://jtimberman.myopenid.com/" } ) Copyright © 2010 Opscode, Inc - All Rights Reserved 125
  126. 126. Upload Roles % knife role from file roles/monitoring.rb WARN: HTTP Request Returned 404 Not Found: Cannot load role monitoring WARN: Updated Role monitoring! % ls roles README base.rb monitoring.rb production.rb webserver.rb % rake roles (in /Users/jtimberman/Development/oscon/chef-repo) WARN: HTTP Request Returned 404 Not Found: Cannot load role base WARN: Updated Role base! WARN: Updated Role monitoring! WARN: HTTP Request Returned 404 Not Found: Cannot load role production WARN: Updated Role production! WARN: HTTP Request Returned 404 Not Found: Cannot load role webserver WARN: Updated Role webserver! Copyright © 2010 Opscode, Inc - All Rights Reserved 126
  127. 127. Launch a new Monitoring Server knife ec2 server create ‘role[monitoring]’ Copyright © 2010 Opscode, Inc - All Rights Reserved 127
  128. 128. Chef runs on your new server sudo chef-client INFO: Starting Chef Run ... INFO: Chef Run complete in 211.852033 seconds Automatically. Copyright © 2010 Opscode, Inc - All Rights Reserved 128
  129. 129. Shef is Chef in IRB Copyright © 2010 Opscode, Inc - All Rights Reserved 129
  130. 130. Resources/Questions www.opscode.com IRC and Mailing lists ‣ irc.freenode.net #chef ‣ lists.opscode.com Twitter: ‣ @opscode, #opschef ‣ @jtimberman, @metaxis Questions? http://bit.ly/chef-oscon2010 Copyright © 2010 Opscode, Inc - All Rights Reserved 130

×