Configuring Nagios with Chef
Bryan McLellan
Technical Program Manager, Open Source
btm@opscode.com / @btmspox
Overview




• Who am I?
• Why automation
• Introduction to Chef
• Nagios Demo
• Questions
Who am I?



• Chef
  Early developer, user, pundit

• 10+ years in Systems
  Administration
  Computer repair, ISPs, Corporate
  IT,
  Web operations

• Event Logistics Volunteer
  Traffic Control, Parking,
  Communications, Networking,
  Emergency Management

• Hacker-Operator
How did we get here?



Bare Metal Deployment
How did we get here?



Bare Metal Deployment
 • Purchasing
How did we get here?



Bare Metal Deployment
 • Purchasing
 • Vendor build
How did we get here?



Bare Metal Deployment
 • Purchasing
 • Vendor build
 • Delivery
How did we get here?



Bare Metal Deployment
 •   Purchasing
 •   Vendor build
 •   Delivery
 •   Installation
How did we get here?



Bare Metal Deployment
 •   Purchasing
 •   Vendor build
 •   Delivery
 •   Installation
 •   OS deployment
How did we get here?



Bare Metal Deployment
 •   Purchasing
 •   Vendor build
 •   Delivery
 •   Installation
 •   OS deployment
 •   Application deployment
How did we get here?



Bare Metal Deployment
 •   Purchasing
 •   Vendor build
 •   Delivery                 Weeks?
 •   Installation
 •   OS deployment
 •   Application deployment
How did we get here?



Cloud or Virtual Deployment
 •   Purchasing
 •   Vendor build
 •   Delivery                 Nearly immediate
 •   Installation
 •   OS deployment
 •   Application deployment
How did we get here?



Cloud or Virtual Deployment
 •   Purchasing
 •   Vendor build
 •   Delivery                 Nearly immediate
 •   Installation
 •   OS deployment
 •   Application deployment
                                   Must be fast
Why automate?


    Good Reasons:
•   More agility and faster scalability
•   Improved infrastructure documentation
•   Better disaster recovery
Why automate?


    Good Reasons:
•   More agility and faster scalability
•   Improved infrastructure documentation
•   Better disaster recovery


    Really Good Reasons:
•   Spend less time on monotonous tasks
•   Spend more time solving interesting
    problems
Why automate?



Operations is responsible for two things:
Why automate?



Operations is responsible for two things:


1. Availability
Why automate?



Operations is responsible for two things:


1. Availability

2. Efficiency
What is Chef?




• Configuration management language
• Systems integration framework
• API for your infrastructure

           http://www.flickr.com/photos/morville/3220961040/
Chef Principles
Chef Principles



• Idempotent
Chef Principles



• Idempotent
• Reasonable
Chef Principles



• Idempotent
• Reasonable
• Primitives
Chef Principles



• Idempotent
• Reasonable
• Primitives
• Scalable
Chef Principles



• Idempotent
• Reasonable
• Primitives
• Scalable
• Hackable
Chef Principles



• Idempotent
• Reasonable
• Primitives
• Scalable
• Hackable
• Shareable
Chef Basics



Chef manages Nodes
Chef Basics



Chef manages Nodes
Nodes have Attributes
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Each node has a Run List
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Each node has a Run List
A Run List is a list of Recipes to run
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Each node has a Run List
A Run List is a list of Recipes to run
A Role also has a Run List
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Each node has a Run List
A Run List is a list of Recipes to run
A Role also has a Run List
Roles can also be added to a Node’s Run List
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Each node has a Run List
A Run List is a list of Recipes to run
A Role also has a Run List
Roles can also be added to a Node’s Run List
Nodes can be in Environments
Chef Basics



Chef manages Nodes
Nodes have Attributes
Users and Nodes authenticate as Clients
Cookbooks contain Recipes
Each node has a Run List
A Run List is a list of Recipes to run
A Role also has a Run List
Roles can also be added to a Node’s Run List
Nodes can be in Environments
Data bags are... bags of data.
Chef Basics Visualized



       client: srv01                     client: srv02                    client: srv03




         node: srv01                      node: srv02                     node: srv03
 run_list: “role[web_server]”     run_list: “role[web_server]”     run_list: “role[db_server]”




              role: web_server                                  role: db_server
run_list: [“recipe[apache2]”, “recipe[php]” ]     run_list: [ “recipe[mysql]”, “recipe[nfs]” ]
Application Programming Interface


 The Meatcloud Manifesto

Give me an API or give me death.
      -- Andrew Clay Shafer (@littleidea)
Chef Stacks



chef-client    knife     chef-shell      chef-solo




                       API




 Open Source       Hosted Chef        Private Chef
Chef 10 Open Source Architecture




  Chef Expander
Resources



•   A Resource is something you manage
    service, package, file, user, execute, git
Resources



•   A Resource is something you manage
    service, package, file, user, execute, git

•   Resources have actions
    start, install, create, deploy

•   Resources can notify of other resources
Resources



•   A Resource is something you manage
    service, package, file, user, execute, git

•   Resources have actions
    start, install, create, deploy

•   Resources can notify of other resources


     cookbook_file “/etc/apache2/apache2.conf” do
       source “apache2.conf”
       owner “root”
       group “root”
       mode 0644
       notifies :restart, “service[apache2]”
     end
Providers



•   A Provider performs the actions specified by the resource
Providers



•   A Provider performs the actions specified by the resource

•   Each Resource can have multiple providers
    package: apt, yum, macports...
    service: upstart, windows, systemd...
Providers



•   A Provider performs the actions specified by the resource

•   Each Resource can have multiple providers
    package: apt, yum, macports...
    service: upstart, windows, systemd...

•   Each platform (OS) has default Providers that can be


     package “sudo” do
       provider Chef::Provider::Package::Yum
       action :install
     end
A basic recipe

package “apache2” do
  action :install
end
A basic recipe

package “apache2” do
  action :install
end

service “apache2” do
  action :enable
end
A basic recipe

package “apache2” do
  action :install
end

service “apache2” do
  action :enable
end

cookbook_file “/etc/apache2/apache2.conf” do
  source “apache2.conf”
  owner “root”
  group “root”
  mode 0644
end
A basic recipe

package “apache2” do
  action :install
end

service “apache2” do
  action :enable
end

cookbook_file “/etc/apache2/apache2.conf” do
  source “apache2.conf”
  owner “root”
  group “root”
  mode 0644
end

service “apache2” do
 action :start
end
A basic recipe

package “apache2” do
  action :install
end

service “apache2” do
  action :enable
  supports [ :restart, :reload, :status ]
end

cookbook_file “/etc/apache2/apache2.conf” do
  source “apache2.conf”
  owner “root”
  group “root”
  mode 0644
  notifies :restart, “service[apache2]”
end

service “apache2” do
 action :start
end
Search


Search is a first class citizen
# Find all nodes in production that are tagged ‘group_d’
search(:node, “tags:group_d AND chef_environment:prod”)

# Find the mail server in this environment
search (:node, “role:mail_server AND chef_environment:corp”)


Search and Ruby are powerful allies
# Find all my nodes that run on HP hardware
nodes = search(:node, “dmi_systems_manufacturer:HP”)

# Dynamically create a config in a template
<% nodes.each do |node| -%>
server <%= node[‘hostname’] %>
<% end -%>
Nagios Demo
•Download nagios server
 cookbooks
•Install nagios server
•Create fake nodes
knife cookbook site install nagios
knife cookbook upload -a
cd chef-repo
knife role from file monitoring.json
knife data bag create users
knife data bag from file users btm.json
knife node run list add chef-demo-server role[monitoring]
sudo chef-client

http://chef-demo-server/nagios3/

for n in {1..5} ; do knife node from file fake${n}.json ; done
sudo chef-client

knife data bag create nagios_hostgroups
knife data bag from file nagios_hostgroups hp_systems.json
sudo chef-client
Questions?

There is lots more to
 learn about Chef at
       http://

Using Nagios with Chef

  • 1.
    Configuring Nagios withChef Bryan McLellan Technical Program Manager, Open Source btm@opscode.com / @btmspox
  • 2.
    Overview • Who amI? • Why automation • Introduction to Chef • Nagios Demo • Questions
  • 3.
    Who am I? •Chef Early developer, user, pundit • 10+ years in Systems Administration Computer repair, ISPs, Corporate IT, Web operations • Event Logistics Volunteer Traffic Control, Parking, Communications, Networking, Emergency Management • Hacker-Operator
  • 4.
    How did weget here? Bare Metal Deployment
  • 5.
    How did weget here? Bare Metal Deployment • Purchasing
  • 6.
    How did weget here? Bare Metal Deployment • Purchasing • Vendor build
  • 7.
    How did weget here? Bare Metal Deployment • Purchasing • Vendor build • Delivery
  • 8.
    How did weget here? Bare Metal Deployment • Purchasing • Vendor build • Delivery • Installation
  • 9.
    How did weget here? Bare Metal Deployment • Purchasing • Vendor build • Delivery • Installation • OS deployment
  • 10.
    How did weget here? Bare Metal Deployment • Purchasing • Vendor build • Delivery • Installation • OS deployment • Application deployment
  • 11.
    How did weget here? Bare Metal Deployment • Purchasing • Vendor build • Delivery Weeks? • Installation • OS deployment • Application deployment
  • 12.
    How did weget here? Cloud or Virtual Deployment • Purchasing • Vendor build • Delivery Nearly immediate • Installation • OS deployment • Application deployment
  • 13.
    How did weget here? Cloud or Virtual Deployment • Purchasing • Vendor build • Delivery Nearly immediate • Installation • OS deployment • Application deployment Must be fast
  • 14.
    Why automate? Good Reasons: • More agility and faster scalability • Improved infrastructure documentation • Better disaster recovery
  • 15.
    Why automate? Good Reasons: • More agility and faster scalability • Improved infrastructure documentation • Better disaster recovery Really Good Reasons: • Spend less time on monotonous tasks • Spend more time solving interesting problems
  • 16.
    Why automate? Operations isresponsible for two things:
  • 17.
    Why automate? Operations isresponsible for two things: 1. Availability
  • 18.
    Why automate? Operations isresponsible for two things: 1. Availability 2. Efficiency
  • 19.
    What is Chef? •Configuration management language • Systems integration framework • API for your infrastructure http://www.flickr.com/photos/morville/3220961040/
  • 20.
  • 21.
  • 22.
  • 23.
    Chef Principles • Idempotent •Reasonable • Primitives
  • 24.
    Chef Principles • Idempotent •Reasonable • Primitives • Scalable
  • 25.
    Chef Principles • Idempotent •Reasonable • Primitives • Scalable • Hackable
  • 26.
    Chef Principles • Idempotent •Reasonable • Primitives • Scalable • Hackable • Shareable
  • 27.
  • 28.
    Chef Basics Chef managesNodes Nodes have Attributes
  • 29.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients
  • 30.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes
  • 31.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes Each node has a Run List
  • 32.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes Each node has a Run List A Run List is a list of Recipes to run
  • 33.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes Each node has a Run List A Run List is a list of Recipes to run A Role also has a Run List
  • 34.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes Each node has a Run List A Run List is a list of Recipes to run A Role also has a Run List Roles can also be added to a Node’s Run List
  • 35.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes Each node has a Run List A Run List is a list of Recipes to run A Role also has a Run List Roles can also be added to a Node’s Run List Nodes can be in Environments
  • 36.
    Chef Basics Chef managesNodes Nodes have Attributes Users and Nodes authenticate as Clients Cookbooks contain Recipes Each node has a Run List A Run List is a list of Recipes to run A Role also has a Run List Roles can also be added to a Node’s Run List Nodes can be in Environments Data bags are... bags of data.
  • 37.
    Chef Basics Visualized client: srv01 client: srv02 client: srv03 node: srv01 node: srv02 node: srv03 run_list: “role[web_server]” run_list: “role[web_server]” run_list: “role[db_server]” role: web_server role: db_server run_list: [“recipe[apache2]”, “recipe[php]” ] run_list: [ “recipe[mysql]”, “recipe[nfs]” ]
  • 38.
    Application Programming Interface The Meatcloud Manifesto Give me an API or give me death. -- Andrew Clay Shafer (@littleidea)
  • 39.
    Chef Stacks chef-client knife chef-shell chef-solo API Open Source Hosted Chef Private Chef
  • 40.
    Chef 10 OpenSource Architecture Chef Expander
  • 41.
    Resources • A Resource is something you manage service, package, file, user, execute, git
  • 42.
    Resources • A Resource is something you manage service, package, file, user, execute, git • Resources have actions start, install, create, deploy • Resources can notify of other resources
  • 43.
    Resources • A Resource is something you manage service, package, file, user, execute, git • Resources have actions start, install, create, deploy • Resources can notify of other resources cookbook_file “/etc/apache2/apache2.conf” do source “apache2.conf” owner “root” group “root” mode 0644 notifies :restart, “service[apache2]” end
  • 44.
    Providers • A Provider performs the actions specified by the resource
  • 45.
    Providers • A Provider performs the actions specified by the resource • Each Resource can have multiple providers package: apt, yum, macports... service: upstart, windows, systemd...
  • 46.
    Providers • A Provider performs the actions specified by the resource • Each Resource can have multiple providers package: apt, yum, macports... service: upstart, windows, systemd... • Each platform (OS) has default Providers that can be package “sudo” do provider Chef::Provider::Package::Yum action :install end
  • 47.
    A basic recipe package“apache2” do action :install end
  • 48.
    A basic recipe package“apache2” do action :install end service “apache2” do action :enable end
  • 49.
    A basic recipe package“apache2” do action :install end service “apache2” do action :enable end cookbook_file “/etc/apache2/apache2.conf” do source “apache2.conf” owner “root” group “root” mode 0644 end
  • 50.
    A basic recipe package“apache2” do action :install end service “apache2” do action :enable end cookbook_file “/etc/apache2/apache2.conf” do source “apache2.conf” owner “root” group “root” mode 0644 end service “apache2” do action :start end
  • 51.
    A basic recipe package“apache2” do action :install end service “apache2” do action :enable supports [ :restart, :reload, :status ] end cookbook_file “/etc/apache2/apache2.conf” do source “apache2.conf” owner “root” group “root” mode 0644 notifies :restart, “service[apache2]” end service “apache2” do action :start end
  • 52.
    Search Search is afirst class citizen # Find all nodes in production that are tagged ‘group_d’ search(:node, “tags:group_d AND chef_environment:prod”) # Find the mail server in this environment search (:node, “role:mail_server AND chef_environment:corp”) Search and Ruby are powerful allies # Find all my nodes that run on HP hardware nodes = search(:node, “dmi_systems_manufacturer:HP”) # Dynamically create a config in a template <% nodes.each do |node| -%> server <%= node[‘hostname’] %> <% end -%>
  • 53.
    Nagios Demo •Download nagiosserver cookbooks •Install nagios server •Create fake nodes
  • 54.
    knife cookbook siteinstall nagios knife cookbook upload -a cd chef-repo knife role from file monitoring.json knife data bag create users knife data bag from file users btm.json knife node run list add chef-demo-server role[monitoring] sudo chef-client http://chef-demo-server/nagios3/ for n in {1..5} ; do knife node from file fake${n}.json ; done sudo chef-client knife data bag create nagios_hostgroups knife data bag from file nagios_hostgroups hp_systems.json sudo chef-client
  • 55.
    Questions? There is lotsmore to learn about Chef at http://

Editor's Notes

  • #2 \n
  • #3 \n
  • #4 Systems\n
  • #5 Who considers there job to be operations or running servers?\n
  • #6 1 week each?\nDifferent departments schedules\n
  • #7 1 week each?\nDifferent departments schedules\n
  • #8 1 week each?\nDifferent departments schedules\n
  • #9 1 week each?\nDifferent departments schedules\n
  • #10 1 week each?\nDifferent departments schedules\n
  • #11 1 week each?\nDifferent departments schedules\n
  • #12 1 week each?\nDifferent departments schedules\n
  • #13 Now application deployment needs to be fast\n
  • #14 Now application deployment needs to be fast\n
  • #15 DR: Bare metal, application data backup, chef repository\n\n
  • #16 DR: Bare metal, application data backup, chef repository\n\n
  • #17 Who works in operations?\n
  • #18 \n
  • #19 \n
  • #20 Who here uses Chef?\nIntrastructure as Code\n\n
  • #21 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #22 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #23 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #24 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #25 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #26 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #27 idempotency: If it is right, do nothing. If it is wrong, fix it.\nreasonable: no surprises to the platform user\nprimitives: the unix way\nscalable: hosted chef\n
  • #28 Nodes are data objects - servers, network switches, Google Compute Engine\n\n
  • #29 Nodes are data objects - servers, network switches, Google Compute Engine\nplatform, platform_version\n
  • #30 Clients are authentication objects\npublic key / private key\n
  • #31 Cookbooks are something you would manage\n
  • #32 Nodes are data objects\nClients are authentication objects\nCookbooks are something you would manage\n
  • #33 Nodes are data objects\nClients are authentication objects\nCookbooks are something you would manage\n
  • #34 Nodes are data objects\nClients are authentication objects\nCookbooks are something you would manage\n
  • #35 Nodes are data objects\nClients are authentication objects\nCookbooks are something you would manage\n
  • #36 Workflow: Environments control cookbook versions\n
  • #37 Nodes are data objects\nClients are authentication objects\nCookbooks are something you would manage\n
  • #38 \n
  • #39 \n
  • #40 Hackable\nshef -&gt; chef-shell\n
  • #41 Service Oriented Architecture\n
  • #42 Primitives\n
  • #43 Primitives\n
  • #44 Idempotent\n
  • #45 \n
  • #46 \n
  • #47 \n
  • #48 \n
  • #49 \n
  • #50 \n
  • #51 \n
  • #52 \n
  • #53 \n
  • #54 \n
  • #55 \n
  • #56 \n