Chef or how to make computers do the work for us

5,016 views

Published on

My KRUG (Kraków Ruby Users Group) presentation about automating boring tasks with Opscode's Chef.

Published in: Technology, Self Improvement
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,016
On SlideShare
0
From Embeds
0
Number of Embeds
661
Actions
Shares
0
Downloads
123
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Chef or how to make computers do the work for us

  1. 1. CHEFor how to make computers do the work for us Marcin Kulik, Lunar Logic Polska KRUG 2011/11/08
  2. 2. Everyday were dealing with mechanical, repetitive tasks... we can automate.
  3. 3. What is Chef?
  4. 4. Automation toolwritten in Ruby
  5. 5. DSL
  6. 6. Created by Opscode
  7. 7. "Chef is an open source systems integration framework built to bring the benefits of configuration management to your entire infrastructure.""You write source code to describe how you want each part ofyour infrastructure to be built, then apply those descriptions to your servers." "The result is a fully automated infrastructure: when a newserver comes on line, the only thing you have to do is tell Chef what role it should play in your architecture."
  8. 8. Why do you need it?
  9. 9. Economics + Efficiency + Scalability
  10. 10. Terms
  11. 11. Noderemote server, local machine...
  12. 12. Roleweb server, database server, ruby dev workstation...
  13. 13. Cookbookmysql, ssh-access, dotfiles...
  14. 14. Recipeinstall mysql-server, create database, add user...
  15. 15. Resourcefile, dir, user, package, service, gem, virtual host...
  16. 16. Run listlist of recipes to run in order
  17. 17. { "run_list": [ "recipe[mysql]", "recipe[git]", "recipe[ruby19]" ]}
  18. 18. Cookbook structure|-- config| |-- node.json| `-- solo.rb|-- cookbooks| |-- book1| | |-- attributes| | |-- files| | |-- metadata.rb| | |-- recipes| | | |-- default.rb| | | `-- source.rb| | `-- templates| |-- book2| | |-- attributes| | | `-- default.rb| | |-- files| | |-- recipes| | | `-- default.rb| | `-- templates
  19. 19. | | `-- templates| | `-- default| | `-- authorized_keys.erb| |-- book3| | |-- attributes| | |-- files| | | `-- default| | | `-- secret-key| | |-- recipes| | | `-- default.rb Installation| | `-- templates|-- config| |-- node.json| `-- solo.rb|-- cookbooks| |-- book1| | |-- attributes| | |-- files| | |-- metadata.rb| | |-- recipes| | | |-- default.rb| | | `-- libs.rb| | `-- templates
  20. 20. $ gem install chef
  21. 21. Modes of operation
  22. 22. Cookbooks storedin central repository (free cookbooks hosting by Opscode: https://manage.opscode.com/)
  23. 23. $ sudo chef-client
  24. 24. Cookbooks stored on the node
  25. 25. $ sudo chef-solo -c /path/to/cfg.rb -j /path/to/node-data.json
  26. 26. Use cases
  27. 27. Configure new machine(in the cloud with Knife) Amazon EC2, Engine Yard, Linode, BrightBox...
  28. 28. Manage config of existing company serversClient demo apps (directory, vhost, god config), developers ssh keys...
  29. 29. Bootstrap workstation! rvm + ruby 1.9, git, mysql, vim/emacs...
  30. 30. Enough with theory!
  31. 31. Lunar Stationhttps://github.com/LunarLogicPolska/lunar-station
  32. 32. Lunar Station is a set of Chef cookbooks and a bash script (???)for bootstrapping developers machines at Lunar Logic Polska.
  33. 33. You need ruby to run Chef
  34. 34. (We assume) you use RVM No need for system ruby for ruby devs nowadays
  35. 35. bootstrap.sh
  36. 36. detects platform (Ubuntu, Fedora, OSX) installs compilers and other RVM dependencies installs RVM & ruby 1.9 & chef gem downloads latest Lunar Station cookbooks runs chef-solo
  37. 37. $ curl -skL http://bit.ly/lunar-station | bashInitializing Lunar Workstation...>> Fedora Linux detected.>> Checking for RVM...>> Fetching latest version of Lunar Station cookbooks...>> Starting chef-solo run...[Mon, 07 Nov 2011 22:19:54 +0100] INFO: *** Chef 0.10.4 ***[Mon, 07 Nov 2011 22:19:54 +0100] INFO: Setting the run_list to...
  38. 38. Nodes
  39. 39. # linux-rubydev.json{ "run_list": [ "role[rubydev]" ]}
  40. 40. # osx-rubydev.json{ "run_list": [ "role[osx]", "role[rubydev]" ]}
  41. 41. Roles
  42. 42. # base.rbrun_list recipe[repos], recipe[curl], recipe[wget], recipe[git], recipe[libxml2], recipe[ack], recipe[vim], recipe[ctags], recipe[skype], recipe[firefox] , recipe[google-chrome]
  43. 43. # rubydev.rbrun_list role[base], recipe[mysql]
  44. 44. # osx.rbrun_list "recipe[homebrew]"
  45. 45. Cookbooks
  46. 46. repos cookbook
  47. 47. # cookbooks/repos/recipes/default.rbcase node[:platform]when fedora path = "/tmp/rpmfusion-free-release-stable.noarch.rpm" bash "download rpmfusion free package" do code "wget http://download1.rpmfusion.org/.../" + "rpmfusion-free-release-stable.noarch.rpm -O #{path}" not_if { File.exist?(path) } end package "rpmfusion-free-release-stable" do source path options "--nogpgcheck" endwhen ubuntu ...end
  48. 48. end# cookbooks/repos/recipes/default.rbcase node[:platform]when fedora ...when ubuntu bash "enable multiverse repo" do code "head -n 1 /etc/apt/sources.list | " + "sed s/main universe/multiverse/ " + ">> /etc/apt/sources.list" not_if "egrep ^deb.+multiverse /etc/apt/sources.list" endend
  49. 49. vim cookbook
  50. 50. # cookbooks/vim/recipes/default.rbcase node[:platform]when "ubuntu" package "vim" package "vim-gnome"when "fedora" package "vim-enhanced" package "vim-X11"when mac_os_x package "macvim"end
  51. 51. skype cookbook
  52. 52. # cookbooks/skype/recipes/default.rbcase node[:platform]when ubuntu include_recipe init::ubuntu # for partner repo package skypewhen mac_os_x dmg_package "Skype" do source "http://www.skype.com/go/getskype-macosx.dmg" action :install endwhen fedora ...end
  53. 53. Lunar Kitchen
  54. 54. Source of LLP servers configuration data and a set of Chef cookbooks
  55. 55. chef-solo invoked on remote machines no chef server
  56. 56. Each server we configure has its corresponding nodeconfiguration file in nodes/ directory of kitchen project that specifies run_list and few other settings
  57. 57. # nodes/deneb.json{ "run_list": [ "recipe[ssh_access]" ], "ssh_access": [ "marcin.kulik", "anna.lesniak", ...], "opened_ports": { "tcp": [80, 443, 22, 8080], "udp": [] }, ...
  58. 58. How do we run chef-solo on remote machine?
  59. 59. Capistrano!
  60. 60. # See the list of configured servers:$ cap -T# Make the changes happen on the server:$ cap configure:deneb
  61. 61. How does Capfile look like?
  62. 62. set :user, chefNODE_LIST = Dir["nodes/*.json"].map do |nodefile| File.basename(nodefile, .json)endNODE_LIST.each do |node| role node.to_sym, nodeendNODE_CONFIG = <<-EOS file_cache_path /tmp/chef-solo cookbook_path /tmp/chef-solo/cookbooks role_path /tmp/chef-solo/rolesEOS...
  63. 63. ...namespace :configure do NODE_LIST.each do |node| desc "Configure #{node}" task node.to_sym, :roles => node.to_sym do run "if [ ! -e /tmp/chef-solo ]; then mkdir /tmp/chef-sol upload("cookbooks", "/tmp/chef-solo/", :via => :scp, :rec upload("roles", "/tmp/chef-solo/", :via => :scp, :recursi upload("nodes/#{node}.json", "/tmp/chef-solo/node.json", put(NODE_CONFIG, "/tmp/chef-solo/solo.rb") run "rvmsudo chef-solo " + "-c /tmp/chef-solo/solo.rb " + "-j /tmp/chef-solo/node.json" end endend
  64. 64. SSH access
  65. 65. ├── Capfile├── config├── cookbooks├── nodes├── README.md├── roles└── ssh_keys ├── anna.lesniak ├── artur.bilski ├── ... └── marcin.kulik
  66. 66. # cookbooks/access/recipes/default.rbusername = devssh_keys = node[:ssh_access].map do |f| File.read("/tmp/chef-solo/ssh_keys/#{f}")endtemplate "/home/#{username}/.ssh/authorized_keys" do source "authorized_keys.erb" owner username group users mode "0600" variables :ssh_keys => ssh_keysend
  67. 67. # cookbooks/access/templates/authorized_keys.erb# Generated by Chef, do not edit!<%= @ssh_keys.join("n") %>
  68. 68. Tips
  69. 69. Learn step by stepEC2 + Chef + Knife + Opscode... = Fuuuuuuuuuuuuuuuuuuuuu
  70. 70. Start with chef-solo
  71. 71. Run on local machine Easy to troubleshoot problems
  72. 72. Use Vagrant http://vagrantup.com/Great for testing cookbooks - doesnt pollute your system
  73. 73. Q?
  74. 74. Thanks!marcin.kulik@llp.pl | @sickill | https://github.com/sickill

×