0
CookbookRefactoringA
CookbookRefactoring... and extracting logic into RubygemsA
sethvargo@opscode.comEbyz
Were Hiring!
Were Hiring!Colorado
New BrandingWere Hiring!
UDO YOU SOMETIMESFEEL LIKETHIS
template /etc/hosts doowner rootgroup rootsource etc/hostsendrecipes/default.rb
# This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fq...
default[etc][hosts] = [] unless node[etc][hosts]attributes/default.rb
# This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fq...
include_attribute hostsfiledefault[etc][hosts] << {ip => 1.2.3.4,host => www.example.com}other_cookbook/attributes/default...
node.default[etc][hosts] << {ip => 1.2.3.4,host => www.example.com}other_cookbook/recipes/default.rb
default_attributes({etc => {hosts => [{ip => 1.2.3.4, host => www.example.com},{ip => 4.5.6.7, host => foo.example.com}]}}...
{"default_attributes": {"etc": {"hosts": [{"ip": "1.2.3.4", "host": "www.example.com"},{"ip": "4.5.6.7", "host": "foo.exam...
node.set[etc][hosts] = {ip: 7.8.9.0,host: bar.example.com})recipes/default.rb
arr = [1,2,3]arr << 4 => [1,2,3,4]arr = 4 => 4
arr = [1,2,3]arr << 4 => [1,2,3,4]arr = 4 => 4Not an Array
TODO:Add infographics# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.1.2.3.4 www.myapp...
TODO:Add infographics# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.1.2.3.4 www.myapp...
Post Mortem
<< =
<< =!=
Post MortemAction Items7
Monkey patch Chef to raise anexception when redefining thatparticular node attribute.
Monkey patch Chef to raise anexception when redefining thatparticular node attribute.t
Create a special cookbook thatuses a threshold value and raises anexception if the size of the arraydoesnt "make sense".
Create a special cookbook thatuses a threshold value and raises anexception if the size of the arraydoesnt "make sense".t
Move all entries to a data bag
Move all entries to a data bagu
Move all entries to a data bag66 Add tests
Data Bags
["1.2.3.4 example.com www.example.com","4.5.6.7 foo.example.com","7.8.9.0 bar.example.com"]data_bags/etc_hosts.json
hosts = data_bag(etc_hosts)template /etc/hosts doowner rootgroup rootsource etc/hostsvariables(hosts: hosts)endrecipes/def...
# This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fq...
Move all entries to a data bag56 Add tests
require chefspecspec/default_spec.rb
require chefspecdescribe hostsfile::default doendspec/default_spec.rb
require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe....
require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe....
require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe....
require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe....
$ rspec cookbooks/hostsfileRunning all specs
$ rspec cookbooks/hostsfileRunning all specs**Finished in 0.0003 seconds2 examples, 0 failures
$ rspec cookbooks/hostsfileRunning all specs**Finished in 0.0003 seconds2 examples, 0 failuresReally Fucking Fast™
#winning
10,000 tests
28 seconds
#winning
⏳⏳
hosts = data_bag(etc_hosts)hosts << search(:node, role:mongo_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endtempl...
hosts = data_bag(etc_hosts)hosts << search(:node, role:mongo_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endhosts...
LWRPs
# List of all actions supported by the provideractions :create, :create_if_missing, :update, :remove# Make create the defa...
action :create do::Chef::Util::FileEdit.search_file_delete_line(entry)::Chef::Util::FileEdit.insert_line_after_match(/n/, ...
hostsfile_entry 1.2.3.4 dohostname example.comendproviders/entry.rb
Chef::Util::FileEdit is slow
Re-writing the file on each run
Provider kept growning
Untested
RefactorA
Move to pure Ruby classes
Ditch Chef::Util::FileEdit andmanage the entire file
Only implement Ruby classes inthe Provider (logic-less Provider)
Test the Ruby code
Test that the Provider implementsthe proper Ruby classes
TODO:Add infographicsclass Entryattr_accessor :ip_address, :hostname, :aliases, :commentdef initialize(options = {})if opt...
TODO:Add infographicsclass Manipulatordef initializecontents = ::File.readlines(hostsfile_path)@entries = contents.collect...
# Creates a new hosts file entry. If an entry already exists, it# will be overwritten by this one.action :create dohostsfi...
RSpec
TODO:Add infographicsdescribe Entry dodescribe .initialize dosubject { Entry.new(ip_address: 2.3.4.5, hostname:www.example...
ChefSpec
ChefSpec
TODO:Add infographicsdescribe hostsfile lwrp dolet(:manipulator) { double(manipulator) }before doManipulator.stub(:new).an...
TODO:Add infographicscontext actions dodescribe :create doit adds the entry domanipulator.should_receive(:add).with({ip_ad...
Open It
Gem It
$ bundle gem hostsfile
$ bundle gem hostsfilecreate hostsfile/Gemfilecreate hostsfile/Rakefilecreate hostsfile/LICENSE.txtcreate hostsfile/README...
entry.rbmanipulator.rb99
9
9?
chef_gem hostsfilerecipes/default.rb
require hostsfileproviders/entry.rb
In another cookbook...
# ...depends hostsfileother_cookbook/metadata.rb
{"run_list": ["recipe[hostsfile]"]}www.myapp.com (Chef Node)
ThankYouz
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Cookbook refactoring & abstracting logic to Ruby(gems)
Upcoming SlideShare
Loading in...5
×

Cookbook refactoring & abstracting logic to Ruby(gems)

3,262

Published on

Chef is awesome, but it’s also very easy to go overboard. In terms of testing and maintainability, sometimes its better to refactor your long recipe into an LWRP. As your infrastructure evolves, so should you cookbooks. But at some point your bound to have a cookbook 500+ lines of antiquated logic. How do you refactor such a large chunk of code that is critical to your infrastructure? How much logic should me moved into other cookbooks? How much logic should be extracted into LWRPs? How much logic should be moved out of Chef, into Ruby, and packaged as a gem?

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,262
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
30
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Cookbook refactoring & abstracting logic to Ruby(gems)"

  1. 1. CookbookRefactoringA
  2. 2. CookbookRefactoring... and extracting logic into RubygemsA
  3. 3. sethvargo@opscode.comEbyz
  4. 4. Were Hiring!
  5. 5. Were Hiring!Colorado
  6. 6. New BrandingWere Hiring!
  7. 7. UDO YOU SOMETIMESFEEL LIKETHIS
  8. 8. template /etc/hosts doowner rootgroup rootsource etc/hostsendrecipes/default.rb
  9. 9. # This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fqdn] %>127.0.0.1!localhost <%= node[fqdn] %>255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhosttemplates/default/etc/hosts.erb
  10. 10. default[etc][hosts] = [] unless node[etc][hosts]attributes/default.rb
  11. 11. # This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fqdn] %>127.0.0.1!localhost <%= node[fqdn] %>255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries<% node[etc][hosts].each do |h| -%><%= h[ip] %> <%= h[host] %><% end -%>templates/default/etc/hosts.erb
  12. 12. include_attribute hostsfiledefault[etc][hosts] << {ip => 1.2.3.4,host => www.example.com}other_cookbook/attributes/default.rb
  13. 13. node.default[etc][hosts] << {ip => 1.2.3.4,host => www.example.com}other_cookbook/recipes/default.rb
  14. 14. default_attributes({etc => {hosts => [{ip => 1.2.3.4, host => www.example.com},{ip => 4.5.6.7, host => foo.example.com}]}})roles/my_role.rb
  15. 15. {"default_attributes": {"etc": {"hosts": [{"ip": "1.2.3.4", "host": "www.example.com"},{"ip": "4.5.6.7", "host": "foo.example.com"}]}}}environments/production.json
  16. 16. node.set[etc][hosts] = {ip: 7.8.9.0,host: bar.example.com})recipes/default.rb
  17. 17. arr = [1,2,3]arr << 4 => [1,2,3,4]arr = 4 => 4
  18. 18. arr = [1,2,3]arr << 4 => [1,2,3,4]arr = 4 => 4Not an Array
  19. 19. TODO:Add infographics# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.1.2.3.4 www.myapp.com127.0.0.1!localhost www.myapp.com255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries1.2.3.4 www.example.com4.5.6.7 foo.example.com7.8.9.0 bar.example.com/etc/hosts
  20. 20. TODO:Add infographics# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.1.2.3.4 www.myapp.com127.0.0.1!localhost www.myapp.com255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries7.8.9.0 bar.example.com/etc/hosts
  21. 21. Post Mortem
  22. 22. << =
  23. 23. << =!=
  24. 24. Post MortemAction Items7
  25. 25. Monkey patch Chef to raise anexception when redefining thatparticular node attribute.
  26. 26. Monkey patch Chef to raise anexception when redefining thatparticular node attribute.t
  27. 27. Create a special cookbook thatuses a threshold value and raises anexception if the size of the arraydoesnt "make sense".
  28. 28. Create a special cookbook thatuses a threshold value and raises anexception if the size of the arraydoesnt "make sense".t
  29. 29. Move all entries to a data bag
  30. 30. Move all entries to a data bagu
  31. 31. Move all entries to a data bag66 Add tests
  32. 32. Data Bags
  33. 33. ["1.2.3.4 example.com www.example.com","4.5.6.7 foo.example.com","7.8.9.0 bar.example.com"]data_bags/etc_hosts.json
  34. 34. hosts = data_bag(etc_hosts)template /etc/hosts doowner rootgroup rootsource etc/hostsvariables(hosts: hosts)endrecipes/default.rb
  35. 35. # This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fqdn] %>127.0.0.1!localhost <%= node[fqdn] %>255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries<%= @hosts.join("n") %>templates/default/etc/hosts.erb
  36. 36. Move all entries to a data bag56 Add tests
  37. 37. require chefspecspec/default_spec.rb
  38. 38. require chefspecdescribe hostsfile::default doendspec/default_spec.rb
  39. 39. require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endendspec/default_spec.rb
  40. 40. require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endlet(:runner) { ChefSpec::ChefRunner.new.converge(hostsfile::default) }endspec/default_spec.rb
  41. 41. require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endlet(:runner) { ChefSpec::ChefRunner.new.converge(hostsfile::default) }it loads the data bag doChef::Recipe.any_instance.should_receive(:data_bag).with(etc_hosts)endendspec/default_spec.rb
  42. 42. require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endlet(:runner) { ChefSpec::ChefRunner.new.converge(hostsfile::default) }it loads the data bag doChef::Recipe.any_instance.should_receive(:data_bag).with(etc_hosts)endit creates the /etc/hosts template doexpect(runner).to create_template(/etc/hosts).with_content(hosts.join("n"))endendspec/default_spec.rb
  43. 43. $ rspec cookbooks/hostsfileRunning all specs
  44. 44. $ rspec cookbooks/hostsfileRunning all specs**Finished in 0.0003 seconds2 examples, 0 failures
  45. 45. $ rspec cookbooks/hostsfileRunning all specs**Finished in 0.0003 seconds2 examples, 0 failuresReally Fucking Fast™
  46. 46. #winning
  47. 47. 10,000 tests
  48. 48. 28 seconds
  49. 49. #winning
  50. 50. ⏳⏳
  51. 51. hosts = data_bag(etc_hosts)hosts << search(:node, role:mongo_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endtemplate /etc/hosts doowner rootgroup rootsource etc/hostsvariables(hosts: hosts)endrecipes/default.rb
  52. 52. hosts = data_bag(etc_hosts)hosts << search(:node, role:mongo_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endhosts << search(:node, role:mysql_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endhosts << search(:node, role:redis_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endtemplate /etc/hosts doowner rootgroup rootrecipes/default.rb
  53. 53. LWRPs
  54. 54. # List of all actions supported by the provideractions :create, :create_if_missing, :update, :remove# Make create the default actiondefault_action :create# Required attributesattribute :ip_address,kind_of: String,name_attribute: true,required: trueattribute :hostname, kind_of: String# Optional attributesattribute :aliases, kind_of: Arrayattribute :comment, kind_of: Stringresources/entry.rb
  55. 55. action :create do::Chef::Util::FileEdit.search_file_delete_line(entry)::Chef::Util::FileEdit.insert_line_after_match(/n/, entry)endprotecteddef entry[new_resource.ip_address, new_resource.hostname,new_resource.aliases.join( )].compact.join( ).squeeze( )endproviders/entry.rb
  56. 56. hostsfile_entry 1.2.3.4 dohostname example.comendproviders/entry.rb
  57. 57. Chef::Util::FileEdit is slow
  58. 58. Re-writing the file on each run
  59. 59. Provider kept growning
  60. 60. Untested
  61. 61. RefactorA
  62. 62. Move to pure Ruby classes
  63. 63. Ditch Chef::Util::FileEdit andmanage the entire file
  64. 64. Only implement Ruby classes inthe Provider (logic-less Provider)
  65. 65. Test the Ruby code
  66. 66. Test that the Provider implementsthe proper Ruby classes
  67. 67. TODO:Add infographicsclass Entryattr_accessor :ip_address, :hostname, :aliases, :commentdef initialize(options = {})if options[:ip_address].nil? || options[:hostname].nil?raise :ip_address and :hostname are both required optionsend@ip_address = options[:ip_address]@hostname = options[:hostname]@aliases = [options[:aliases]].flatten@comment = options[:comment]end# ...endlibraries/entry.rb
  68. 68. TODO:Add infographicsclass Manipulatordef initializecontents = ::File.readlines(hostsfile_path)@entries = contents.collect do |line|Entry.parse(line) unless line.strip.nil? || line.strip.empty?end.compactenddef add(options = {})@entries << Entry.new(ip_address: options[:ip_address],hostname: options[:hostname],aliases: options[:aliases],comment: options[:comment])endendlibraries/manipulator.rb
  69. 69. # Creates a new hosts file entry. If an entry already exists, it# will be overwritten by this one.action :create dohostsfile.add(ip_address: new_resource.ip_address,hostname: new_resource.hostname,aliases: new_resource.aliases,comment: new_resource.comment)new_resource.updated_by_last_action(true) if hostsfile.saveendproviders/entry.rb
  70. 70. RSpec
  71. 71. TODO:Add infographicsdescribe Entry dodescribe .initialize dosubject { Entry.new(ip_address: 2.3.4.5, hostname:www.example.com, aliases: [foo, bar], comment: This is acomment!, priority: 100) }it raises an exception if :ip_address is missing doexpect {Entry.new(hostname: www.example.com)}.to raise_error(ArgumentError)endit sets the ip_address doexpect(subject.ip_address).to eq(2.3.4.5)endendspec/entry_spec.rb
  72. 72. ChefSpec
  73. 73. ChefSpec
  74. 74. TODO:Add infographicsdescribe hostsfile lwrp dolet(:manipulator) { double(manipulator) }before doManipulator.stub(:new).and_return(manipulator)Manipulator.should_receive(:new).with(kind_of(Chef::Node)).and_return(manipulator)manipulator.should_receive(:save!)endlet(:chef_run) {ChefSpec::ChefRunner.new(cookbook_path: $cookbook_paths,step_into: [hostsfile_entry])}spec/default_spec.rb
  75. 75. TODO:Add infographicscontext actions dodescribe :create doit adds the entry domanipulator.should_receive(:add).with({ip_address: 2.3.4.5,hostname: www.example.com,aliases: nil,comment: nil,priority: nil})chef_run.converge(fake::create)endendendend
  76. 76. Open It
  77. 77. Gem It
  78. 78. $ bundle gem hostsfile
  79. 79. $ bundle gem hostsfilecreate hostsfile/Gemfilecreate hostsfile/Rakefilecreate hostsfile/LICENSE.txtcreate hostsfile/README.mdcreate hostsfile/.gitignorecreate hostsfile/hostsfile.gemspeccreate hostsfile/lib/hostsfile.rbcreate hostsfile/lib/hostsfile/version.rbInitializating git repo in ~Development/hostsfile
  80. 80. entry.rbmanipulator.rb99
  81. 81. 9
  82. 82. 9?
  83. 83. chef_gem hostsfilerecipes/default.rb
  84. 84. require hostsfileproviders/entry.rb
  85. 85. In another cookbook...
  86. 86. # ...depends hostsfileother_cookbook/metadata.rb
  87. 87. {"run_list": ["recipe[hostsfile]"]}www.myapp.com (Chef Node)
  88. 88. ThankYouz
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×