• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Cookbook refactoring & abstracting logic to Ruby(gems)
 

Cookbook refactoring & abstracting logic to Ruby(gems)

on

  • 2,466 views

Chef is awesome, but it’s also very easy to go overboard. In terms of testing and maintainability, sometimes its better to refactor your long recipe into an LWRP. As your infrastructure evolves, so ...

Chef is awesome, but it’s also very easy to go overboard. In terms of testing and maintainability, sometimes its better to refactor your long recipe into an LWRP. As your infrastructure evolves, so should you cookbooks. But at some point your bound to have a cookbook 500+ lines of antiquated logic. How do you refactor such a large chunk of code that is critical to your infrastructure? How much logic should me moved into other cookbooks? How much logic should be extracted into LWRPs? How much logic should be moved out of Chef, into Ruby, and packaged as a gem?

Statistics

Views

Total Views
2,466
Views on SlideShare
1,336
Embed Views
1,130

Actions

Likes
3
Downloads
19
Comments
0

3 Embeds 1,130

http://www.getchef.com 718
http://www.opscode.com 411
https://www.google.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Cookbook refactoring & abstracting logic to Ruby(gems) Cookbook refactoring & abstracting logic to Ruby(gems) Presentation Transcript

    • CookbookRefactoringA
    • CookbookRefactoring... and extracting logic into RubygemsA
    • sethvargo@opscode.comEbyz
    • Were Hiring!
    • Were Hiring!Colorado
    • New BrandingWere Hiring!
    • UDO YOU SOMETIMESFEEL LIKETHIS
    • template /etc/hosts doowner rootgroup rootsource etc/hostsendrecipes/default.rb
    • # This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fqdn] %>127.0.0.1!localhost <%= node[fqdn] %>255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhosttemplates/default/etc/hosts.erb
    • default[etc][hosts] = [] unless node[etc][hosts]attributes/default.rb
    • # This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fqdn] %>127.0.0.1!localhost <%= node[fqdn] %>255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries<% node[etc][hosts].each do |h| -%><%= h[ip] %> <%= h[host] %><% end -%>templates/default/etc/hosts.erb
    • include_attribute hostsfiledefault[etc][hosts] << {ip => 1.2.3.4,host => www.example.com}other_cookbook/attributes/default.rb
    • node.default[etc][hosts] << {ip => 1.2.3.4,host => www.example.com}other_cookbook/recipes/default.rb
    • default_attributes({etc => {hosts => [{ip => 1.2.3.4, host => www.example.com},{ip => 4.5.6.7, host => foo.example.com}]}})roles/my_role.rb
    • {"default_attributes": {"etc": {"hosts": [{"ip": "1.2.3.4", "host": "www.example.com"},{"ip": "4.5.6.7", "host": "foo.example.com"}]}}}environments/production.json
    • node.set[etc][hosts] = {ip: 7.8.9.0,host: bar.example.com})recipes/default.rb
    • arr = [1,2,3]arr << 4 => [1,2,3,4]arr = 4 => 4
    • arr = [1,2,3]arr << 4 => [1,2,3,4]arr = 4 => 4Not an Array
    • TODO:Add infographics# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.1.2.3.4 www.myapp.com127.0.0.1!localhost www.myapp.com255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries1.2.3.4 www.example.com4.5.6.7 foo.example.com7.8.9.0 bar.example.com/etc/hosts
    • TODO:Add infographics# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.1.2.3.4 www.myapp.com127.0.0.1!localhost www.myapp.com255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries7.8.9.0 bar.example.com/etc/hosts
    • Post Mortem
    • << =
    • << =!=
    • Post MortemAction Items7
    • Monkey patch Chef to raise anexception when redefining thatparticular node attribute.
    • Monkey patch Chef to raise anexception when redefining thatparticular node attribute.t
    • Create a special cookbook thatuses a threshold value and raises anexception if the size of the arraydoesnt "make sense".
    • Create a special cookbook thatuses a threshold value and raises anexception if the size of the arraydoesnt "make sense".t
    • Move all entries to a data bag
    • Move all entries to a data bagu
    • Move all entries to a data bag66 Add tests
    • Data Bags
    • ["1.2.3.4 example.com www.example.com","4.5.6.7 foo.example.com","7.8.9.0 bar.example.com"]data_bags/etc_hosts.json
    • hosts = data_bag(etc_hosts)template /etc/hosts doowner rootgroup rootsource etc/hostsvariables(hosts: hosts)endrecipes/default.rb
    • # This file is managed by Chef for "<%= node[fqdn] %>"# Do NOT modify this file by hand.<%= node[ipaddress] %> <%= node[fqdn] %>127.0.0.1!localhost <%= node[fqdn] %>255.255.255.255!broadcasthost::1 localhostfe80::1%lo0! localhost# Custom Entries<%= @hosts.join("n") %>templates/default/etc/hosts.erb
    • Move all entries to a data bag56 Add tests
    • require chefspecspec/default_spec.rb
    • require chefspecdescribe hostsfile::default doendspec/default_spec.rb
    • require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endendspec/default_spec.rb
    • require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endlet(:runner) { ChefSpec::ChefRunner.new.converge(hostsfile::default) }endspec/default_spec.rb
    • require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endlet(:runner) { ChefSpec::ChefRunner.new.converge(hostsfile::default) }it loads the data bag doChef::Recipe.any_instance.should_receive(:data_bag).with(etc_hosts)endendspec/default_spec.rb
    • require chefspecdescribe hostsfile::default dolet(:hosts) { [1.2.3.4 example.com, 4.5.6.7 bar.com] }before doChef::Recipe.any_instance.stub(:data_bag).with(etc_hosts).and_return(hosts)endlet(:runner) { ChefSpec::ChefRunner.new.converge(hostsfile::default) }it loads the data bag doChef::Recipe.any_instance.should_receive(:data_bag).with(etc_hosts)endit creates the /etc/hosts template doexpect(runner).to create_template(/etc/hosts).with_content(hosts.join("n"))endendspec/default_spec.rb
    • $ rspec cookbooks/hostsfileRunning all specs
    • $ rspec cookbooks/hostsfileRunning all specs**Finished in 0.0003 seconds2 examples, 0 failures
    • $ rspec cookbooks/hostsfileRunning all specs**Finished in 0.0003 seconds2 examples, 0 failuresReally Fucking Fast™
    • #winning
    • 10,000 tests
    • 28 seconds
    • #winning
    • ⏳⏳
    • hosts = data_bag(etc_hosts)hosts << search(:node, role:mongo_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endtemplate /etc/hosts doowner rootgroup rootsource etc/hostsvariables(hosts: hosts)endrecipes/default.rb
    • hosts = data_bag(etc_hosts)hosts << search(:node, role:mongo_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endhosts << search(:node, role:mysql_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endhosts << search(:node, role:redis_master).first.tap do |n|"#{n[ip_address]} #{n[fqdn]}"endtemplate /etc/hosts doowner rootgroup rootrecipes/default.rb
    • LWRPs
    • # List of all actions supported by the provideractions :create, :create_if_missing, :update, :remove# Make create the default actiondefault_action :create# Required attributesattribute :ip_address,kind_of: String,name_attribute: true,required: trueattribute :hostname, kind_of: String# Optional attributesattribute :aliases, kind_of: Arrayattribute :comment, kind_of: Stringresources/entry.rb
    • action :create do::Chef::Util::FileEdit.search_file_delete_line(entry)::Chef::Util::FileEdit.insert_line_after_match(/n/, entry)endprotecteddef entry[new_resource.ip_address, new_resource.hostname,new_resource.aliases.join( )].compact.join( ).squeeze( )endproviders/entry.rb
    • hostsfile_entry 1.2.3.4 dohostname example.comendproviders/entry.rb
    • Chef::Util::FileEdit is slow
    • Re-writing the file on each run
    • Provider kept growning
    • Untested
    • RefactorA
    • Move to pure Ruby classes
    • Ditch Chef::Util::FileEdit andmanage the entire file
    • Only implement Ruby classes inthe Provider (logic-less Provider)
    • Test the Ruby code
    • Test that the Provider implementsthe proper Ruby classes
    • TODO:Add infographicsclass Entryattr_accessor :ip_address, :hostname, :aliases, :commentdef initialize(options = {})if options[:ip_address].nil? || options[:hostname].nil?raise :ip_address and :hostname are both required optionsend@ip_address = options[:ip_address]@hostname = options[:hostname]@aliases = [options[:aliases]].flatten@comment = options[:comment]end# ...endlibraries/entry.rb
    • TODO:Add infographicsclass Manipulatordef initializecontents = ::File.readlines(hostsfile_path)@entries = contents.collect do |line|Entry.parse(line) unless line.strip.nil? || line.strip.empty?end.compactenddef add(options = {})@entries << Entry.new(ip_address: options[:ip_address],hostname: options[:hostname],aliases: options[:aliases],comment: options[:comment])endendlibraries/manipulator.rb
    • # Creates a new hosts file entry. If an entry already exists, it# will be overwritten by this one.action :create dohostsfile.add(ip_address: new_resource.ip_address,hostname: new_resource.hostname,aliases: new_resource.aliases,comment: new_resource.comment)new_resource.updated_by_last_action(true) if hostsfile.saveendproviders/entry.rb
    • RSpec
    • TODO:Add infographicsdescribe Entry dodescribe .initialize dosubject { Entry.new(ip_address: 2.3.4.5, hostname:www.example.com, aliases: [foo, bar], comment: This is acomment!, priority: 100) }it raises an exception if :ip_address is missing doexpect {Entry.new(hostname: www.example.com)}.to raise_error(ArgumentError)endit sets the ip_address doexpect(subject.ip_address).to eq(2.3.4.5)endendspec/entry_spec.rb
    • ChefSpec
    • ChefSpec
    • TODO:Add infographicsdescribe hostsfile lwrp dolet(:manipulator) { double(manipulator) }before doManipulator.stub(:new).and_return(manipulator)Manipulator.should_receive(:new).with(kind_of(Chef::Node)).and_return(manipulator)manipulator.should_receive(:save!)endlet(:chef_run) {ChefSpec::ChefRunner.new(cookbook_path: $cookbook_paths,step_into: [hostsfile_entry])}spec/default_spec.rb
    • TODO:Add infographicscontext actions dodescribe :create doit adds the entry domanipulator.should_receive(:add).with({ip_address: 2.3.4.5,hostname: www.example.com,aliases: nil,comment: nil,priority: nil})chef_run.converge(fake::create)endendendend
    • Open It
    • Gem It
    • $ bundle gem hostsfile
    • $ bundle gem hostsfilecreate hostsfile/Gemfilecreate hostsfile/Rakefilecreate hostsfile/LICENSE.txtcreate hostsfile/README.mdcreate hostsfile/.gitignorecreate hostsfile/hostsfile.gemspeccreate hostsfile/lib/hostsfile.rbcreate hostsfile/lib/hostsfile/version.rbInitializating git repo in ~Development/hostsfile
    • entry.rbmanipulator.rb99
    • 9
    • 9?
    • chef_gem hostsfilerecipes/default.rb
    • require hostsfileproviders/entry.rb
    • In another cookbook...
    • # ...depends hostsfileother_cookbook/metadata.rb
    • {"run_list": ["recipe[hostsfile]"]}www.myapp.com (Chef Node)
    • ThankYouz