An overview of the Chef, automation testing, and best practices.

1. Dive into CHEF
Eduardo S. Scarpellini
@escarpellini
Aug, 2016

2. Agenda
- Architecture;
- Main tools;
- Cooking;
- cookbooks, recipes, resources, templates, etc;
- Knife;
- LWRPs;
- Tests;
- Best practices;

3. Architecture

4. Main tools
● Development workstation:
○ chef-dk;
■ cookbook standardized skeletons, linting and testing tools, style checking;
○ knife;
■ swiss knife: cookbook/node management, search, many plugins (AWS, OpenStack,
VMWare, etc);
○ chef-kitchen (or any other testing tool);
■ runs acceptance criterias in a brand new/isolated environment (docker, vagrant, etc);
● Nodes:
○ chef-client;
○ ohai;
■ node attributes: platform and kernel details, network configuration, hostname, cpu and
memory data, etc;

5. Cookbook building blocks
● Metadata;
○ author, version, dependencies, documentation, etc;
● Resources;
○ desired state, several types (package, service), name+parameters, notifications/subscriptions;
● Attributes;
○ global hash table, precedence/override;
● Files and templates;
○ static or dynamic files (erb templates) that should be synchronized with nodes;
● Recipes;
○ collection of resources;
● Libraries, definitions and LWRPs;
○ custom resource types, ruby functions;

6. Cookbook anatomy
chef generate <cookbook, recipe, attribute, lwrp, ...>
attributes
files repository
recipes
templates repository
cookbook

7. Recipes: basics
● Ruby DSL;
● Collection of resources;
○ package, cookbook_file, directory,
template, service, execute, user,
group, cron, dsc_resource, etc.
● Interaction with attributes;
○ user defined or ohai;
● Everything is evaluated in the
same order that they are
declared;
● Can include other recipes;
{yum install …
apt-get install …
msiexec.exe /i …
type + name
}parameters
notification
{service …
systemctl …
sc.exe …
myapp/recipes/default.rb

8. Recipes: a bit more complex
ohai attribute
Powerful of Ruby: syntax,
variables, data structures,
conditionals, loopings,
libraries, gems, etc.
include external code
local variables
attribute
notification timer
Requires attribute definition:
myapp/recipes/default.rb
myapp/attributes/default.rb

9. Templates
ohai attributeCombine plain text with
Ruby code (<% … %>) for
variable substitution and
flow control. loop + attribute +
local variables
myapp/templates/default/haproxy.cfg.erb
myapp/attributes/default.rb
haproxy.cfg

10. What else...
● Attributes:
○ defined/accessed in/from attribute files, recipes, roles, environments and/or node JSONs;
○ precedence/overwritten (node.override > node.set > node.default);
○ accessed via node['key'] or node.key
● Data bags:
○ global variables stored as JSON files;
○ can be encrypted (suitable for passwords, shared keys, etc);
○ accessed via Chef::EncryptedDataBagItem.load('bag_name', 'bag_item')
● Roles describes nodes (group cookbooks, overwrite attributes);
● Nodes (JSON):
○ Node attributes (ohai + defined by cookbooks);
○ run_list: list (array) of roles, cookbooks and/or recipes that will run on the node.

11. Knife
● Provides an interface between a local chef-repo and the Chef server;
● Extensible;
● Helps to manage:
○ nodes:
■ knife node <create, edit, delete ,...>
○ cookbooks, roles and data bags:
■ knife cookbook <create, delete, upload, ...>
■ knife role <create, edit, delete, ...>
■ knife data bag <create, edit, delete, ...>
○ cloud/hypervisors/servers (plugins):
■ knife ec2 server create…
■ knife bootstrap <FQDN>…

12. LWRPs
● Is a simple extension of Chef;
● Is implemented as part of a cookbook;
● Follows easy, repeatable syntax patterns;
● Effectively leverages resources that are built into Chef;
● Is reusable in the same way as built-in resources;

13. LWRPs
myapp/providers/hello_world.rbmyapp/resources/hello_world.rb
resource parameter
usage:
resource name
resource action
resource
attributes

14. Kitchen
● Test cookbook data across any combination of platforms and test suites;
● Defined in a YAML file (.kitchen.yaml);
● Uses a driver plugin architecture;
○ many cloud providers and virtualization technologies
■ AWS, VMWare, Docker, Vagrant, openstack, etc;
○ automatic chef-server provisioning;
■ environments, roles, data bags, dependencies (via Berkshelf), etc;
● Supports all common testing frameworks that are used by the Ruby
community;
○ bats (bash), minitest, rspec, serverspec;

15. Kitchen will run all
.bats files against all
specified platforms
Kitchen
myapp/.kitchen.yml
test/integration/default/bats/lb_haproxy.bats
{
driver plugin
run_list +
attributes
bash commands

16. Kitchen
kitchen test
successful
fail
platform/instance

17. Best practices
● Plan in advance;
○ environments, roles, data bags and attributes hierarchy, naming conventions;
● Use foodcritic integrated to your favorite editor (sublime, atom);
● Versioning your cookbooks (at scm and at chef-server level);
○ use scm tags;
○ use one repository for each cookbook;
○ freeze stable versions on chef-server;
○ specify versions of the dependencies on metadata.rb;
○ specify versions of the application cookbooks either on roles or run_list;
● Use Berkshelf to manage your dependencies;
● Use search(...) instead hard-coded attributes whenever it's possible;
○ load-balancer members, cluster members, etc;

18. Best practices
● Build cookbooks that support external templates;
● Avoid repetition: use LWRPs;
● Don't reinvent the wheel: https://supermarket.chef.io/
○ don't fork;
○ build wrapper cookbooks that match your needs;
○ replace original templates when it's needed;
● Write/run tests;
○ rely on chef built-in resources and write tests only for your code;
● Let your CI tool take care of your publishing process;
○ checkout + run tests + version bump + cookbook upload;

19. Going further
● Getting started:
○ https://docs.chef.io/chef_overview.html
○ https://www.linode.com/docs/applications/chef/beginners-guide-chef
○ https://www.digitalocean.com/community/tutorial_series/getting-started-managing-your-infrastr
ucture-using-chef
● Books:
○ Chef Infrastructure Automation Cookbook;
○ Learning Chef: A Guide to Configuration Management and Automation
○ Customizing Chef;

20. Questions?