Module 5 of Chef for Beginners Workshop at Optum-Chef DevOps Transformation Day in Minneapolis on April 21st 2013

1. Testing Cookbooks
Validating Our Recipes in Virtual Environments
Slide 1 of 88

2. After completing this module, you should be able to:
Use Test Kitchen to verify your recipes converge on a
virtual instance
Read the ServerSpec documentation
Write and execute tests
Objectives
Slide 2 of 88

3. As we start to define our infrastructure as code we also
need to start thinking about testing it.
Can We Test Cookbooks?
Slide 3 of 88

4. As we start to define our infrastructure as code we also
need to start thinking about testing it.
Will the recipes that we created work on
another system similar to this one? Will they
work in production?
Can We Test Cookbooks?
Slide 4 of 88

5. As we start to define our infrastructure as code we also
need to start thinking about testing it.
Will the recipes that we created work on
another system similar to this one? Will they
work in production?
"Works on my machine".
Can We Test Cookbooks?
Slide 5 of 88

6. As we start to define our infrastructure as code we also
need to start thinking about testing it.
Will the recipes that we created work on
another system similar to this one? Will they
work in production?
"Works on my machine".
So how could we solve a problem like this?
Can We Test Cookbooks?
Slide 6 of 88

7. What steps would it take to test one of the cookbooks that
we have created?
Mandating Testing
Slide 7 of 88

8. Slide 8 of 88

9. We can start by first mandating that all cookbooks are
tested.
Testing Cookbooks
Slide 9 of 88

10. We can start by first mandating that all cookbooks are
tested.
Considerations--
Testing Cookbooks
Slide 10 of 88

11. We can start by first mandating that all cookbooks are
tested.
Considerations--
How often should you test your cookbook?
Testing Cookbooks
Slide 11 of 88

12. We can start by first mandating that all cookbooks are
tested.
Considerations--
How often should you test your cookbook?
How often do you think changes will occur?
Testing Cookbooks
Slide 12 of 88

13. We can start by first mandating that all cookbooks are
tested.
Considerations--
How often should you test your cookbook?
How often do you think changes will occur?
What happens when the rate of cookbook
changes exceed the time interval it takes to
verify the cookbook?
Testing Cookbooks
Slide 13 of 88

14. An automated way to ensure code accomplishes the
intended goal and help the team understand its intent.
Code Testing
Slide 14 of 88

15. "What are we running in production? Maybe I could test
the cookbook against a virtual machine."
Test Configuration
Slide 15 of 88

16. "What are we running in production? Maybe I could test
the cookbook against a virtual machine."
Objective:
Configure the workstation cookbook to test against the
centos-6.7 platform
Test the workstation cookbook on a virtual machine
Test Configuration
Slide 16 of 88

17. Slide 17 of 88

18. What Can kitchen Do?
$kitchen--help
Commands:
kitchenconsole #KitchenConsole!
kitchenconverge[INSTANCE|REGEXP|all] #Convergeoneormoreinstances
kitchencreate[INSTANCE|REGEXP|all] #Createoneormoreinstances
kitchendestroy[INSTANCE|REGEXP|all] #Destroyoneormoreinstances
...
kitchenhelp[COMMAND] #Describeavailablecommandsoronespecif...
kitcheninit #Addssomeconfigurationtoyourcookbook...
kitchenlist[INSTANCE|REGEXP|all] #Listsoneormoreinstances
kitchensetup[INSTANCE|REGEXP|all] #Setuponeormoreinstances
kitchentest[INSTANCE|REGEXP|all] #Testoneormoreinstances
kitchenverify[INSTANCE|REGEXP|all] #Verifyoneormoreinstances
kitchenversion #PrintKitchen'sversioninformation
Slide 18 of 88

19. What Can kitchen init Do?
$kitchenhelpinit
Usage:
kitcheninit
-D,[--driver=onetwothree] #OneormoreKitchenDrivergems...
#Default:kitchen-vagrant
-P,[--provisioner=PROVISIONER] #ThedefaultKitchenProvisionertouse
#Default:chef_solo
[--create-gemfile],[--no-create-gemfile] #WhetherornottocreateaGemfi...
Description:
InitwilladdTestKitchensupporttoanexistingprojectforconvergence
integrationtesting.Adefault.kitchen.ymlfile(whichisintendedtobe
customized)iscreatedintheproject'srootdirectoryandoneormoregems
willbeaddedtotheproject'sGemfile.
Slide 19 of 88

20. Do We Have a .kitchen.yml?
$treecookbooks/workstation-a-I.git
workstation
├──Berksfile
├──chefignore
├──.gitignore
├──.kitchen.yml
├──metadata.rb
├──README.md
├──recipes
│ ├──default.rb
│ └──setup.rb
├──spec
│ ├──spec_helper.rb
│ └──unit
...
Slide 20 of 88

21. What is Inside .kitchen.yml?
$catcookbooks/workstation/.kitchen.yml
---
driver:
name:vagrant
provisioner:
name:chef_zero
platforms:
-name:ubuntu-14.04
-name:centos-7.1
suites:
-name:default
Slide 21 of 88

22. When chef generates a cookbook, a default
.kitchen.ymlis created.
It contains kitchen configuration for the driver,
provisioner, platform, and suites.
.kitchen.yml
http://kitchen.ci/docs/getting-started/creating-cookbook
Slide 22 of 88

23. Demo: The kitchen Driver
~/cookbooks/workstation/.kitchen.yml
---
driver:
name:vagrant
provisioner:
name:chef_zero
platforms:
-name:ubuntu-14.04
-name:centos-7.1
...
The driver is responsible for creating a machine that we'll use to test our
cookbook.
Example Drivers:
docker
vagrant
Slide 23 of 88

24. Demo: The kitchen Provisioner
~/cookbooks/workstation/.kitchen.yml
---
driver:
name:vagrant
provisioner:
name:chef_zero
platforms:
-name:ubuntu-14.04
-name:centos-7.1
...
This tells Test Kitchen how to run Chef to apply the code in our cookbook to
the machine under test.
The default and simplest approach is to use chef_zero.
Slide 24 of 88

25. Demo: The kitchen Platforms
~/cookbooks/workstation/.kitchen.yml
---
driver:
name:vagrant
provisioner:
name:chef_zero
platforms:
-name:ubuntu-14.04
-name:centos-7.1
...
This is a list of operation systems on which we want to run our code.
Slide 25 of 88

26. Demo: The kitchen Suites
~/cookbooks/workstation/.kitchen.yml
...
suites:
-name:default
run_list:
-recipe[workstation::default]
attributes:
This section defines what we want to test. It includes the Chef run-listof
recipes that we want to test.
We define a single suite named default.
Slide 26 of 88

27. Demo: The kitchen Suites
~/cookbooks/workstation/.kitchen.yml
...
suites:
-name:default
run_list:
-recipe[workstation::default]
attributes:
The suite named default defines a run_list.
Run the workstation cookbook's default recipe file.
Slide 27 of 88

28. Kitchen defines a list of instances, or test matrix,
based on the platforms multiplied by the suites.
PLATFORMS x SUITES
Running kitchen list will show that matrix.
Kitchen Test Matrix
Slide 28 of 88

29. suites:
-name:default
run_list:
-recipe[workstation::default]
attributes:
platforms:
-name:ubuntu-12.04
-name:centos-6.5
Example: Kitchen Test Matrix
$kitchenlist
Instance Driver Provisioner Verifier TransportLastAction
default-ubuntu-1204 Vagrant ChefZero Busser Ssh <NotCreated>
default-centos-65 Vagrant ChefZero Busser Ssh <NotCreated>
Slide 29 of 88

30. suites:
-name:default
run_list:
-recipe[workstation::default]
attributes:
platforms:
-name:ubuntu-12.04
-name:centos-6.5
Example: Kitchen Test Matrix
$kitchenlist
Instance Driver Provisioner Verifier TransportLastAction
default-ubuntu-1204 Vagrant ChefZero Busser Ssh <NotCreated>
default-centos-65 Vagrant ChefZero Busser Ssh <NotCreated>
Slide 30 of 88

31. "What are we running in production? Maybe I could test
the cookbook against a virtual machine."
Group Exercise: Test Configuration
Slide 31 of 88

32. "What are we running in production? Maybe I could test
the cookbook against a virtual machine."
Objective:
Configure the workstation cookbook's .kitchen.ymlto
use the Docker driver and centos 6.7 platform.
Use kitchen converge to apply the recipe on a virtual
machine.
Group Exercise: Test Configuration
Slide 32 of 88

33. Group Exercise: Move into the Cookbook's Directory
$cd~/cookbooks/workstation
Slide 33 of 88

34. Group Exercise: Edit the Kitchen Configuration File
~/cookbooks/workstation/.kitchen.yml
---
driver:
name:docker
provisioner:
name:chef_zero
platforms:
-name:centos-6.7
suites:
#...REMAINDEROFFILE...
https://github.com/portertech/kitchen-docker
Slide 34 of 88

35. Group Exercise: Edit the Kitchen Configuration File
~/cookbooks/workstation/.kitchen.yml
---
driver:
name:docker
provisioner:
name:chef_zero
platforms:
-name:centos-6.7
suites:
#...REMAINDEROFFILE...
https://www.centos.org
Slide 35 of 88

36. Group Exercise: Look at the Test Matrix
$kitchenlist
Instance Driver Provisioner Verifier Transport LastAction
default-centos-67 Docker ChefZero Busser Ssh <NotCreated>
Slide 36 of 88

37. "Before I add features it really would be nice to test these
cookbooks against the environments that resemble
production."
Converging a Cookbook
Slide 37 of 88

38. "Before I add features it really would be nice to test these
cookbooks against the environments that resemble
production."
Objective:
Configure the workstation cookbook's .kitchen.ymlto
use the Docker driver and centos-6.7 platform
Use kitchen converge to apply the recipe on a virtual
machine
Converging a Cookbook
Slide 38 of 88

39. Kitchen Create
$kitchencreate[INSTANCE|REGEXP|all]
Create one or more instances.
Slide 39 of 88

40. Group Exercise: Kitchen Converge
$kitchenconverge[INSTANCE|REGEXP|all]
Create the instance (if necessary) and then apply the run list to one or more
instances.
Slide 40 of 88

41. Group Exercise: Converge the Cookbook
$cd~/cookbooks/workstation
$kitchenconverge
----->StartingKitchen(v1.4.0)
----->Creating<default-centos-67>...
SendingbuildcontexttoDockerdaemon 2.56kB
(skipping)
-----> Finishedcreating<default-centos-67>(1m18.32s).
----->Converging<default-centos-67>...
$$$$$$Runninglegacyconvergefor'Docker'Driver
(skipping)
SynchronizingCookbooks:
-workstation
CompilingCookbooks...
Converging0resources
Runninghandlers:
Slide 41 of 88

42. We want to validate that our run-list installs correctly.
Within the apache cookbook, use kitchen converge for
the default suite on the centos 6.7 platform.
Lab: Converge the Recipe for apache
Slide 42 of 88

43. Lab: Configuring Test Kitchen for apache
~/cookbooks/apache/.kitchen.yml
---
driver:
name:docker
provisioner:
name:chef_zero
platforms:
-name:centos-6.7
suites:
-name:default
run_list:
-recipe[apache::default]
attributes:
Slide 43 of 88

44. Lab: Converge the apache Cookbook
$cd~/cookbooks/apache
$kitchenconverge
----->StartingKitchen(v1.4.0)
----->Creating<default-centos-67>...
SendingbuildcontexttoDockerdaemon 2.56kB
SendingbuildcontexttoDockerdaemon
(skipping)
InstallingChef
installingwithrpm...
warning:/tmp/install.sh.23/chef-12.4.1-1.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID83ef826a:NOKEY
(skipping)
SynchronizingCookbooks:
-apache
CompilingCookbooks...
Slide 44 of 88

45. What is being tested when kitchen converges a recipe
without error?
Test Kitchen
Slide 45 of 88

46. What is being tested when kitchen converges a recipe
without error?
What is NOT being tested when kitchen converges the
recipe without error?
Test Kitchen
Slide 46 of 88

47. What is being tested when kitchen converges a recipe
without error?
What is NOT being tested when kitchen converges the
recipe without error?
What is left to validate to ensure that the cookbook
successfully applied the policy defined in the recipe?
Test Kitchen
Slide 47 of 88

48. "Converging seems to validate that the recipe runs
successfully. But does it assert what actually is installed?"
The First Test
Slide 48 of 88

49. "Converging seems to validate that the recipe runs
successfully. But does it assert what actually is installed?"
Objective:
In a few minutes we'll write and execute a test that
asserts that the tree package is installed when the
workstation cookbook's default recipe is applied.
The First Test
Slide 49 of 88

50. Kitchen Verify
$kitchenverify[INSTANCE|REGEXP|all]
Create, converge, and verify one or more instances.
Slide 50 of 88

51. Kitchen Destroy
$kitchendestroy[INSTANCE|REGEXP|all]
Destroys one or more instances.
Slide 51 of 88

52. Kitchen Test
$kitchentest[INSTANCE|REGEXP|all]
Destroys (for clean-up), creates, converges, verifies and then destroys one or
more instances.
Slide 52 of 88

53. Serverspec tests your servers' actual state by
executing command locally, via SSH, via WinRM, via
Docker API and so on.
So you don't need to install any agent software on
your servers and can use any configuration
management tools, Puppet, Chef, CFEngine, Itamae
and so on.
Serverspec
http://serverspec.org
Slide 53 of 88

54. Example
Is the tree package installed?
describepackage('tree')do
it{shouldbe_installed}
end
I expect the package tree should be installed.
http://serverspec.org/resource_types.html#package
Slide 54 of 88

55. Group Exercise: Requiring a Test Helper
~/cookbooks/workstation/test/integration/default/serverspec/default_spec.rb
require'spec_helper'
describe'workstation::default'do
describepackage('tree')do
it{shouldbe_installed}
end
end
Loads a helper file with that name in the same directory.
http://kitchen.ci/docs/getting-started/writing-test
Slide 55 of 88

56. Group Exercise: Describing the Test Context
~/cookbooks/workstation/test/integration/default/serverspec/default_spec.rb
require'spec_helper'
describe'workstation::default'do
describepackage('tree')do
it{shouldbe_installed}
end
end
Describes a body of tests for the workstation cookbook's default recipe.
https://relishapp.com/rspec/rspec-core/v/3-3/docs
Slide 56 of 88

57. Group Exercise: Our Assertion in a spec File
~/cookbooks/workstation/test/integration/default/serverspec/default_spec.rb
require'spec_helper'
describe'workstation::default'do
describepackage('tree')do
it{shouldbe_installed}
end
end
When we converge the workstation cookbook's default recipe we expect the
tree package to be installed.
http://serverspec.org/resource_types.html#package
Slide 57 of 88

58. workstation/test/ /default/serverspec/default_spec.rb
Test Kitchen will look for tests to run under this directory.
It allows you to put unit or other tests in test/unit, spec,
acceptance, or wherever without mixing them up. This is
configurable, if desired.
Where do Tests Live?
integration
http://kitchen.ci/docs/getting-started/writing-test
Slide 58 of 88

59. workstation/test/integration/ /serverspec/default_spec.rb
This corresponds to the name of the test suite that is
defined in the .kitchen.ymlfile.
In our case the name of the suite is default so when
Test Kitchen performs a kitchen verify for the default
suite it will look within the defaultfolder for the
specifications to run.
Where do Tests Live?
default
http://kitchen.ci/docs/getting-started/writing-test
Slide 59 of 88

60. workstation/test/integration/default/ /default_spec.rb
This tells Test Kitchen that we wish to use Serverspec
framework for testing.
Where do Tests Live?
serverspec
http://kitchen.ci/docs/getting-started/writing-test
Slide 60 of 88

61. workstation/test/integration/default/serverspec/
All test files (or specs) are named after the recipe they
test and end with the suffix _spec.rb.
A spec missing that will not be found when executing
kitchen verify.
Where do Tests Live?
default_spec.rb
http://kitchen.ci/docs/getting-started/writing-test
Slide 61 of 88

62. Group Exercise: Move into the Cookbook
$cd~/cookbooks/workstation
Slide 62 of 88

63. Group Exercise: Running the Specification
$kitchenverify
----->StartingKitchen(v1.4.0)
----->Converging<default-centos-67>...
$$$$$$Runninglegacyconvergefor'Docker'Driver
(skipping)
----->ChefOmnibusinstallationdetected(installonlyifmissing)
Transferringfilesto<default-centos-67>
StartingChefClient,version12.4.1
(skipping)
Runninghandlers:
Runninghandlerscomplete
ChefClientfinished,6/6resourcesupdatedin64.426896317seconds
Finishedconverging<default-centos-67>(1m9.02s).
----->Kitchenisfinished.(1m9.69s)
Slide 63 of 88

64. Group Exercise: Commit Your Work
$cd~/cookbooks/workstation
$gitadd.
$gitstatus
$gitcommit-m"Addedfirsttestforthedefaultrecipe"
Slide 64 of 88

65. More Tests
What are other resources within the recipe that we could test?
Slide 65 of 88

66. Serverspec can help us assert different characteristics
about files on the file system. Like if it is a file,
directory, socket or symlink.
The file's mode owner or group. If the file is readable,
writeable, or executable. It is even able to verify the
data contained within the file.
Testing a File
http://serverspec.org/resource_types.html#file
Slide 66 of 88

67. Example: The File Contains Data
describefile('/etc/passwd')do
it{shouldbe_file}
end
I expect the file named /etc/passwdto be a file (as opposed to a directory,
socket or symlink).
http://serverspec.org/resource_types.html#file
Slide 67 of 88

68. Example: The File Contains Specific Content
describefile('/etc/httpd/conf/httpd.conf')do
its(:content){shouldmatch/ServerNamewww.example.jp/}
end
I expect the file named /etc/httpd/conf/httpd.confto have content that
matches ServerNamewww.example.jp
http://serverspec.org/resource_types.html#file
Slide 68 of 88

69. Example: The File is Owned by a Particular User
describefile('/etc/sudoers')do
it{shouldbe_owned_by'root'}
end
I expect the file named /etc/sudoersto be owned by the rootuser.
Slide 69 of 88

70. Add tests that validate that the remaining package
resources have been installed
(http://serverspec.org/resource_types.html#package)
Add tests that validate the file resource
(http://serverspec.org/resource_types.html#file)
Run kitchen verify to validate the test meets the
expectations that you defined
Commit your changes
Lab: More Tests
Slide 70 of 88

71. Lab: Our Assertion in a spec File
~/cookbooks/workstation/test/integration/default/serverspec/default_spec.rb
require'spec_helper'
describe'workstation::default'do
#...othertestsforpackages...
describepackage('tree')do
it{shouldbe_installed}
end
describepackage('git')do
it{shouldbe_installed}
end
end
The package named git is installed.
http://serverspec.org/resource_types.html#package
Slide 71 of 88

72. Lab: Our Assertion in a spec File
~/cookbooks/workstation/test/integration/default/serverspec/default_spec.rb
...
describepackage('git')do
it{shouldbe_installed}
end
describefile('/etc/motd')do
it{shouldbe_owned_by'root'}
end
end
The file named /etc/motdshould be owned by root.
http://serverspec.org/resource_types.html#file
Slide 72 of 88

73. Group Exercise: Return to the Cookbook Directory
$cd~/cookbooks/workstation
Slide 73 of 88

74. Lab: Running the Specification
$kitchenverify
----->StartingKitchen(v1.4.0)
----->Converging<default-centos-67>...
$$$$$$Runninglegacyconvergefor'Docker'Driver
(skipping)
----->ChefOmnibusinstallationdetected(installonlyifmissing)
Transferringfilesto<default-centos-67>
StartingChefClient,version12.4.1
(skipping)
Runninghandlers:
Runninghandlerscomplete
ChefClientfinished,6/6resourcesupdatedin64.426896317seconds
Finishedconverging<default-centos-67>(1m9.02s).
----->Kitchenisfinished.(1m9.69s)
Slide 74 of 88

75. Lab: Commit Your Work
$cd~/cookbooks/workstation
$gitadd.
$gitstatus
$gitcommit-m"Addedadditionaltestsfordefaultrecipe"
Slide 75 of 88

76. "I would love to know that the webserver is installed and
running correctly."
Testing Our Webserver
Slide 76 of 88

77. "I would love to know that the webserver is installed and
running correctly."
Objective:
Discuss and decide what should be tested with the
apache cookbook
Testing Our Webserver
Slide 77 of 88

78. What are some things we could test to validate our
web server has deployed correctly?
What manual tests do we use now to validate a
working web server?
Testing
Slide 78 of 88

79. Create a test file for the apache cookbook's default
recipe
Add tests that validate a working web server
http://serverspec.org/resource_types.html#port
http://serverspec.org/resource_types.html#command
Run kitchen verify
Commit your changes
Lab: Testing Apache
Slide 79 of 88

80. Lab: Switch to the apache cookbook
$cd~/cookbooks/apache
Slide 80 of 88

81. Lab: What Does the Webserver Say?
~/cookbooks/apache/test/integration/default/serverspec/default_spec.rb
require'spec_helper'
describe'apache::default'do
describeport(80)do
it{shouldbe_listening}
end
describecommand('curlhttp://localhost')do
its(:stdout){shouldmatch/Hello,world!/}
end
end
Port 80 should be listening.
The standard out from the command curl http://localhost should match
'Hello, world!'
Slide 81 of 88

82. Lab: Commit Your Work
$cd~/cookbooks/apache
$gitadd.
$gitstatus
$gitcommit-m"Addedtestsforthedefaultrecipe"
Slide 82 of 88

83. Why do you have to run kitchen within the directory
of the cookbook?
Discussion
Slide 83 of 88

84. Why do you have to run kitchen within the directory
of the cookbook?
Where would you define additional platforms?
Discussion
Slide 84 of 88

85. Why do you have to run kitchen within the directory
of the cookbook?
Where would you define additional platforms?
Why would you define a new test suite?
Discussion
Slide 85 of 88

86. Why do you have to run kitchen within the directory
of the cookbook?
Where would you define additional platforms?
Why would you define a new test suite?
What are the limitations of using Test Kitchen to
validate recipes?
Discussion
Slide 86 of 88

87. Q&A
What questions can we help you answer?
Test Kitchen
kitchen commands
kitchen configuration
Serverspec
Slide 87 of 88

88. Slide 88 of 88