Test Kitchen is a test harness to execute infrastructure code on one or more platforms in isolation. Easily create, automate, verify, and destroy infrastructure using Test Kitchen in your automation engineering development process. InSpec is compliance as code. Turn your compliance, security, and other policy requirements into automated tests. This talk provides a brief introduction to the Test Kitchen lifecycle and explores using InSpec to validate the infrastructure created by Test Kitchen. There is also a brief exploration of using InSpec for validating compliance controls. This talk was given at the Baltimore Devops Meetup in July, 2018 - https://www.meetup.com/BaltimoreDevOps/events/wcrxlpyxlblb/ Videos are embedded where there would normally be live demo examples. Test Kitchen - https://kitchen.ci/ InSpec - https://www.inspec.io/ Code repositories used - https://github.com/nathenharvey/intro-to-test-kitchen - https://github.com/nathenharvey/testing-ansible-with-inspec

1. @nathenharvey
Introduction to Test Kitchen

2. @nathenharvey
Hello!
Nathen Harvey
VP, Community
Development
Chef
@nathenharvey

3. @nathenharvey
Why Testing?
• Move fast with a safety net
• Decrease feedback cycles
• Increase confidence
• Prevent regressions
• Scale

4. @nathenharvey
Developing Infrastructure Automation
• Write the automation
• Static analysis
• Unit testing
• Integration testing
• Commit to version control
• Submit to pipeline

5. @nathenharvey
Zoom-in on Integration Testing
• Create infrastructure that matches production
• Run the automation
• Verify the results
• Destroy the infrastructure

6. @nathenharvey
Test Kitchen
Test harness to execute
infrastructure code on one or
more platforms in isolation.

7. @nathenharvey
Before we continue…
$ kitchen verify -c 2

8. @nathenharvey
The Test Kitchen
platforms:
- name: ubuntu-16.04
- name: centos-7
platforms

9. @nathenharvey
The Test Kitchen
platforms:
- name: ubuntu-16.04
- name: centos-7
kitchen create
platforms

10. @nathenharvey
The Test Kitchen
driver:
name: gce
platforms:
- name: ubuntu-16.04
- name: centos-7
kitchen create driver
platforms

11. @nathenharvey
Zoom-in on Integration Testing
Create infrastructure that matches production
• Run the automation
• Verify the results
• Destroy the infrastructure

12. @nathenharvey
The Test Kitchen
driver:
name: vagrant
platforms:
- name: ubuntu-16.04
- name: centos-7
kitchen create driver
platforms
kitchen converge

13. @nathenharvey
The Test Kitchen
driver:
name: vagrant
platforms:
- name: ubuntu-16.04
- name: centos-7
provisioner:
name: chef_zero
kitchen create driver
platforms
kitchen converge provisioner

14. @nathenharvey
Chef Recipe
package 'git' do
action :install
end

15. @nathenharvey
Zoom-in on Integration Testing
Create infrastructure that matches production
Run the automation
• Verify the results
• Destroy the infrastructure

17. @nathenharvey
InSpec
• Open-source framework
• Infrastructure testing
• Make assertions about state of resources in the infrastructure

18. @nathenharvey
Sample InSpec Code
describe service('apache2') do
it { should be_running }
end
describe port(80) do
it { should be_listening }
end
describe http('http://localhost', enable_remote_worker: true) do
its('status') { should cmp 200 }
its('body') { should match /Welcome to / }
end

19. @nathenharvey
The Test Kitchen
driver:
name: vagrant
platforms:
- name: ubuntu-16.04
- name: centos-7
provisioner:
name: chef_zero
kitchen create driver
platforms
kitchen converge provisioner
kitchen verify

20. @nathenharvey
The Test Kitchen
driver:
name: vagrant
platforms:
- name: ubuntu-16.04
- name: centos-7
provisioner:
name: chef_zero
verifier:
name: inspec
kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier

21. @nathenharvey
Verify the Results with InSpec
describe package('git') do
it { should be_installed }
end
describe command('git') do
it { should exist }
end
describe command('which git') do
its('exit_status') { should eq 0 }
end

22. @nathenharvey
Zoom-in on Integration Testing
Create infrastructure that matches production
Run the automation
Verify the results
• Destroy the infrastructure

23. @nathenharvey
The Test Kitchen
driver:
name: vagrant
platforms:
- name: ubuntu-16.04
- name: centos-7
provisioner:
name: chef_zero
verifier:
name: inspec
kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy

24. @nathenharvey
Zoom-in on Integration Testing
Create infrastructure that matches production
Run the automation
Verify the results
Destroy the infrastructure

25. @nathenharvey
The Test Kitchen
kitchen test kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy

26. @nathenharvey
The Test Kitchen
kitchen destroy
kitchen test kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy

27. @nathenharvey
The Test Kitchen
kitchen destroy
kitchen create
kitchen test kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy

28. @nathenharvey
The Test Kitchen
kitchen destroy
kitchen create
kitchen converge
kitchen test kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy

29. @nathenharvey
The Test Kitchen
kitchen destroy
kitchen create
kitchen converge
kitchen verify
kitchen test kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy

30. @nathenharvey
The Test Kitchen
kitchen create driver
platforms
kitchen converge provisioner
kitchen verify verifier
kitchen destroy
kitchen destroy
kitchen create
kitchen converge
kitchen verify
kitchen destroy
kitchen test

31. @nathenharvey
More Test Kitchen <3

32. @nathenharvey
The Test Kitchen
driver:
name: kitchen create driver

33. @nathenharvey
Remember this?
$ kitchen verify -c 2
Let’s go check-in on it…
$ kitchen list

34. @nathenharvey
Kitchen List

35. @nathenharvey
The Test Kitchen
provisioner:
name:
kitchen converge provisioner

36. @nathenharvey
The Test Kitchen
verifier:
name:
kitchen verify verifier

37. @nathenharvey
Compliance Testing

38. @nathenharvey

39. @nathenharvey
InSpec to Detect Policy Violations
• InSpec is great for integration testing
• But it can also be used for security or compliance checks

40. Automate Test Execution
describe ini('/etc/tac_plus/tac_plus.conf') do
its('key') { should_not be_nil }
end
404.3.5:
Communication
between network
devices and central
authentication systems
must be encrypted at
all times.

41. Map Documentation to Controls
control 'sox-404.3.5' do
title 'Network Device to Central Auth Encryption'
impact 1.0
desc "
All communication between network devices and
central auth must be encrypted. Our TACACS+ servers
encrypt all the time and the presence of a
pre-shared key proves it."
describe ini('/etc/tac_plus/tac_plus.conf') do
its('key') { should_not be_nil }
end
end
404.3.5:
Communication
between network
devices and central
authentication systems
must be encrypted at
all times.

42. Share Context
control 'sox-404.3.5' do
title 'Network Device to Central Auth Encryption'
impact 1.0
desc "
All communication between network devices and
central auth must be encrypted. Our TACACS+ servers
encrypt all the time and the presence of a
pre-shared key proves it."
describe ini('/etc/tac_plus/tac_plus.conf') do
its('key') { should_not be_nil }
end
end
404.3.5:
Communication
between network
devices and central
authentication systems
must be encrypted at
all times.

43. @nathenharvey
dev-sec.io

44. @nathenharvey
Add Linux Baseline to Test Kitchen
suites:
- name: default
verifier:
inspec_tests:
- test/integration/default
- https://github.com/dev-sec/linux-baseline

45. @nathenharvey
So many failures
• Stop when the build breaks
• We need to get to green
• What is the best way to get the build green?

46. @nathenharvey
Wrap it up
• Create a TODO list
• One measure of technical debt
• Get to green by commenting out tests?!

47. @nathenharvey
Wrapper Profile
name: my-linux-baseline
title: InSpec Profile
maintainer: The Authors
copyright: The Authors
copyright_email: you@example.com
license: Apache-2.0
summary: An InSpec Compliance Profile
version: 0.1.0
depends:
- name: linux-baseline
url: https://github.com/dev-sec/linux-baseline/archive/master.tar.gz

48. @nathenharvey
Wrapper Profile
include_controls 'linux-baseline' do
skip_control 'os-05'
skip_control 'package-08'
skip_control 'sysctl-05'
...
end

49. @nathenharvey
Hardening Playbook

50. @nathenharvey
- hosts: all
become: true
become_user: root
become_method: sudo
roles:
- { role: apache }
- { role: dev-sec.os-hardening }
Add to the Playbook

51. @nathenharvey
One Commit

52. @nathenharvey

53. @nathenharvey

54. @nathenharvey
Use InSpec to Verify Terraform-created Infrastructure
https://www.slideshare.net/nathenharvey/testing-terraform-102777946

56. @nathenharvey
September 10 & 11

57. @nathenharvey

58. @nathenharvey

59. @nathenharvey
Get Started with Test Kitchen
• Install Chef Development Kit - https://downloads.chef.io/chefdk
Test Kitchen
InSpec
• Install Driver Requirements
Vagrant – VirtualBox & Vagrant
Docker – Docker
GCE – None, but best to Google Cloud SDK installed
EC2 – None, but you need an AWS account

60. @nathenharvey
Use, Share, Contribute!
• Test Kitchen
https://kitchen.ci/
https://github.com/test-kitchen
• InSpec
https://www.inspec.io/
https://github.com/chef/inspec
• Code from this presentation
https://github.com/nathenharvey/intro-to-test-kitchen
https://github.com/nathenharvey/testing-ansible-with-inspec
https://github.com/nathenharvey/testing-terraform

61. @nathenharvey
Join us on Slack
• http://community-slack.chef.io
• #general (for Chef stuff)
• #test-kitchen
• #inspec
The Chef community
believes that diversity is one
of our biggest strengths!
YOU are welcome here!

62. @nathenharvey
Local Technology Slacks
• Baltimore
https://baltimoretech-slack.herokuapp.com/
• Washington DC
http://www.dctechslack.com/
Join a local technology
slack, or two, to help
maintain connections across
the community!

63. @nathenharvey
What questions can I answer for you?
Nathen Harvey
VP, Community
Development
Chef
@nathenharvey