Including Compliance & Security
Checks in the Delivery Pipeline
Clinton Wolfe | Chef - Sr. Software Engineer
Nicole Johnson | Chef - Manager, Solutions Architects
East
Chef is...
● An automation framework that enables Infrastructure as Code
● A robust set of tooling for testing Chef code
● A large library of reusable patterns (supermarket.chef.io)
● Available for Linux variants, Unix variants, and Windows, all as first class
citizens.
CHEF IS INFRASTRUCTURE AS CODE
Programmatically provision
and configure components
Treat like any other
code base
Reconstruct business from code repository,
data backup, and compute resources
Chef
▪ Manages deployment
and on-going automation
▪ Define reusable resources
and infrastructure state as code
▪ Scale elegantly from one to tens of
thousands of managed nodes across
multiple complex environments
▪ Community, Certified Partner, and
Chef supported content available
for all common automation tasks
Infrastructure automation
and delivery at scale
windows_feature ‘IIS-WebServerRole’ do
action :install
end
windows_feature ‘IIS-ASPNET’ do
action :install
end
iis_pool FooBarPool do
runtime_version “4.0”
action :add
end
package "apache" do
action :install
end
template “/etc/httpd/https.conf” do
source “httpd.conf.erb”
mode 0075
owner “root”
group “root”
end
service “apache2” do
action :start
end
PART OF A PROCESS OF CONTINUOUS COMPLIANCE
Scan for
Compliance
Build & Test
Locally
Build & Test
CI/CD Remediate Verify
A SIMPLE EXAMPLE OF AN INSPEC CIS RULE
InSpec
▪ Translate compliance into Code
▪ Clearly express statements of policy
▪ Move risk to build/test from runtime
▪ Find issues early
▪ Write code quickly
▪ Run code anywhere
▪ Inspect machines, data and APIs
Turn security and
compliance into code
control ‘cis-1.4.1’ do
title ‘1.4.1 Enable SELinux in /etc/grub.conf’
desc ‘
Do not disable SELinux and enforcing in your
GRUB configuration. These are important security features that
prevent attackers from escalating their access to your systems.
For reference see …
‘
impact 1.0
expect(grub_conf.param ‘selinux’).to_not eq ‘0’
expect(grub_conf.param ‘enforcing’).to_not eq ‘0’
end
InSpec
Compliance as Code
The Old Way
People working directly on machines
InSpec - Security meets Operations
The Modern Way
People automating machines using code
DEVOPS
SECURITY
DEVOPS
COMPLIANCE
The New Way
Shared tooling across organizations
Differences in verifying compliance policy
> grep "^Protocol"
/etc/ssh/sshd_config | sed
's/Protocol //'
2
DOCUMENTATION
SSH supports two different protocol versions. The original version, SSHv1, was subject to a number of
security issues. Please use SSHv2 instead to avoid these.
describe sshd_config do
its('Protocol') { should eq 2 }
end
control 'ssh-1234' do
impact 1.0
title 'Server: Set protocol version to SSHv2'
desc "
Set the SSH protocol version to 2. Don't use legacy
insecure SSHv1 connections anymore...
"
describe sshd_config do
its('Protocol') { should eq 2 }
end
end
SCRIPTING TOOLS
TESTING LANGUAGE
COMPLIANCE LANGUAGE
ONE LANGUAGE
InSpec
InSpec on Windows Example
control 'windows-base-201' do
impact 1.0
title 'Strong Windows NTLMv2 Authentication Enabled; Weak LM
Disabled'
desc '
@link: http://support.microsoft.com/en-us/kb/823659
'
describe registry_key
('HKLMSystemCurrentControlSetControlLsa') do
it { should exist }
its('LmCompatibilityLevel') { should eq 4 }
end
end
InSpec
● Baremetal
● VMs
● Containers
ONE LANGUAGE
> inspec exec test.rb
Test a machine remotely via SSH
> inspec exec test.rb -i identity.key -t ssh://root@172.17.0.1
Test your machine locally
> inspec exec test.rb -t winrm://Admin@192.168.1.2 --password super
Test Docker Container
> inspec exec test.rb -t docker://5cc8837bb6a8
Test a machine remotely via WinRM
AGENTLESS
InSpec
Database Testing
describe mysql_session.query("SELECT user,host
FROM mysql.user WHERE host = '%'") do
its(:stdout) { should be empty }
end
Cloud Provider Testing
security_groups.each do |security_group|
describe security_group do
it { should_not
have_inbound_rule().with_source('0.0.0.0/0') }
end
end
ONE LANGUAGE
● Databases
● API endpoints
(e.g. cloud)
● Baremetal
● VMs
● Containers
Jumpstart your compliance test coverage
Compliance in production
Amazon Linux
2014.09 / 2015.03
CentOS
6 / 7
HP UX
11i
IBM AIX
5.3 / 6.1 / 7.1
RHEL
6 / 7
SLES
11 / 12
Ubuntu Server
12.04 / 14.04
Windows
7 / 8 / 10 / 2012 / 2012R2
Chef Automate ships with profiles for:
● Write some code
● Write and run some unit tests
● Go to pre-production
● Run some integration/acceptance tests
● Go to production
● Lowered chance of production failure
Test Driven Development
Add a test
Run the tests
Make a little change
Run the tests
pass
[development
continues]fail
fail
pass
pass
[development
stops]
Detect Correct
Continuous Workflow
InSpec for Platforms
chef/inspec-vmwarechef/inspec-azure chef/inspec-aws
InSpec for AWS
describe aws_iam_user(’iam_user') do
its('has_mfa_enabled?') { should be false }
its('has_console_password?') { should be false }
end
InSpec for Azure
describe azure_virtual_machine(name: 'Linux-Internal-VM',
resource_group: 'Inspec-Azure') do
its('sku') { should eq '16.04.0-LTS' }
its('publisher') { should eq 'Canonical' }
its('offer') { should eq 'UbuntuServer' }
its('size') { should eq 'Standard_DS2_v2' }
its('location') { should eq 'westeurope' }
its('admin_username') { should eq 'azure' }
end
InSpec for VmWare
control 'vmware-7.3.3' do
impact 0.7
title 'Ensure that the vSwitch Promiscuous
Mode policy is set to reject.'
describe vmhost_vswitch(datacenter: 'vm001',
host: 'localhost.localdomain',vswitch:
'vSwitch0') do
its('allowPromiscuous') { should be false }
end
end
Chef Automate
One platform that delivers DevOps workflow, automated
compliance, and end-to-end pipeline visibility
The Chef Automate Platform
Continuous Automation for High Velocity IT
Workflow • Local development • Integration • Tooling (APIs & SDKs)
COLLABORATE
▪ Package
▪ Test
▪ Approve
BUILD
▪ Provision
▪ Configure
▪ Execute
▪ Update
DEPLOY
▪ Secure
▪ Comply
▪ Audit
▪ Measure
▪ Log
MANAGE
Infrastructure Automation Compliance AutomationApplication Automation
OSS AUTOMATION ENGINES
Increase Speed
▪ Package infrastructure and app
configuration as code
▪ Continuously automate
infrastructure and app updates
Improve Efficiency
▪ Define and execute standard
workflows and automation
▪ Audit and measure effectiveness of
automation
Decrease Risk
▪ Define compliance rules as code
▪ Deliver continuous compliance as
part of standard workflow
Appendix
● Programmatically provision and configure components
● Declarative DSL with the flexibility
● Built on Ruby
● Extensible through Ruby
The Chef DSL (domain specific language)
Core Chef Concepts
Cookbooks
Recipes
Resources
package 'httpd' do
action :install
end
windows_feature 'IIS-WebServerRole' do
action :install
end
● A Resource is a system state you define
○ Example: Package installed, state of a service, configuration file existing
● You declare what the state of the resource is
○ Chef automatically determine HOW that state is achieved
Building Blocks: What is a Resource?
● A recipe is a collection of Resources
● Resources are executed in the order they are listed
Building Blocks: What is a Recipe?
On Linux based OSes:
windows_feature 'IIS-WebServerRole' do
action :install
end
template 'c:inetpubwwwrootDefault.htm' do
source 'Default.htm.erb'
rights :read, 'Everyone'
end
service 'w3svc' do
action [ :enable, :start ]
end
package 'httpd' do
action :install
end
template '/var/www/index.html' do
source 'index.html.erb'
mode '0644'
end
service 'httpd' do
action [ :enable, :start ]
end
● A cookbook is a set of recipes
● A cookbook is a defined set of
items and different outcomes
that you expect to address
○ A cookbook could have a recipe
to install apache2/httpd but also
another set of recipes to activate
modules required.
Building Blocks: What is a Cookbook?
./attributes
./attributes/default.rb
./CHANGELOG.md
./metadata.rb
./README.md
./recipes
./recipes/application.rb
./recipes/balancer.rb
./recipes/database.rb
./recipes/default.rb
./recipes/webserver.rb
./templates
./templates/default
./templates/default/mysite.conf.erb
Environments
More Chef Concepts
Roles
Cookbooks
Recipes
Resources
Review: Detect
➔ Inspec Profiles
• provide definitions of standards and expectations.
• Validate systems against those standards
➔ Chef Client
• Executes a run list, consisting of configurations defined in
cookbooks.
• Reports results back to Chef Automate
➔ Chef Automate
• Provides pre-loaded inspec profiles
• Stores user-generated inspec profiles
• Creates compliance reports when chef client is run with the
Audit cookbook.
Software Testing and Why it Matters
quantity
lesstimemoretime
manual
tests
automated
GUI tests
integration
tests
unit
tests
quantity
lesstimemoretime
manual
session based
tests
automated
GUI tests
automated
API tests
automated
unit testsautomated
integration
tests
automated
component
tests
Testing builds
safety through
feedback loops
Inexpensive
experiments to
provide validation
Reduces risk
Optimize Testing: Do
more of the inexpensive
testing first!
● Write some code
● Write and run some unit tests
● Go to pre-production
● Run some integration/acceptance tests
● Go to production
● Lowered chance of production failure
Test Driven Development
Add a test
Run the tests
Make a little change
Run the tests
pass
[development
continues]fail
fail
pass
pass
[development
stops]
Chef DK - The Chef Development Kit
▪ Validate your Chef code against Chef best
practices
▪ Extend with rules to enforce organizational
Chef development best practices
▪ Enforce compliance & security practices
Foodcritic
Test Your “Chef Style”
▪ Validate your Chef code against Ruby best
practices
▪ Identify potential Ruby errors
○ Unclosed strings, etc.
▪ Identify style/convention that helps write
better code
○ Single quotes vs. double quotes
CookStyle
Validate your Ruby
▪ Validate your Chef code will run
▪ Testing for more Chef advanced use cases
▪ Useful for regression testing
ChefSpec
Simulate Chef
▪ Executes your Chef code on an instance or container
▪ Integrates with Cloud and Virtualization providers
▪ Validate your Chef code locally before sharing
▪ Speed development of Chef Cookbooks
Test Kitchen
Let’s do this (almost) for real
▪ Assert the intention of your Chef code
▪ Verify on live systems that your Chef code produced the correct result
▪ Confirm your Chef code didn’t not produce compliance drift
InSpec
Verify automation results & ensure compliance
FAST INEXPENSIVE TESTING
DEEP INTEGRATION TESTING
Definitive tooling for local development of Chef code & Infrastructure as Code development
Chef
Infrastructure as Code
● The chef-client is an agent that
runs on our nodes to apply
configuration changes, and report
results to Chef Automate
● The Chef Supermarket provides
pre-written cookbooks to get you
started.
● The audit cookbook allows us to
run inspec scans on a regular
basis.
Philly security shell meetup

Philly security shell meetup

  • 1.
    Including Compliance &Security Checks in the Delivery Pipeline Clinton Wolfe | Chef - Sr. Software Engineer Nicole Johnson | Chef - Manager, Solutions Architects East
  • 2.
    Chef is... ● Anautomation framework that enables Infrastructure as Code ● A robust set of tooling for testing Chef code ● A large library of reusable patterns (supermarket.chef.io) ● Available for Linux variants, Unix variants, and Windows, all as first class citizens. CHEF IS INFRASTRUCTURE AS CODE Programmatically provision and configure components Treat like any other code base Reconstruct business from code repository, data backup, and compute resources
  • 3.
    Chef ▪ Manages deployment andon-going automation ▪ Define reusable resources and infrastructure state as code ▪ Scale elegantly from one to tens of thousands of managed nodes across multiple complex environments ▪ Community, Certified Partner, and Chef supported content available for all common automation tasks Infrastructure automation and delivery at scale windows_feature ‘IIS-WebServerRole’ do action :install end windows_feature ‘IIS-ASPNET’ do action :install end iis_pool FooBarPool do runtime_version “4.0” action :add end package "apache" do action :install end template “/etc/httpd/https.conf” do source “httpd.conf.erb” mode 0075 owner “root” group “root” end service “apache2” do action :start end
  • 4.
    PART OF APROCESS OF CONTINUOUS COMPLIANCE Scan for Compliance Build & Test Locally Build & Test CI/CD Remediate Verify A SIMPLE EXAMPLE OF AN INSPEC CIS RULE InSpec ▪ Translate compliance into Code ▪ Clearly express statements of policy ▪ Move risk to build/test from runtime ▪ Find issues early ▪ Write code quickly ▪ Run code anywhere ▪ Inspect machines, data and APIs Turn security and compliance into code control ‘cis-1.4.1’ do title ‘1.4.1 Enable SELinux in /etc/grub.conf’ desc ‘ Do not disable SELinux and enforcing in your GRUB configuration. These are important security features that prevent attackers from escalating their access to your systems. For reference see … ‘ impact 1.0 expect(grub_conf.param ‘selinux’).to_not eq ‘0’ expect(grub_conf.param ‘enforcing’).to_not eq ‘0’ end
  • 5.
  • 6.
    The Old Way Peopleworking directly on machines InSpec - Security meets Operations The Modern Way People automating machines using code DEVOPS SECURITY DEVOPS COMPLIANCE The New Way Shared tooling across organizations
  • 7.
    Differences in verifyingcompliance policy > grep "^Protocol" /etc/ssh/sshd_config | sed 's/Protocol //' 2 DOCUMENTATION SSH supports two different protocol versions. The original version, SSHv1, was subject to a number of security issues. Please use SSHv2 instead to avoid these. describe sshd_config do its('Protocol') { should eq 2 } end control 'ssh-1234' do impact 1.0 title 'Server: Set protocol version to SSHv2' desc " Set the SSH protocol version to 2. Don't use legacy insecure SSHv1 connections anymore... " describe sshd_config do its('Protocol') { should eq 2 } end end SCRIPTING TOOLS TESTING LANGUAGE COMPLIANCE LANGUAGE
  • 8.
    ONE LANGUAGE InSpec InSpec onWindows Example control 'windows-base-201' do impact 1.0 title 'Strong Windows NTLMv2 Authentication Enabled; Weak LM Disabled' desc ' @link: http://support.microsoft.com/en-us/kb/823659 ' describe registry_key ('HKLMSystemCurrentControlSetControlLsa') do it { should exist } its('LmCompatibilityLevel') { should eq 4 } end end
  • 9.
    InSpec ● Baremetal ● VMs ●Containers ONE LANGUAGE > inspec exec test.rb Test a machine remotely via SSH > inspec exec test.rb -i identity.key -t ssh://root@172.17.0.1 Test your machine locally > inspec exec test.rb -t winrm://Admin@192.168.1.2 --password super Test Docker Container > inspec exec test.rb -t docker://5cc8837bb6a8 Test a machine remotely via WinRM AGENTLESS
  • 10.
    InSpec Database Testing describe mysql_session.query("SELECTuser,host FROM mysql.user WHERE host = '%'") do its(:stdout) { should be empty } end Cloud Provider Testing security_groups.each do |security_group| describe security_group do it { should_not have_inbound_rule().with_source('0.0.0.0/0') } end end ONE LANGUAGE ● Databases ● API endpoints (e.g. cloud) ● Baremetal ● VMs ● Containers
  • 11.
    Jumpstart your compliancetest coverage Compliance in production Amazon Linux 2014.09 / 2015.03 CentOS 6 / 7 HP UX 11i IBM AIX 5.3 / 6.1 / 7.1 RHEL 6 / 7 SLES 11 / 12 Ubuntu Server 12.04 / 14.04 Windows 7 / 8 / 10 / 2012 / 2012R2 Chef Automate ships with profiles for:
  • 12.
    ● Write somecode ● Write and run some unit tests ● Go to pre-production ● Run some integration/acceptance tests ● Go to production ● Lowered chance of production failure Test Driven Development Add a test Run the tests Make a little change Run the tests pass [development continues]fail fail pass pass [development stops]
  • 13.
  • 14.
  • 15.
    InSpec for AWS describeaws_iam_user(’iam_user') do its('has_mfa_enabled?') { should be false } its('has_console_password?') { should be false } end
  • 16.
    InSpec for Azure describeazure_virtual_machine(name: 'Linux-Internal-VM', resource_group: 'Inspec-Azure') do its('sku') { should eq '16.04.0-LTS' } its('publisher') { should eq 'Canonical' } its('offer') { should eq 'UbuntuServer' } its('size') { should eq 'Standard_DS2_v2' } its('location') { should eq 'westeurope' } its('admin_username') { should eq 'azure' } end
  • 17.
    InSpec for VmWare control'vmware-7.3.3' do impact 0.7 title 'Ensure that the vSwitch Promiscuous Mode policy is set to reject.' describe vmhost_vswitch(datacenter: 'vm001', host: 'localhost.localdomain',vswitch: 'vSwitch0') do its('allowPromiscuous') { should be false } end end
  • 18.
    Chef Automate One platformthat delivers DevOps workflow, automated compliance, and end-to-end pipeline visibility
  • 19.
    The Chef AutomatePlatform Continuous Automation for High Velocity IT Workflow • Local development • Integration • Tooling (APIs & SDKs) COLLABORATE ▪ Package ▪ Test ▪ Approve BUILD ▪ Provision ▪ Configure ▪ Execute ▪ Update DEPLOY ▪ Secure ▪ Comply ▪ Audit ▪ Measure ▪ Log MANAGE Infrastructure Automation Compliance AutomationApplication Automation OSS AUTOMATION ENGINES Increase Speed ▪ Package infrastructure and app configuration as code ▪ Continuously automate infrastructure and app updates Improve Efficiency ▪ Define and execute standard workflows and automation ▪ Audit and measure effectiveness of automation Decrease Risk ▪ Define compliance rules as code ▪ Deliver continuous compliance as part of standard workflow
  • 20.
  • 21.
    ● Programmatically provisionand configure components ● Declarative DSL with the flexibility ● Built on Ruby ● Extensible through Ruby The Chef DSL (domain specific language)
  • 22.
  • 23.
    package 'httpd' do action:install end windows_feature 'IIS-WebServerRole' do action :install end ● A Resource is a system state you define ○ Example: Package installed, state of a service, configuration file existing ● You declare what the state of the resource is ○ Chef automatically determine HOW that state is achieved Building Blocks: What is a Resource?
  • 24.
    ● A recipeis a collection of Resources ● Resources are executed in the order they are listed Building Blocks: What is a Recipe? On Linux based OSes: windows_feature 'IIS-WebServerRole' do action :install end template 'c:inetpubwwwrootDefault.htm' do source 'Default.htm.erb' rights :read, 'Everyone' end service 'w3svc' do action [ :enable, :start ] end package 'httpd' do action :install end template '/var/www/index.html' do source 'index.html.erb' mode '0644' end service 'httpd' do action [ :enable, :start ] end
  • 25.
    ● A cookbookis a set of recipes ● A cookbook is a defined set of items and different outcomes that you expect to address ○ A cookbook could have a recipe to install apache2/httpd but also another set of recipes to activate modules required. Building Blocks: What is a Cookbook? ./attributes ./attributes/default.rb ./CHANGELOG.md ./metadata.rb ./README.md ./recipes ./recipes/application.rb ./recipes/balancer.rb ./recipes/database.rb ./recipes/default.rb ./recipes/webserver.rb ./templates ./templates/default ./templates/default/mysite.conf.erb
  • 26.
  • 27.
    Review: Detect ➔ InspecProfiles • provide definitions of standards and expectations. • Validate systems against those standards ➔ Chef Client • Executes a run list, consisting of configurations defined in cookbooks. • Reports results back to Chef Automate ➔ Chef Automate • Provides pre-loaded inspec profiles • Stores user-generated inspec profiles • Creates compliance reports when chef client is run with the Audit cookbook.
  • 28.
    Software Testing andWhy it Matters quantity lesstimemoretime manual tests automated GUI tests integration tests unit tests quantity lesstimemoretime manual session based tests automated GUI tests automated API tests automated unit testsautomated integration tests automated component tests Testing builds safety through feedback loops Inexpensive experiments to provide validation Reduces risk Optimize Testing: Do more of the inexpensive testing first!
  • 29.
    ● Write somecode ● Write and run some unit tests ● Go to pre-production ● Run some integration/acceptance tests ● Go to production ● Lowered chance of production failure Test Driven Development Add a test Run the tests Make a little change Run the tests pass [development continues]fail fail pass pass [development stops]
  • 30.
    Chef DK -The Chef Development Kit ▪ Validate your Chef code against Chef best practices ▪ Extend with rules to enforce organizational Chef development best practices ▪ Enforce compliance & security practices Foodcritic Test Your “Chef Style” ▪ Validate your Chef code against Ruby best practices ▪ Identify potential Ruby errors ○ Unclosed strings, etc. ▪ Identify style/convention that helps write better code ○ Single quotes vs. double quotes CookStyle Validate your Ruby ▪ Validate your Chef code will run ▪ Testing for more Chef advanced use cases ▪ Useful for regression testing ChefSpec Simulate Chef ▪ Executes your Chef code on an instance or container ▪ Integrates with Cloud and Virtualization providers ▪ Validate your Chef code locally before sharing ▪ Speed development of Chef Cookbooks Test Kitchen Let’s do this (almost) for real ▪ Assert the intention of your Chef code ▪ Verify on live systems that your Chef code produced the correct result ▪ Confirm your Chef code didn’t not produce compliance drift InSpec Verify automation results & ensure compliance FAST INEXPENSIVE TESTING DEEP INTEGRATION TESTING Definitive tooling for local development of Chef code & Infrastructure as Code development
  • 31.
    Chef Infrastructure as Code ●The chef-client is an agent that runs on our nodes to apply configuration changes, and report results to Chef Automate ● The Chef Supermarket provides pre-written cookbooks to get you started. ● The audit cookbook allows us to run inspec scans on a regular basis.