'Intro to Infrastructure as Code' - DevOps Belfast

1. Infrastructure as Code
And Quick Introduction to Chef
DevOps Belfast
Chef Fundamentals by Chef Software, Inc. is licensed under a
Creative Commons Attribution-ShareAlike 4.0 International License.

2. 14 months working at Chef
>4 years working with Chef
Who Am I?
John Fitzpatrick
johnfitzpatrick
jfitzpatrick@getchef.com
@jhnftzptrck
Belfast, Northern Ireland
Curriculum Development
Training
Community stuff

3. Hello!
• System Administrator?

4. Hello!
• System Administrator?
• Developer?

5. Hello!
• System Administrator?
• Developer?
• Business Person (non-technical)?

6. Hello!
• System Administrator?
• Developer?
• Business Person (non-technical)?
• Experience with Infrastructure as Code or
Configuration Management?

7. Hello!
• System Administrator?
• Developer?
• Business Person (non-technical)?
• Experience with Infrastructure as Code or
Configuration Management?
•Familiar with DevOps?

8. Why Configuration Management?

9. IT is revolutionizing every sector
• All companies in every sector have their own IT Departments
automating their product offerings
• They're each striving for
• Automation
• Faster Speed to Market
• Consistent/Predictable Delivery
• Cost Efficiency

10. Manufacturing CAD/CAM
• Faster Design, Develop
& Deliver cycles
• Lower costs
• 24 hour shifts!
• Faster time to market

11. Online Banking
• Convenience
• Accuracy
• Instant Access

12. Online Retail
• Convenient
• Cheaper than a high
street presence
• Global reach
• Faster time to market

13. IT is revolutionizing every sector
• Accounting
• Advertising
• Aerospace
• Agriculture
• Airline
• Apparel & Accessories
• Automotive
• Banking
• Biotechnology
• Broadcasting
• Brokerage
• Call Centers
• Cargo Handling
• Chemical
• Consulting
• Defense
• Department Stores
• Education
• Electronics
• Energy
• Entertainment & Leisure
• Executive Search
• Financial Services
• Food, Beverage & Tobacco
• Grocery
• Health Care
• Internet Publishing
• Investment Banking
• Legal
• Manufacturing
• Motion Picture & Video
• Music
• Newspaper Publishers
• Online Auctions
• Pension Funds
• Pharmaceuticals
• Private Equity
• Publishing
• Real Estate
• Retail & Wholesale
• Securities & Commodity
Exchanges
• Service
• Telecommunications
• Television
• Transportation
• Trucking
• Venture Capital

14. What about IT industry itself?
• IT is still a manual process
• Stuck in the 1990's

15. IT Sector
• Companies IT departments
have been too busy
providing solutions for their
Companies' core business
• But they've neglected their
own back yard!
Dev QA Prod
• IT industry itself is the bottleneck - no one was attempting to
automate the actual IT environment itself

16. IT Industry
• Software companies provide products that feed
into these IT solutions
• But sometimes they add complexity of solution to be
maintained, and not ease of management

17. Dimensions of Scale

18. Case Study: Online Retail
• You place an order
• Card payment is taken
• Item is packed & shipped
• Meanwhile, stock level is debited, & reordered from suppliers
if it falls below a certain level
• Totally automated – no human intervention!
• However the s/w allowing this takes long development
cycles (maybe months), through planning, development,
testing/QA, and into production

19. High Maintenance
• Herculean task to keep these systems running,
patched, upgraded, etc

20. Waterfall Method – Silos!
Business/Sales Lead
Software Architect
Development Team
QA Team
Sys Admin

21. Silos are complex structures

22. Option 1
• Spend money improving your Silo
• Looks pretty but entrenches the problem

23. Is there a better way?
•We're in an era of unlimited compute
• Speed is the new currency

24. Option 2
• Pull them down

25. Option 2
• Or at least let them become more latterly cross
functional

26. DevOps!
• Implement DevOps culture
• Communication
• Collaboration
• Integration
• End Goal: CD Pipeline!

27. Automate everything!

28. Integrated Approach
• Historically
• Developers wrote software
• Operations (Sys Admins) installed and maintained it
• Now developers need to consider deploying their
code while writing they're writing it

29. CD Pipeline – Sample Workflow
Automate
dBuild and
Unit Tests
Developmen
t
Release
to
Productio
n
GitHub Automate
d
Acceptanc
e Test
Manual
Code
Review (?)
Feedback Check-in
(Linting)
Feedback
Trigger
Check-in
Feedback
(Linting)
Trigger
Feedback
Trigger
Check-in
Feedback
(Linting)
Feedback
Trigger
Trigger
Trigger Trigger Feedback
Feedback

30. Developers with Pagers? Whoah!

31. Benefits of Automation

32. So where do I start?
•The first step to establish a fully automated CD
pipeline is to define your Infrastructure as
Code

33. Infrastructure as Code
A SysAdmins Journey

34. Typical Sys Admin Journey
From a vanilla image…
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful

35. Typical Sys Admin Journey
From a vanilla image…
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful
Log into machine

36. Typical Sys Admin Journey
From a vanilla image…
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful
Install a few packages

37. Typical Sys Admin Journey
From a vanilla image…
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful
Pull in some content

38. Typical Sys Admin Journey
From a vanilla image…
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful
Manipulate directories &
content

39. Typical Sys Admin Journey
From a vanilla image…
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful
Re/start services

40. Typical Sys Admin Journey
ssh into machine
$ yum install httpd
$ yum install wget
$ yum install unzip
$ wget https://somewhere/master.zip
$ unzip master.zip
$ cd myapp
$ sudo mv html /var/www/
$ sudo su root
$ python myappinstall.py
$ apachectl graceful
• All commands are
manual
• Have different
syntaxes
• They're platform
specific (RHEL,
Debian, Windows, …)
From a vanilla image…

41. Typical Sys Admin Journey
• Store notes in ~/server.txt

42. Typical Sys Admin Journey
• Store notes in ~/server.txt
• Move notes to the wiki

43. Typical Sys Admin Journey
• Store notes in ~/server.txt
• Move notes to the wiki
•Write some scripts (setup.sh, fixit.sh, etc.)

44. Typical Sys Admin Journey
• Store notes in ~/server.txt
• Move notes to the wiki
•Write some scripts (setup.sh, fixit.sh, etc.)
• setup.sh.BAK
• fixit.sh.OLD

45. Typical Sys Admin Journey
• Store notes in ~/server.txt
• Move notes to the wiki
•Write some scripts (setup.sh, fixit.sh, etc.)
• setup.sh.BAK
• fixit.sh.OLD
• Golden images and snapshots

46. Sample Infrastructure
Graphite Nagios
JBoss
Memcache
Postgres Slaves
Postgres Master

47. New Compliance Mandate!
Graphite Nagios
JBoss
Memcache
Postgres Slaves
Postgres Master
• Move SSH off port 22
• Let’s put it on 2022

48. 6 Golden Images to Update
Graphite Nagios
JBoss
Memcache
Postgres Slaves
Postgres Master
1
3
4
5
2
6
/etc/ssh/sshd_config
--- a/sshd_config
+++ b/sshd_config
-Port 22
+Port 2202

49. 12 Instances to replace
Graphite Nagios
JBoss
Memcache
Postgres Slaves
Postgres Master
1
3
2
4 5 6 7
8
10
9
11
12
• Launch
• Delete
• Repeat
• Typically manually
• High stakes
• Late hours
• Risky change

50. The Infrastructure Code
• Consistent DSL to manage any configuration
component of a server
• packages
• files
• users
• …
• Platform agnostic
• Complex implementation code abstracted out

51. Treat Infrastructure like any code base
• Infrastructure configuration files stored in version
control, e.g. GitHub
• Infrastructure becomes as testable & repeatable
as the application code you're delivering

52. So what does this code look like?
Lets drill down and look at how Chef implements this

53. What is a Resource?
• 'Resource', n., the basic unit of configuration in Chef
• Represents a piece of the system & its desired state
• A package to be installed
• A service to be running
• A file to be generated
• A user to be managed
• etc

54. Example: 'package' Resource
•Manage software packages
• Install
• Upgrade
• Remove
package "apache2" do
action :install
end

55. Resources – Test and Repair
• Resources use a test and repair model
• Resource currently in the desired state?
• Yes – Do nothing
• No – Bring the resource into the desired state (repair)

56. Example: 'file' Resource
• Create a static file on disk
• Add
• Delete
• Permissions
• etc
file "/var/www/html/index.html" do
content "Hello, Belfast!"
owner "root"
group "root"
end

57. Example: 'template' Resource
• Create a dynamic/templated file on disk
• Install
• Upgrade
• Remove
• etc
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode "0644"
notifies :restart, "service[httpd]"
variables(
:document_root => node["docroot"],
:port => "80"
)
end

58. Example: 'service' Resource
•Manage services on the machine
• start
• stop
• reload
• etc
service "apache2" do
action [ :enable, :start ]
end

59. Other Resources
• deploy
• cron
• directory
• mount
• user
• group
• dsc_resource
•powershell_script
• registry_key
• remote_directory
• route
• and many more…
• Or build your own!

60. Declarative Interface
• Resources are platform agnostic
• Policy declares what state each resource should be
in, but not how to get there
package "ntp"
• Chef decides how to implement this:-
• OSX: brew install ntp
• RHEL: yum install ntp
• Ubuntu: agt-get install ntp
• …

61. What is a Chef Recipe?
• 'Recipe', n., a file containing one or more
Resources

62. Recipe: An ordered list of resources
package "apache2" do
file "/var/www/html/index.html" do
content "Hello, Belfast!"
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
notifies :restart, "service[httpd]"
variables(
:document_root => node["docroot"],
:port => "80"
)
end
action :install
end
service "apache2" do
action [:enable,:start]
end

63. Recipe: An ordered list of resources
package "apache2" do
file "/var/www/html/index.html" do
content "Hello, Belfast!"
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
notifies :restart, "service[httpd]"
variables(
:document_root => node["docroot"],
:port => "80"
)
end
action :install
end
service "apache2" do
action [:enable,:start]
end
Order is critical!

64. Chef Provisioning Resources (aka Chef Metal)
• Provision servers in any our multiple locations
with_driver 'aws'
num_nodes = 100
1.upto(num_nodes) do |i|
machine "hadoop#{i}" do
recipe 'hadoop::default'
recipe 'ntp'
converge true
tag 'hadoop'
end
end

65. Chef Provisioning Resources (aka Chef Metal)
• Provision servers in any our multiple locations
with_driver 'aws'
num_nodes = 100
1.upto(num_nodes) do |i|
machine "hadoop#{i}" do
recipe 'hadoop::default'
recipe 'ntp'
converge true
tag 'hadoop'
end
end
• Cloud:
• FOG: EC2, DigitalOcean, OpenStack, etc.
• Virtualization:
• Vagrant: VirtualBox, VMWare Fusion, etc.
• Vsphere
• Containers:
• LXC
• Docker
• Bare Metal:
• SSH

66. Extending the Ruby DSL
• Recipes can include arbitrary Ruby code
search(:node, "ipaddress:10*").each.uniq do|node|
file "node_#{node[ipaddress]}" do
content #{node[user]}
end
end
%w{mysql apache2 ntp}.each do |pkg|
package pkg do
action :install
end
end
1.upto(10) do |i|
file "file#{i}" do
action :create
end
end

67. Extending the Ruby DSL
• Recipes can include arbitrary Ruby code
search(:node, "ipaddress:10*").each.uniq do|node|
file "node_#{node[ipaddress]}" do
content #{node[user]}
end
end
%w{mysql apache2 ntp}.each do |pkg|
package pkg do
action :install
end
end
1.upto(10) do |i|
file "file#{i}" do
action :create
end
end

68. Extending the Ruby DSL
• Recipes can include arbitrary Ruby code
search(:node, "ipaddress:10*").each.uniq do|node|
file "node_#{node[ipaddress]}" do
content #{node[user]}
end
end
%w{mysql apache2 ntp}.each do |pkg|
package pkg do
action :install
end
end
1.upto(10) do |i|
file "file#{i}" do
action :create
end
end
Returns a Ruby array
Std Ruby array methods

69. Extending the Ruby DSL
• Recipes can include arbitrary Ruby code
search(:node, "ipaddress:10*").each.uniq do|node|
file "node_#{node[ipaddress]}" do
content #{node[user]}
end
end
%w{mysql apache2 ntp}.each do |pkg|
package pkg do
action :install
end
end
1.upto(10) do |i|
file "file#{i}" do
action :create
end
end
Declare an array
explicitly
Std Ruby array method

70. Extending the Ruby DSL
• Recipes can include arbitrary Ruby code
search(:node, "ipaddress:10*").each.uniq do|node|
file "node_#{node[ipaddress]}" do
content #{node[user]}
end
end
%w{mysql apache2 ntp}.each do |pkg|
package pkg do
action :install
end
end
1.upto(10) do |i|
file "file#{i}" do
action :create
end
end
Std Ruby array method

71. What is a Cookbook?
• 'Cookbook', n., a collection of recipes & supporting
files
• These supporting files could be
• attributes
• templates
• etc
• Naming convention: 'cookbookname::recipename'

72. What is a run_list?
• 'run_list', n., a list of recipes to be run in a given
'chef-client' run
run_list [
"recipe[ntp::client]"
"recipe[users::default]"
"role[webserver]"
]

73. What is a run_list?
• 'run_list', n., a list of recipes to be run in a given
'chef-client' run
run_list [
"recipe[ntp::client]"
"recipe[users::default]"
"role[webserver]"
]
Order is critical!
cookbook recipe

74. chef–client queries Chef Server for runlist
Node
Chef
Server

75. chef–client pulls node runlist from Chef Server
Node
Chef
Server
"recipe[ntp::client]"
"recipe[users::default]"
"role[webserver]"

76. chef-client invokes the runlist
Chef
Server
"recipe[ntp::client]"
"recipe[users::default]"
"role[webserver]"

77. Distributed Architecture
• Highly scalable distributed system
• No processing performed on the Chef Server
• All processing is performed on the node itself

78. What can you manage?
•Nodes represent any infrastructure component
• Physical servers or virtual servers
• Local hardware
•Compute instances in a public or private cloud
• Employee workstations
• Could also be network hardware - switches,
routers, etc

79. Chef Server
• Stores policy files and other configuration data
• Maintains a searchable index of node data

80. Use Case - Search
Bringing it all Together

81. Search
• Chef Server maintains a searchable index of
node data
• Recipes can search for other nodes with specific
• Roles
• IP addresses
• Hostnames
• FQDNs
• etc
http://www.flickr.com/photos/kathycsus/2686772625

82. Search for Nodes
pool_members = search("node","role:webserver")
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end

83. Webservers
HAProxy Configuration
HA Proxy

84. pool_members = search("node","role:webserver")
Webservers
HAProxy Load Balancer
Chef
Server
HA Proxy

85. pool_members = search("node","role:webserver")
Webservers
HAProxy Load Balancer
Chef
Server
HA Proxy

86. pool_members = search("node","role:webserver")
Webservers
HAProxy Load Balancer
Chef
Server
HA Proxy

87. pool_members = search("node","role:webserver")
Webservers
HAProxy Load Balancer
Chef
Server
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
HA Proxy

88. pool_members = search("node","role:webserver")
Webservers
HAProxy Load Balancer
Chef
Server
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
HA Proxy

89. pool_members = search("node","role:webserver")
Webservers
HAProxy Load Balancer
HA Proxy
Chef
Server
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}

90. HAProxy Configuration
Webservers
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
HA Proxy

91. HAProxy Configuration
haproxy.cfg
server web01 10.1.1.1 weight 1 maxconn 1 check
server web02 10.1.1.2 weight 1 maxconn 1 check
server web03 10.1.1.3 weight 1 maxconn 1 check
server web04 10.1.1.4 weight 1 maxconn 1 check
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
Webservers
HA Proxy

92. HAProxy Configuration
haproxy.cfg
server web01 10.1.1.1 weight 1 maxconn 1 check
server web02 10.1.1.2 weight 1 maxconn 1 check
server web03 10.1.1.3 weight 1 maxconn 1 check
server web04 10.1.1.4 weight 1 maxconn 1 check
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
Webservers
HA Proxy

93. HAProxy Configuration
haproxy.cfg
server web01 10.1.1.1 weight 1 maxconn 1 check
server web02 10.1.1.2 weight 1 maxconn 1 check
server web03 10.1.1.3 weight 1 maxconn 1 check
server web04 10.1.1.4 weight 1 maxconn 1 check
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
Webservers
HA Proxy

94. HAProxy Configuration
haproxy.cfg
server web01 10.1.1.1 weight 1 maxconn 1 check
server web02 10.1.1.2 weight 1 maxconn 1 check
server web03 10.1.1.3 weight 1 maxconn 1 check
server web04 10.1.1.4 weight 1 maxconn 1 check
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
}
Webservers
HA Proxy

95. So when this…
9
Jboss App
Graphite Nagios
Memcache
Postgres Slaves
Postgres Master

96. Jboss App
Graphite Nagios
Memcache
Postgres Slaves
Postgres Master
…becomes this

97. ...this can happen automatically
Graphite Nagios
Memcache
Postgres Slaves
Postgres Master
Jboss App

98. Wrapping Up & Demo

99. Power of Infrastructure as Code
• Reconstruct business from code repository,
data backup, and compute resources

100. We’ve only scratched the surface
https://www.getchef.com/chef/

101. We’ve only scratched the surface…
• actions
• alerting
• analytics
• auditing
• Berkshelf
• bootstrap
• built-in and custom
• CD/CI frameworks
(ant, maven,
Jenkins, etc.)
• chef (executable)
• chef-apply
• chef-client
• chef-metal
• chef-shell
• ChefSpec
• cookbook metadata
• cookbook
versioning
• cookbooks
• data bags
• definitions
• delete validation
keys
• diagnostics
• encrypted
• environments
• event logging
• handlers
• HEC signup
• HWRP
• idempotence
• kitchen
• knife
• knife-ec2 and
friends
• LWRP
• monitoring
• multiple data
centers
• node attributes
• nodes
• ohai
• orchestration
• performance
testing
• provisioning
• push jobs
• recipes
• reporting
• resource
notifications
• resources
• roles
• rubocop
• food critic
• Serverspec
• scaling
• search
• server/data locality
• source control
(GitHub, etc.)
• standard & custom
plugin
• Starter Kit
• Supermarket
• TDD
• templates

102. Build Anything Automatically
• Simple internal applications
•Workstations
• Hadoop clusters
• IaaS infrastructure
• PaaS infrastructure
• SaaS applications
• Storage systems
• You name it
http://www.flickr.com/photos/hyku/245010680/

103. And Manage it Simply
•Automatically reconfigure
everything
• Linux, Windows, Unixes,
BSDs
• Load balancers
•Monitoring systems
• Cloud migrations become
trivial
http://www.flickr.com/photos/helico/404640681/

104. Automate everything!

105. Chef Training
Chef Fundamentals - 25/26 September, London
Ping me for 10% Discount

107. Demo
Deploying Apache web application in EC2 using
Chef Provisioning

108. Demo
• https://github.com/johnfitzpatrick/DevOps-
Belfast/tree/18nov14