Chef is an open source configuration management and automation framework used to configure, deploy and manage infrastructure of every type. Deploying to the cloud has made it easy to run large numbers of servers and Chef makes it even easier to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. This talk will provide a quick introduction to Chef and is intended for sysadmins and developers familiar with the concepts behind managing applications and infrastructure in the cloud, without diving too deeply into technical specifics.

2. Chef: The Swiss Army Knife
of Cloud Infrastructure
Matt Ray
SCALE Build a Cloud Day
February 21, 2014

3. Introductions
• Matt Ray
• Director of Cloud
Integrations at Chef
• matt@getchef.com
• mattray GitHub|IRC|
Twitter

4. We have a problem…

5. Complexity
http://www.flickr.com/photos/michaelheiss/3090102907/

6. Items of Manipulation (Resources)
• Nodes
• Networking
• Files
• Directories
• Symlinks
• Mounts
• Routes
• Users
• Groups
• Packages
• Services
• Filesystems

7. A tale of growth...
Application

8. Add a database
Application
Application Database

9. Make database redundant
Application
App Databases

10. Application server redundancy
App Servers
App Databases

11. Add a load balancer
App LB
App Servers
App Databases

12. Webscale!
App LBs
App Servers
App Databases

13. Now we need a caching layer
App LBs
App Servers
App DB Cache
App DBs

14. Infrastructure has a Topology
App LBs
App Servers
App DB Cache
App DBs

15. Your Infrastructure is a Snowflake
Round Robin DNS
App Servers
App DB Cache
Floating IP?
App DBs

16. Complexity Increases Quickly
App LBs
Cache
App Servers
DB Cache
< Shiny!
A
we
re
nito
mo
g??
rin
DB slaves
DBs

17. ™

18. The Chef Framework
• Reasonability
• Flexibility
• Library & Primitives
• TIMTOWTDI
• Sane defaults
http://www.flickr.com/photos/wonderlane/3609342683/sizes/l/in/photostream/

19. The Chef Tool(s)
• ohai
• chef-client
• chef-shell
• knife
• The Ruby language
Om
nibu
s-F
Nati
ull S
ve P
ta c k
ack
age
s

20. The Chef API
• HTTPS, RESTful API w/ JSON, RSA key auth
• Infrastructure data store such as node data
• Search Service
• Derivative Services?
http://www.flickr.com/photos/core-materials/4419853626/sizes/o/in/photostream/

21. The Chef Community
• Apache License, Version 2.0
• Thousands of Individual and Corporate contributors.
• Thousands of cookbooks available from the
community
• http://community.opscode.com

22. How does it work?
http://i3.kym-cdn.com/photos/images/original/000/046/123/magnets.jpg

23. Chef is Infrastructure as Code
http://www.flickr.com/photos/louisb/4555295187/
• Programmatically
provision and
configure
• Treat like any other
code base
• Reconstruct business
from code repository,
data backup, and bare
metal resources.

24. Programs
• Chef generates
configurations directly
on nodes from their run
list
• Reduce management
complexity through
abstraction
• Store the configuration
of your programs in
version control
http://www.flickr.com/photos/ssoosay/5126146763/

25. Declarative Interface to Resources
• Define Policy
• Say what, not how
• Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/

26. That Looks Like This
package "apache2"
template "/etc/apache2/apache2.conf" do!
source "apache2.conf.erb"!
owner "root"!
group "root"!
mode "0644"!
variables(:allow_override => "All")!
notifies :reload, "service[apache2]"!
end
service "apache2" do!
action [:enable,:start]!
supports :reload => true!
end

27. Chef Nodes
• Chef runs on nodes
• Chef nodes do the heavy lifting
• Authority about themselves
• Stored on the server when using Chef Server
• Indexed for search
http://www.flickr.com/photos/peterrosbjerg/3913766224/

28. Search
• Search for nodes with Roles
• Find Topology Data
!
• IP addresses
• Hostnames
• FQDNs
http://www.flickr.com/photos/kathycsus/2686772625

29. So when this...
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

30. ...becomes this
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

31. ...this can happen automatically
Graphite
Nagios
Jboss App
!
!
!
Memcache
!
!
Postgres Slaves
Postgres Master

32. Count the Resources
•
Graphite
Nagios
•
•
•
Jboss App
!
!
•
!
•
Memcache
!
•
!
Postgres Slaves
•
•
•
• 12+ resource changes for 1 node addition •
Load balancer config
Nagios host ping
Nagios host ssh
Nagios host HTTP
Nagios host app health
Graphite CPU
Graphite Memory
Graphite Disk
Graphite SNMP
Memcache firewall
Postgres firewall
Postgres authZ config

33. Landscape of Chef-managed Infrastructure

34. knife

35. knife - with the Chef Server
• knife node
• create / edit / delete
• list
• knife cookbook ...
• knife role ...
• knife environment ...

36. knife bootstrap
$ knife bootstrap SERVER -r ‘role[webserver]’
• SSH to the machine given existing credentials
• Install the Chef Client
• Register with the Chef Server
• Run the initial Run List
• Now managed with Chef!

37. Chef & CloudStack
gem install knife-cloudstack

38. knife-cloudstack 1/4
knife cs
• ** CS COMMANDS **
• knife cs aag list (options)
• knife cs account list (options)
• knife cs cluster list (options)
• knife cs config list (options)
• knife cs disk list (options)
• knife cs domain list (options)
• knife cs firewallrule create hostname 8080:8090:TCP:10.0.0.0/24
• knife cs firewallrule list (options)
• knife cs forwardrule create hostname 8080:8090:TCP
• knife cs host list (options)

39. knife-cloudstack 2/4
knife cs
• knife cs hosts
• knife cs iso list (options)
• knife cs keypair create KEY_NAME (options)
• knife cs keypair delete KEY_NAME (options)
• knife cs keypair list (options)
• knife cs network list (options)
• knife cs oscategory list (options)
• knife cs ostype list (options)
• knife cs pod list (options)
• knife cs project list (options)

40. knife-cloudstack 3/4
knife cs
• knife cs publicip list (options)
• knife cs router list (options)
• knife cs securitygroup list (options)
• knife cs server add nic SERVERID NETWORKID (--ipaddress X.X.X.X)
• knife cs server create [SERVER_NAME] (options)
• knife cs server delete SERVER_NAME [SERVER_NAME ...] (options)
• knife cs server list (options)
• knife cs server reboot SERVER_NAME [SERVER_NAME ...] (options)
• knife cs server remove nic SERVERID NICID
• knife cs server start SERVER_NAME [SERVER_NAME ...] (options)

41. knife-cloudstack 4/4
knife cs
• knife cs server stop SERVER_NAME [SERVER_NAME ...] (options)
• knife cs service list (options)
• knife cs stack create JSON_FILE (options)
• knife cs stack delete JSON_FILE (options)
• knife cs template create NAME (options)
• knife cs template extract NAME (options)
• knife cs template list (options)
• knife cs template register NAME (options)
• knife cs user list (options)
• knife cs volume create NAME (options)
• knife cs volume list (options)
• knife cs zone list (options)

42. Why the Cloud?
• Instant infrastructure
• Unlimited capacity
• Autoscaling
• No commitment
• Immediate replacement

43. Why not the Cloud?
• Reliability
• Performance
• Security
• Price

44. Data Gravity

45. Know our escape plan
for every infrastructure
provider

46. Chef for Infrastructure Portability
• knife rackspace
• knife openstack
• knife azure
• knife cloudstack
• knife ec2
• knife google
• knife hp
• knife vsphere
• ...and many others

47. Desktop, Virtualization & Cloud
•
•
•
•
•
•
Vagrant
VMware
CloudStack
Eucalyptus
OpenStack
bare metal
•
•
•
•
•
•
AWS
Rackspace
HP
Google
Azure
many others
™

48. Desktop, Virtualization & Cloud
•
•
•
•
•
•
Vagrant
VMware
CloudStack
Eucalyptus
OpenStack
bare metal
•
•
•
•
•
•
AWS
Rackspace
HP
Google
Azure
many others
™

49. What does this all mean?
•Every infrastructure is a unique
snowflake
•Understand the costs associated with
the features of your platform(s) of choice.
•Chef enables Infrastructure Portability
•Use the same infrastructure code for
wherever you deploy
•"Data Gravity" is a concern
™

50. There’s a lot more...
• Attributes
• Environments
• Roles
• Chef Handlers
• LWRPs
• Workflow Helpers
• Testing Frameworks
• and even more awesome

51. Thanks!
Matt Ray
matt@getchef.com
@mattray