Your SlideShare is downloading. ×
0
Chef: The Swiss Army Knife
of Cloud Infrastructure
Matt Ray
SCALE Build a Cloud Day
February 21, 2014
Introductions
• Matt Ray
• Director of Cloud
Integrations at Chef
• matt@getchef.com
• mattray GitHub|IRC|
Twitter
We have a problem…
Complexity

http://www.flickr.com/photos/michaelheiss/3090102907/
Items of Manipulation (Resources)
• Nodes
• Networking
• Files
• Directories
• Symlinks
• Mounts

• Routes
• Users
• Group...
A tale of growth...
Application
Add a database
Application

Application Database
Make database redundant
Application

App Databases
Application server redundancy
App Servers

App Databases
Add a load balancer
App LB

App Servers

App Databases
Webscale!
App LBs

App Servers

App Databases
Now we need a caching layer
App LBs
App Servers
App DB Cache
App DBs
Infrastructure has a Topology
App LBs
App Servers
App DB Cache
App DBs
Your Infrastructure is a Snowflake
Round Robin DNS
App Servers
App DB Cache
Floating IP?
App DBs
Complexity Increases Quickly
App LBs
Cache
App Servers
DB Cache

< Shiny!

A

we
re

nito
mo

g??
rin

DB slaves
DBs
™
The Chef Framework
• Reasonability
• Flexibility
• Library & Primitives
• TIMTOWTDI
• Sane defaults

http://www.flickr.com/...
The Chef Tool(s)
• ohai
• chef-client
• chef-shell
• knife
• The Ruby language

Om
nibu
s-F
Nati
ull S
ve P
ta c k
ack
age...
The Chef API
• HTTPS, RESTful API w/ JSON, RSA key auth
• Infrastructure data store such as node data
• Search Service
• D...
The Chef Community
• Apache License, Version 2.0
• Thousands of Individual and Corporate contributors.
• Thousands of cook...
How does it work?

http://i3.kym-cdn.com/photos/images/original/000/046/123/magnets.jpg
Chef is Infrastructure as Code

http://www.flickr.com/photos/louisb/4555295187/

• Programmatically
provision and
configure...
Programs
• Chef generates
configurations directly
on nodes from their run
list
• Reduce management
complexity through
abst...
Declarative Interface to Resources
• Define Policy
• Say what, not how
• Pull not Push

http://www.flickr.com/photos/bixent...
That Looks Like This
package "apache2"

template "/etc/apache2/apache2.conf" do!
source "apache2.conf.erb"!
owner "root"!
...
Chef Nodes
• Chef runs on nodes
• Chef nodes do the heavy lifting
• Authority about themselves
• Stored on the server when...
Search
• Search for nodes with Roles
• Find Topology Data
!

• IP addresses
• Hostnames
• FQDNs

http://www.flickr.com/phot...
So when this...
Graphite

Nagios

Jboss App
Memcache
Postgres Slaves
Postgres Master
...becomes this
Graphite

Nagios

Jboss App
Memcache
Postgres Slaves
Postgres Master
...this can happen automatically
Graphite

Nagios

	

Jboss App

!
!
!

	


Memcache

!
!

Postgres Slaves
Postgres Master
Count the Resources
•
Graphite
Nagios
•
•
	

•
Jboss App
!
!
•
!
•
Memcache
	

!
•
!
Postgres Slaves
•
•
•
• 12+ resource ...
Landscape of Chef-managed Infrastructure
knife
knife - with the Chef Server
• knife node
• create / edit / delete
• list
• knife cookbook ...
• knife role ...
• knife en...
knife bootstrap
$ knife bootstrap SERVER -r ‘role[webserver]’

• SSH to the machine given existing credentials
• Install t...
Chef & CloudStack
gem install knife-cloudstack
knife-cloudstack 1/4
knife cs
• ** CS COMMANDS **
• knife cs aag list (options)
• knife cs account list (options)
• knife ...
knife-cloudstack 2/4
knife cs
• knife cs hosts
• knife cs iso list (options)
• knife cs keypair create KEY_NAME (options)
...
knife-cloudstack 3/4
knife cs
• knife cs publicip list (options)
• knife cs router list (options)
• knife cs securitygroup...
knife-cloudstack 4/4
knife cs
• knife cs server stop SERVER_NAME [SERVER_NAME ...] (options)
• knife cs service list (opti...
Why the Cloud?
• Instant infrastructure
• Unlimited capacity
• Autoscaling
• No commitment
• Immediate replacement
Why not the Cloud?
• Reliability
• Performance
• Security
• Price
Data Gravity
Know our escape plan
for every infrastructure
provider
Chef for Infrastructure Portability

• knife rackspace
• knife openstack
• knife azure
• knife cloudstack
• knife ec2

• k...
Desktop, Virtualization & Cloud
•
•
•
•
•
•

Vagrant
VMware
CloudStack
Eucalyptus
OpenStack
bare metal

•
•
•
•
•
•

AWS
R...
Desktop, Virtualization & Cloud
•
•
•
•
•
•

Vagrant
VMware
CloudStack
Eucalyptus
OpenStack
bare metal

•
•
•
•
•
•

AWS
R...
What does this all mean?
•Every infrastructure is a unique

snowflake
•Understand the costs associated with
the features o...
There’s a lot more...
• Attributes
• Environments
• Roles
• Chef Handlers
• LWRPs
• Workflow Helpers
• Testing Frameworks
...
Thanks!
Matt Ray
matt@getchef.com
@mattray
SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud Infrastructure
Upcoming SlideShare
Loading in...5
×

SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud Infrastructure

1,963

Published on

Chef is an open source configuration management and automation framework used to configure, deploy and manage infrastructure of every type. Deploying to the cloud has made it easy to run large numbers of
servers and Chef makes it even easier to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. This talk will provide a quick introduction to Chef and is intended for sysadmins and developers familiar with the concepts behind managing applications and infrastructure in the cloud, without diving too deeply into technical specifics.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,963
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
13
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud Infrastructure"

  1. 1. Chef: The Swiss Army Knife of Cloud Infrastructure Matt Ray SCALE Build a Cloud Day February 21, 2014
  2. 2. Introductions • Matt Ray • Director of Cloud Integrations at Chef • matt@getchef.com • mattray GitHub|IRC| Twitter
  3. 3. We have a problem…
  4. 4. Complexity http://www.flickr.com/photos/michaelheiss/3090102907/
  5. 5. Items of Manipulation (Resources) • Nodes • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Packages • Services • Filesystems
  6. 6. A tale of growth... Application
  7. 7. Add a database Application Application Database
  8. 8. Make database redundant Application App Databases
  9. 9. Application server redundancy App Servers App Databases
  10. 10. Add a load balancer App LB App Servers App Databases
  11. 11. Webscale! App LBs App Servers App Databases
  12. 12. Now we need a caching layer App LBs App Servers App DB Cache App DBs
  13. 13. Infrastructure has a Topology App LBs App Servers App DB Cache App DBs
  14. 14. Your Infrastructure is a Snowflake Round Robin DNS App Servers App DB Cache Floating IP? App DBs
  15. 15. Complexity Increases Quickly App LBs Cache App Servers DB Cache < Shiny! A we re nito mo g?? rin DB slaves DBs
  16. 16.
  17. 17. The Chef Framework • Reasonability • Flexibility • Library & Primitives • TIMTOWTDI • Sane defaults http://www.flickr.com/photos/wonderlane/3609342683/sizes/l/in/photostream/
  18. 18. The Chef Tool(s) • ohai • chef-client • chef-shell • knife • The Ruby language Om nibu s-F Nati ull S ve P ta c k ack age s
  19. 19. The Chef API • HTTPS, RESTful API w/ JSON, RSA key auth • Infrastructure data store such as node data • Search Service • Derivative Services? http://www.flickr.com/photos/core-materials/4419853626/sizes/o/in/photostream/
  20. 20. The Chef Community • Apache License, Version 2.0 • Thousands of Individual and Corporate contributors. • Thousands of cookbooks available from the community • http://community.opscode.com
  21. 21. How does it work? http://i3.kym-cdn.com/photos/images/original/000/046/123/magnets.jpg
  22. 22. Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.
  23. 23. Programs • Chef generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/
  24. 24. Declarative Interface to Resources • Define Policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/
  25. 25. That Looks Like This package "apache2" template "/etc/apache2/apache2.conf" do! source "apache2.conf.erb"! owner "root"! group "root"! mode "0644"! variables(:allow_override => "All")! notifies :reload, "service[apache2]"! end service "apache2" do! action [:enable,:start]! supports :reload => true! end
  26. 26. Chef Nodes • Chef runs on nodes • Chef nodes do the heavy lifting • Authority about themselves • Stored on the server when using Chef Server • Indexed for search http://www.flickr.com/photos/peterrosbjerg/3913766224/
  27. 27. Search • Search for nodes with Roles • Find Topology Data ! • IP addresses • Hostnames • FQDNs http://www.flickr.com/photos/kathycsus/2686772625
  28. 28. So when this... Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  29. 29. ...becomes this Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  30. 30. ...this can happen automatically Graphite Nagios Jboss App ! ! ! Memcache ! ! Postgres Slaves Postgres Master
  31. 31. Count the Resources • Graphite Nagios • • • Jboss App ! ! • ! • Memcache ! • ! Postgres Slaves • • • • 12+ resource changes for 1 node addition • Load balancer config Nagios host ping Nagios host ssh Nagios host HTTP Nagios host app health Graphite CPU Graphite Memory Graphite Disk Graphite SNMP Memcache firewall Postgres firewall Postgres authZ config
  32. 32. Landscape of Chef-managed Infrastructure
  33. 33. knife
  34. 34. knife - with the Chef Server • knife node • create / edit / delete • list • knife cookbook ... • knife role ... • knife environment ...
  35. 35. knife bootstrap $ knife bootstrap SERVER -r ‘role[webserver]’ • SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef!
  36. 36. Chef & CloudStack gem install knife-cloudstack
  37. 37. knife-cloudstack 1/4 knife cs • ** CS COMMANDS ** • knife cs aag list (options) • knife cs account list (options) • knife cs cluster list (options) • knife cs config list (options) • knife cs disk list (options) • knife cs domain list (options) • knife cs firewallrule create hostname 8080:8090:TCP:10.0.0.0/24 • knife cs firewallrule list (options) • knife cs forwardrule create hostname 8080:8090:TCP • knife cs host list (options)
  38. 38. knife-cloudstack 2/4 knife cs • knife cs hosts • knife cs iso list (options) • knife cs keypair create KEY_NAME (options) • knife cs keypair delete KEY_NAME (options) • knife cs keypair list (options) • knife cs network list (options) • knife cs oscategory list (options) • knife cs ostype list (options) • knife cs pod list (options) • knife cs project list (options)
  39. 39. knife-cloudstack 3/4 knife cs • knife cs publicip list (options) • knife cs router list (options) • knife cs securitygroup list (options) • knife cs server add nic SERVERID NETWORKID (--ipaddress X.X.X.X) • knife cs server create [SERVER_NAME] (options) • knife cs server delete SERVER_NAME [SERVER_NAME ...] (options) • knife cs server list (options) • knife cs server reboot SERVER_NAME [SERVER_NAME ...] (options) • knife cs server remove nic SERVERID NICID • knife cs server start SERVER_NAME [SERVER_NAME ...] (options)
  40. 40. knife-cloudstack 4/4 knife cs • knife cs server stop SERVER_NAME [SERVER_NAME ...] (options) • knife cs service list (options) • knife cs stack create JSON_FILE (options) • knife cs stack delete JSON_FILE (options) • knife cs template create NAME (options) • knife cs template extract NAME (options) • knife cs template list (options) • knife cs template register NAME (options) • knife cs user list (options) • knife cs volume create NAME (options) • knife cs volume list (options) • knife cs zone list (options)
  41. 41. Why the Cloud? • Instant infrastructure • Unlimited capacity • Autoscaling • No commitment • Immediate replacement
  42. 42. Why not the Cloud? • Reliability • Performance • Security • Price
  43. 43. Data Gravity
  44. 44. Know our escape plan for every infrastructure provider
  45. 45. Chef for Infrastructure Portability • knife rackspace • knife openstack • knife azure • knife cloudstack • knife ec2 • knife google • knife hp • knife vsphere • ...and many others
  46. 46. Desktop, Virtualization & Cloud • • • • • • Vagrant VMware CloudStack Eucalyptus OpenStack bare metal • • • • • • AWS Rackspace HP Google Azure many others ™
  47. 47. Desktop, Virtualization & Cloud • • • • • • Vagrant VMware CloudStack Eucalyptus OpenStack bare metal • • • • • • AWS Rackspace HP Google Azure many others ™
  48. 48. What does this all mean? •Every infrastructure is a unique snowflake •Understand the costs associated with the features of your platform(s) of choice. •Chef enables Infrastructure Portability •Use the same infrastructure code for wherever you deploy •"Data Gravity" is a concern ™
  49. 49. There’s a lot more... • Attributes • Environments • Roles • Chef Handlers • LWRPs • Workflow Helpers • Testing Frameworks • and even more awesome
  50. 50. Thanks! Matt Ray matt@getchef.com @mattray
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×