vBACD - Introduction to Opscode Chef - 2/29

Automating Cloud Deployments with Chef [email_address] www.opscode.com Twitter/IRC/GitHub: mattray

U has a cloud
Now what?
http://www.flickr.com/photos/ian_munroe/4758240536/ Congratulations!!!

But then what? http://www.flickr.com/photos/doctorow/2698336843

You need to configure them

APIs are awesome!
You can provision compute resources in seconds
You can provision storage resources in seconds
That’ s cool.
http://www.flickr.com/photos/jdhancock/3634246981/

Chef can help with that
knife ec2 server create
knife rackspace server create
knife terremark server create
knife voxel server create
knife cloudstack server create
knife kvm server create
knife vsphere server create
knife eucalyptus server create
knife openstack server create
http://www.flickr.com/photos/kyz/3122499444/

See Node Application Server

See Nodes Application Server Application Database

See Nodes Grow Application Server Application Databases

Application Servers Application Databases See Nodes Grow

Application Servers Application Databases Load Balancer See Nodes Grow

See Nodes Grow Application Servers Application Databases Load Balancers

See Nodes Grow Application Servers Application Database Cache Load Balancers Application Databases

Tied together with Config Application Servers Application Database Cache Load Balancers Application Databases

Infrastructure is a Snowflake Application Servers Application Database Cache Load Balancers Floating IP? Application Databases

Evolving Complexity Load Balancers Application Servers NoSQL Database Slaves Application Cache Database Cache Database

Complexity Grows Quickly DC1 DC3 DC2

http://www.flickr.com/photos/ 16339684 @N00/2681435235/ And it Continues to Evolve

Golden Images are not the answer
Gold is heavy
Hard to transport
Hard to mold
Easy to lose configuration detail
http://www.flickr.com/photos/garysoup/2977173063/

Typical Boring Infrastructure Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite

Move SSH off port 22
Lets put it on 2022
New Compliance Mandate Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite

edit /etc/ssh/sshd_config
6 Golden Image Updates Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 3 4 5 6

Delete, launch
Repeat
Typically manually
8 12 Instance Replacements Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 3 4 5 6 7 9 10 11 12

Don't break anything!
In a Maintenance Window 5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3

Invalid Configs
Bob just got fired :(
With Different IP Addresses? Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite

http://www.flickr.com/photos/francoforeshock/5716969942/ Configuration Desperation

Configuration Management and Automated Systems Integration is the Answer http://www.flickr.com/photos/philliecasablanca/3354734116/

Chef Solves This Problem
But you already guessed that, didn ’t you?

Collections of Resources
Networking
Files
Directories
Symlinks
Mounts
Routes
Users
Groups
Tasks
Packages
Software
Services
Configurations
Other Stuff
http://www.flickr.com/photos/stevekeys/3123167585/

http://www.flickr.com/photos/glowjangles/4081048126/ Acting in Concert

http://www.flickr.com/photos/28309157@N08/3743455858/ To Provide a Service

Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/
Programmatically provision and configure
Treat like any other code base
Reconstruct business from code repository, data backup, and bare metal resources.

Declarative Interface to Resources
Define policy
Say what, not how
Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/

That looks like this extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end

Or this search(:users, '*:*') do |u| user u['id'] do uid u['uid'] shell u['shell'] home "/home/#{u['id']}" end directory "#{home_dir}/.ssh" do owner u['id'] group u['gid'] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['id'] mode "0600" variables :ssh_keys => u['ssh_keys'] end end

Recipes and Cookbooks
Recipes are collections of Resources
Cookbooks contain recipes, templates, files, custom resources, etc
Code re-use and modularity
Hundreds already on Community.opscode.com
http://www.flickr.com/photos/shutterhacks/4474421855/

Chef-Client generates configurations directly on nodes from their run list
Reduce management complexity through abstraction
Store the configuration of your programs in version control
http://www.flickr.com/photos/ssoosay/5126146763/ Nodes

Upload your infrastructure knife cookbook upload chef-client knife cookbook upload java knife cookbook upload jpackage knife cookbook upload ntp knife cookbook upload sudo knife cookbook upload tomcat knife cookbook upload users knife cookbook upload sample knife role from file base.rb knife role from file tc.rb knife role from file sample.rb knife data bag create users knife data bag from file users mray.json

Build it somewhere #EC2 knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce -f m1.small -d omnibus -r 'role[base],role[tc],role[sample] ’ #Rackspace knife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -r 'role[base],role[tc],role[sample] ’ #CloudStack knife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i ~/.ssh/mray.pem -d omnibus -r 'role[base],role[tc],role[sample] ’ #Ubuntu Linux VM knife bootstrap test.lab -i ~/.ssh/mray.pem -x ubuntu --sudo -d omnibus -r 'role[base],role[tc],role[sample]'

Tomcat stack deployed ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Chef Run complete in 125.548799554 seconds ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Running report handlers ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Report handlers complete Instance ID: i-ee18148b Flavor: m1.small Image: ami-0c6ebd65 Region: us-east-1 Availability Zone: us-east-1b Security Groups: default SSH Key: mray Root Device Type: instance-store Public DNS Name: ec2-107-21-179-169.compute-1.amazonaws.com Public IP Address: 107.21.179.169 Private DNS Name: ip-10-120-255-91.ec2.internal Private IP Address: 10.120.255.91 Environment: _default Run List: role[base], role[tc], role[sample]

Tomcat stack deployed

Update the sample cookbook $ knife cookbook upload sample Uploading sample [0.0.2] upload complete

Update the nodes $ knife ssh "role:base" "sudo chef-client" -i ~/.ssh/mray.pem -x ubuntu -a cloud.public_ipv4 107.21.179.169 [Thu, 23 Feb 2012 03:30:22 +0000] INFO: *** Chef 0.10.8 *** 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Run List is [role[base], role[tc], role[sample]] 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Run List expands to [chef-client, ntp, sudo, users::sysadmins, java, tomcat, sample] 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Starting Chef Run for i-ee18148b 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Running start handlers 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Start handlers complete. 107.21.179.169 [Thu, 23 Feb 2012 03:30:36 +0000] INFO: Loading cookbooks [chef-client, java, jpackage, ntp, sample, sudo, tomcat, users] 107.21.179.169 [Thu, 23 Feb 2012 03:30:36 +0000] INFO: Storing updated cookbooks/sample/recipes/default.rb in the cache. <SNIP> 107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: template[/var/lib/tomcat6/webapps/sample/index.html] updated content 107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: template[/var/lib/tomcat6/webapps/sample/index.html] sending restart action to service[tomcat] (immediate) 107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: Processing service[tomcat] action restart (tomcat::default line 42) 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: service[tomcat] restarted 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Chef Run complete in 15.170097638 seconds 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Running report handlers 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Report handlers complete

Tomcat stack updated

http://www.flickr.com/photos/kathycsus/2686772625
IP addresses
Hostnames
FQDNs
Search for nodes with Roles
Find configuration data
Search

pool_members = search("node","role:webserver ”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb ” owner "root" group "root ” mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy] ” end Pass results into Templates

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |memb| -%> server <%= memb[:hostname] %> <%= memb[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates

Jboss App Memcache Postgres Slaves Postgres Master So when this Nagios Graphite

Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite Becomes this

Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite Updates can be automatic

Nagios Graphite Count the resources Jboss App Memcache Postgres Slaves
Load balancer config
Nagios host ping
Nagios host ssh
Nagios host HTTP
Nagios host app health
Graphite CPU
Graphite Memory
Graphite Disk
Graphite SNMP
Memcache firewall
Postgres firewall
Postgres authZ config
12+ resource changes for 1 node addition

http://www.flickr.com/photos/evelynishere/2798236471/ CLONING CANNOT COPE WITH THIS
Chef can.

Build anything
Simple internal applications
Complex external applications
Workstations
Hadoop clusters
IaaS infrastructure
PaaS infrastructure
SaaS applications
Storage systems
You name it
http://www.flickr.com/photos/hyku/245010680/

And manage it simply http://www.flickr.com/photos/helico/404640681/
Automatically reconfigure everything
Linux, Windows, Unixes, BSDs
Load balancers
Metrics collection systems
Monitoring systems
Cloud migrations become trivial

The Chef Community
Apache License, Version 2.0
550+ Individual contributors
100+ Corporate contributors
Dell, Rackspace,VMware, RightScale, Heroku, and many more
400+ Community cookbooks
http://community.opscode.com

Questions? http://www.flickr.com/photos/mrchippy/443960682/ Questions?

Thanks! [email_address] www.opscode.com Twitter/IRC/GitHub: mattray

vBACD - Introduction to Opscode Chef - 2/29

by CloudStack - Open Source Cloud Computing Project on Feb 23, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

vBACD - Introduction to Opscode Chef - 2/29 Presentation Transcript