SCALE 10x Build a Cloud Day

Introduction to ChefSCALE Build a Cloud Day matt@opscode.com www.opscode.com @mattray

Congratulations!!! • U has a cloud • Now what? http://www.ﬂickr.com/photos/ian_munroe/4758240536/

APIs are awesome!• You can provision compute resources in seconds• You can provision storage resources in seconds• That’s cool. http://www.ﬂickr.com/photos/jdhancock/3634246981/

Chef can help with that• knife ec2 server • knife cloudstack create server create• knife rackspace • knife kvm server server create create• knife terremark • knife vsphere server create server create• knife voxel • knife eucalyptus server create server create• knife gandi • knife openstack server create server createhttp://www.ﬂickr.com/photos/kyz/3122499444/

But then what?http://www.ﬂickr.com/photos/doctorow/2698336843

You need to configure them

See NodeApplication Server

See NodesApplication ServerApplication Database

See Nodes GrowApplication ServerApplication Databases

See Nodes GrowApplication ServersApplication Databases

See Nodes GrowLoad BalancerApplication ServersApplication Databases

See Nodes GrowLoad Balancers Application ServersApplication Databases

See Nodes GrowLoad Balancers Application ServersApplication Database CacheApplication Databases

Tied together with Config Load Balancers Application Servers Application Database Cache Application Databases

Infrastructure is a Snowflake Load Balancers Application Servers Application Database Cache Floating IP? Application Databases

Evolving Complexity Load Balancers ApplicationCache Application ServersNoSQL Database Cache Database Slaves Database

Complexity Grows Quickly DC2DC1 DC3

And it Continues to Evolve http://www.ﬂickr.com/photos/16339684@N00/2681435235/

Golden Images are not the answer• Gold is heavy• Hard to transport• Hard to mold• Easy to lose configuration detail http://www.ﬂickr.com/photos/garysoup/2977173063/

Typical Boring InfrastructureGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master

New Compliance Mandate Graphite Nagios Jboss App• Move SSH off port 22• Lets put it on 2022 Memcache Postgres Slaves Postgres Master

6 Golden Image Updates Graphite 1 2 Nagios 3 Jboss App• edit /etc/ssh/sshd_config 4 Memcache 5 Postgres Slaves 6 Postgres Master

12 Instance Replacements Graphite 1 2 Nagios 3 4 5 6 7 Jboss App• Delete, launch 8• Repeat 9 Memcache• Typically manually 10 11 Postgres Slaves 12 Postgres Master

In a Maintenance Window Graphite 1 2 Nagios 3 4 5 6 7 Jboss App• Dont break anything! 8 9 Memcache 5 10 11 Postgres Slaves 12 Postgres Master

With Different IP Addresses? Graphite Nagios Jboss App• Invalid Configs• Bob just got fired :( Memcache Postgres Slaves Postgres Master

Configuration Desperation http://www.ﬂickr.com/photos/francoforeshock/5716969942/

Configuration Management andAutomated Systems Integration is the Answer http://www.ﬂickr.com/photos/philliecasablanca/3354734116/

Chef Solves This Problem • But you already guessed that, didn’t you?

Collections of Resources• Networking • Routes • Users• Files • Groups• Directories • Tasks• Symlinks • Packages• Mounts • Software • Services • Configurations • Other Stuff http://www.ﬂickr.com/photos/stevekeys/3123167585/

Acting in Concert http://www.ﬂickr.com/photos/glowjangles/4081048126/

To Provide a Service http://www.ﬂickr.com/photos/28309157@N08/3743455858/

Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. http://www.ﬂickr.com/photos/louisb/4555295187/

Declarative Interface to Resources• Define policy• Say what, not how• Pull not Pushhttp://www.ﬂickr.com/photos/bixentro/2591838509/

That looks like thisextra_packages = case node[platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend

Or thissearch(:users, *:*) do |u| user u[id] do uid u[uid] shell u[shell] home "/home/#{u[id]}" end directory "#{home_dir}/.ssh" do owner u[id] group u[gid] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u[id] group u[id] mode "0600" variables :ssh_keys => u[ssh_keys] endend

Recipes and Cookbooks• Recipes are collections of Resources• Cookbooks contain recipes, templates, files, custom resources, etc• Code re-use and modularity• Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/

Nodes • Chef-Client generates configurations directly on nodes • Reduce management complexity through abstraction • Store the configuration of your programs in version controlhttp://www.flickr.com/photos/ssoosay/5126146763/

Upload your infrastructureknife cookbook upload aptknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload jpackageknife cookbook upload ntpknife cookbook upload sudoknife cookbook upload tomcatknife cookbook upload usersknife cookbook upload sampleknife role from file base-cloud.rbknife role from file tc.rbknife role from file sample.rbknife data bag create usersknife data bag from file users mray.json

Build it somewhere#EC2knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce-f m1.small -d omnibus -r role[base-cloud],role[tc],role[sample]#Rackspaceknife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -rrole[base-cloud],role[tc],role[sample]#CloudStackknife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i~/.ssh/mray.pem -d omnibus -r role[base-cloud],role[tc],role[sample]#Ubuntu Linuxknife bootstrap test.lab -r role[webserver] -i ~/.ssh/mray.pem -x ubuntu --sudo -domnibus -r role[base-cloud],role[tc],role[sample]

Search• Search for nodes with Roles• Find configuration data• IP addresses• Hostnames• FQDNs http://www.ﬂickr.com/photos/kathycsus/2686772625

Pass results into Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end

Pass results into Templates# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>

So when thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master

Becomes thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master

Updates can be automaticGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master

Count the resources • Load balancer config Graphite Nagios • Nagios host ping • Nagios host ssh Jboss App • Nagios host HTTP • Nagios host app health Memcache • Graphite CPU • Graphite Memory Postgres Slaves • Graphite Disk • Graphite SNMP • Memcache firewall• 12+ resource changes for 1 node addition • Postgres firewall Postgres authZ config

CLONING CANNOT COPE WITH THIS • Chef can. http://www.ﬂickr.com/photos/evelynishere/2798236471/

Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name ithttp://www.ﬂickr.com/photos/hyku/245010680/

And manage it simply• Automatically reconfigure everything• Linux, Windows, Unixes, BSDs• Load balancers• Metrics collection systems• Monitoring systems• Cloud migrations become trivial http://www.ﬂickr.com/photos/helico/404640681/

The Chef Community• Apache License, Version 2.0• 550+ Individual contributors• 100+ Corporate contributors • Dell, Rackspace,VMware, RightScale, Heroku, and many more• Nearly 400 cookbooks• http://community.opscode.com

Questions? http://www.ﬂickr.com/photos/mrchippy/443960682/

Thanks! matt@opscode.com www.opscode.com

SCALE 10x Build a Cloud Day

by Opscode, Inc. on Jan 23, 2012

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 11

Statistics

SCALE 10x Build a Cloud Day — Presentation Transcript

http://a0.twimg.com	10
http://blog.sahsu.mobi	1