SCALE 10x Build a Cloud Day
Upcoming SlideShare
Loading in...5
×
 

SCALE 10x Build a Cloud Day

on

  • 163,702 views

Matt Ray's Introduction to Chef talk from the 2012 Southern California Linux Expo's Build a Cloud Day.

Matt Ray's Introduction to Chef talk from the 2012 Southern California Linux Expo's Build a Cloud Day.

Statistics

Views

Total Views
163,702
Views on SlideShare
163,691
Embed Views
11

Actions

Likes
8
Downloads
88
Comments
0

2 Embeds 11

http://a0.twimg.com 10
http://blog.sahsu.mobi 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    SCALE 10x Build a Cloud Day SCALE 10x Build a Cloud Day Presentation Transcript

    • Introduction to ChefSCALE Build a Cloud Day matt@opscode.com www.opscode.com @mattray
    • Congratulations!!! • U has a cloud • Now what? http://www.flickr.com/photos/ian_munroe/4758240536/
    • APIs are awesome!• You can provision compute resources in seconds• You can provision storage resources in seconds• That’s cool. http://www.flickr.com/photos/jdhancock/3634246981/
    • Chef can help with that• knife ec2 server • knife cloudstack create server create• knife rackspace • knife kvm server server create create• knife terremark • knife vsphere server create server create• knife voxel • knife eucalyptus server create server create• knife gandi • knife openstack server create server createhttp://www.flickr.com/photos/kyz/3122499444/
    • But then what?http://www.flickr.com/photos/doctorow/2698336843
    • You need to configure them
    • See NodeApplication Server
    • See NodesApplication ServerApplication Database
    • See Nodes GrowApplication ServerApplication Databases
    • See Nodes GrowApplication ServersApplication Databases
    • See Nodes GrowLoad BalancerApplication ServersApplication Databases
    • See Nodes GrowLoad Balancers Application ServersApplication Databases
    • See Nodes GrowLoad Balancers Application ServersApplication Database CacheApplication Databases
    • Tied together with Config Load Balancers Application Servers Application Database Cache Application Databases
    • Infrastructure is a Snowflake Load Balancers Application Servers Application Database Cache Floating IP? Application Databases
    • Evolving Complexity Load Balancers ApplicationCache Application ServersNoSQL Database Cache Database Slaves Database
    • Complexity Grows Quickly DC2DC1 DC3
    • And it Continues to Evolve http://www.flickr.com/photos/16339684@N00/2681435235/
    • Golden Images are not the answer• Gold is heavy• Hard to transport• Hard to mold• Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
    • Typical Boring InfrastructureGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
    • New Compliance Mandate Graphite Nagios Jboss App• Move SSH off port 22• Lets put it on 2022 Memcache Postgres Slaves Postgres Master
    • 6 Golden Image Updates Graphite 1 2 Nagios 3 Jboss App• edit /etc/ssh/sshd_config 4 Memcache 5 Postgres Slaves 6 Postgres Master
    • 12 Instance Replacements Graphite 1 2 Nagios 3 4 5 6 7 Jboss App• Delete, launch 8• Repeat 9 Memcache• Typically manually 10 11 Postgres Slaves 12 Postgres Master
    • In a Maintenance Window Graphite 1 2 Nagios 3 4 5 6 7 Jboss App• Dont break anything! 8 9 Memcache 5 10 11 Postgres Slaves 12 Postgres Master
    • With Different IP Addresses? Graphite Nagios Jboss App• Invalid Configs• Bob just got fired :( Memcache Postgres Slaves Postgres Master
    • Configuration Desperation http://www.flickr.com/photos/francoforeshock/5716969942/
    • Configuration Management andAutomated Systems Integration is the Answer http://www.flickr.com/photos/philliecasablanca/3354734116/
    • Chef Solves This Problem • But you already guessed that, didn’t you?
    • Collections of Resources• Networking • Routes • Users• Files • Groups• Directories • Tasks• Symlinks • Packages• Mounts • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/
    • Acting in Concert http://www.flickr.com/photos/glowjangles/4081048126/
    • To Provide a Service http://www.flickr.com/photos/28309157@N08/3743455858/
    • Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
    • Declarative Interface to Resources• Define policy• Say what, not how• Pull not Pushhttp://www.flickr.com/photos/bixentro/2591838509/
    • That looks like thisextra_packages = case node[platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend
    • Or thissearch(:users, *:*) do |u| user u[id] do uid u[uid] shell u[shell] home "/home/#{u[id]}" end directory "#{home_dir}/.ssh" do owner u[id] group u[gid] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u[id] group u[id] mode "0600" variables :ssh_keys => u[ssh_keys] endend
    • Recipes and Cookbooks• Recipes are collections of Resources• Cookbooks contain recipes, templates, files, custom resources, etc• Code re-use and modularity• Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
    • Nodes • Chef-Client generates configurations directly on nodes • Reduce management complexity through abstraction • Store the configuration of your programs in version controlhttp://www.flickr.com/photos/ssoosay/5126146763/
    • Upload your infrastructureknife cookbook upload aptknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload jpackageknife cookbook upload ntpknife cookbook upload sudoknife cookbook upload tomcatknife cookbook upload usersknife cookbook upload sampleknife role from file base-cloud.rbknife role from file tc.rbknife role from file sample.rbknife data bag create usersknife data bag from file users mray.json
    • Build it somewhere#EC2knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce-f m1.small -d omnibus -r role[base-cloud],role[tc],role[sample]#Rackspaceknife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -rrole[base-cloud],role[tc],role[sample]#CloudStackknife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i~/.ssh/mray.pem -d omnibus -r role[base-cloud],role[tc],role[sample]#Ubuntu Linuxknife bootstrap test.lab -r role[webserver] -i ~/.ssh/mray.pem -x ubuntu --sudo -domnibus -r role[base-cloud],role[tc],role[sample]
    • Search• Search for nodes with Roles• Find configuration data• IP addresses• Hostnames• FQDNs http://www.flickr.com/photos/kathycsus/2686772625
    • Pass results into Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end
    • Pass results into Templates# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>
    • So when thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
    • Becomes thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
    • Updates can be automaticGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
    • Count the resources • Load balancer config Graphite Nagios • Nagios host ping • Nagios host ssh Jboss App • Nagios host HTTP • Nagios host app health Memcache • Graphite CPU • Graphite Memory Postgres Slaves • Graphite Disk • Graphite SNMP • Memcache firewall• 12+ resource changes for 1 node addition • Postgres firewall Postgres authZ config
    • CLONING CANNOT COPE WITH THIS • Chef can. http://www.flickr.com/photos/evelynishere/2798236471/
    • Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name ithttp://www.flickr.com/photos/hyku/245010680/
    • And manage it simply• Automatically reconfigure everything• Linux, Windows, Unixes, BSDs• Load balancers• Metrics collection systems• Monitoring systems• Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
    • The Chef Community• Apache License, Version 2.0• 550+ Individual contributors• 100+ Corporate contributors • Dell, Rackspace,VMware, RightScale, Heroku, and many more• Nearly 400 cookbooks• http://community.opscode.com
    • Questions? http://www.flickr.com/photos/mrchippy/443960682/
    • Thanks! matt@opscode.com www.opscode.com