Achieving Infrastructure Portability with Chef


Published on

Deploying to the cloud has made it easy to run large numbers of servers, but users may become dissatisfied with their particular cloud platform for reasons such as price, support and performance. There are a number of vendor lock-ins to avoid, this talk discusses how to do so with the open source configuration management and infrastructure automation platform Chef. Chef makes it easy to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. Chef may also be used to deploy cloud infrastructures such as OpenStack, Eucalyptus or CloudStack. By abstracting away the platform, infrastructure becomes portable and you are free to deploy wherever necessary.

Published in: Technology, Self Improvement
1 Comment
  • 22-Título = Venha se divertir de verdade !!!
    [b]Ola,sou o Pr1nceMG,Divulgador oficial do Servidor de Mu online Season 6 do Brasil
    ESTÁ ON HÁ MAIS DE 5 ANOS,Produzindo sua Diversão com qualidade.
    TODOS OS SERVERS ficam ON 24 horas por dia, Sempre Buscamos o Melhor para os Gamers.
    São varios Server esperando por você :
    * MuWar' 1000x/1500x
    * Super - 10.000x ** Pvp 15.000x
    * Very Easy - 5.000x
    * Hard 100 x
    * Extreme 10x
    * Novo servidor Phoenix: Free 3000x | Vip: 4000x Phoenix
    Os Melhores itens e kits mais tops de um server De MU Online:
    * Novas asas level 4
    * Novos Kits DEVASTADOR
    * Novos Kits DIAMOND v2 + Kit Mystical (a combinação da super força)
    * Novos Sets especiais de TIME.
    Esperamos pela sua visita.Sejam todos muito benvindos ao nosso Servidor.
    *Um mês de grandes eventos e Promoções do dia das Crianças e Sorteio de 1 iPad e 2.000.000 de Golds!
    E obrigado pela atençao de todos voces !!!
    Conheça também animes cloud São mais de 20.000 mil videos online.
    By:Pr1nceMG divulgador oficial do
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Achieving Infrastructure Portability with Chef

  1. Achieving Infrastructure Portabilitywith Chef CloudOpen 2012 Matt Ray IRC/Twitter/GitHub: mattray
  2. Data Centers...
  3. To the Cloud!
  4. Why the Cloud?• Instant infrastructure• Unlimited capacity• Autoscaling• No commitment• Immediate replacement
  5. Cloud Differentiation• Each cloud defines themselves against Amazon• Entry into the market is easier• Not a lot of price competition• Feature parity is growing
  6. The Dark Side of the Cloud
  7. Why not the Cloud?• Reliability• Performance• Security• Price
  8. Data Gravity
  9. Know our escape planfor every infrastructure provider
  10. Why Chef?
  11. See NodeApplication Server
  12. See NodesApplication ServerApplication Database
  13. See Nodes GrowApplication ServerApplication Databases
  14. See Nodes GrowApplication ServersApplication Databases
  15. See Nodes GrowLoad BalancerApplication ServersApplication Databases
  16. See Nodes GrowLoad Balancers Application ServersApplication Databases
  17. See Nodes GrowLoad Balancers Application ServersApplication Database CacheApplication Databases
  18. Tied together with ConfigLoad Balancers Application ServersApplication Database CacheApplication Databases
  19. Infrastructure is a SnowflakeLoad Balancers Application ServersApplication Database CacheFloating IP?Application Databases
  20. Evolving Complexity Load Balancers ApplicationCache Application ServersNoSQL Database Cache Database Slaves Database
  21. Complexity Grows Quickly DC2DC1 DC3
  22. And it Continues to EvolveThats great and all, but tell me about Chef!
  23. Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.
  24. Nodes • Chef-Client generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control
  25. Collections of Resources• Networking • Routes • Users• Files • Groups• Directories • Tasks• Symlinks • Packages• Mounts • Software • Services • Configurations • Other Stuff
  26. Declarative Interface to Resources• Define policy• Say what, not how• Pull not Push
  27. Ruby!extra_packages = case node[platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend
  28. Recipes and Cookbooks• Recipes are collections of Resources• Cookbooks contain recipes, templates, files, custom resources, etc• Code re-use and modularity• Hundreds already on
  29. Search• Search for nodes with Roles• Find configuration data• IP addresses• Hostnames• FQDNs
  30. Pass Results to Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end
  31. Pass Results to Templates# Set up application listeners here.listen application balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin mode http stats uri /<% end -%>
  32. So when thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  33. Becomes thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  34. Updates can be automaticGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  35. Count the resources • Load balancer config Graphite Nagios • Nagios host ping • Nagios host ssh Jboss App • Nagios host HTTP • Nagios host app health Memcache • Graphite CPU • Graphite Memory Postgres Slaves • Graphite Disk • Graphite SNMP • Memcache firewall• 12+ resource changes for 1 node addition • Postgres firewall Postgres authZ config
  36. Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it
  37. And manage it simply• Automatically reconfigure everything• Linux, Windows, Unixes, BSDs• Load balancers• Metrics collection systems• Monitoring systems• Cloud migrations become trivial
  38. knife
  39. knife with the Chef Server• knife node • create/delete/edit • list• knife cookbook ...• knife role ...• knife environment ...
  40. knife bootstrap knife bootstrap SERVER -r role[webserver] -i ~/.ssh/id_rsa• SSH to the machine given existing credentials• Install the Chef Client• Register with the Chef Server• Run the initial Run List• Now managed with Chef!
  41. knife ec2$ knife ec2Available ec2 subcommands: (for details, knife SUB-COMMAND --help)** EC2 COMMANDS **knife ec2 flavor list (options)knife ec2 instance data (options)knife ec2 server create (options)knife ec2 server delete SERVER [SERVER] (options)knife ec2 server list (options)$ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu-I ami-4721882e -f m1.small -r role[webserver]
  42. knife openstack$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)$ knife openstack server create -S keypair -i ~/.ssh/id_rsa-x ubuntu -I 1231 -f standard.small -r role[webserver]
  43. Chef for Infrastructure Portability• knife ec2 • knife cloudstack• knife rackspace • knife openstack• knife hp • knife vsphere• knife google • ... and many• knife azure others
  44. The Chef Community• Apache License, Version 2.0• 850+ Individual contributors• 150+ Corporate contributors • HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more• 550+ cookbooks•
  45. Desktop, Virtualization, Private & Public Clouds• Vagrant • AWS• VMware • Rackspace• CloudStack • HP• Eucalyptus • Google• OpenStack • Azure• bare metal • many others
  46. Desktop, Virtualization, Private & Public Clouds• Vagrant • AWS• VMware • Rackspace• CloudStack • HP• Eucalyptus • Google• OpenStack • Azure• bare metal • many others
  47. Tale of the Tape• Artur Bergman, CEO at Fastly • Its All About Speed •• Jason Stowe, CEO at Cycle Computing • CycleCloud + Chef = 50,000-core Utility Supercomputer for Science •
  48. Chef and Abstractions• Resources and Providers• Cookbooks (may) normalize deployment• Knife treats APIs the same• Chef strives to not be opinionated
  49. Environments• Lock down versions of cookbooks • stable releases vs. development• Enforce attributes for deployment • Ports, addresses, etc.• Different run lists based on environment • Debugging enabled?
  50. Environments• Use the same infrastructure code for wherever you deploy• Development, QA, Pre-Production, Prod• Role-based Access Controls to restrict the promotion of deployment code
  51. TL;DL• Every infrastructure is a unique snowflake• Understand the costs associated with the features of your platform(s) of choice.• Chef enables Infrastructure Portability• "Data Gravity" is the primary concern
  52. Thanks! Matt Ray IRC/Twitter/GitHub: mattray