Deep Dive into Opscode Chef


Published on

Opscode Chef is the leader in open source cloud infrastructure automation. This Session covers:
- Chef Fundamentals
- Infrastructure as Code
- How Chef works?
- Core concepts: Nodes, Recipes, Roles, Databags & Environments
- Knife - the "swiss army knife" of infrastructure management tools.
- Build anything with Chef!

Published in: Technology

Deep Dive into Opscode Chef

  1. 1. Opscode Chef – Deep DiveKalpak ShahCEO, Clogeny Technologies
  2. 2. IntroductionsKalpak Shah - CEO, Clogeny • Leading strategy and product management for cutting-edge product development in cloud, DevOps & storage space.You? • Developers? • System Administrators? • Architects? • Managers? • Executives?
  3. 3. Your application / infrastructureAppLBs App ServersApp DB CacheApp DBs
  4. 4. Collection of Resources• Nodes • Tasks• Networking • Packages• Files • Software• Directories • Services• Symlinks • Configurations• Mounts • And more..• Routes• Users• Groups
  5. 5. Acting in concert
  6. 6. To provide a Service
  7. 7. And it evolves….
  8. 8. Complexity increases quickly App LBs Cache App ServersNoSQ DB CacheL DB slaves DBs
  9. 9. Complexity increases very quickly
  10. 10. New Compliance Mandate Graphite Nagios Jboss App• Move SSH Memcache off port 22 Postgres Slaves• Lets put it on 2022 Postgres Master
  11. 11. Different IP Addresses? Graphite Nagios Jboss App Memcache Postgres Slaves• Invalid configs! Postgres Master
  12. 12. Chef – 10,000 feet view• A library for configuration management• A configuration management system• A system integration platform• An API for your entire infrastructure• Idempotent, Data-driven, Sane defaults, Extensibility (Hackability)Chef enables Infrastructure as Code
  13. 13. Infrastructure as Code??• Manage infrastructure as idempotent Resources• Put them together in Recipes• Configure your servers and run your integrated infrastructure• Track and manage like source code• Reconstruct business from code repository, data backup, and bare metal resources.• Ruby DSL (Domain Specific Language)
  14. 14. A Resource… 20 package “sudo” do• Is of a given type 21 version „1.6.8p12‟• Has a name 22 action :install 23 end• Has attributes• Takes actions to bring the 60 service „apache2‟ do resource into 61 action [ :enable, :start] desired state 62 end
  15. 15. Declarative Interface to Resources • Define policy • Say what, not how • Pull not Push
  16. 16. Providers• Resources take actions through providers• You say “what” is to be done not “how”• Multiple providers per resource type • Apt, yum, rubygems, portage, macports, etcResources > Platform > Provider
  17. 17. Chef Resources package "haproxy" do action :install• Have a type. end• Have a name. template "/etc/haproxy/haproxy.cfg" do• Have parameters. source "haproxy.cfg.erb" owner "root"• Take action to put the group "root" resource in the mode 0644 declared state. notifies :restart, "service[haproxy]" end• Can send notifications to other service "haproxy" do resources. supports :restart => true action [:enable, :start] end
  18. 18. Recipes• Recipes are a collection of resources• Code re-use & modularityLet‟s look atapache2recipe…
  19. 19. How do you think of your servers?Similar infrastructure will be replicated acrossdev, QA, pre-prod & production environments Role=LB App LBs Cache Role=AppServer App Servers NoSQ DB Cache Role=DBCache L DB slaves Role=DBSlave DBs Role=DB
  20. 20. Roles• Roles describe what a node should be• Roles are searchable• Roles consist of a run_list • What Roles & Recipes to apply in order• Roles are applied to nodesRoles define the personality of your server –thereby what gets installed/configured onyour server.
  21. 21. Cookbooks• Distributable• Infrastructure as Code• Version control repository• Contains • Recipes • Assets (files/templates) – static & dynamic • Attributes • Metadata
  22. 22. Community Cookbooks• 700+ cookbooks for everything – databases, web servers, applications, CMS, package management, Hadoop, Utilities, Cloud deployments
  23. 23. Environments• Use same infrastructure code wherever you deploy• Dev, QA, Pre-production, Production• Role Based Access Control(RBAC) to restrict promotion of deployment code• Examples: • Dev & QA in Amazon EC2 cloud while pre-prod & prod in on-premise infrastructure • Different server sizes & performance settings in dev & prod environments
  24. 24. Data Driven!!• Separation of policy & data• Attributes • Attributes can be stored/applied to attribute files, nodes, roles, cookbooks, environments • For example: generic “ssh” cookbook uses “ssh- port” attribute with 22. But environment “production” overrides it with 2022. • Precedence rules allow absolutely generic policy configurations and all of the specific data to be in attributes.• Data bags • Arbitrary store of globally available JSON data • Use in recipes & search • Can be encrypted
  25. 25. KnifeKnife is the command-line tool forChefKnife is the "swiss army knife" ofinfrastructure management tools.- manage the local Chef repository- interact with the Chef Server API- interact with cloud computing providers APIs- extend with custom plugins/libraries
  26. 26. Search• Search for nodes• Find configuration data• Discoverable infrastructure - self- configuring
  27. 27. Search for nodes – self-configuring systempool_members =search("node", "role:webserver”)template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members =>pool_members.uniq notifies :restart, "service[haproxy]“end
  28. 28. So when thisLB Nagios Jboss App Memcache Postgres Slaves Postgres Master
  29. 29. Becomes thisLB Nagios Jboss App Memcache Postgres Slaves Postgres Master
  30. 30. This can happen automaticallyLB Nagios Jboss App Memcach e Postgres Slaves Postgres Master
  31. 31. Count the resources • Load balancer config • Nagios host ping LB Nagios • Nagios host ssh • Nagios host HTTP Jboss App • Nagios host app health Memcach e • Graphite CPU Postgres • Graphite Memory Slaves • Graphite Disk • Graphite SNMP• 12+ resource changes for 1 node addition • Memcache firewall • Postgres firewall
  32. 32. Connect the dots…. Chef Server Fetch the runlist, attributes & Node assets data, Roles, Cookbooks, Se arch Index, Environments Nodes to Manage your be configurations & configured Infrastructure as Code Running chef-client Knife Knife Workstation WorkstationSCM / Chef Repo Devs & Sysads
  33. 33. Opscode Server Types• Opscode Hosted Chef • • Hosted SaaS version of Chef• Opscode Private Chef • Private deployments of Opscode Chef server • On-premise deployments• Open Source Chef
  34. 34. Opscode Server Types• Opscode Hosted Chef • • Hosted SaaS version of Chef• Opscode Private Chef • Private deployments of Opscode Chef server • On-premise deployments• Open Source Chef
  35. 35. Salient Features• Clouds supported by Knife • EC2, Rackspace, HP, Google, Azure, CloudStack, OpenStack, vSphere, vCloud, Bluelock • Provision & bootstrap cloud servers • Provision & configure cloud assets• Extensible • Implement own Resources & Providers • Access all of your infrastructure through APIs• Bootstrap thousands of machines using SpiceWeasel• AD Integration
  36. 36. Build Anything• Simple internal applications• Complex multi-tier & distributed applications• Workstations• Hadoop Clusters• IaaS, PaaS infrastructure• Continuous Delivery Pipelines• Storage Systems
  37. 37. Questions?Thank You!! Kalpak Shah,