Your SlideShare is downloading. ×
Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co

478

Published on

This talk was held at a Zühlke internal meetup of architects in 2012. It explains the motivation behind DevOps and Infrastructure-as-Code, then introduces Chef, looking at the essential Chef concepts …

This talk was held at a Zühlke internal meetup of architects in 2012. It explains the motivation behind DevOps and Infrastructure-as-Code, then introduces Chef, looking at the essential Chef concepts and the Chef recipe DSL. Next comes a brief outlook to Vagrant for managing VMs, Librarian for dependency management, and various Chef testing tools. Finally, different options for cloud deployments with Chef are discussed: AWS CloudFormation, Mccloud and knife-ec2.

Published in: Software, Business, Travel
1 Comment
5 Likes
Statistics
Notes
  • Nice explanation ... easily understand...
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
478
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
26
Comments
1
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © Zühlke 2012 Torben Knerr Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co 20. Juli 2012 Slide 1
  • 2. © Zühlke 2012 Topics Motivation  DevOps & Infrastructure-As-Code Introduction to Chef  Terms & Concepts (with Examples)  Chef vs. Puppet Cloud Deployment  Different Approaches for Cloud Provisioning Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 2
  • 3. © Zühlke 2012 Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co Torben Knerr Motivation DevOps & Infrastructure-As-Code 20. Juli 2012 Slide 3
  • 4. © Zühlke 2012 Motivation DevOps & Infrastructure-As-Code Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr DevOps Drivers:  Cloud & Virtualisation  Mass Server Management  Agile & Continous Delivery Philosophy:  Collaboration between Devs and Ops  Devs understand the needs of Ops  Ops learn from Devs (Infrastructure-As-Code)  Ops are part of the (SCRUM) team Disciplines:  Operations, Monitoring  Automation  Programming (Ruby)  Configuration Management Goals:  Traceability, Auditability  Maintainability  Reproducability  Speed / Time-To-Market Infrastructure- As-Code 20. Juli 2012 Slide 4
  • 5. © Zühlke 2012 Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co Torben Knerr Introduction to Chef Knife, Cookbooks, Recipes, etc… 20. Juli 2012 Slide 5
  • 6. © Zühlke 2012 Introduction to Chef What‘s in a Chef Repository? Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr chef-repo 20. Juli 2012 Slide 6
  • 7. © Zühlke 2012 Introduction to Chef What‘s in a Chef Repository? Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr chef-repo cookbooks databags environments roles 20. Juli 2012 Slide 7
  • 8. © Zühlke 2012 Introduction to Chef What‘s in a Chef Repository? Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr recipes attributes files templates chef-repo cookbooks databags environments roles 20. Juli 2012 Slide 8
  • 9. © Zühlke 2012 Introduction to Chef Chef Solo vs. Chef Server Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr node couchdb cookbooks roles databags nodes …Chef Server node node node node chef-client poll for node’s run_list and converge node node cookbooks roles databags solo.rb chef-solo read run_list from solo.rb and converge node Chef Solo  Runs chef-solo locally  No server required  Easier to get started with  Good for few nodes, onetime bootstraps chef- repo upload to node 20. Juli 2012 Slide 9
  • 10. © Zühlke 2012 Introduction to Chef Chef Solo vs. Chef Server Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr node couchdb cookbooks roles databags nodes …Chef Server node node node node chef-client poll for node’s run_list and converge node node cookbooks roles databags solo.rb chef-solo read run_list from solo.rb and converge node Chef Server  chef-client polls regularly  More complex workflow  Additional features: Search, WebUI  Good for ongoing operations of many nodes chef- repo upload to chef server 20. Juli 2012 Slide 10
  • 11. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr The Chef Repository is the place where cookbooks, roles, config files and other artifacts for managing systems with Chef will live, typically stored in a version control system. 20. Juli 2012 Slide 11
  • 12. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Cookbooks are the fundamental units of distribution in Chef. They encapsulate all the resources you need to automate your infrastructure and are easily sharable with other Chef users. 20. Juli 2012 Slide 12
  • 13. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Attributes are node data such as the IP address, hostname, etc... New attributes can be defined on cookbook, environment, role and node level. cookbooks/apache2/attributes/default.rb 20. Juli 2012 Slide 13
  • 14. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Recipes encapsulate collections of resources which are executed in order to configure the system. cookbooks/apache2/recipes/default.rb 20. Juli 2012 Slide 14
  • 15. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Recipes encapsulate collections of resources which are executed in order to configure the system. Resources cookbooks/apache2/recipes/default.rb 20. Juli 2012 Slide 15
  • 16. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Resources are the cross platform abstraction of the thing you're configuring on the node. Examples: 20. Juli 2012 Slide 16
  • 17. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Resources are the cross platform abstraction of the thing you're configuring on the node. Examples: 20. Juli 2012 Slide 17
  • 18. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Resources are the cross platform abstraction of the thing you're configuring on the node. Examples: 20. Juli 2012 Slide 18
  • 19. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Resources are the cross platform abstraction of the thing you're configuring on the node. Examples: 20. Juli 2012 Slide 19
  • 20. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Resources are the cross platform abstraction of the thing you're configuring on the node. Examples: 20. Juli 2012 Slide 20
  • 21. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Resources are the cross platform abstraction of the thing you're configuring on the node. Examples:  Many, many more in the Opscode Wiki: http://wiki.opscode.com/display/chef/Resources  You can also write your own: http://wiki.opscode.com/display/chef/Lightweight+Resources+and+Providers+ %28LWRP%29 20. Juli 2012 Slide 21
  • 22. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Data bags provide an arbitrary stores of globally available JSON data… chef-repo/data_bags/users/bofh.json 20. Juli 2012 Slide 22
  • 23. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr …which are then used in Recipes: cookbooks/users/recipes/default.rb 20. Juli 2012 Slide 23
  • 24. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr A role provides a means of grouping similar features of similar nodes, providing a mechanism for easily composing sets of functionality. chef-repo/roles/webserver.rb 20. Juli 2012 Slide 24
  • 25. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Environments in Chef provide a mechanism for managing different environments such as production, staging, development, testing, etc chef-repo/environments/dev.rb 20. Juli 2012 Slide 25
  • 26. © Zühlke 2012  Chef Repo  Cookbooks  Attributes  Recipes  Resources  Databags  Roles  Environments  Nodes Introduction to Chef Terms & Concepts Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Nodes are the thing that Recipes and Roles are applied to. The primary features of a node, from Chef's point of view, are its Attributes and its run list. chef-repo/nodes/web.example.com.rb 20. Juli 2012 Slide 26
  • 27. © Zühlke 2012 How to get started? • If you are working on Windows use Bill’s Kitchen https://ikm.zuehlke.com/topics/Pages/Bill's%20Kitchen.aspx • Start with Vagrant and Chef-Solo (you will love Vagrant…) http://vagrantup.com/v1/docs/provisioners/chef_solo.html • Get familiar with cookbook development and testing https://github.com/tknerr/bills-kitchen/blob/master/COOKBOOK_DEVELOPMENT.md • Set up a Chef Server using knife-server http://fnichol.github.com/knife-server/ • Use Vagrant with Chef-Server Provisioning http://vagrantup.com/v1/docs/provisioners/chef_server.html • Get familiar with knife and use advanced features like search, bootstrap, etc.. http://wiki.opscode.com/display/chef/Knife Introduction to Chef Getting Started with Chef Development Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 27
  • 28. © Zühlke 2012 Introduction to Chef VirtualBox Automation with Vagrant Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Vagranfile 20. Juli 2012 Slide 28
  • 29. © Zühlke 2012 Introduction to Chef Cookbook Dependency Management with Librarian Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Cheffile 20. Juli 2012 Slide 29
  • 30. © Zühlke 2012 Introduction to Chef Test-Driven Infrastrucure! Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Lint Checking • foodcritic Unit-Level Testing • chefspec • fauxhai Smoke Tests (from the inside) • chef-minitest-handler Acceptance Testing (from the outside) • cucumber-nagios 20. Juli 2012 Slide 30
  • 31. © Zühlke 2012 It doesn’t really matter (for us)  Same concepts, slightly different terminology  External vs. internal DSL (like Maven vs. Gradle)  Both have solo / standalone mode  There are more Puppet books than Chef books  Both have a very active community (mailing list, IRC, etc..)  Do you have in-house knowledge of Chef/Puppet? Introduction to Chef Chef vs. Puppet Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 31
  • 32. © Zühlke 2012 It doesn’t really matter (for us)  Same concepts, slightly different terminology  External vs. internal DSL (like Maven vs. Gradle)  Both have solo / standalone mode  There are more Puppet books than Chef books  Both have a very active community (mailing list, IRC, etc..)  Do you have in-house knowledge of Chef/Puppet? Introduction to Chef Chef vs. Puppet Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Use the tool that “operations” understands! 20. Juli 2012 Slide 32
  • 33. © Zühlke 2012 Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co Torben Knerr Cloud Deployment Different Approaches for Provisioning Cloud Instances 20. Juli 2012 Slide 33
  • 34. © Zühlke 2012 Cloud Deployment The Basic Mechanism: EC2 User Data You can pass arbitrary user data when launching an instance… …and access it on the instance via the metadata service: Notes:  User data is limited to 16K in raw form (before base64 encoding) Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 34
  • 35. © Zühlke 2012 Cloud Deployment Option 1: Provisioning with Shell Scripts Convention: if the user data script starts with a `#!` it will be run at first boot Notes:  Might not be supported on all AMIs (for the official Ubuntu AMIs it is) Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr path/to/bootstrap.sh 20. Juli 2012 Slide 35
  • 36. © Zühlke 2012 Cloud Deployment Option 1: Provisioning with Shell Scripts Convention: if the user data script starts with a `#!` it will be run at first boot Notes:  Might not be supported on all AMIs (for the official Ubuntu AMIs it is) Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr path/to/bootstrap.sh  easiest method  good if bootstrapping is simple  one-off bootstrap (first boot) 20. Juli 2012 Slide 36
  • 37. © Zühlke 2012  VM instances are bootstrapped with Chef/Puppet via Cloud-init  The rest is just AWS CloudFormation Templates  Supports Chef Server, Chef Solo and Puppet Client Server Cloud Deployment Option 2: AWS CloudFormation Templates Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 37
  • 38. © Zühlke 2012  VM instances are bootstrapped with Chef/Puppet via Cloud-init  The rest is just AWS CloudFormation Templates  Supports Chef Server, Chef Solo and Puppet Client Server Cloud Deployment Option 2: AWS CloudFormation Templates Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr  good for more complex scenarios  official, AWS supported documentation available  not meant for humans to read 20. Juli 2012 Slide 38
  • 39. © Zühlke 2012  Vagrant-like approach for managing VMs in the cloud  Supports Chef, Puppet and Shell provisioning  Currently focused on AWS but other providers possible  Supports stacks via AWS CloudFormation Templates Cloud Deployment Option 3: Mccloud Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 39
  • 40. © Zühlke 2012  Vagrant-like approach for managing VMs in the cloud  Supports Chef, Puppet and Shell provisioning  Currently focused on AWS but other providers possible  Supports stacks via AWS CloudFormation Templates Cloud Deployment Option 3: Mccloud Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Yay! :-) 20. Juli 2012 Slide 40
  • 41. © Zühlke 2012  knife-ec2  knife plugin to let you launch, bootstrap and provision EC2 VMs with Chef  Works only with Chef Server – no Chef Solo support  Puppet Cloud Provisioner  Puppet extension to launch, bootstrap and provision EC2 VMs with Puppet  Works only with Puppet Master – no Puppet Standalone mode (?) Cloud Deployment Option 4: Native Chef / Puppet Tools Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 41
  • 42. © Zühlke 2012  knife-ec2  knife plugin to let you launch, bootstrap and provision EC2 VMs with Chef  Works only with Chef Server – no Chef Solo support  Puppet Cloud Provisioner  Puppet extension to launch, bootstrap and provision EC2 VMs with Puppet  Works only with Puppet Master – no Puppet Standalone mode (?) Cloud Deployment Option 4: Native Chef / Puppet Tools Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr  You can’t have both: it’s Chef OR Puppet  Fallback if the generic tools (Option 2 & 3) don’t work out as expected  No Solo / Standalone mode 20. Juli 2012 Slide 42
  • 43. © Zühlke 2012 Vagrant – create and configure lightweight, reproducible, and portable development environments Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr Vagranfile 20. Juli 2012 Slide 43
  • 44. © Zühlke 2012  EC2 User Data Scripts http://alestic.com/2009/06/ec2-user-data-scripts https://help.ubuntu.com/community/CloudInit http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html  AWS CloudFormation Templates with Chef/Puppet https://s3.amazonaws.com/cloudformation-examples/IntegratingAWSCloudFormationWithOpscodeChef.pdf https://s3.amazonaws.com/cloudformation-examples/IntegratingAWSCloudFormationWithPuppet.pdf  Mccloud https://github.com/jedi4ever/mccloud https://github.com/jedi4ever/mccloud-demo https://gist.github.com/3175267  knife-ec2 http://wiki.opscode.com/display/chef/Launch+Cloud+Instances+with+Knife  Puppet Cloud Provisioner http://docs.puppetlabs.com/guides/cloud_pack_getting_started.html Cloud Deployment Overview & Resources Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co | Torben Knerr 20. Juli 2012 Slide 44
  • 45. © Zühlke 2012 Infrastructure-As-Code and Cloud Deployments with Opscode Chef & Co Torben Knerr Discussion Time! Puppet, Mccloud, Foo, Bar, Baz DevOps, NoOps, Blablabla Vagrant, Chef, Deployment Stuff Cloudy Cloud! 20. Juli 2012 Slide 45
  • 46. Thank You!  And let’s continue the discussion here:

×