Infrastructure Automation with Chef

  • 14,193 views
Uploaded on

Introduction to Chef, from Velocity 2010.

Introduction to Chef, from Velocity 2010.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
14,193
On Slideshare
0
From Embeds
0
Number of Embeds
10

Actions

Shares
Downloads
774
Comments
3
Likes
51

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide











































  • Roles are another abstraction that describe a set of configuration functionality about nodes.
  • webserver, dbserver, batch processing server
    Attributes applied to node, but stay with the role, add/remove.
    Searchable
  • webserver, dbserver, batch processing server
    Attributes applied to node, but stay with the role, add/remove.
    Searchable
  • Role ruby DSL file.



  • Resources are an abstraction we feed data into. When you write recipes in Chef, you create resources of things you want to configure.





















  • type: template
    name: grants.sql
    parameters: source,owner,group,mode
    action: create
  • The abstraction over the commands to run for this platform.






  • Platform determines provider, but has sane defaults otherwise




























  • Cookbooks encapsulate all the components needed to configure the infrastructure.
  • Distribute cookbooks as tarballs either to/from server, with solo, or share with others
    They contain the code and assets that configure infrastructure
    Keep them in a version control repository
  • Distribute cookbooks as tarballs either to/from server, with solo, or share with others
    They contain the code and assets that configure infrastructure
    Keep them in a version control repository
  • Distribute cookbooks as tarballs either to/from server, with solo, or share with others
    They contain the code and assets that configure infrastructure
    Keep them in a version control repository
  • Cookbooks are a directory of code components
  • Cookbooks are a directory of code components
  • Cookbooks are a directory of code components
  • Cookbooks are a directory of code components
  • Cookbook directory with common components.
    - metadata can be written in ruby DSL, converted to JSON.
  • non-dynamic, like tarballs or graphics
    remote_file resource
    file specificity based on node’s FQDN, platform, or default
  • non-dynamic, like tarballs or graphics
    remote_file resource
    file specificity based on node’s FQDN, platform, or default


  • Attributes in cookbooks apply to the node object directly.
    Two ways to set in cookbooks, ‘set’ - the cookbook is authoritative, ‘set_unless’ - node is authoritative
  • Attributes in cookbooks apply to the node object directly.
    Two ways to set in cookbooks, ‘set’ - the cookbook is authoritative, ‘set_unless’ - node is authoritative
  • Metadata provides hints about the cookbook like packaging system’s. Primarily, used to distribute dependencies. Metadata as Ruby code, converted to JSON to upload.
  • Metadata provides hints about the cookbook like packaging system’s. Primarily, used to distribute dependencies. Metadata as Ruby code, converted to JSON to upload.
  • Metadata provides hints about the cookbook like packaging system’s. Primarily, used to distribute dependencies. Metadata as Ruby code, converted to JSON to upload.
  • Other components of cookbooks that are very useful but not commonly used
  • Other components of cookbooks that are very useful but not commonly used





















Transcript

  • 1. Infrastructure Automation Speakers: with Chef Adam Jacob CTO Christopher Brown VP Engineering Joshua Timberman Sr. Solution Engineer Copyright © 2010 Opscode, Inc - All Rights Reserved 1
  • 2. • Adam Jacob • Christopher Brown • Joshua Timberman http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/
  • 3. http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 4. • Developers? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 5. • Developers? • Systems Administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 6. • Developers? • Systems Administrators? • “Business” People? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 7. • Developers? • Systems Administrators? • “Business” People? • Executives? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 8. http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg
  • 9. At a High Level... http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 10. At a High Level... • A library for configuration management http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 11. At a High Level... • A library for configuration management • A configuration management system http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 12. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 13. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform • An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 14. http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 15. Principles • Idempotent http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 16. Principles • Idempotent • Reasonability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 17. Principles • Idempotent • Reasonability • Sane defaults, easily changed http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 18. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 19. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability • TMTOWTDI http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 20. http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 21. The world moves pretty fast • Primitives http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 22. The world moves pretty fast • Primitives • Enable you to solve your problems http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 23. The world moves pretty fast • Primitives • Enable you to solve your problems • Your best skill is knowing your systems http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 24. http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 25. So you better be ready • You need a 3rd generation language http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 26. So you better be ready • You need a 3rd generation language • You’re already a programmer http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 27. So you better be ready • You need a 3rd generation language • You’re already a programmer • You’ve just been lied to http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 28. Chef enables Infrastructure as Code Manage configuration as idempotent Resources. Put them together in Recipes. Track it like source code. Configure your servers.
  • 29. Infrastructure as Code http://www.flickr.com/photos/wonderlane/2306082998/ Copyright © 2010 Opscode, Inc - All Rights Reserved 10
  • 30. Infrastructure as Code is... http://www.flickr.com/photos/kwerfeldein/2634561264/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 11
  • 31. Infrastructure as Code is... A technical domain revolving around building and managing infrastructure programmatically http://www.flickr.com/photos/kwerfeldein/2634561264/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 11
  • 32. Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal resources. Copyright © 2010 Opscode, Inc - All Rights Reserved 12
  • 33. Theory of Constraints http://www.flickr.com/photos/mabar/313860092/ Copyright © 2010 Opscode, Inc - All Rights Reserved 13
  • 34. Your Prime Constraint Should Be http://www.flickr.com/photos/visualage/2126833132/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 14
  • 35. Your Prime Constraint Should Be The time it takes to restore your application data http://www.flickr.com/photos/visualage/2126833132/sizes/o/ Copyright © 2010 Opscode, Inc - All Rights Reserved 14
  • 36. Philosophy Sidebar
  • 37. Wax Philosophical • We are craftsmen and artists • You need great brushes and tools • But nobody remembers Gaudi’s hammer http://www.flickr.com/photos/vgm8383/2686128924/sizes/l/
  • 38. • Can only be measured by the final solution http://www.flickr.com/photos/aloshbennett/619307160/sizes/l/
  • 39. A Tour of Chef Copyright © 2010 Opscode, Inc - All Rights Reserved 18
  • 40. Chef Client runs on your systems Copyright © 2010 Opscode, Inc - All Rights Reserved 19
  • 41. They talk to Chef Servers Copyright © 2010 Opscode, Inc - All Rights Reserved 20
  • 42. They can also stand alone - Chef Solo Copyright © 2010 Opscode, Inc - All Rights Reserved 21
  • 43. We call each system you configure a Node Copyright © 2010 Opscode, Inc - All Rights Reserved 22
  • 44. Nodes have Attributes Copyright © 2010 Opscode, Inc - All Rights Reserved 23
  • 45. Attributes are Searchable $ knife search node ‘platform:mac_os_x’ search(:node, ‘platform:mac_os_x’) Copyright © 2010 Opscode, Inc - All Rights Reserved 24
  • 46. Nodes have a Run List What Roles or Recipes to apply in Order Copyright © 2010 Opscode, Inc - All Rights Reserved 25
  • 47. Nodes have a Run List Copyright © 2010 Opscode, Inc - All Rights Reserved 26
  • 48. Nodes have Roles Copyright © 2010 Opscode, Inc - All Rights Reserved 27
  • 49. Roles Copyright © 2010 Opscode, Inc - All Rights Reserved 28
  • 50. Roles Describe what a node should be Copyright © 2010 Opscode, Inc - All Rights Reserved 28
  • 51. Roles Describe what a node should be Copyright © 2010 Opscode, Inc - All Rights Reserved 28
  • 52. Copyright © 2010 Opscode, Inc - All Rights Reserved 29
  • 53. Roles are Searchable $ knife search role ‘max_children:50’ search(:role, ‘max_children:50’) Copyright © 2010 Opscode, Inc - All Rights Reserved 30
  • 54. Roles have a Run List What Roles or Recipes to apply in Order Copyright © 2010 Opscode, Inc - All Rights Reserved 31
  • 55. Run Lists on Roles expand Copyright © 2010 Opscode, Inc - All Rights Reserved 32
  • 56. Chef manages Resources on Nodes Copyright © 2010 Opscode, Inc - All Rights Reserved 33
  • 57. A Resource... Is a declarative description of the state you desire a part of your node to be in. http://www.flickr.com/photos/xiaming/382205902/sizes/l/
  • 58. A Resource... Is a declarative description of the state you desire a part of your node to be in. http://www.flickr.com/photos/xiaming/382205902/sizes/l/
  • 59. A Resource... • Is of a given type. Is a declarative description of the state you desire a part of your node to be in. http://www.flickr.com/photos/xiaming/382205902/sizes/l/
  • 60. A Resource... • Is of a given type. • Has a name. Is a declarative description of the state you desire a part of your node to be in. http://www.flickr.com/photos/xiaming/382205902/sizes/l/
  • 61. A Resource... • Is of a given type. • Has a name. • Has attributes. Is a declarative description of the state you desire a part of your node to be in. http://www.flickr.com/photos/xiaming/382205902/sizes/l/
  • 62. A Resource... • Is of a given type. • Has a name. • Has attributes. • Takes actions to bring the resource to a declared state. Is a declarative description of the state you desire a part of your node to be in. http://www.flickr.com/photos/xiaming/382205902/sizes/l/
  • 63. Copyright © 2010 Opscode, Inc - All Rights Reserved 35
  • 64. Resources take action through Providers Copyright © 2010 Opscode, Inc - All Rights Reserved 36
  • 65. A Provider... Knows how to actually perform the actions specified by a resource. http://www.flickr.com/photos/affableslinky/562950216/
  • 66. A Provider... Knows how to actually perform the actions specified by a resource. Apt,Yum, Rubygems, Multiple providers per resource type. Portage, Macports, FreeBSD Ports, etc. http://www.flickr.com/photos/affableslinky/562950216/
  • 67. A Provider... Knows how to actually perform the actions specified by a resource. Apt,Yum, Rubygems, Multiple providers per resource type. Portage, Macports, FreeBSD Ports, etc. Can be overridden with the provider attribute on a resource. http://www.flickr.com/photos/affableslinky/562950216/
  • 68. Copyright © 2010 Opscode, Inc - All Rights Reserved 38
  • 69. Resources Platform Provider http://www.flickr.com/photos/acurbelo/2628837104/sizes/o/
  • 70. Recipes are lists of Resources Copyright © 2010 Opscode, Inc - All Rights Reserved 40
  • 71. A Recipe... Applies resources in the order they are specified http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 72. A Recipe... Applies resources in the order they are specified http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 73. A Recipe... Applies resources in the order they are specified 1 • Evaluates resources in the order they appear. 2 http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 74. A Recipe... Applies resources in the order they are specified • Evaluates resources in the order they appear. 1 2 • Adds each resource object to the Resource Collection http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 75. A Recipe... Applies resources in the order they are specified http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 76. A Recipe... Applies resources in the order they are specified • Can include other recipes. http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 77. A Recipe... Applies resources in the order they are specified • Can include other recipes. • Is just Ruby. http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/
  • 78. Order Matters Copyright © 2010 Opscode, Inc - All Rights Reserved 43
  • 79. Cookbooks are packages for Recipes Copyright © 2010 Opscode, Inc - All Rights Reserved 44
  • 80. Cookbooks Copyright © 2010 Opscode, Inc - All Rights Reserved 45
  • 81. Cookbooks Distributable Copyright © 2010 Opscode, Inc - All Rights Reserved 45
  • 82. Cookbooks Distributable Infrastructure as Code Copyright © 2010 Opscode, Inc - All Rights Reserved 45
  • 83. Cookbooks Distributable Infrastructure as Code Version control repository Copyright © 2010 Opscode, Inc - All Rights Reserved 45
  • 84. Common Cookbook Components Copyright © 2010 Opscode, Inc - All Rights Reserved 46
  • 85. Common Cookbook Components Recipes Copyright © 2010 Opscode, Inc - All Rights Reserved 46
  • 86. Common Cookbook Components Recipes Assets (files/templates) Copyright © 2010 Opscode, Inc - All Rights Reserved 46
  • 87. Common Cookbook Components Recipes Assets (files/templates) Attributes Copyright © 2010 Opscode, Inc - All Rights Reserved 46
  • 88. Common Cookbook Components Recipes Assets (files/templates) Attributes Metadata Copyright © 2010 Opscode, Inc - All Rights Reserved 46
  • 89. Cookbook directory Copyright © 2010 Opscode, Inc - All Rights Reserved 47
  • 90. Cookbook assets Copyright © 2010 Opscode, Inc - All Rights Reserved 48
  • 91. Cookbook assets Files Copyright © 2010 Opscode, Inc - All Rights Reserved 48
  • 92. Cookbook assets Files ‣ Static assets ‣ Downloaded via remote_file ‣ File specificity Copyright © 2010 Opscode, Inc - All Rights Reserved 48
  • 93. Cookbook assets Copyright © 2010 Opscode, Inc - All Rights Reserved 49
  • 94. Cookbook assets Templates Copyright © 2010 Opscode, Inc - All Rights Reserved 49
  • 95. Cookbook assets Templates ‣ Dynamic assets ‣ ERB (erubis) ‣ File specificity Copyright © 2010 Opscode, Inc - All Rights Reserved 49
  • 96. Cookbooks Copyright © 2010 Opscode, Inc - All Rights Reserved 50
  • 97. Cookbooks Attributes Copyright © 2010 Opscode, Inc - All Rights Reserved 50
  • 98. Cookbooks Attributes ‣ Node attributes Copyright © 2010 Opscode, Inc - All Rights Reserved 50
  • 99. Cookbooks Copyright © 2010 Opscode, Inc - All Rights Reserved 51
  • 100. Cookbooks Metadata Copyright © 2010 Opscode, Inc - All Rights Reserved 51
  • 101. Cookbooks Metadata cookbooks/god/metadata.rb Copyright © 2010 Opscode, Inc - All Rights Reserved 51
  • 102. Cookbooks Copyright © 2010 Opscode, Inc - All Rights Reserved 52
  • 103. Cookbooks Other Components Copyright © 2010 Opscode, Inc - All Rights Reserved 52
  • 104. Cookbooks Other Components ‣ definitions ‣ libraries ‣ resources ‣ providers Copyright © 2010 Opscode, Inc - All Rights Reserved 52
  • 105. Cookbooks are shareable! Copyright © 2010 Opscode, Inc - All Rights Reserved 53
  • 106. Data bags store arbitrary data Copyright © 2010 Opscode, Inc - All Rights Reserved 54
  • 107. A user $ knife data bag show users adam Copyright © 2010 Opscode, Inc - All Rights Reserved 55
  • 108. Data Bags are Searchable $ knife search users ‘shell:/usr/bin/zsh’ search(:users, ‘/usr/bin/zsh’) Copyright © 2010 Opscode, Inc - All Rights Reserved 56
  • 109. Data bags make recipes awesome-r (that’s totally a word) Copyright © 2010 Opscode, Inc - All Rights Reserved 57
  • 110. Data bags make recipes awesome-r (that’s totally a word) Copyright © 2010 Opscode, Inc - All Rights Reserved 57
  • 111. Working with Chef Copyright © 2010 Opscode, Inc - All Rights Reserved 58
  • 112. Create your Chef Repostiory $ git clone http://github.com/opscode/chef- repo.git Copyright © 2010 Opscode, Inc - All Rights Reserved 59
  • 113. Configure knife Everything you can do in Chef, you can do from the command line with Knife. Per-directory configurations! Copyright © 2010 Opscode, Inc - All Rights Reserved 60
  • 114. Download some cookbooks $ knife cookbook site vendor rails -d INFO: Downloading rails from the cookbooks site at version 0.8.0 ... INFO: Cookbook rails version 0.8.0 successfully vendored! INFO: Downloading apache2 from the cookbooks site at version 0.12.0 ... INFO: Cookbook apache2 version 0.12.0 successfully vendored! INFO: Downloading ruby from the cookbooks site at version 0.7.0 ... INFO: Cookbook ruby version 0.7.0 successfully vendored! Uses the “vendor branch” pattern, so you can make changes and track the upstream Copyright © 2010 Opscode, Inc - All Rights Reserved 61
  • 115. Check out the Rails recipe... $ vim cookbooks/rails/recipes/default.rb Copyright © 2010 Opscode, Inc - All Rights Reserved 62
  • 116. Upload them! $ knife cookbook upload -a These run as root, kids. Let’s not blindly trust the upstream too much, ok? Copyright © 2010 Opscode, Inc - All Rights Reserved 63
  • 117. Build a role $ knife role create rails Copyright © 2010 Opscode, Inc - All Rights Reserved 64
  • 118. Add your Cloudy credentials to knife $ vi ~/.chef/knife.rb Copyright © 2010 Opscode, Inc - All Rights Reserved 65
  • 119. Launch a new Rails Server $ knife rackspace server create 'role[rails]' $ knife ec2 server create ‘role[rails]’ Or just install Chef on your existing servers Copyright © 2010 Opscode, Inc - All Rights Reserved 66
  • 120. Chef runs on your new server $ sudo chef-client INFO: Starting Chef Run ... INFO: Chef Run complete in 11.852033 seconds Automatically. Copyright © 2010 Opscode, Inc - All Rights Reserved 67
  • 121. Marvel at Rails being installed $ knife ssh role:rails ‘gem list | grep rails’ Knife SSH! “Search for all nodes with role rails” Run the following command Executes in parallel Supports screen, tmux, and macterm Copyright © 2010 Opscode, Inc - All Rights Reserved 68
  • 122. Decide you want Rails 3 instead $ vim cookbooks/rails/default The vendor pattern means that you can just change it, and safely merge any upstream changes later on. Copyright © 2010 Opscode, Inc - All Rights Reserved 69
  • 123. Upload your changed cookbook $ knife cookbook upload rails Only uploads the changed file. Copyright © 2010 Opscode, Inc - All Rights Reserved 70
  • 124. Run chef and verify $ knife ssh role:rails ‘sudo chef-client && gem list | grep rails’ cook-prod-i78f2e213.opscode.us rails (3.0.0- beta4) Run chef on all the Rails systems Then verify the changes. Copyright © 2010 Opscode, Inc - All Rights Reserved 71
  • 125. Watch now as Mr. Timberman rocks it live. Copyright © 2010 Opscode, Inc - All Rights Reserved 72
  • 126. Questions? www.opscode.com ‣ Twitter: @opscode, #opschef ‣ irc.freenode.net #chef ‣ lists.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 73