Your SlideShare is downloading. ×
An Introduction to DevOps with Chef
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

An Introduction to DevOps with Chef

2,601

Published on

Published in: Technology, Business
1 Comment
11 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,601
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
104
Comments
1
Likes
11
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Rethinking IT: An Introduction to DevOps with Chef Julian C. Dunn Senior Consultant, Opscode <jdunn@opscode.com> Wednesday, October 2, 13
  • 2. Topics • Rethinking IT with DevOps • What Chef is and how it helps • Flavors of Chef • Use cases and demo Wednesday, October 2, 13
  • 3. I came to hear about Chef. Wednesday, October 2, 13
  • 4. Why are you throwing DevOps into this? Wednesday, October 2, 13
  • 5. It’s how Chef works best • Chef is a tool. • It works best when developers and operations work well together • You don’t need “DevOps” for Chef to be successful. • But it helps. Wednesday, October 2, 13
  • 6. What is this DevOps? Sounds like a buzzword. Wednesday, October 2, 13
  • 7. DevOps: Aligning Objectives • Cultural and Professional movement • Development and Operations working together • Leveraging ideas & processes from other industries • To enable the business Wednesday, October 2, 13
  • 8. Technology: Business Engaging Customers Wednesday, October 2, 13
  • 9. Speed of Globalization • 40 years for container ships to move 70% of seaborne trade • 22 years for internet access to reach 78% penetration in North America Wednesday, October 2, 13
  • 10. Globalization • Online retail sales are 7% of all retail sales • 75% of 2011 Thanksgiving shoppers did so online • 42% of all retail purchases were influenced by online research – accounting for ~50% of total retail spending.WTO Trends in Globalization http://www.wto.org/english/res_e/booksp_e/anrep_e/wtr08-2b_e.pdf http://www.flickr.com/photos/duke_raoul/2261478794/sizes/l/in/photostream/ Wednesday, October 2, 13
  • 11. Mobile Devices • 42% are smartphones • 58% will be on next purchase • 4.2 Billion phones globally for 7.09 Billion people (USCB) http://ssiknowledgewatch.com/2012/05/09/cell-phones-approach-total-penetration-globally-with-smartphones-moving-toward-market-dominance-2/ http://www.brightsideofnews.com/news/2011/1/26/digital-divide-global-household-penetration-rates-for-technology.aspx?pageid=1 Wednesday, October 2, 13
  • 12. Software: The Interface for Consumtpion Wednesday, October 2, 13
  • 13. Wednesday, October 2, 13
  • 14. Wednesday, October 2, 13
  • 15. Wednesday, October 2, 13
  • 16. Wednesday, October 2, 13
  • 17. Wednesday, October 2, 13
  • 18. Drivers of IT Innovation The Result: The Coded Business How: Redefinition of how to use technology to create business value Why: To rapidly deliver experiences, goods and services to customers What: Consumer-facing businesses Wednesday, October 2, 13
  • 19. The Rise of the Coded Business • Speed of change > speed of skills development • IT is moving from the back office to the front office • Digital consumption • Technology directly supports customer interactions • Companies must move faster to compete Wednesday, October 2, 13
  • 20. Patterns of the Coded Business Business Agility Development Velocity and Consistency Continuous Delivery IT IT enables Business Agility and becomes a strategic advantage rather than a cost center. Wednesday, October 2, 13
  • 21. The Coded Business: Coming to Every Enterprise Manufacturing Financial Services Retail Media and Entertainment High Technology Healthcare Wednesday, October 2, 13
  • 22. Scale x Complexity > Skills Wednesday, October 2, 13
  • 23. DevOps... • Cultural and professional movement • Grew directly from people who have experienced this transition • Applies directly to traditional IT • Applies directly to other systems of survival in our culture Wednesday, October 2, 13
  • 24. DevOps: CAMS • Culture • Automation • Measurement • Sharing Wednesday, October 2, 13
  • 25. Open Communication • Developers & Operations talk and listen to one another • Production & build metrics are available to all • Current infrastructure is documented Wednesday, October 2, 13
  • 26. Incentive & Responsibility Align • Create awesome customer experiences • Responsibility & accountability, not authority • Responsible for your own uptime Wednesday, October 2, 13
  • 27. Respect • You don’t have to be everyone’s best friend • But you do need to recognize contributions and treat each other well • No asshole rule Wednesday, October 2, 13
  • 28. Culture: Trust • Trust that everyone is competent & working toward the common goals • Without trust, the tools don’t matter Wednesday, October 2, 13
  • 29. DevOps: CAMS • Culture • Automation • Measurement • Sharing • Chef is (part of) “A”. Works best with C, M and S. Wednesday, October 2, 13
  • 30. Infrastructure Automation with Chef Wednesday, October 2, 13
  • 31. http://www.flickr.com/photos/steffenz/337700069/ http://www.flickr.com/photos/kky/704056791/ Applications Wednesday, October 2, 13
  • 32. Infrastructure http://www.flickr.com/photos/sbh/462754460/ Wednesday, October 2, 13
  • 33. http://www.flickr.com/photos/philliecasablanca/3354734116/ • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configuration • Other Stuff Collection of Resources Wednesday, October 2, 13
  • 34. Acting in Concert http://www.flickr.com/photos/glowjangles/4081048126/ Wednesday, October 2, 13
  • 35. To Provide a Service http://www.flickr.com/photos/28309157@N08/3743455858/ Wednesday, October 2, 13
  • 36. And it Evolves http://www.flickr.com/photos/16339684@N00/2681435235/ Wednesday, October 2, 13
  • 37. Application Server See Node Wednesday, October 2, 13
  • 38. Application Server Application Database See Nodes Wednesday, October 2, 13
  • 39. Application Server Application Databases See Nodes Grow Wednesday, October 2, 13
  • 40. Application Servers Application Databases See Nodes Grow Wednesday, October 2, 13
  • 41. Application Servers Application Databases Load Balancer See Nodes Grow Wednesday, October 2, 13
  • 42. Application Servers Application Databases Load Balancers See Nodes Grow Wednesday, October 2, 13
  • 43. Application Servers Application Database Cache Load Balancers Application Databases See Nodes Grow Wednesday, October 2, 13
  • 44. Application Servers Application Database Cache Load Balancers Application Databases Tied Together with Configuration Wednesday, October 2, 13
  • 45. Application Servers Application Database Cache Load Balancers Floating IP? Application Databases Infrastructure is a Snowflake Wednesday, October 2, 13
  • 46. Load Balancers Application Servers NoSQL Database Slaves Application Cache Database Cache Database Evolving Complexity Wednesday, October 2, 13
  • 47. Complexity Increases Very Quickly DC1 DC3 DC2 Wednesday, October 2, 13
  • 48. Configuration Management http://www.flickr.com/photos/philliecasablanca/3354734116/ Wednesday, October 2, 13
  • 49. Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/ Wednesday, October 2, 13
  • 50. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite Typical Infrastructure Wednesday, October 2, 13
  • 51. JBoss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Move SSH off port 22 • Let’s put it on 2022 New Compliance Mandate! Wednesday, October 2, 13
  • 52. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6 6 Golden Image Updates Wednesday, October 2, 13
  • 53. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually 12 Instance Replacements Wednesday, October 2, 13
  • 54. • Don’t break anything! • Bob just got fired =( 5 JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite 1 2 4 5 6 7 8 9 10 11 12 3 Done in Maintenance Windows Wednesday, October 2, 13
  • 55. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite • Invalid configs! Different IP Addresses? Wednesday, October 2, 13
  • 56. Configuration Desperation http://www.flickr.com/photos/francoforeshock/5716969942/ Wednesday, October 2, 13
  • 57. Chef Solves This Problem • But you already guessed that, didn’t you? Wednesday, October 2, 13
  • 58. Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.http://www.flickr.com/photos/louisb/4555295187/ Wednesday, October 2, 13
  • 59. Programs • Chef generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ Wednesday, October 2, 13
  • 60. Declarative Interface to Resources • Define Policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ Wednesday, October 2, 13
  • 61. That Looks Like This package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end Wednesday, October 2, 13
  • 62. Ohai "languages": { "ruby": { }, "perl": { "version": "5.14.2", "archname": "x86_64- linux-gnu-thread-multi" }, "python": { "version": "2.7.3", "builddate": "Aug 1 2012, 05:14:39" }, "php": { "version": "5.3.10-1ubuntu3.6", "builddate": "(cli) (built: Mar" } }, "network": { "interfaces": { "lo": { "mtu": "16436", "flags": [ "LOOPBACK", “UP","LOWER_UP" ], "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "netmask": "255.0.0.0", "scope": "Node" }, "::1": { "family": "inet6", "scope": "Node" } }, }, "eth0": { "type": "eth", "number": "0", "kernel": { "name": "Linux", "release": "3.2.0-32-virtual", "version": "#51-Ubuntu SMP Wed Sep 26 21:53:42 UTC 2012", "machine": "x86_64", "modules": { "isofs": { "size": "40257", "refcount": "0" }, "acpiphp": { "size": "24231", "refcount": "0" } }, "os": "GNU/Linux" }, "os": "linux", "os_version": "3.2.0-32-virtual", "ohai_time": 1369328621.3456137, Wednesday, October 2, 13
  • 63. execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end Decide what to declare Wednesday, October 2, 13
  • 64. execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end Decide what to declare Wednesday, October 2, 13
  • 65. execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end Decide what to declare Wednesday, October 2, 13
  • 66. Recipes and Cookbooks • Recipes are collections of Resources • Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity http://www.flickr.com/photos/shutterhacks/4474421855/ Wednesday, October 2, 13
  • 67. Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]” node ntp client.rb Wednesday, October 2, 13
  • 68. Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]” node ntp client.rb openssh server.rb Wednesday, October 2, 13
  • 69. Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb Wednesday, October 2, 13
  • 70. Roles name "base" description "base" run_list [ "recipe[selinux::disabled]", "recipe[etchosts]", "recipe[yum::epel]", "recipe[debugtools]" ] name "webserver" description "webserver server" run_list [ "role[base]", "recipe[nginx::server]" ] Wednesday, October 2, 13
  • 71. Roles Role Recipe Recipe Recipe Role Role Recipe Recipe RecipeRole Recipe Server ServerServer Server chef-server API Knife Wednesday, October 2, 13
  • 72. Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb Wednesday, October 2, 13
  • 73. Roles Server ServerServer Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Wednesday, October 2, 13
  • 74. Roles Server ServerServer Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb Wednesday, October 2, 13
  • 75. http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search for nodes with Roles • Find configuration data Search: Dynamic Infrastructure Wednesday, October 2, 13
  • 76. Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Wednesday, October 2, 13
  • 77. Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Wednesday, October 2, 13
  • 78. Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Wednesday, October 2, 13
  • 79. Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Wednesday, October 2, 13
  • 80. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite So when this... Wednesday, October 2, 13
  • 81. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite ... becomes this ... Wednesday, October 2, 13
  • 82. JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite ...this can happen automatically Wednesday, October 2, 13
  • 83. NagiosGraphite JBoss App Memcache PostgreSQL Slaves To Add a New Server… • 2x Web Server Configurations • 2 Web Server Restarts • 4x Database Configurations • 8x Firewall Configurations • DNS Service • Network Configuration • Deployer • 8x Monitoring Changes 20+ Changes 12+ New Infrastructure Dependencies 4+ Hours Count the Resources Wednesday, October 2, 13
  • 84. NagiosGraphite JBoss App Memcache PostgreSQL Slaves To Add a New Server… • 2x Web Server Configurations • 2 Web Server Restarts • 4x Database Configurations • 8x Firewall Configurations • DNS Service • Network Configuration • Deployer • 8x Monitoring Changes 20+ Changes 12+ New Infrastructure Dependencies 4+ Hours Count the Resources Add 1 server 20+ Changes Wednesday, October 2, 13
  • 85. Managing Complexity Later Wednesday, October 2, 13
  • 86. Managing Complexity Later We added: • Load Balancers • MemCache • Search Appliances • Lots of VM’s • More Scale Exponential Increase In: • Configuration Changes • Infrastructure Dependencies • Skills Needed • Greater Risk Wednesday, October 2, 13
  • 87. And at Greater Scale... Wednesday, October 2, 13
  • 88. And at Greater Scale... How Do we Manage This at Cloud Scale? • Thousands of infrastructure dependencies and configurations needed for each change. • Huge Amounts of Time • Increased Cost of Correction of Manual Errors • Huge Need for Talent • Risk of Critical Skills Shortage Wednesday, October 2, 13
  • 89. Automation is a Journey Full Automation Common Automation Tasks: Scripts, OS Compliance, Updates, etc. Configuration ManagementDiscovery and Visibility Application Management Continuous Deployment The Path to the Coded Business Wednesday, October 2, 13
  • 90. Flavors of Chef Server Wednesday, October 2, 13
  • 91. Landscape of Chef-Managed Infrastructure Wednesday, October 2, 13
  • 92. Flavors of Chef Server • SaaS, hosted by Opscode • Manage up to 50,000 servers • Industry-leading SLAs • 24x7x365 Support Options • Get up and running quickly • Pay/grow as you need Hosted Enterprise Chef Wednesday, October 2, 13
  • 93. Flavors of Chef Server • All the power of Hosted Enterprise, behind the firewall • Delivered as enterprise software • Implementation consulting customized to customer’s needs Enterprise Chef Wednesday, October 2, 13
  • 94. Flavors of Chef Server • Core components of Chef Server developed as open source • No enterprise features (LDAP, HA, etc.) but many people use it successfully • Community support Open Source Chef Server Wednesday, October 2, 13
  • 95. Learn Chef! Wednesday, October 2, 13
  • 96. Let’s Learn Chef! • https://learnchef.opscode.com/ • Next meetup: Hands on. Bring laptops! • Feel free to get started early Wednesday, October 2, 13
  • 97. Thanks • Julian Dunn • jdunn@opscode.com • @julian_dunn • www.juliandunn.net Wednesday, October 2, 13

×