• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
An Introduction to DevOps with Chef
 

An Introduction to DevOps with Chef

on

  • 1,601 views

 

Statistics

Views

Total Views
1,601
Views on SlideShare
1,601
Embed Views
0

Actions

Likes
8
Downloads
35
Comments
1

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    An Introduction to DevOps with Chef An Introduction to DevOps with Chef Presentation Transcript

    • Rethinking IT: An Introduction to DevOps with Chef Julian C. Dunn Senior Consultant, Opscode <jdunn@opscode.com> Wednesday, October 2, 13
    • Topics • Rethinking IT with DevOps • What Chef is and how it helps • Flavors of Chef • Use cases and demo Wednesday, October 2, 13
    • I came to hear about Chef. Wednesday, October 2, 13
    • Why are you throwing DevOps into this? Wednesday, October 2, 13
    • It’s how Chef works best • Chef is a tool. • It works best when developers and operations work well together • You don’t need “DevOps” for Chef to be successful. • But it helps. Wednesday, October 2, 13
    • What is this DevOps? Sounds like a buzzword. Wednesday, October 2, 13
    • DevOps: Aligning Objectives • Cultural and Professional movement • Development and Operations working together • Leveraging ideas & processes from other industries • To enable the business Wednesday, October 2, 13
    • Technology: Business Engaging Customers Wednesday, October 2, 13
    • Speed of Globalization • 40 years for container ships to move 70% of seaborne trade • 22 years for internet access to reach 78% penetration in North America Wednesday, October 2, 13
    • Globalization • Online retail sales are 7% of all retail sales • 75% of 2011 Thanksgiving shoppers did so online • 42% of all retail purchases were influenced by online research – accounting for ~50% of total retail spending.WTO Trends in Globalization http://www.wto.org/english/res_e/booksp_e/anrep_e/wtr08-2b_e.pdf http://www.flickr.com/photos/duke_raoul/2261478794/sizes/l/in/photostream/ Wednesday, October 2, 13
    • Mobile Devices • 42% are smartphones • 58% will be on next purchase • 4.2 Billion phones globally for 7.09 Billion people (USCB) http://ssiknowledgewatch.com/2012/05/09/cell-phones-approach-total-penetration-globally-with-smartphones-moving-toward-market-dominance-2/ http://www.brightsideofnews.com/news/2011/1/26/digital-divide-global-household-penetration-rates-for-technology.aspx?pageid=1 Wednesday, October 2, 13
    • Software: The Interface for Consumtpion Wednesday, October 2, 13
    • Wednesday, October 2, 13
    • Wednesday, October 2, 13
    • Wednesday, October 2, 13
    • Wednesday, October 2, 13
    • Wednesday, October 2, 13
    • Drivers of IT Innovation The Result: The Coded Business How: Redefinition of how to use technology to create business value Why: To rapidly deliver experiences, goods and services to customers What: Consumer-facing businesses Wednesday, October 2, 13
    • The Rise of the Coded Business • Speed of change > speed of skills development • IT is moving from the back office to the front office • Digital consumption • Technology directly supports customer interactions • Companies must move faster to compete Wednesday, October 2, 13
    • Patterns of the Coded Business Business Agility Development Velocity and Consistency Continuous Delivery IT IT enables Business Agility and becomes a strategic advantage rather than a cost center. Wednesday, October 2, 13
    • The Coded Business: Coming to Every Enterprise Manufacturing Financial Services Retail Media and Entertainment High Technology Healthcare Wednesday, October 2, 13
    • Scale x Complexity > Skills Wednesday, October 2, 13
    • DevOps... • Cultural and professional movement • Grew directly from people who have experienced this transition • Applies directly to traditional IT • Applies directly to other systems of survival in our culture Wednesday, October 2, 13
    • DevOps: CAMS • Culture • Automation • Measurement • Sharing Wednesday, October 2, 13
    • Open Communication • Developers & Operations talk and listen to one another • Production & build metrics are available to all • Current infrastructure is documented Wednesday, October 2, 13
    • Incentive & Responsibility Align • Create awesome customer experiences • Responsibility & accountability, not authority • Responsible for your own uptime Wednesday, October 2, 13
    • Respect • You don’t have to be everyone’s best friend • But you do need to recognize contributions and treat each other well • No asshole rule Wednesday, October 2, 13
    • Culture: Trust • Trust that everyone is competent & working toward the common goals • Without trust, the tools don’t matter Wednesday, October 2, 13
    • DevOps: CAMS • Culture • Automation • Measurement • Sharing • Chef is (part of) “A”. Works best with C, M and S. Wednesday, October 2, 13
    • Infrastructure Automation with Chef Wednesday, October 2, 13
    • http://www.flickr.com/photos/steffenz/337700069/ http://www.flickr.com/photos/kky/704056791/ Applications Wednesday, October 2, 13
    • Infrastructure http://www.flickr.com/photos/sbh/462754460/ Wednesday, October 2, 13
    • http://www.flickr.com/photos/philliecasablanca/3354734116/ • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configuration • Other Stuff Collection of Resources Wednesday, October 2, 13
    • Acting in Concert http://www.flickr.com/photos/glowjangles/4081048126/ Wednesday, October 2, 13
    • To Provide a Service http://www.flickr.com/photos/28309157@N08/3743455858/ Wednesday, October 2, 13
    • And it Evolves http://www.flickr.com/photos/16339684@N00/2681435235/ Wednesday, October 2, 13
    • Application Server See Node Wednesday, October 2, 13
    • Application Server Application Database See Nodes Wednesday, October 2, 13
    • Application Server Application Databases See Nodes Grow Wednesday, October 2, 13
    • Application Servers Application Databases See Nodes Grow Wednesday, October 2, 13
    • Application Servers Application Databases Load Balancer See Nodes Grow Wednesday, October 2, 13
    • Application Servers Application Databases Load Balancers See Nodes Grow Wednesday, October 2, 13
    • Application Servers Application Database Cache Load Balancers Application Databases See Nodes Grow Wednesday, October 2, 13
    • Application Servers Application Database Cache Load Balancers Application Databases Tied Together with Configuration Wednesday, October 2, 13
    • Application Servers Application Database Cache Load Balancers Floating IP? Application Databases Infrastructure is a Snowflake Wednesday, October 2, 13
    • Load Balancers Application Servers NoSQL Database Slaves Application Cache Database Cache Database Evolving Complexity Wednesday, October 2, 13
    • Complexity Increases Very Quickly DC1 DC3 DC2 Wednesday, October 2, 13
    • Configuration Management http://www.flickr.com/photos/philliecasablanca/3354734116/ Wednesday, October 2, 13
    • Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/ Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite Typical Infrastructure Wednesday, October 2, 13
    • JBoss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Move SSH off port 22 • Let’s put it on 2022 New Compliance Mandate! Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6 6 Golden Image Updates Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually 12 Instance Replacements Wednesday, October 2, 13
    • • Don’t break anything! • Bob just got fired =( 5 JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite 1 2 4 5 6 7 8 9 10 11 12 3 Done in Maintenance Windows Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite • Invalid configs! Different IP Addresses? Wednesday, October 2, 13
    • Configuration Desperation http://www.flickr.com/photos/francoforeshock/5716969942/ Wednesday, October 2, 13
    • Chef Solves This Problem • But you already guessed that, didn’t you? Wednesday, October 2, 13
    • Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.http://www.flickr.com/photos/louisb/4555295187/ Wednesday, October 2, 13
    • Programs • Chef generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ Wednesday, October 2, 13
    • Declarative Interface to Resources • Define Policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ Wednesday, October 2, 13
    • That Looks Like This package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end Wednesday, October 2, 13
    • Ohai "languages": { "ruby": { }, "perl": { "version": "5.14.2", "archname": "x86_64- linux-gnu-thread-multi" }, "python": { "version": "2.7.3", "builddate": "Aug 1 2012, 05:14:39" }, "php": { "version": "5.3.10-1ubuntu3.6", "builddate": "(cli) (built: Mar" } }, "network": { "interfaces": { "lo": { "mtu": "16436", "flags": [ "LOOPBACK", “UP","LOWER_UP" ], "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "netmask": "255.0.0.0", "scope": "Node" }, "::1": { "family": "inet6", "scope": "Node" } }, }, "eth0": { "type": "eth", "number": "0", "kernel": { "name": "Linux", "release": "3.2.0-32-virtual", "version": "#51-Ubuntu SMP Wed Sep 26 21:53:42 UTC 2012", "machine": "x86_64", "modules": { "isofs": { "size": "40257", "refcount": "0" }, "acpiphp": { "size": "24231", "refcount": "0" } }, "os": "GNU/Linux" }, "os": "linux", "os_version": "3.2.0-32-virtual", "ohai_time": 1369328621.3456137, Wednesday, October 2, 13
    • execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end Decide what to declare Wednesday, October 2, 13
    • execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end Decide what to declare Wednesday, October 2, 13
    • execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end Decide what to declare Wednesday, October 2, 13
    • Recipes and Cookbooks • Recipes are collections of Resources • Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity http://www.flickr.com/photos/shutterhacks/4474421855/ Wednesday, October 2, 13
    • Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]” node ntp client.rb Wednesday, October 2, 13
    • Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]” node ntp client.rb openssh server.rb Wednesday, October 2, 13
    • Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb Wednesday, October 2, 13
    • Roles name "base" description "base" run_list [ "recipe[selinux::disabled]", "recipe[etchosts]", "recipe[yum::epel]", "recipe[debugtools]" ] name "webserver" description "webserver server" run_list [ "role[base]", "recipe[nginx::server]" ] Wednesday, October 2, 13
    • Roles Role Recipe Recipe Recipe Role Role Recipe Recipe RecipeRole Recipe Server ServerServer Server chef-server API Knife Wednesday, October 2, 13
    • Run Lists Server ServerServer Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb Wednesday, October 2, 13
    • Roles Server ServerServer Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Wednesday, October 2, 13
    • Roles Server ServerServer Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb Wednesday, October 2, 13
    • http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search for nodes with Roles • Find configuration data Search: Dynamic Infrastructure Wednesday, October 2, 13
    • Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Wednesday, October 2, 13
    • Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Wednesday, October 2, 13
    • Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Wednesday, October 2, 13
    • Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite So when this... Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite ... becomes this ... Wednesday, October 2, 13
    • JBoss App Memcache PostgreSQL Slaves PostgreSQL Master NagiosGraphite ...this can happen automatically Wednesday, October 2, 13
    • NagiosGraphite JBoss App Memcache PostgreSQL Slaves To Add a New Server… • 2x Web Server Configurations • 2 Web Server Restarts • 4x Database Configurations • 8x Firewall Configurations • DNS Service • Network Configuration • Deployer • 8x Monitoring Changes 20+ Changes 12+ New Infrastructure Dependencies 4+ Hours Count the Resources Wednesday, October 2, 13
    • NagiosGraphite JBoss App Memcache PostgreSQL Slaves To Add a New Server… • 2x Web Server Configurations • 2 Web Server Restarts • 4x Database Configurations • 8x Firewall Configurations • DNS Service • Network Configuration • Deployer • 8x Monitoring Changes 20+ Changes 12+ New Infrastructure Dependencies 4+ Hours Count the Resources Add 1 server 20+ Changes Wednesday, October 2, 13
    • Managing Complexity Later Wednesday, October 2, 13
    • Managing Complexity Later We added: • Load Balancers • MemCache • Search Appliances • Lots of VM’s • More Scale Exponential Increase In: • Configuration Changes • Infrastructure Dependencies • Skills Needed • Greater Risk Wednesday, October 2, 13
    • And at Greater Scale... Wednesday, October 2, 13
    • And at Greater Scale... How Do we Manage This at Cloud Scale? • Thousands of infrastructure dependencies and configurations needed for each change. • Huge Amounts of Time • Increased Cost of Correction of Manual Errors • Huge Need for Talent • Risk of Critical Skills Shortage Wednesday, October 2, 13
    • Automation is a Journey Full Automation Common Automation Tasks: Scripts, OS Compliance, Updates, etc. Configuration ManagementDiscovery and Visibility Application Management Continuous Deployment The Path to the Coded Business Wednesday, October 2, 13
    • Flavors of Chef Server Wednesday, October 2, 13
    • Landscape of Chef-Managed Infrastructure Wednesday, October 2, 13
    • Flavors of Chef Server • SaaS, hosted by Opscode • Manage up to 50,000 servers • Industry-leading SLAs • 24x7x365 Support Options • Get up and running quickly • Pay/grow as you need Hosted Enterprise Chef Wednesday, October 2, 13
    • Flavors of Chef Server • All the power of Hosted Enterprise, behind the firewall • Delivered as enterprise software • Implementation consulting customized to customer’s needs Enterprise Chef Wednesday, October 2, 13
    • Flavors of Chef Server • Core components of Chef Server developed as open source • No enterprise features (LDAP, HA, etc.) but many people use it successfully • Community support Open Source Chef Server Wednesday, October 2, 13
    • Learn Chef! Wednesday, October 2, 13
    • Let’s Learn Chef! • https://learnchef.opscode.com/ • Next meetup: Hands on. Bring laptops! • Feel free to get started early Wednesday, October 2, 13
    • Thanks • Julian Dunn • jdunn@opscode.com • @julian_dunn • www.juliandunn.net Wednesday, October 2, 13