SELF 2011: Deploying Django Application Stacks with Chef
Upcoming SlideShare
Loading in...5
×
 

SELF 2011: Deploying Django Application Stacks with Chef

on

  • 3,871 views

 

Statistics

Views

Total Views
3,871
Views on SlideShare
3,869
Embed Views
2

Actions

Likes
7
Downloads
79
Comments
0

1 Embed 2

http://www.techgig.com 2

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Managing infrastructure with Chef.\n
  • How’s and why’s of managing infrastructure with Chef.\nA quick overview of Chef\nWe’ll talk about the steps involved for automatically building our n-tier Django stack\nWe’ll discuss how your Chef code is reusable across environments\n\n
  • How’s and why’s of managing infrastructure with Chef.\nA quick overview of Chef\nWe’ll talk about the steps involved for automatically building our n-tier Django stack\nWe’ll discuss how your Chef code is reusable across environments\n\n
  • How’s and why’s of managing infrastructure with Chef.\nA quick overview of Chef\nWe’ll talk about the steps involved for automatically building our n-tier Django stack\nWe’ll discuss how your Chef code is reusable across environments\n\n
  • How’s and why’s of managing infrastructure with Chef.\nA quick overview of Chef\nWe’ll talk about the steps involved for automatically building our n-tier Django stack\nWe’ll discuss how your Chef code is reusable across environments\n\n
  • Why not just use some shell scripts?\n
  • \n
  • This is the dream. When everyone’s 2nd girlfriend ‘nagios’ wakes you up in the middle of the night you are ready for the worst.\n
  • Keep track of all the steps required to take bare metal systems to doing their job in the infrastructure.\n\nIt is all about the policy. This can be lo-fi (ie a wiki page and the ‘meat’ cloud) or fully automated with something like Chef.\n\nUsually this is the context of a single node.\n
  • Taking all the systems that have been configured to do their job, and make them work together to actually run the infrastructure.\n\nWRT Chef, we talk about Fully Automated Infrastructure.\n\nThis brings all the single nodes together.\n
  • Introducing Chef.\n\nMaybe you’ve already met!\n
  • Chef provides a framework for fully automating infrastructure, and has some important design principles.\n
  • Chef makes it easy to reason about your infrastructure, at scale. The declarative Ruby configuration language is easy to read, and the predictable ordering makes it easy to understand what’s going on.\n\nChef is flexible, and designed to allow you to build infrastructure. “Tim Toady”\n
  • Chef makes it easy to reason about your infrastructure, at scale. The declarative Ruby configuration language is easy to read, and the predictable ordering makes it easy to understand what’s going on.\n\nChef is flexible, and designed to allow you to build infrastructure. “Tim Toady”\n
  • Chef makes it easy to reason about your infrastructure, at scale. The declarative Ruby configuration language is easy to read, and the predictable ordering makes it easy to understand what’s going on.\n\nChef is flexible, and designed to allow you to build infrastructure. “Tim Toady”\n
  • Chef makes it easy to reason about your infrastructure, at scale. The declarative Ruby configuration language is easy to read, and the predictable ordering makes it easy to understand what’s going on.\n\nChef is flexible, and designed to allow you to build infrastructure. “Tim Toady”\n
  • Chef makes it easy to reason about your infrastructure, at scale. The declarative Ruby configuration language is easy to read, and the predictable ordering makes it easy to understand what’s going on.\n\nChef is flexible, and designed to allow you to build infrastructure. “Tim Toady”\n
  • Idempotent: Consistency, known system state, easy to build\nChef takes idempotent actions to put the system in the desired state.\n
  • Chef makes it easy to reason about your infrastructure.\n\nWe build an API, with primitives and libraries, and let you model your data. We don’t try to discover because we might make an incorrect assumption about the system.\n\nYou need an API to manage your infrastructure. The standard tools on Unix and Windows already use APIs to the system. Use an API to manage your infrastructure.\n
  • Chef comes from the perspective that the tool should do pretty much what you expect, hence sane defaults. To get there we had express some opinions. Those opinions can be changed easily, whether that is modifying the configuration of the chef-client, or how Apache is set up.\n
  • Chef is a flexible framework. You can make it do pretty much anything you want. Just like Perl doesn’t tell the programmer how to program, Chef doesn’t tell the system administrator how to manage the system. You can extend it, mold it and modify it to do what you need.\n\nBeing written in Ruby, using Ruby for the main language was very purposeful. The tool shouldn’t get in your way.\n
  • \n
  • -Chef is a library for building and managing infrastructure, and as such it happens to have some tools we wrote that help with that.\n-Ohai gathers data about your nodes.\n-Chef client runs on your nodes to configure them.\n-Knife is the CLI tool used to access the API.\n-Shef is an interactive console debugger.\n
  • -Chef is a library for building and managing infrastructure, and as such it happens to have some tools we wrote that help with that.\n-Ohai gathers data about your nodes.\n-Chef client runs on your nodes to configure them.\n-Knife is the CLI tool used to access the API.\n-Shef is an interactive console debugger.\n
  • -Chef is a library for building and managing infrastructure, and as such it happens to have some tools we wrote that help with that.\n-Ohai gathers data about your nodes.\n-Chef client runs on your nodes to configure them.\n-Knife is the CLI tool used to access the API.\n-Shef is an interactive console debugger.\n
  • -Chef is a library for building and managing infrastructure, and as such it happens to have some tools we wrote that help with that.\n-Ohai gathers data about your nodes.\n-Chef client runs on your nodes to configure them.\n-Knife is the CLI tool used to access the API.\n-Shef is an interactive console debugger.\n
  • \n
  • -thin server...fat clients => all execution happens down on the nodes...server is dumb REST/file server\n-easy to scale horizontally (like a stateless web app)\n
  • -thin server...fat clients => all execution happens down on the nodes...server is dumb REST/file server\n-easy to scale horizontally (like a stateless web app)\n
  • -thin server...fat clients => all execution happens down on the nodes...server is dumb REST/file server\n-easy to scale horizontally (like a stateless web app)\n
  • -thin server...fat clients => all execution happens down on the nodes...server is dumb REST/file server\n-easy to scale horizontally (like a stateless web app)\n
  • \n
  • Community is important.\n\nhttp://apache.org/licenses/LICENSE-2.0.html\nhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/\nhttp://wiki.opscode.com/display/chef/How+to+Contribute\nhttp://wiki.opscode.com/display/chef/Approved+Contributors\n\n
  • Community is important.\n\nhttp://apache.org/licenses/LICENSE-2.0.html\nhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/\nhttp://wiki.opscode.com/display/chef/How+to+Contribute\nhttp://wiki.opscode.com/display/chef/Approved+Contributors\n\n
  • Community is important.\n\nhttp://apache.org/licenses/LICENSE-2.0.html\nhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/\nhttp://wiki.opscode.com/display/chef/How+to+Contribute\nhttp://wiki.opscode.com/display/chef/Approved+Contributors\n\n
  • Community is important.\n\nhttp://apache.org/licenses/LICENSE-2.0.html\nhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/\nhttp://wiki.opscode.com/display/chef/How+to+Contribute\nhttp://wiki.opscode.com/display/chef/Approved+Contributors\n\n
  • Community is important.\n\nhttp://apache.org/licenses/LICENSE-2.0.html\nhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/\nhttp://wiki.opscode.com/display/chef/How+to+Contribute\nhttp://wiki.opscode.com/display/chef/Approved+Contributors\n\n
  • Community is important.\n\nhttp://apache.org/licenses/LICENSE-2.0.html\nhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/\nhttp://wiki.opscode.com/display/chef/How+to+Contribute\nhttp://wiki.opscode.com/display/chef/Approved+Contributors\n\n
  • \n
  • \n
  • Communication with the server is secure. Clients hold the private key, the server has the public key. The server is not a centralized store of authentication information, pushing the responsibility to the nodes.\n
  • \n
  • \n
  • \n
  • Manage system configuration as resources.\nPut resources together in recipes.\nDistribute recipes with cookbooks.\nAssign recipes to systems through roles.\nStore arbitrary data in data bags.\nTrack it all like source code.\n
  • Resources are an abstraction we feed data into. When you write recipes in Chef, you create resources of things you want to configure.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • Resources declare a description of the state a part of the node should be in.\n
  • The abstraction over the commands or API calls that will configure the resource to be in the state you have defined.\n
  • Providers know how to actually configure the resources to be in the declared state\n\n
  • \n
  • \n
  • Just like recipes themselves are processed in order, the recipes included are processed in order, so when you include a recipe, all its resources are added to the resource collection, then Chef continues to the next.\n\n\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • -the cpan, PyPI or rubygems.org of the cookbbook world!\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Data bags are containers of grouped information.\n\nUsers, application information, network info, cabinet/rack locations. Describe components of your infrastructure with data, and use that data to configure systems.\n\nProcess data bags in recipes to configure systems dynamically.\n
  • \n
  • \n
  • In our example we are deploy Django CMS but this would apply to any Django application.\n
  • Provisioning is the first step. We need some computers on the internet. They’re going to be load balancers, webservers, database servers. Launch those with a cloud API. Every cloud does this.\n
  • You need to do something and you manage the configuration. Install packages, create users, etc. You can use pre-built images for this, but then just for a simple infrastructure (w/o monitoring/trending, middleware, etc) you have four discrete images to maintain for updates both for the OS and app updates.\n
  • This is the last mile. The systems are configured, but also need to be configured to talk to each other.\n
  • \n
  • \n
  • \n
  • \n
  • Every database backed application needs a master database. For this simple example we haven’t done any complex setup of master/slave replication, but the recipes are built such that this would be relatively easy to add.\n
  • The database master recipe will read the application information from the data bag and use it to create the database so the application can store its data.\n
  • all done fully automated\n
  • all done fully automated\n
  • all done fully automated\n
  • We start with one but scale horizontally as needed. We can even scale up and down based on demands.\n
  • The main thing in this role is the application recipe. \n\nThe recipe will read in data from the data bag (in a predefined format) to determine what kind of application to deploy, the repository where it lives, details on where to put it, what roles to search for to find the database, and many more customizable properties.\n\nWe launched two of these to have something to load balance :).\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • -acquire the code (remote file or scm)...create required config files (database.yml, local_settings.py etc.)\n-install the server software...create an application config for server\n
  • You no longer need to track which system has an IP that should be applied as the database master. We can just use its fqdn from a search.\n
  • \n
  • We’re using haproxy, and we’ll search for a specific application to load balance. The recipe is written to search for the django_cms role to find systems that should be pool members.\n
  • \n
  • -This search occurs everytime Chef runs on the load balancer.\n-If you add more app servers to the mix, the load balancer will *automatically* become aware!\n
  • A single set of Chef cookbooks, roles and databags allows you to deploy your application stack in numerous ways.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • https://github.com/opscode/cookbooks/tree/master/application\nhttp://www.opscode.com/blog/category/quickstarts/\n
  • Before we go to questions, here’s some resources to be aware of.\n
  • \n
  • \n

SELF 2011: Deploying Django Application Stacks with Chef SELF 2011: Deploying Django Application Stacks with Chef Presentation Transcript

  • Deploying Fully Automated, Highly AvailableDjango Application Stacks with Chef schisamo@opscode.com www.opscode.com
  • Who am I?
  • Who am I?• Seth Chisamore Senior Technical Consultant/Evangelist Opscode, Inc.
  • Who am I?• Seth Chisamore Senior Technical Consultant/Evangelist Opscode, Inc.• Project lead for many Opscode sponsored open source projects: • Opscode Cookbooks • Knife cloud provisioning plugins (ec2, rackspace & openstack) • Knife Windows plugin (knife-windows)
  • Who are you? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • Who are you?• System administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • Who are you?• System administrators?• Developers? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • Who are you?• System administrators?• Developers?• “Business” People? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • What are we talking about? http://www.flickr.com/photos/peterkaminski/2174679908/
  • Agendahttp://www.flickr.com/photos/koalazymonkey/3590953001/
  • Agenda• How’s and Why’s http://www.flickr.com/photos/koalazymonkey/3590953001/
  • Agenda• How’s and Why’s• Chef 101 http://www.flickr.com/photos/koalazymonkey/3590953001/
  • Agenda• How’s and Why’s• Chef 101• Deploying n-tier Django app stacks with Chef http://www.flickr.com/photos/koalazymonkey/3590953001/
  • Agenda• How’s and Why’s• Chef 101• Deploying n-tier Django app stacks with Chef• Leveraging the same Chef code to deploy different configurations of our stack http://www.flickr.com/photos/koalazymonkey/3590953001/
  • Infrastructure as Code
  • A technical domainrevolving aroundbuilding andmanaginginfrastructureprogrammatically
  • Enable the reconstruction of the business fromnothing but a source coderepository, an application data backup, and bare metal resources.
  • ConfigurationManagement
  • ConfigurationManagement
  • System Integration http://www.flickr.com/photos/opalsson/3773629074/
  • The Chef Framework With thanks (and apologies) to Stephen Nelson-Smith
  • The Chef Framework
  • The Chef Framework• Idempotent
  • The Chef Framework• Idempotent• Library & Primitives
  • The Chef Framework• Idempotent• Library & Primitives• Flexibility
  • The Chef Framework• Idempotent• Library & Primitives• Flexibility• Reasonability
  • The Chef Framework• Idempotent• Library & Primitives• Flexibility• Reasonability• TIMTOWTDI
  • Multiple applications of an operation do not change the result
  • We start with APIs, you supply data
  • Defaults are sane, but easily changed
  • There is more than one way to do it
  • The Chef Tool(s) With thanks (and apologies) to Stephen Nelson-Smith
  • The Chef Tool(s)
  • The Chef Tool(s)• ohai
  • The Chef Tool(s)• ohai• chef-client
  • The Chef Tool(s)• ohai• chef-client• knife
  • The Chef Tool(s)• ohai• chef-client• knife• shef
  • The Chef APIWith thanks (and apologies) to Stephen Nelson-Smith
  • The Chef API
  • The Chef API• Fat Client/Thin Server
  • The Chef API• Fat Client/Thin Server• RESTful API w/ JSON
  • The Chef API• Fat Client/Thin Server• RESTful API w/ JSON• Search Service
  • The Chef API• Fat Client/Thin Server• RESTful API w/ JSON• Search Service• Derivative Services (mash-up your infrastructure)
  • The Chef Community With thanks (and apologies) to Stephen Nelson-Smith
  • The Chef Community
  • The Chef Community• Apache License, Version 2.0
  • The Chef Community• Apache License, Version 2.0• 360+ Individual contributors
  • The Chef Community• Apache License, Version 2.0• 360+ Individual contributors• 70+ Corporate contributors
  • The Chef Community• Apache License, Version 2.0• 360+ Individual contributors• 70+ Corporate contributors • Dell, Rackspace,VMware, RightScale, Heroku, and more
  • The Chef Community• Apache License, Version 2.0• 360+ Individual contributors• 70+ Corporate contributors • Dell, Rackspace,VMware, RightScale, Heroku, and more• 240+ cookbooks
  • The Chef Community• Apache License, Version 2.0• 360+ Individual contributors• 70+ Corporate contributors • Dell, Rackspace,VMware, RightScale, Heroku, and more• 240+ cookbooks• http://community.opscode.com
  • Chef client runs on your systems
  • Clients talk to a Chef server
  • Clients authenticate with RSA keys
  • Each system youconfigure is a managed node
  • Nodes have attributes{ "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", "os": "darwin", "current_user": "schisamo", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "helsinki", "fqdn": "helsinki.local", "uptime_seconds": 1619358}
  • Nodes are Searchableknife search node ‘platform:mac_os_x’ search(:node, ‘platform:mac_os_x’)
  • Chef Enables Infrastructure as Code package "haproxy" do action :install end• Resources template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb"• Recipes owner "root" group "root"• Cookbooks mode 0644• notifies :restart, "service[haproxy]" Roles end• Data Bags service "haproxy" do• Source Code supports :restart => true action [:enable, :start] end
  • Chef managesResources on Nodes
  • Chef Resourcespackage "haproxy" do action :installendtemplate "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]"endservice "haproxy" do supports :restart => true action [:enable, :start]end
  • Chef Resources package "haproxy" do action :install end• Have a type. template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => true action [:enable, :start] end
  • Chef Resources package "haproxy" do action :install end• Have a type. template "/etc/haproxy/haproxy.cfg" do• source "haproxy.cfg.erb" Have a name. owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => true action [:enable, :start] end
  • Chef Resources package "haproxy" do action :install end• Have a type. template "/etc/haproxy/haproxy.cfg" do• source "haproxy.cfg.erb" Have a name. owner "root"• Have parameters. group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => true action [:enable, :start] end
  • Chef Resources package "haproxy" do action :install end• Have a type. template "/etc/haproxy/haproxy.cfg" do• source "haproxy.cfg.erb" Have a name. owner "root"• Have parameters. group "root" mode 0644• Take action to put the resource notifies :restart, "service[haproxy]" end in the declared state. service "haproxy" do supports :restart => true action [:enable, :start] end
  • Chef Resources package "haproxy" do action :install end• Have a type. template "/etc/haproxy/haproxy.cfg" do• source "haproxy.cfg.erb" Have a name. owner "root"• Have parameters. group "root" mode 0644• Take action to put the resource notifies :restart, "service[haproxy]" end in the declared state.• Can send notifications to other service "haproxy" do supports :restart => true resources. action [:enable, :start] end
  • Resources take action through Providers
  • Chef Providers• Multiple providers per resource type.• For example the `package` resource has providers for: Apt, Yum, Rubygems, Portage, Macports, FreeBSD Ports, etc.• If you are developer think of it as an interface and an implementation.
  • Recipes are collections of Resources
  • Chef Recipes package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb"• Recipes are evaluated for owner "root" resources in the order they group "root" mode 0644 appear. notifies :restart, "service[haproxy]"• Each resource object is added end to the Resource Collection. service "haproxy" do supports :restart => true action [:enable, :start] end
  • Chef Recipes• Recipes can include other include_recipe include_recipe "apache2" "apache2::mod_rewrite" recipes. include_recipe "apache2::mod_deflate"• Included recipes are include_recipe include_recipe "apache2::mod_headers" "apache2::mod_php5" processed in order.
  • Chef Recipes%w{ python python-dev }.each do |pkg| package pkg do action :install endendpool_members = search("node", "role:django_cms")template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, "service[haproxy]"end
  • Chef Recipes %w{ python python-dev }.each do |pkg| package pkg do action :install end end• Extend recipes with pool_members = search("node", "role:django_cms") Ruby. template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, "service[haproxy]" end
  • Chef Recipes %w{ python python-dev }.each do |pkg| package pkg do action :install end end• Extend recipes with pool_members = search("node", "role:django_cms") Ruby.• Dynamic configuration template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" through search. owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, "service[haproxy]" end
  • Cookbooks arepackages for recipes
  • Cookbook Metadata• Cookbooks are like packages.• Cookbooks have dependencies.
  • Cookbooks are easy to share. community.opscode.com
  • Chef Rolesname "django_cms"description "django cms app server"run_list( "recipe[mysql::client]", "recipe[application]")name "django_cms_load_balancer"description "Django CMS load balancer"run_list( "recipe[haproxy::app_lb]")override_attributes( "haproxy" => { "app_server_role" => "django_cms" })
  • Chef Roles name "django_cms" description "django cms app server" run_list( "recipe[mysql::client]", "recipe[application]" )• Roles describe nodes. name "django_cms_load_balancer" description "Django CMS load balancer" run_list( "recipe[haproxy::app_lb]" ) override_attributes( "haproxy" => { "app_server_role" => "django_cms" } )
  • Chef Roles name "django_cms" description "django cms app server" run_list( "recipe[mysql::client]", "recipe[application]" )• Roles describe nodes. name "django_cms_load_balancer"• Roles have a run list. description "Django CMS load balancer" run_list( "recipe[haproxy::app_lb]" ) override_attributes( "haproxy" => { "app_server_role" => "django_cms" } )
  • Chef Roles name "django_cms" description "django cms app server" run_list( "recipe[mysql::client]", "recipe[application]" )• Roles describe nodes. name "django_cms_load_balancer"• Roles have a run list. description "Django CMS load balancer" run_list(• Roles can have attributes. ) "recipe[haproxy::app_lb]" override_attributes( "haproxy" => { "app_server_role" => "django_cms" } )
  • Data bags store arbitrary data
  • Data Bags% knife data bag show apps django_cmsdatabase_master_role: ["django_cms_database_master"]databases: production: adapter: mysql database: django_cms_production encoding: utf8 password: awesome_password port: 3306 username: django_cmsdeploy_to: /srv/django_cmsgroup: nogroupid: django_cmslocal_settings_file: local_settings.pymysql_root_password: production: mysql_rootowner: nobodypackages: curl:pips: django: 1.2.5repository: https://github.com/opscode/django-quick-start-cms.gitrevision: production: masterserver_roles: ["django_cms"]type: django_cms: django gunicorn
  • Track it like source code...% git logcommit d640a8c6b370134d7043991894107d806595cc35Author: jtimberman <joshua@opscode.com> Import nagios version 1.0.0commit c40c818498710e78cf73c7f71e722e971fa574e7Author: jtimberman <joshua@opscode.com> installation and usage instruction docscommit 99d0efb024314de17888f6b359c14414fda7bb91Author: jtimberman <joshua@opscode.com> Import haproxy version 1.0.1commit c89d0975ad3f4b152426df219fee0bfb8eafb7e4Author: jtimberman <joshua@opscode.com> add mediawiki cookbookcommit 89c0545cc03b9be26f1db246c9ba4ce9d58a6700Author: jtimberman <joshua@opscode.com> multiple environments in data bag for mediawiki
  • Ponies ,(green )Unicorns and n-tier oh my!
  • N-Tier FTW• Provision
  • N-Tier FTW Load BalancerApp Server App Server • Provision • Configure Database Master
  • N-Tier FTW Load Balancer • ProvisionApp Server App Server • Configure • Integrate Database Master
  • Chef Enables Systems Integration
  • Chef Enables Systems Integration• Chef searches allow nodes to be aware of each other and self configure
  • Chef Enables Systems Integration• Chef searches allow nodes to be aware of each other and self configure• A shared application data bag allows:
  • Chef Enables Systems Integration• Chef searches allow nodes to be aware of each other and self configure• A shared application data bag allows: • many app servers to checkout the same application code during initial bootstrapping
  • Chef Enables Systems Integration• Chef searches allow nodes to be aware of each other and self configure• A shared application data bag allows: • many app servers to checkout the same application code during initial bootstrapping • app server and database servers to initialize off of the same database config
  • Database Master
  • Database Master% knife role show django_cms_database_masterdefault_attributes: {}description: db master for the django_cms app.env_run_lists: {}json_class: Chef::Rolename: django_cms_database_masteroverride_attributes: {}run_list: recipe[database::master]
  • Database Master
  • Database Master• Install and configure MySQL
  • Database Master• Install and configure MySQL• Create the application database
  • Database Master• Install and configure MySQL• Create the application database• Grant the application user access
  • Django App Servers (1..n)
  • Django App Servers% knife role show django_cmschef_type: roledefault_attributes: {}description: django_cms front end application server.env_run_lists: {}json_class: Chef::Rolename: django_cmsoverride_attributes: {}run_list: recipe[mysql::client], recipe[application]
  • Django App Servers
  • Django App Servers• Deploy the Code
  • Django App Servers• Deploy the Code • checkout the code with git
  • Django App Servers• Deploy the Code • checkout the code with git • install required packages (os and python)
  • Django App Servers• Deploy the Code • checkout the code with git • install required packages (os and python) • create environment specific config files
  • Django App Servers• Deploy the Code • checkout the code with git • install required packages (os and python) • create environment specific config files• Serve the Code
  • Django App Servers• Deploy the Code • checkout the code with git • install required packages (os and python) • create environment specific config files• Serve the Code • install Gunicorn (or apache2/mod_wsgi)
  • Django App Servers• Deploy the Code • checkout the code with git • install required packages (os and python) • create environment specific config files• Serve the Code • install Gunicorn (or apache2/mod_wsgi) • create an app specific config file (vhost)
  • Search for Database Masterresults = search(:node, "role:django_cms_database_master")template "/srv/django_cms/shared/local_settings.py" do source "settings.py.erb" mode "644" variables( :host => results[0][fqdn] )end
  • Load Balancer
  • Load Balancer% knife role show django_cms_load_balancerchef_type: roledefault_attributes: {}description: Django CMS load balancerenv_run_lists: {}json_class: Chef::Rolename: django_cms_load_balanceroverride_attributes: haproxy: app_server_role: django_cmsrun_list: recipe[haproxy::app_lb]
  • Load Balancer• The balancing pool should grow as more application servers are brought online
  • Search for App Serverspool_members = search(:node, "role:django_cms")template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, "service[haproxy]"end
  • One set of codez.... ...to rule them all
  • Same Code Different Configurations
  • Same Code Different Configurations• We can deploy the same app stack on:
  • Same Code Different Configurations• We can deploy the same app stack on: • a single node or a fully redundant cluster of nodes.
  • Same Code Different Configurations• We can deploy the same app stack on: • a single node or a fully redundant cluster of nodes. • physical hardware or any number of public/private cloud providers.
  • Same Code Different Configurations• We can deploy the same app stack on: • a single node or a fully redundant cluster of nodes. • physical hardware or any number of public/private cloud providers. • a developer workstation fully virtualized with Vagrant + VirtualBox
  • Staging/QA - Single Node# launch stack in your data center% knife bootstrap 123.123.3.3 -r ‘role[django_cms_database_master]’, ‘role[django_cms]’, ‘role[django_cms_load_balancer]’# launch stack in EC2% knife ec2 server create -r ‘role[django_cms_database_master]’, ‘role[django_cms]’, ‘role[django_cms_load_balancer]’# launch stack in Rackspace% knife rackspace server create -r ‘role[django_cms_database_master]’, ‘role[django_cms]’, ‘role[django_cms_load_balancer]’
  • Production - Multi-Machine Cluster% knife ec2 server create -r ‘role[django_cms_database_master]’% knife ec2 server create -r ‘role[django_cms]’% knife ec2 server create -r ‘role[django_cms]’% knife ec2 server create -r ‘role[django_cms_load_balancer]’
  • Developer Workstation% cat VagrantfileVagrant::Config.run do |config| config.vm.box = base_box config.vm.provision :chef_server do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/foo" chef.validation_key_path = "~/.chef/private-pems/foo-validator.pem" chef.validation_client_name = "#{ENV[ORGNAME]}-validator" chef.provisioning_path = "/etc/chef" chef.run_list = [ role[django_cms_database_master], role[django_cms], role[django_cms_load_balancer] ] endend% vagrant up
  • BONUS: deploy [insert your favorite webframework here] stacks
  • Application Cookbook• Supports the following web stacks: • Django • Java Webapps • Php • Rails • Node.js (coming soon)• Opscode contributed quick starts walk you through example deployments
  • Further Resources & Closing• www.opscode.com• wiki.opscode.com• @opscode, #opschef• irc.freenode.net, #chef, #chef-hacking• lists.opscode.com
  • Questions? http://www.flickr.com/photos/oberazzi/318947873/
  • Thanks! schisamo@opscode.com www.opscode.com