TXLF: Automated Deployment of OpenStack with Chef
Upcoming SlideShare
Loading in...5
×
 

TXLF: Automated Deployment of OpenStack with Chef

on

  • 4,435 views

Automated Deployment of OpenStack with Chef talk given April 2, 2011 at the Texas Linux Fest.

Automated Deployment of OpenStack with Chef talk given April 2, 2011 at the Texas Linux Fest.

Statistics

Views

Total Views
4,435
Views on SlideShare
4,375
Embed Views
60

Actions

Likes
5
Downloads
228
Comments
1

5 Embeds 60

http://www.opscode.com 51
http://www.getchef.com 6
http://www.linkedin.com 1
https://www.linkedin.com 1
http://localhost 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • good review of crowbar and chef
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

TXLF: Automated Deployment of OpenStack with Chef TXLF: Automated Deployment of OpenStack with Chef Presentation Transcript

  • Automated Deploymentof OpenStack with Chef Texas Linux Fest April 2, 2011 1
  • Introductions Matt Ray Senior Technical Evangelist matt@opscode.com @mattray GitHub:mattray 2
  • What is OpenStack? 3
  • Founders operate atmassive scale NASA 4
  • OpenStack: The Mission "To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable." 5
  • OpenStack Founding Principles Apache 2.0 license (OSI), open development process Open design process, 2x year public Design Summits Publicly available open source code repositories Open community processes documented and transparent Commitment to drive and adopt open standards Modular design for deployment flexibility via APIs 6
  • Community with Broad Support 7
  • Software to provision virtual machines on standard hardware at massive scaleOpenStack Computecreating open source software to build public and private clouds Software to reliably store billions of objects distributed across standard hardware OpenStackObject Storage 8
  • OpenStack Compute Key Features ReST-based API Asynchronous eventually consistent communication Horizontally and massively scalable Hypervisor agnostic: support for Xen ,XenServer, Hyper-V, KVM, UML and ESX is coming Hardware agnostic: standard hardware, RAID not required 9
  • User ManagerCloud Controller: Global state ofsystem, talks to LDAP, OpenStackObject Storage, and node/storageworkers through a queue ATAoE / iSCSIAPI: Receives HTTP requests,converts commands to/from APIformat, and sends requests to cloudcontroller Host Machines: workers that spawn instances Glance: HTTP + OpenStack ObjectOpenStack Compute Storage for server images 10
  • Hardware Requirements OpenStack is designed to run on industry standard hardware, with flexible configurations Compute x86 Server (Hardware Virt. recommended) Storage flexible (Local, SAN, NAS) Object Storage x86 Server (other architectures possible) Do not deploy with RAID (can use controller for cache) 11
  • Why is OpenStack important? Open eliminates vendor lock-in Working together, we all go faster Freedom to federate, or move between clouds 12
  • What is Chef? 13
  • Chef enables Infrastructure as Code Manage configuration as idempotent Resources. Put them together in Recipes. Track it like Source Code. Configure your servers. 14
  • At a High Level Library for configuration management Configuration management system Systems integration platform API for your entire Infrastructure 15
  • Fully automated Infrastructure 16
  • Principles Idempotent Data-driven Sane defaults Hackability TMTOWTDI 17
  • Open Source and Community Apache 2 licensed Large and active community Over 300 individual contributors (60+ corporate) Community is Important! 18
  • 19
  • How does it Work? 20
  • How does it Work? Miracles! 21
  • How does it Work? Miracles! (no really) 22
  • Chef Client runs on your System 23
  • Chef Client runs on your System ohai! 24
  • Clients talk to the Chef Server 25
  • The Opscode Platformis a hosted Chef Server 26
  • We call each system you configure a Node 27
  • Nodes have Attributes{ "kernel": { "machine": "x86_64", "name": "Darwin", Kernel info! "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;root:xnu-1504.7.4~1/RELEASE_I386", }, "release": "10.4.0" Platform info! "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", "os": "darwin", "current_user": "mray", "ohai_time": 1278602661.60043, "os_version": "10.4.0", Hostname and IP! "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "morbo", "fqdn": "morbomorbo.local", "uptime_seconds": 1619358} 28
  • Nodes have a Run List What Roles and Recipes to Apply in Order 29
  • Nodes have Roleswebserver, database, monitoring, etc. 30
  • Roles have a Run List What Roles and Recipes to Apply in Order 31
  • name "webserver"description "Systems that serve HTTP traffic"run_list( "role[base]", "recipe[apache2]", "recipe[apache2::mod_ssl]")default_attributes( "apache" => { "listen_ports" => [ "80", "443" ] })override_attributes( "apache" => { "max_children" => "50" }) 32 32
  • name "webserver"description "Systems that serve HTTP traffic"run_list( "role[base]", Can include "recipe[apache2]", other roles! "recipe[apache2::mod_ssl]")default_attributes( "apache" => { "listen_ports" => [ "80", "443" ] })override_attributes( "apache" => { "max_children" => "50" }) 32 32
  • Chef managesResources on Nodes 33
  • Resources Declare a description of the state a part of the node should be in 34
  • Resources package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34
  • Resources ‣ Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34
  • Resources ‣ Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install ‣ Have a name end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34
  • Resources ‣ Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install ‣ Have a name end template "/etc/apache2/apache2.conf" do ‣ Have parameters source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34
  • Resources ‣ Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install ‣ Have a name end template "/etc/apache2/apache2.conf" do ‣ Have parameters source "apache2.conf.erb" owner "root" ‣ Take action to put the group "root" mode 0644 resource in the action :create declared state end Declare a description of the state a part of the node should be in 34
  • Resources take action through Providers 35
  • Recipes are lists of Resources 36
  • Recipes 1 package "apache2" do version "2.2.11-2ubuntu2.6" action :install end Evaluate and apply template "/etc/apache2/apache2.conf" do Resources in the order source "apache2.conf.erb" owner "root" they appear group "root" mode 0644 action :create2 end 37
  • Order Matters 38
  • Recipes are just Ruby!extra_packages = case node[:platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend 39
  • Cookbooks arepackages for Recipes 40
  • Cookbooks Distributable, shareable comunity.opscode.com Infrastructure as Code Versioned Hundreds 41
  • Cookbooks Recipes Files Templates Attributes Metadata 42
  • Data bags store arbitrary data 43
  • A user data bag item...% knife data bag show users mray{ "comment": "Matt Ray", "groups": "sysadmin", "ssh_keys": "ssh-rsa SUPERSEKRATS mray@morbo", "files": { ".bashrc": { "mode": "0644", "source": "dot-bashrc" }, ".emacs": { "mode": "0644", "source": "dot-emacs" } }, "id": "mray", "uid": 7004, "shell": "/usr/bin/bash" } 44
  • Environments manageversioned infrastructure 45
  • Command-line API utility, Knife http://www.flickr.com/photos/myklroventine/3474391066/ Copyright © 2011 Opscode, Inc - All Rights Reserved 46 46
  • Search $ knife search node platform:ubuntu‣ CLI or in Ruby search(:node, ‘platform:centos’)‣ Nodes are searchable $ knife search role max_children:50‣ Roles are searchable search(:role, ‘max_children:50’)‣ Recipes are $ knife search node ‘role:webserver’ searchable search(:node, ‘role:webserver’)‣ Data bags are $ knife users ‘shell:/bin/bash’ searchable search (:users, ‘group:sysadmins’) 47
  • 48
  • HOW TO: Turn Racks ofStandard Hardware Into a Cloud with OpenStack 49
  • What Works Today? 50
  • Compute (Nova) Single machine installation ‣ Role: nova-single-machine ‣ MySQL, RabbitMQ ‣ Nova-(api|scheduler|network|objectstore|compute) Multi-machine ‣ Role: nova-multi-controller (1) ‣ Role: nova-multi-compute (N) 51
  • Role: nova-single-machinename "nova-single-machine-install"description "Installs everything required to run Nova on a singlemachine"run_list( "role[nova-multi-controller]", "role[nova-multi-compute]" ) 52
  • Role: nova-multi-controllername "nova-multi-controller"description "Installs requirements to run the Controller node in aNova cluster"run_list( "role[nova-support-server]", "role[nova-head]", "role[nova-cloud-controller]", "role[nova-super-user-setup]" ) 53
  • Role: nova-multi-computename "nova-multi-compute"description "Installs requirements to run a Compute node in a Novacluster"run_list( "recipe[nova::compute]") 54
  • What does this look like? 55
  • Crowbar‣ Codename for the OpenStack installer from Dell ‣ Dell is releasing this under the Apache 2 license ‣ Extension of the Chef server ‣ Jointly developed by Dell, Rackspace and Opscode 56
  • Crowbar - What does it Do?‣ Crowbar is a PXE state machine‣ starts with bare metal hardware‣ manages and configures BIOS and network settings‣ network boot and installation‣ nodes are configured with Chef‣ deploys OpenStack, could be used for anything 57
  • OpenStack Installation‣ Cookbooks uploaded $ $ knife cookbook upload -a knife cookbook list $ rake roles‣ Roles uploaded $ knife role list $ knife node list‣ Nodes ready 58
  • AMIsname "nova-ami-urls"description "Feed in a list URLs for AMIs to download"default_attributes( "nova" => { "images" =>["http://192.168.11.7/ubuntu1010-UEC-localuser-image.tar.gz”] } )$ knife role from file roles/nova-ami-urls.rb‣ Use an existing AMI‣ Update URL to your own 59
  • Assign the Roles$ knife node run_list add crushinator.localdomain "role[nova-ami-urls]"{ "run_list": [ "role[nova-ami-urls]" ]}$ knife node run_list add crushinator.localdomain "role[nova-single-machine-install]"{ "run_list": [ "role[nova-ami-urls]" "role[nova-single-machine-install]", ]} 60
  • chef-clientmray@ubuntu1010:~$ sudo chef-client[Fri, 25 Feb 2011 11:52:59 -0800] INFO: Starting Chef Run (Version0.9.12)...[Fri, 25 Feb 2011 11:56:05 -0800] INFO: Chef Run complete in5.911955 seconds[Fri, 25 Feb 2011 11:56:05 -0800] INFO: cleaning the checksum cache[Fri, 25 Feb 2011 11:56:05 -0800] INFO: Running report handlers[Fri, 25 Feb 2011 11:56:05 -0800] INFO: Report handlers complete 61
  • The Moment of Truthnova@$ nova-manage service listnova@$ euca-describe-imagesnova@$ euca-run-instances ami-h8wh0j17 -k mykey -t m1.tinynova@$ euca-describe-instancesnova@$ ssh -i mykey.priv ubuntu@10.0.0.2Linux i-00000001 2.6.35-24-virtual #42-Ubuntu SMP Thu Mar 3005:15:26 UTC 2011 x86_64 GNU/LinuxUbuntu 10.10Welcome to Ubuntu!<SNIP>See "man sudo_root" for details.ubuntu@i-00000001:~$ 62
  • How Did We Get Here? 63
  • Forked from Anso Labs’ Cookbooks Bootstrapped by Opscode Chef Solo/Vagrant installs for Developers http://github.com/ansolabs/openstack-cookbooks 64
  • Who’s involvedso far? 65
  • What’s Next? 66
  • Nova needed enhancements Pluggable/Modular Roles ‣ Database ‣ ObjectStore ‣ Network ‣ Virtualization Swift and Glance integration 67
  • 68
  • Dashboard 69
  • Knife ‣ http://github.com/opscode/knife-openstack ‣ Nova has same API as Amazon ‣ Fog supports OpenStack already ‣ knife openstack server create ‘role [base]’ -i ami-a403f6xd -f m1.micro - A “OpenStack instance” 70
  • Object Storage (Swift) ‣ Recipes originated from Anso Labs’ repository ‣ Will be managed with Chef and Crowbar ‣ Included in the ‘bexar’ branch ‣ Untested so far (Cactus will tackle) 71
  • Image Registry (Glance) ‣ Recipes originated from Anso Labs’ repository ‣ Will be managed with Chef and Crowbar ‣ Included in the ‘bexar’ branch ‣ Untested so far (Cactus!) 72
  • Scaling changes howwe deploy OpenStack! 73
  • Deployment Scenarios ‣ Single machine is special case of multi-install ‣ Controller + Compute nodes is a known quantity for small installations ‣ Nova + Swift + Glance in large installations ‣ Services separated and HA configurations supported ‣ Documentation and Chef Roles will be the solution 74
  • Cactus, Diablo, ... Development continues... Branches for each stable release Design Summit later this month Design Summit in the Fall 75
  • Rackspace Cloud Builders Commercial support and Training for OpenStack ‣ Opscode ‣ Dell ‣ Equinix ‣ Cloudscaling ‣ Citrix 76
  • Get Involved! https://github.com/mattray/openstack-cookbooks/tree/bexar http://lists.openstack.org http://lists.opscode.com #chef on irc.freenode.net #openstack on irc.freenode.net matt@opscode.com jordan@openstack.com 77