Chef and OpenStack Workshop from ChefConf 2013
Upcoming SlideShare
Loading in...5
×
 

Chef and OpenStack Workshop from ChefConf 2013

on

  • 4,373 views

Slides from the ChefConf 2013 Chef and OpenStack workshop. Attendees were at many different levels of

Slides from the ChefConf 2013 Chef and OpenStack workshop. Attendees were at many different levels of

Statistics

Views

Total Views
4,373
Views on SlideShare
4,088
Embed Views
285

Actions

Likes
9
Downloads
335
Comments
0

8 Embeds 285

https://twitter.com 195
http://www.scoop.it 76
http://irq.tumblr.com 9
https://web.tweetdeck.com 1
http://192.168.1.2 1
http://moderation.local 1
http://www.newsblur.com 1
http://safe.txmblr.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Chef and OpenStack Workshop from ChefConf 2013 Chef and OpenStack Workshop from ChefConf 2013 Presentation Transcript

  • Chef and OpenStackMatt RayApril 24, 2013
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • Introductions• Matt Ray• matt@opscode.com• mattray IRC, GitHub• @mattrayhttp://upload.wikimedia.org/wikipedia/commons/2/27/Hi_How_Are_You_Austin_2005.jpgPresenter notes: Austin, San Antonio, Santa Clara, Boston, San Francisco, San Diego, PortlandBexar and onMercado Libre, Dell, Rackspace, HP, DreamHost
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • "To produce the ubiquitous Open Source cloud computingplatform that will meet the needs of public and privatecloud providers regardless of size, by being simple toimplement and massively scalable."Mission Statement
  • Why OpenStack?‣ Control. Open source, no vendor lock in. Apache 2 license.‣ Flexibility. Modular design integrates legacy and third party technologies.‣ Emerging Industry Standard. More than 180 technology industry leaders backing it andmajor public clouds built on OpenStack. ‣ Proven. Originally built for scale and redundancy at NASA and Rackspace. More than200 large-scale deployments worldwide.‣ Compatible and Connected. Enables portability.Control.  Open  source  means  you’re  never  locked  to  a  proprietary  vendor.  You  always  have  visibility  and  the  ability  to  directly  influence  the  roadmap  through  the  open  design  process.Flexibility.  Modular  design  can  integrate  with  legacy  systems  and  third-­‐party  technologies,  so  you  dont  have  to  rip-­‐and-­‐replace  your  exisAng  infrastructure.  Emerging  Industry  Standard.    More  than  170  leading  technology  companies  across  the  globe  are  developing  and  building  tools  for  OpenStack,  including  AT&T,  Cisco,  Dell,  HP,  Intel,  IBM,  MicrosoP  and  Red  Hat,  and  new  OpenStack  clouds  are  coming  online  daily.Proven  and  Scalable.  OpenStack  was  built  for  scale  and  redundancy.  You  can  run  the  same  soPware  that  today  powers  some  of  the  worlds  largest  public  and  private  clouds.Compa<ble  and  Connected.  CompaAbility  with  public  OpenStack  clouds  means  enterprises  are  prepared  for  the  future—making  it  easy  to  migrate  data  and  applicaAons  to  public  clouds  when  condiAons  are  right.
  • OpenStack Controls Compute, Storage & Networking
  • Architecture
  • Architecture: Nova
  • Compute: Nova‣ Virtual Machines‣ Provision and manage large pools of on-demand computing resources(hypervisors & instances)‣ KVM‣ Xen‣ LXC‣ Hyper-V‣ VMware‣ Bare-metalCompute (codenamed "Nova") provides virtual servers upon demand. Rackspace and HP provide commercial compute services builton Nova and it is used internally at companies like Mercado Libre, AT&T and NASA (where it originated).
  • Architecture: Cinder
  • Block Storage: Cinder‣ Virtual Block Storage Devices‣ Volumes on commodity storage gear‣ Drivers for more advanced systems like NetApp, Solidfire, Ceph and Nexenta‣ Released in Folsom Fall 2012Block Storage (codenamed "Cinder") provides persistent block storage to guest VMs. This project was born from code originally inNova (the nova-volume service described below). Please note that this is block storage (or volumes) not filesystems like NFS or CIFSshare.Cinder is new for the Folsom release.
  • Architecture: Quantum
  • Networking as a Service: Quantum‣ Virtualized Networking‣ Software Defined Networking (SDN)‣ Automation of hardware & software‣ Define network connectivity & addressing used by devices from other services‣ Drivers for Ryu, Floodlight, Nicira, Midokura, Cisco and many more‣ Released in Folsom Fall 2012Network (codenamed "Quantum") provides "network connectivity as a service" between interface devices managed by otherOpenStack services (most likely Nova). The service works by allowing users to create their own networks and then attach interfaces tothem. Quantum has a pluggable architecture to support many popular networking vendors and technologies. Quantum is new in theFolsom release.
  • Architecture: Glance
  • Image Registry: Glance‣ Multi-format virtual disk image registry & catalog‣ Delivery of images to Nova Compute‣ Allows uploads of private and public images in a wide variety of formats‣ Machine (kernel/ramdisk outside of image, a.k.a. AMI)‣ qcow2 (Qemu/KVM)‣ VMDK (VMWare)‣ OVF (VMWare, others)‣ And moreImage (codenamed "Glance") provides a catalog and repository for virtual disk images. These disk images are mostly commonly usedin OpenStack Compute. While this service is technically optional, any cloud of size will require it.
  • Architecture: Keystone
  • Identity: Keystone‣ Unifies all core projects with common authentication system‣ Provides authorization for multiple log-in credentials‣ Username/password‣ Token-based‣ AWS-style logins‣ Integrate with existing systemsIdentity (codenamed "Keystone") provides authentication and authorization for all the OpenStack services. It also provides a servicecatalog of services within a particular OpenStack cloud.
  • Architecture: Swift
  • Object Storage: Swift‣ Redundant, resilient, horizontally scalable object storage‣ Petabytes of reliable storage on standard gear‣ Examples include virtual machine images, photo storage, email storageand backup archiving‣ Rackspace Cloud FilesObject Store (codenamed "Swift") allows you to store or retrieve files (but not mount directories like a fileserver). Several companiesprovide commercial storage services based on Swift. These include KT, Rackspace (from which Swift originated) and Internap. Swift isalso used internally at many large companies to store their data.
  • Architecture: Horizon
  • Web Dashboard: Horizon‣ Self-service, role-based web interface for users and administrators‣ Provision cloud-based resources through a self-service portal‣ Create and manage projects and users, defining resources available tothem‣ Extensible design makes it easy to plug in and expose third partyproducts and services‣ Django application that consumes APIsDashboard (codenamed "Horizon") provides a modular web-based user interface for all the OpenStack services. With this webGUI, you can perform most operations on your cloud like launching an instance, assigning IP addresses and setting accesscontrols.
  • OpenStack Community‣ OpenStack Summits (Spring & Fall)‣ IRC (irc.freenode.net)‣ #openstack, #openstack-meetings, #openstack-chef, many more‣ Mailing Lists (lists.openstack.org)‣ OpenStack.org‣ Blog, Docs, Wiki‣ Twitter @OpenStack
  • OpenStack Code‣ GitHub‣ https://github.com/openstack/‣ Gerrit‣ https://review.openstack.org‣ Jenkins‣ https://jenkins.openstack.org/‣ DevStack.org
  • OpenStack FoundationLed by Executive Director, Jonathan Bryce, the Foundation is hiring 10-12 employees who, under the strategic direction of the Board, willhelp carry out the OpenStack mission. Specific responsibilities include coordinating the projects infrastructure, such as systems for testingthe software at scale, community building activities, and managing the OpenStack trademark, which was transferred from Rackspacefollowing the first board meeting.
  • OpenStack Distributions‣ Linux Distributions‣ Debian‣ Fedora‣ Red Hat‣ SUSE‣ Ubuntu‣ Commercial Offerings‣ CloudScaling‣ Mirantis‣ Nebula‣ Piston‣ Rackspace‣ ...many more
  • Grizzly Release‣ 550 Developers‣ 2500 conference attendees‣ Red Hat, IBM, Rackspace‣ Incubating:‣ Commons: Oslo‣ Metering: Ceilometer‣ Orchestration: Heat• Ceilometer is a metering project. The project offers metering information and the ability to code more ways to know what has happened on an OpenStack cloud.While it provides metering, it is not a billing project. A full billing solution requires metering, rating, and billing. Metering lets you know what actions have takenplace, rating enables pricing and line items, and billing gathers the line items to create a bill to send to the consumer and collect payment. Ceilometer is availableas a preview.• Heat provides a REST API to orchestrate multiple cloud applications implementing standards such as AWS CloudFormation.
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • Why Chef?Im here to tell you
  • Chef is Infrastructure as Codehttp://www.flickr.com/photos/louisb/4555295187/• Programmaticallyprovision and configure• Treat like any other codebase• Reconstruct business fromcode repository, databackup, and bare metalresources.When dealing with Chef, need to literally “think outside the box”, by shifting your thinking about configuration away from asingle system, to that of an Application Infrastructure. The concept of an Infrastructure is an abstract one with a specifictechnical meaning. When we talk about Infrastructure, we mean..
  • Declarative Interface to Resources• Define policy• Say what, not how• Pull not Pushhttp://www.flickr.com/photos/bixentro/2591838509/Chef gives you declarative interfaces into the Resources on those Nodes.Being declarative means that you say what you want to do, instead of how to do it.For example, you declare that package foobar-1.2.3 should be installed, or that the directory /var/log/foobar should exist.Chef pulls down policy from the chef-server, ensuring that a node down for maintenance will receive its policy update when itcomes back online.
  • Ruby!extra_packages = case node[platform]when "ubuntu","debian"%w{ruby1.8ruby1.8-devrdoc1.8ri1.8libopenssl-ruby}endextra_packages.each do |pkg|package pkg doaction :installendendBecause we use a 3GL for the recipe config files, we can use features of ruby like case statements and iterative loops.Sysadmins don’t need to be afraid of Ruby, they’ve been dealing with sub-standard programming languages likeconfiguration files for years.They’re also not limited to what the language tells them they can do.
  • Recipes and Cookbooks• Recipes are collections ofResources• Cookbooks containrecipes, templates, files,custom resources, etc• Code re-use andmodularity• Hundreds already onCommunity.opscode.comhttp://www.flickr.com/photos/shutterhacks/4474421855/
  • The Chef Community• Apache License,Version 2.0• 1300+ Individual contributors• 200+ Corporate contributors• Dell, DreamHost, HP, Rackspace,VMware, SUSE and many more• 900+ cookbooks• http://community.opscode.comChef is hackable! Permissive Apache2 license, vibrant community of awesome folks.Community is very important to us.Thats why were here.
  • http://www.flickr.com/photos/16339684@N00/2681435235/Chef and OpenStack?Thats great and all,but tell me aboutOpenStack!And a basic fact about Infrastructure -- it EVOLVES.
  • Deploying OpenStack• Chef ties it all together automatically• Scaling changes how we deploy• Interchangeable components• Configurations shared, supported &documented• Licensing makes it available to everyoneWeve learned a lot of things.
  • Overview & Status ofChef for OpenStack
  • Chef for OpenStack:Who• Arista• AT&T• Baremetal Cloud• Calxeda• Dell• DreamHost• HP• HubSpot• IBM• Intel• Internap• Mercado Libre• Mirantis• NTT• Nebula• Nicira• Piston Cloud• Rackspace• SUSE• TryStack.org• Voxel• ...and moreThese companies are currently involved to some extent. Some are paying customers that weve done engagements with.
  • Chef for OpenStack:Why• Community for the automated deploymentand management of OpenStack• Reduce fragmentation and encouragecollaboration• Deploying OpenStack is not "secret sauce"• Project not a product• Apache 2 license
  • Chef for OpenStack:What• Chef Repository for Deploying OpenStack• Documentation for Chef for OpenStack• Cookbooks• Keystone• Glance• Nova• Horizon• Swift• Quantum• Cinder• knife-openstack
  • Chef for OpenStack:Where• #openstack-chef on irc.freenode.net• github.com/opscode/openstack-chef-repo• github.com/opscode-cookbooks/• keystone, glance, nova, horizon,swift,quantum,cinder• github.com/opscode/knife-openstack• github.com/mattray/openstack-chef-docs• groups.google.com/group/opscode-chef-openstack• @chefopenstack
  • • Chef repo for Essex/Grizzly• Operating Systems (Ubuntu 12.04)• Hypervisors (KVM, LXC)• Databases (MySQL)• Nova network FlatDHCP HA & VLAN• Quantum Nicira plugin available• Test Kitchen integrationChef for OpenStack:When (Today)
  • Chef for OpenStack:When (Tomorrow)• Grizzly sprint scheduled in 2weeks• Merging AT&T, DreamHost,HubSpot and Rackspace code• Documentation(docs.opscode.com)
  • • Build packages from source• Continuous integration• Hypervisors (Hyper-V, bare metal)• Databases (PostgreSQL)• Cinder (Ceph)• Quantum (Midokura)• Operating Systems (RHEL, Debian, SUSE)• Documentation (docs.opscode.com)• HA ConfigurationsChef for OpenStack:When (Roadmap)
  • ResourcesChef for OpenStackEcosystemMercado LibreHPRackspaceDellTrystackVoxelOpenStack
  • AT&T• github.com/att-cloud/• primary Folsom merge source• openstack-common• cinder (lvm, netapp, rbd)• lots of support cookbooks
  • Crowbar• Hardware provisioning and applicationmanagement platform• crowbar.github.com• Dell, SUSE, others• likely Swift cookbook source combinedwith Rackspaces
  • DreamHost• github.com/dreamhost• ceph• ceilometer• quantum
  • • Nicira NVP cookbook• Open vSwitch cookbook• Development in progress by Opscode• github.com/gmiranda23/nvp-cookbookNicira
  • Rackspace Private Cloud• www.rackspace.com/cloud/private/• github.com/rcbops/chef-cookbooks• primary Essex merge source• likely Red Hat source
  • • Cookbooks reusable outside ofOpenStack• Test Kitchen• knife-rackspace/hp• Crowbar, pxe_dust & Razor• Arista EOS cookbook• Berkshelf & Librarian• Spiceweasel & SputnikChef for OpenStack "Halo Effect"
  • Why the Cloud?Why OpenStack?The solution to this perceived impediment to resources
  • • Instant infrastructure• Unlimited capacity• Autoscaling• No commitment• Immediate replacementWhy the Cloud?Enforces good architectureNo long term commitmentCloud benefits – instant infrastructureImmediate replacement, no sparing etc.Unlimited storage, snapshots (1 TB volume limit)Provisioning APIs, autoscaling, EU storage, geodistPublic, private, hybrid
  • • Real Open Source• Anyone can play• Choice of features• Features achieving parity/accelerating aheadWhy OpenStack?
  • Know our escape planfor every infrastructureprovider"Drink the cloud Kool-aid, but only drink our Kool-aid"If there are problems that you have with your cloud provider...Not just the cloud
  • Chef for Infrastructure Portability• knife ec2• knife rackspace• knife hp• knife google• knife azure• knife cloudstack• knife openstack• knife vcloud• ... and manyothersFrom EC2 to Rackspace, HP or any other OpenStack provider
  • • Vagrant• VMware• CloudStack• Eucalyptus• OpenStack• bare metal• AWS• Rackspace• HP• Google• Azure• many othersDesktop,Virtualization, Private & Public CloudsChef is hackable! Permissive Apache2 license, vibrant community of awesome folks.More than 360 individual contributors, over 70 corporate contributors.Community is very important to us.Thats why were here.
  • • Vagrant• VMware• CloudStack• Eucalyptus• OpenStack• bare metalDesktop,Virtualization, Private & Public Clouds• AWS• Rackspace• HP• Google• Azure• many othersChef is hackable! Permissive Apache2 license, vibrant community of awesome folks.More than 360 individual contributors, over 70 corporate contributors.Community is very important to us.Thats why were here.
  • Chef for OpenStack TL;DL• Project, not a product• Lots of contributors with realdeployments in a vibrantecosystem• Essex works, Grizzly soon• Features driven by demand• Documentation with examples• Do real work with OpenStackFrom EC2 to Rackspace, HP or any other OpenStack provider
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • knife openstackKnife is our command line tool, literally a swiss army knife of cloud APIsIt talks to the Chef server to manage your infrastructure, but it also talks to APIs like the OpenStack oneSo even if youre not managing your OpenStack layer, you have Chef to manage the components on top of it.
  • knife openstack$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack group list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)This is a supported knife plugin for Chef, so we have ticket tracking and everything for it.It has the basics, server creation, deletion and listing available images and servers
  • knife openstack flavor list$ knife openstack flavor listID Name Virtual CPUs RAM Disk1 m1.tiny 1 512 MB 0 GB2 m1.small 1 2048 MB 10 GB3 m1.medium 2 4096 MB 10 GB4 m1.large 4 8192 MB 10 GB5 m1.xlarge 8 16384 MB 10 GBupdate your knife.rb with### Note: If you are not proxying HTTPS to the OpenStack EC2 API port, the schemeshould be HTTP, and the PORT is 8773.you can get these from "knife node show blahblah -a nova"
  • knife openstack group list$ knife openstack group listName Protocol From To CIDR Descriptiondefault tcp 22 22 0.0.0.0/0 defaultdefault icmp -1 -1 0.0.0.0/0 defaulthaproxy tcp 22002 22002 0.0.0.0/0 22022
  • knife openstack image list$ knife openstack image listID Name4a197431-503d-4b85-b61e-84af21ca8654 cirros-imagef8ebb842-c0c0-4be3-8c4c-f72f48edec50 precise-imageupdate your knife.rb with### Note: If you are not proxying HTTPS to the OpenStack EC2 API port, the schemeshould be HTTP, and the PORT is 8773.you can get these from "knife node show blahblah -a nova"
  • knife openstack server create -hknife openstack server create (options)--bootstrap-version VERSION The version of Chef to install-N, --node-name NAME The Chef node name for your new node-s, --server-url URL Chef Server URL-k, --key KEY API Client Key--[no-]color Use colored output, defaults to enabled-c, --config CONFIG The configuration file to use--defaults Accept default values for all questions--disable-editing Do not open EDITOR, just accept the data as is-d, --distro DISTRO Bootstrap a distro using a template; default is chef-full-e, --editor EDITOR Set the editor to use for interactive commands-E, --environment ENVIRONMENT Set the Chef environment-f, --flavor FLAVOR_ID The flavor ID of server (m1.small, m1.medium, etc)-a, --floating-ip [IP] Request to associate a floating IP address to the new OpenStack node. Assumes IPshave been allocated to the project. Specific IP is optional.-F, --format FORMAT Which format to use for output--[no-]host-key-verify Verify host key, enabled by default-i IDENTITY_FILE, The SSH identity file used for authentication--identity-file-I, --image IMAGE_ID The image ID for the server-u, --user USER API Client Username--openstack-api-endpoint ENDPOINTYour OpenStack API endpoint--insecure Ignore SSL certificate on the Auth URL-K, --openstack-password SECRET Your OpenStack Password-T, --openstack-tenant NAME Your OpenStack Tenant NAME-A, --openstack-username KEY Your OpenStack Username--prerelease Install the pre-release chef gems--print-after Show the data after a destructive operation--private-network Use the private IP for bootstrapping rather than the public IP-r, --run-list RUN_LIST Comma separated list of roles/recipes to apply-G, --groups X,Y,Z The security groups for this server-S, --ssh-key KEY The OpenStack SSH keypair id-P, --ssh-password PASSWORD The ssh password-x, --ssh-user USERNAME The ssh username--template-file TEMPLATE Full path to location of template to use-V, --verbose More verbose output. Use twice for max verbosity-v, --version Show chef version-y, --yes Say yes to all prompts for confirmation-h, --help Show this messageupdate your knife.rb with### Note: If you are not proxying HTTPS to the OpenStack EC2 API port, the schemeshould be HTTP, and the PORT is 8773.you can get these from "knife node show blahblah -a nova"
  • $ knife openstack server listInstance ID Name Public IP Private IP Flavor Image Keypair State08f2d9f7-eeb0-45e7-8562-63aed8f096cc os-45539345723309377 50.56.12.229 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active43c6bbf5-b397-4986-8aec-392d955ce5b1 os-9924426691020416 50.56.12.232 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo activec1b9e3df-e566-4378-8a52-ed998b516608 os-553425714287088 50.56.12.230 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo activef3edc5da-ef99-4acb-a141-d957e09809e3 os-07459550287500682 50.56.12.231 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo activeknife openstack server listHow did we get to the point where we can build a multi-tiered, monitored infrastructure?
  • knife openstack server create -a -f 2 -I 737969f8-6091-4896-ba9c-f3cf63bd25c5 -Srs-demo -i ~/.ssh/rs-demo.pem -x ubuntu -r "role[base]"knife openstack server createHow did we get to the point where we can build a multi-tiered, monitored infrastructure?
  • knife openstack server createInstance Name: os-45539345723309377Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096ccWaiting for server.........Flavor: 2Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5SSH Identity File: /Users/mray/.ssh/rs-demo.pemSSH Keypair: rs-demoPublic IP Address: 10.241.0.12Floating IP Address: 50.56.12.229Waiting for sshd.....doneBootstrapping Chef on 50.56.12.229Instance Name: os-45539345723309377Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096ccFlavor: 2Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5SSH Keypair: rs-demoPublic IP Address: 50.56.12.229Environment: _defaultRun List: role[base]update your knife.rb with### Note: If you are not proxying HTTPS to the OpenStack EC2 API port, the schemeshould be HTTP, and the PORT is 8773.you can get these from "knife node show blahblah -a nova"
  • How did we get to the point where we can build a multi-tiered, monitored infrastructure?
  • How did we get to the point where we can build a multi-tiered, monitored infrastructure?
  • knife-openstack compatibility• Uses the OpenStack API• Diablo, Essex, Folsom, Grizzly• Cloudscaling• Crowbar• DreamHost• Nebula• Piston• Rackspace Private Cloud
  • knife-openstack Roadmap• github.com/opscode/knife-openstack• docs.opscode.com/plugin_knife_openstack.html• tickets.opscode.com/browse/KNIFE/component/• windows support• floating IP address management• quantum network management
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • Who was there• AT&T• Dell• DreamHost• HubSpot• KT• Midokura• Opscode• Rackspace• SUSEThere had been several days of conversations, these companies were all represented in the meeting. Missing: eNovance, HP, IBM,KIO, Mirantis
  • Where• #openstack-chef on irc.freenode.net• groups.google.com/group/opscode-chef-openstack• github.com/mattray/openstack-chef-docs• @chefopenstackThe resources were using
  • Licensing• Apache 2• Opscode CLA/CCLA required• http://wiki.opscode.com/display/chef/How+to+Contribute• http://wiki.opscode.com/display/chef/Approved+ContributorsAttendees were all covered already
  • Where on GitHub• http://github.com/osops• chef-repo/• berkshelf, not git submodules• cookbooks all end in "-cookbook"• ie. "nova-cookbook"• "operations" cookbooks outside scope• ie. logging, monitoring, provisioningMove to community GitHub repo, not Opscodes. Opscode will upstream from this repo.
  • Cookbooks• cinder• glance• horizon• keystone• nova• quantum• swift• ceilometer & heat eventuallyThe core OpenStack services
  • Goal of incorporating into OpenStack• Get on StackForge• will provide CI• which everyone will probably slave• Gerrit for code reviews• well sort out reviewers once we start• Testing with TestKitchen initially• Grenade? Kong? SmokeStack?We want to go into "mainline" OpenStack
  • • support alternative package sources• source-built coming ("VanillaStack")• packaging recipes before configuration• ie. "nova/recipes/nova-compute-packages"PackagesUse distro packages were applicable, but not everyone wants to use them. Build from source will come in eventually.
  • Chef Style Guide• Chef 11 target release• partial search• partial templates• Full-stack Chef-client compatible• Ruby 1.9.x• Upstream community cookbooks• Foodcritic as much as possibleTable stakes
  • Chef Style Guide• openstack-common instead of osops-utils• Attribute injection• attributes may short-circuit search• few, if any, attributes in roles• environment-driven attributes• Chef Solo not actively supported• platform logic in attributes filesAlready using these patterns
  • • May release "2013.1.0"• Chef repo for Grizzly• Operating Systems (Ubuntu 12.04)• Databases (MySQL)• Hypervisors (KVM, LXC)• Nova network FlatDHCP HA & VLANInitial osops releaseOpscode employee Matt Ray and Chris McClimans are getting together after ChefConf to work on cleaning up Grizzly. Sourceswill be AT&T, Dell, HubSpot and Rackspace primarily.
  • • Operating Systems (RHEL, SUSE)• Databases (Postgres)• Hypervisors (Xen, bare metal)• Cinder (Ceph, LVM, NetApp)• Quantum (Bridge, Midokura, Nicira, OVS)• Folsom backport• HA Configurations may be stretch goalbecause of differing implementationsGrizzly RoadmapSUSE: SLES, OpenSUSE, PostgresKT: XenHubSpot: bare metalDreamHost: CephAT&T: LVMRackspace: Bridge, NetApp, OVS, RHELOpscode: NiciraMidokura: Folsom, MidoNetAT&T, SUSE, Rackspace different HA setups
  • knife-openstack v0.7.0$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack group list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)Currently supported features.
  • knife-openstack compatibility• Uses the OpenStack API• Diablo, Essex, Folsom, Grizzly• Cloudscaling• Crowbar• DreamHost• Nebula• Piston• Rackspace Private CloudContinue to test for compatibility, will build out CI testing for Opscode-supported knife plugins.
  • knife-openstack Roadmap• github.com/opscode/knife-openstack• docs.opscode.com/plugin_knife_openstack.html• tickets.opscode.com/browse/KNIFE/component/• Continues to be managed by Opscode• Test against multiple OpenStack deploymentsfor compatibility• next major release v0.8.0 (May)• floating IP address management• network assignment on server createsMore features will undoubtedly show up
  • • Submit talk "Chef for OpenStack Fall 2013Overview & Status"• Review this deck• Report progress• Submit for developer track session as well• See you in Hong Kong!Fall 2013 OpenStack SummitWell see what actually happens in November.
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • Berkshelf• http://berkshelf.com/• tool for managing your Chef Cookbooks andtheir dependencies• Community site, Git, local development• Berksfile managed in version control
  • Spiceweasel• https://github.com/mattray/spiceweasel• manages your Chef repositories and creatingreproducible infrastructure• nodes, cookbooks, roles, data bags &environments with a version controlledmanifest• validates dependencies• allows extraction and creation of infrastructure• lightweight orchestration and clustermanagement• Sputnik Cloud Launcher• fills gap between the documentation anddeployment of your Chef repository &infrastructure
  • Rackspace Private Cloud• http://www.rackspace.com/cloud/private/• https://github.com/rcbops-cookbooks/• Session: Deploying OpenStack with Chefand Operational Tooling
  • Test Kitchen• kitchen-openstack• https://github.com/RoboticCheese/kitchen-openstack• Session: Test Kitchen: Multi-Platform Integration Testing for the Masses
  • pxe_dust• Provisioning solution for hardware• Initially developed by Matt Ray• https://github.com/opscode-cookbooks/pxe_dust• pxe_dust::bootstrap_template• pxe_dust::installers• pxe_dust::server
  • Crowbar• Provisioning solution for hardware• Initially developed by Dell• crowbar.github.com• Barclamps!• 1.0 vs. 2.0
  • Razor• Provisioning solution for hardware• Initially developed by EMC and Puppet• Open sourced as a Puppet Labs project• Install using Puppet, Chef, or manual• Auto-Discovered Real-Time Inventory Data• Dynamic Image Selection• Model-Based Provisioning• APIs and Plug-in Architecture• Metal-to-Cloud Application Lifecycle Management• Session: Harnessing the Power of Bare Metal with Razor and Chef Server
  • OpenStack Baremetal• https://wiki.openstack.org/wiki/Baremetal• driver to allow OpenStack Compute to manage hardware directly (Grizzly)• provisioned via PXE and managed via IPMI• OpenStack Compute manages them via the Dashboard, CLI and API• OpenStack on OpenStack (aka "Triple-O")• authentication, authorization, quotas, a dashboard and an API provided by OpenStack• roadmap has device discovery, network management and additional hardware features
  • Agenda• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs
  • currently Folsom release (v3.0.1)open source Chef 11 server embeddedhttp://www.rackspace.com/cloud/private/https://github.com/rcbopshttps://github.com/rcbops-cookbooks/Rackspace Private Cloud
  • Rackspace Private Cloud
  • github.com/rcbops/chef-cookbooks/tree/v3.0.1roles/environments/cookbooks/Rackspace Private Cloud
  • cookbooks/apache2/apt/aws/build-essential/cinder/collectd/collectd-graphite/collectd-plugins/corosync/database/dsh/erlang/exerstack/glance/graphite/haproxy/hardware/horizon/keepalived/keystone/kong/memcached/modules/monit/monitoring/mysql/mysql-openstack/nova/nova-network/ntp/openssh/openssl/openstack-ha/osops-utils/postgresql/rabbitmq/rabbitmq-openstack/rsyslog/runit/selinux/sosreport/swift/sysctl/xfs/yum/
  • environments/example.json{"name": "example_environment","description": "Chef-server example environment","cookbook_versions": {},"json_class": "Chef::Environment","chef_type": "environment","default_attributes": {},"override_attributes": {"developer_mode": false,"monitoring" : {"metric_provider" : "collectd","procmon_provider" : "monit"},"glance": {"image_upload": true,"images": ["cirros", "precise"]},
  • environments/example.json"nova": {"ratelimit" : {"api" : { "enabled" : true },"volume" : { "enabled" : true }},"libvirt": { "virt_type": "qemu" },"networks": [{"label": "public","ipv4_cidr": "10.10.100.0/24","num_networks": "1","network_size": "255","bridge": "br100","bridge_dev": "eth0.100","dns1": "8.8.8.8","dns2": "8.8.4.4"},{"label": "private","ipv4_cidr": "172.16.101.0/24","num_networks": "1","network_size": "255","bridge": "br101","bridge_dev": "eth0.101","dns1": "8.8.8.8","dns2": "8.8.4.4"}]},
  • environments/example.json"mysql": {"allow_remote_root": true,"root_network_acl": "%"},"osops_networks": {"nova": "192.168.1.0/24","public": "192.168.1.0/24","management": "192.168.1.0/24"},"package_component": "folsom"}}
  • roles/allinone.rbbase.rbcinder-all.rbcinder-api.rbcinder-scheduler.rbcinder-setup.rbcinder-volume.rbcollectd-client.rbcollectd-server.rbglance-api.rbglance-registry.rbglance-setup.rbglance.rbgraphite.rbha-controller1.rbha-controller2.rbhaproxy.rbhorizon-server.rbjenkins-allinone.rbjenkins-compute.rbjenkins-controller.rbjenkins-glance.rbjenkins-mysql-master.rbjenkins-nova-api.rbkeystone-api.rbkeystone.rbmysql-master.rbnova-api-ec2.rbnova-api-os-compute.rbnova-api.rbnova-cert.rbnova-controller.rbnova-misc-services.rbnova-network-compute.rbnova-network-controller.rbnova-network.rbnova-scheduler.rbnova-setup.rbnova-vncproxy.rbnova-volume.rbopenstack-ha.rbquantum-network-manager.rbrabbitmq-server.rbrsyslog-client.rbrsyslog-server.rbsingle-compute.rbsingle-controller.rbswift-account-server.rbswift-all-in-one.rbswift-container-server.rbswift-management-server.rbswift-object-server.rbswift-proxy-server.rbswift-setup.rb
  • roles/allinone.rbname "allinone"description "This will create an all-in-one Openstackcluster"run_list("role[single-controller]","role[single-compute]")
  • expanded run listosops-utils::packages, openssh, ntp, sosreport, rsyslog::default,hardware, osops-utils::default, mysql-openstack::server,erlang::default, rabbitmq-openstack::server, keystone::server,keystone::keystone-api, glance::setup, glance::registry,glance::api, nova::nova-setup, nova-network::nova-controller,nova::scheduler, nova::api-ec2, nova::api-os-compute,nova::volume, nova::nova-cert, nova::vncproxy, mysql::client,mysql::ruby, horizon::server, nova-network::nova-compute,nova::compute
  • roles/single-controllername "single-controller"description "Nova Controller (non-HA)"run_list("role[base]","role[mysql-master]","role[rabbitmq-server]","role[keystone]","role[glance-setup]","role[glance-registry]","role[glance-api]","role[nova-setup]","role[nova-network-controller]","role[nova-scheduler]","role[nova-api-ec2]","role[nova-api-os-compute]","role[nova-volume]","role[nova-cert]","role[nova-vncproxy]","role[horizon-server]")
  • roles/basename "base"description "Base role for a server"run_list("recipe[osops-utils::packages]","recipe[openssh]","recipe[ntp]","recipe[sosreport]","recipe[rsyslog::default]","recipe[hardware]","recipe[osops-utils::default]")default_attributes("ntp" => {"servers" => ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org"]})
  • roles/mysql-mastername "mysql-master"description "Installs mysql and sets up replication (if 2nodes with role)"run_list("role[base]","recipe[mysql-openstack::server]")
  • roles/rabbitmq-servername "rabbitmq-server"description "RabbitMQ Server (non-ha)"run_list("role[base]","recipe[erlang::default]","recipe[rabbitmq-openstack::server]")
  • roles/keystonename "keystone"description "Keystone server"run_list("role[base]","recipe[keystone::server]","role[keystone-api]")
  • roles/glance-setupname "glance-setup"description "sets up glance registry db and passwords"run_list("role[base]","recipe[glance::setup]")
  • roles/glance-registryname "glance-registry"description "Glance Registry server"run_list("role[base]","recipe[glance::registry]")
  • roles/glance-apiname "glance-api"description "Glance API server"run_list("role[base]","recipe[glance::api]")
  • roles/nova-setupname "nova-setup"description "Where the setup operations for nova get run"run_list("recipe[nova::nova-setup]")
  • /nova-network-controllername "nova-network-controller"description "Setup nova-networking for controller node"run_list("recipe[nova-network::nova-controller]")
  • roles/nova-schedulername "nova-scheduler"description "Nova scheduler"run_list("role[base]","recipe[nova::scheduler]")
  • roles/nova-api-ec2name "nova-api-ec2"description "Nova API EC2"run_list("role[base]","recipe[nova::api-ec2]")
  • /nova-api-os-computename "nova-api-os-compute"description "Nova API for Compute"run_list("role[base]","recipe[nova::api-os-compute]")
  • roles/nova-volumename "nova-volume"description "Nova Volume Service"run_list("role[base]","recipe[nova::volume]")
  • roles/nova-certname "nova-cert"description "Nova Certificate Service"run_list("role[base]","recipe[nova::nova-cert]")
  • roles/nova-vncproxyname "nova-vncproxy"description "Nova VNC Proxy"run_list("role[base]","recipe[nova::vncproxy]")
  • roles/horizon-servername "horizon-server"description "Horizon server"run_list("role[base]","recipe[mysql::client]","recipe[mysql::ruby]","recipe[horizon::server]")
  • roles/single-computename "single-compute"description "Nova compute (with non-HA Controller)"run_list("role[base]","role[nova-network-compute]","recipe[nova::compute]")
  • Thanks!Matt Raymatt@opscode.com@mattray