Chef for OpenStack
Deployment Workshop
May 14, 2014
!
Justin Shepherd
Matt Ray
Agenda
• Background
• Instructions: http://bit.ly/ATLChef
• Deep-dive walkthrough
Introductions
• Justin Shepherd
• Rackspace
• Principal Architect
• justin.shepherd@rackspace.com
• GitHub: galstrom21
• I...
Overview & Current Status
Chef for OpenStack: Project
• Developer & Operator Community around the automated
deployment and management of OpenStack
•...
Community
• #openstack-chef on irc.freenode.net
• groups.google.com/group/opscode-chef-openstack
• @chefopenstack
• Weekly...
Who's Involved?
• AT&T
• Blue Box
• Dell
• DreamHost
• Gap
• HP
• HubSpot
• IBM
• Korea Telecom
• Opscode
• Rackspace
• SU...
Chef Requirements
• Chef 11
• Ruby 1.9.x
• Foodcritic, ChefSpec, Rubocop for
testing
• attribute-driven by Environments
• ...
StackForge: Cookbooks
• "Official" OpenStack StackForge repositories
•github.com/stackforge/cookbook-openstack-*
•gated by...
StackForge: Deployment
• Chef repository for deploying Grizzly, Havana or Icehouse
•example Environments and Roles
•exampl...
Reference Implementation
• Deployment examples in documentation
•All-in-One Compute
•Single Controller + N Compute
•more c...
docs.opscode.com/openstack.html
Documentation
• docs.opscode.com/openstack.html
•Architecture
•Deployment Prerequisites
•Installation
•Development
•Cookbo...
Example Deployments
• Vagrant "All-in-One" for development/testing
•nova-network or Neutron
•Ubuntu 12.04 or CentOS 6.5
• ...
StackForge: Grizzly Status
• branch ‘stable/grizzly’
• Operating Systems: Ubuntu 12.04, SLES 11 SP2
• Databases: MySQL, SQ...
StackForge: Havana Status
• branch ‘stable/havana’
• Operating Systems: RHEL 6.x, Ubuntu 12.04, SLES 11
SP2
• Databases: D...
StackForge: Icehouse Roadmap
• ‘master' branch currently on 'Icehouse'
•Ceph
•Trove
•Sahara
•Heat enhancements
•ml2 linuxb...
StackForge: Potential Roadmap
• Operating Systems: Debian
• Compute: Bare metal, Docker, Xen
• Messaging: ZeroMQ
• Network...
StackForge: Infra Roadmap
• ChefDK
• Test Kitchen and ServerSpec
• Spiceweasel/Chef Metal reference
deployments
Chef Community Summit
• Developer/Community unconference
• October 2 & 3 in Seattle, WA
knife openstack
Available openstack subcommands: (for details, knife
SUB-COMMAND --help)!
!
** OPENSTACK COMMANDS **!
knife openstack flav...
Name ID Virtual CPUs RAM Disk!
m1.large 4 4 8192 MB 80 GB!
m1.medium 3 2 4096 MB 40 GB!
m1.small 2 1 2048 MB 20 GB!
m1.tin...
Name ID Snapshot!
centos-6.5 68555833-8497-4d14-88ca-c9062e25f14b no!
cirros-test ecc21974-c0f7-4da4-a433-ab826890f4a4 no!...
Name ID Tenant Shared!
external 06dc9d5a-f55a-410d-a7fd-4c7cb34ad927 5da25cc3853f4c54850898f9614c20bb true!
internal ba0fd...
Name Protocol From To CIDR Description!
haproxy tcp 22002 22002 0.0.0.0/0 haproxy!
ssh tcp 22 22 0.0.0.0/0 ssh access!
web...
Name Instance ID Zone Public IP Private IP Flavor Image Keypair State!
OC-4424-chef-client f3302b74-1542-4af8-bc64-bd172ad...
knife openstack server create (options)!
-Z ZONE_NAME, The availability zone for this server!
--availability-zone!
--boots...
knife openstack server create
$ knife openstack server delete -P -y os-9723024061589451!
Instance Name: os-9723024061589451!
Instance ID: b6cb66fd-e42c-...
knife openstack Compatibility
• Uses the OpenStack API
• Diablo, Essex, Folsom, Grizzly, Havana, Icehouse, trunk
• Blue Bo...
knife openstack Resources
• knife openstack --help
• docs.opscode.com/
plugin_knife_openstack.html
• github.com/opscode/kn...
knife openstack 0.10.0
• Specify metadata during server
create
• Select network IDs to attach and
bootstrap
• Support avai...
knife openstack Roadmap
• more network and UUID cleanups
• knife-hp/knife-rackspace consolidation
• knife-cloud common bas...
Test Kitchen
Test Kitchen
• Integration tool for developing and testing
infrastructure code and software on isolated target
platforms
•...
driver:!
name: openstack!
openstack_username: [YOUR OPENSTACK USERNAME]!
openstack_api_key: [YOUR OPENSTACK API KEY]!
open...
Test Kitchen: OpenStack
• Need blueprints for development
• need a busser for Tempest
• Possibly use RefStack for testing ...
Chef Metal
• Chef recipes for deploying infrastructure
• Libraries for repeatably creating machines and
deployments with C...
Chef Metal: Providers
• Cloud
• Digital Ocean, EC2, Fog, OpenStack
• Virtualization
• Vagrant (VirtualBox, Fusion), VSpher...
machine 'mario' do!
recipe 'postgresql'!
recipe 'mydb'!
tag 'mydb_master'!
end!
!
num_webservers = 1!
!
1.upto(num_webserv...
Vagrant All-in-One Walkthrough
Setup
• Instructions: http://bit.ly/ATLChef
• ChefDK, Vagrant, Virtualbox installed
"The Plan"
• Setup
• Tools
• Vagrantfile
• Environment
• Roles
• Cookbooks
• Dashboard
• knife
Tools used
• Bento
•JEOS images
•github.com/opscode/bento
• Packer
•image builder
•packer.io
• Chef Zero
• Berkshelf
Vagrantfile
• Vagrant plugins
•vagrant-chef-zero
•vagrant-omnibus
• chef-client provider
•environment = Vagrant-aio-nova
•...
Environment
• vagrant setup for all-in-one nova-
network developer_mode = true
• services each have attributes
• network s...
Roles
• allinone-compute
• os-compute-single-controller
• os-compute-worker
os-compute-single-controller
• os-base
• os-ops-database
• openstack-ops-database::openstack-db
• os-ops-messaging
• os-id...
os-compute-single-controller 2
• os-compute-setup
• os-compute-conductor
• os-compute-scheduler
• os-compute-api
• os-bloc...
os-compute-worker
• os-base
• openstack-compute::compute
Dashboard
• https://localhost:8443
• admin/admin
chef_server_url 'http://10.10.6.135:4002'!
node_name 'mray'!
client_key '.chef/mray.pem'!
knife[:openstack_username] = "ad...
knife-openstack
• Chef Zero creds
•knife node list -c zero.rb
• OpenStack creds
•knife openstack -c zero.rb
Thanks!
Justin Shepherd
justin.shepherd@rackspace.com
!
Matt Ray
matt@opscode.com
Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop
Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop
Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop
Upcoming SlideShare
Loading in...5
×

Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop

1,341

Published on

The session at the Atlanta 2014 OpenStack Summit is for those already familiar with Chef and interested in deploying and managing OpenStack. We cover the state of the deploying OpenStack with Chef and deploying infrastructure on top of OpenStack with Chef. The second half of the talk is a deep-dive walkthrough of the Vagrant deployment, the instructions are here: http://bit.ly/ATLChef

http://openstacksummitmay2014atlanta.sched.org/event/39587e0e47a20323c6389e136c954ecf

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,341
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
39
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop

  1. 1. Chef for OpenStack Deployment Workshop May 14, 2014 ! Justin Shepherd Matt Ray
  2. 2. Agenda • Background • Instructions: http://bit.ly/ATLChef • Deep-dive walkthrough
  3. 3. Introductions • Justin Shepherd • Rackspace • Principal Architect • justin.shepherd@rackspace.com • GitHub: galstrom21 • IRC: galstrom • Matt Ray • Chef • Director of Partner Integration • matt@getchef.com • GitHub: mattray • IRC: mattray • Twitter: mattray
  4. 4. Overview & Current Status
  5. 5. Chef for OpenStack: Project • Developer & Operator Community around the automated deployment and management of OpenStack • Reduce fragmentation and increase collaboration • Deploying OpenStack is not "Secret Sauce" • Community Project, not a 'Product' • Apache 2 License
  6. 6. Community • #openstack-chef on irc.freenode.net • groups.google.com/group/opscode-chef-openstack • @chefopenstack • Weekly Status Hangouts (Monday 11am EST) • Stackalytics (stackforge->chef-group)
  7. 7. Who's Involved? • AT&T • Blue Box • Dell • DreamHost • Gap • HP • HubSpot • IBM • Korea Telecom • Opscode • Rackspace • SUSE • and many more
  8. 8. Chef Requirements • Chef 11 • Ruby 1.9.x • Foodcritic, ChefSpec, Rubocop for testing • attribute-driven by Environments • platform logic in attributes • currently packages-only installation
  9. 9. StackForge: Cookbooks • "Official" OpenStack StackForge repositories •github.com/stackforge/cookbook-openstack-* •gated by review.openstack.org • OpenStack services for Grizzly, Havana and Icehouse cookbooks •block-storage, common, compute, dashboard, identity, image, telemetry, network, object-storage, orchestration, test-integration • Operational support cookbooks •ceph, ops-database, ops-messaging
  10. 10. StackForge: Deployment • Chef repository for deploying Grizzly, Havana or Icehouse •example Environments and Roles •example "All-in-One" Vagrant deployments •github.com/stackforge/openstack-chef-repo • Gated by review.openstack.org •More single and multi-node testing coming
  11. 11. Reference Implementation • Deployment examples in documentation •All-in-One Compute •Single Controller + N Compute •more coming • Will provide example HA configurations • Operations outside of scope of core repository •logging, monitoring, provisioning
  12. 12. docs.opscode.com/openstack.html
  13. 13. Documentation • docs.opscode.com/openstack.html •Architecture •Deployment Prerequisites •Installation •Development •Cookbooks and Repositories •Example Deployments • github.com/opscode/chef-docs •Creative Commons, no CLA required
  14. 14. Example Deployments • Vagrant "All-in-One" for development/testing •nova-network or Neutron •Ubuntu 12.04 or CentOS 6.5 • Developer lab deployment "1+N" •Single controller, N compute boxes •5 boxes, consumer-grade hardware
  15. 15. StackForge: Grizzly Status • branch ‘stable/grizzly’ • Operating Systems: Ubuntu 12.04, SLES 11 SP2 • Databases: MySQL, SQLite (testing) • Messaging: RabbitMQ • Compute: KVM, LXC, Qemu • Network: Nova + Quantum (Open vSwitch) • Block Storage: LVM • Object Storage: Swift • Dashboard: Apache or Nginx
  16. 16. StackForge: Havana Status • branch ‘stable/havana’ • Operating Systems: RHEL 6.x, Ubuntu 12.04, SLES 11 SP2 • Databases: DB2, MySQL, Postgres, SQLite (testing) • Messaging: RabbitMQ, Qpid • Compute: ESX, Hyper-V, KVM, LXC, Qemu • Network: Nova + Neutron (Open vSwitch, Linux bridge) • Block Storage: Ceph, EMC, IBM, LVM, NetApp • Object Storage: Swift • Dashboard: Apache or Nginx
  17. 17. StackForge: Icehouse Roadmap • ‘master' branch currently on 'Icehouse' •Ceph •Trove •Sahara •Heat enhancements •ml2 linuxbridge L2/openvswitch L3 networking • Juno branch (J3, August)
  18. 18. StackForge: Potential Roadmap • Operating Systems: Debian • Compute: Bare metal, Docker, Xen • Messaging: ZeroMQ • Network: NSX, OpenDaylight • Block Storage: NetApp • Object Storage: Ceph • Source builds via Omnibus
  19. 19. StackForge: Infra Roadmap • ChefDK • Test Kitchen and ServerSpec • Spiceweasel/Chef Metal reference deployments
  20. 20. Chef Community Summit • Developer/Community unconference • October 2 & 3 in Seattle, WA
  21. 21. knife openstack
  22. 22. Available openstack subcommands: (for details, knife SUB-COMMAND --help)! ! ** OPENSTACK COMMANDS **! knife openstack flavor list (options)! knife openstack group list (options)! knife openstack image list (options)! knife openstack network list (options)! knife openstack server create (options)! knife openstack server delete SERVER [SERVER] (options)! knife openstack server list (options) $ knife openstack
  23. 23. Name ID Virtual CPUs RAM Disk! m1.large 4 4 8192 MB 80 GB! m1.medium 3 2 4096 MB 40 GB! m1.small 2 1 2048 MB 20 GB! m1.tiny 1 1 512 MB 10 GB! m1.xlarge 5 8 16384 MB 160 GB $ knife openstack flavor list
  24. 24. Name ID Snapshot! centos-6.5 68555833-8497-4d14-88ca-c9062e25f14b no! cirros-test ecc21974-c0f7-4da4-a433-ab826890f4a4 no! coreos 83d37ea5-d9ae-44cd-9110-d4d39ad997ce no! fedora-19 9add7e14-25e3-41d8-963a-ca744d081f2e no! fedora-20 acb6eba5-226a-4ed5-8db6-33a6fd8cf20d no! freebsd-10.0 0e270df7-1a02-4e91-9fc3-6f5311c58193 no! ubuntu-12.04 ce268db5-ceda-4a90-93c8-3b987ac3705f no! ubuntu-13.04 28d61273-3b8b-4943-8a6f-66630d7d4ef0 no! ubuntu-14.04 4a4f85bf-f164-4e54-83d8-8b2e7d0712b2 no! Windows Server 2012 R2 Std Eval 64e7cba7-7a50-443f-8fa6-a065406e0b04 no $ knife openstack image list
  25. 25. Name ID Tenant Shared! external 06dc9d5a-f55a-410d-a7fd-4c7cb34ad927 5da25cc3853f4c54850898f9614c20bb true! internal ba0fdd03-72b5-41eb-bb67-fef437fd6cb4 5da25cc3853f4c54850898f9614c20bb true! $ knife openstack network list
  26. 26. Name Protocol From To CIDR Description! haproxy tcp 22002 22002 0.0.0.0/0 haproxy! ssh tcp 22 22 0.0.0.0/0 ssh access! web tcp 443 443 0.0.0.0/0 web stuff! web tcp 80 80 0.0.0.0/0 web stuff! web tcp 8080 8080 0.0.0.0/0 web stuff $ knife openstack group list
  27. 27. Name Instance ID Zone Public IP Private IP Flavor Image Keypair State! OC-4424-chef-client f3302b74-1542-4af8-bc64-bd172ad3de50 172.31.6.79 17 89c4181f-6e6c-470e-baa7-d84162112153 shutoff! bb-test d2a9ceff-bf84-4396-9bf3-87b153ca4446 172.31.6.113 10 89c4181f-6e6c-470e-baa7-d84162112153 shutoff! isa-ubu.opscode.us 1bc5212c-3ad1-409c-9881-87fefac78bce 172.31.6.195 7 4a7263a1-3bf7-4b52-be71-6c28339853b9 farniki_pub active! os-3712471938967755 646347a4-4c3a-4559-a193-b352ed85db8e 172.31.6.249 2 967a39b4-b061-4515-94ad-f96717583277 mray-ops active! os-8162382405504458 2b336930-12ba-460d-b6f2-b29a5e38fb74 172.31.6.253 2 967a39b4-b061-4515-94ad-f96717583277 openstack-key active! os-837952636687383 21a81f5b-f9bc-4b14-9f21-298195fcbcbe 172.31.6.250 2 967a39b4-b061-4515-94ad-f96717583277 mray-ops shutoff! os-883820551180086 15b32e62-5cd9-4a15-87d1-e0f4b7fee2ae 172.31.6.252 2 967a39b4-b061-4515-94ad-f96717583277 mray-ops shutoff! sean-test-the-chef f1c171ec-5175-4a61-94ad-cc722278cdce 172.31.6.213 13 663656ce-2fe4-4164-b842-214f221cff55 seanh-support-gen active $ knife openstack server list
  28. 28. knife openstack server create (options)! -Z ZONE_NAME, The availability zone for this server! --availability-zone! --bootstrap-network NAME Specify network for bootstrapping. Default is 'public'.! --bootstrap-protocol protocol! Protocol to bootstrap Windows servers. options: winrm! --bootstrap-proxy PROXY_URL The proxy server for the node being bootstrapped! --bootstrap-version VERSION The version of Chef to install! --ca-trust-file CA_TRUST_FILE! The Certificate Authority (CA) trust file used for SSL transport! -N, --node-name NAME The Chef node name for your new node! -s, --server-url URL Chef Server URL! --chef-zero-port PORT Port to start chef-zero on! -k, --key KEY API Client Key! --[no-]color Use colored output, defaults to false on Windows, true otherwise! -c, --config CONFIG The configuration file to use! --defaults Accept default values for all questions! $ knife openstack server create
  29. 29. knife openstack server create
  30. 30. $ knife openstack server delete -P -y os-9723024061589451! Instance Name: os-9723024061589451! Instance ID: b6cb66fd-e42c-48dc-8893-89bdc644e06e! Flavor: 2! Image: ce268db5-ceda-4a90-93c8-3b987ac3705f! Network: internal! IP Address: 10.230.7.148! Availability Zone: nova! ! WARNING: Deleted server b6cb66fd-e42c-48dc-8893-89bdc644e06e! WARNING: Deleted node os-9723024061589451! WARNING: Deleted client os-9723024061589451 $ knife openstack server delete
  31. 31. knife openstack Compatibility • Uses the OpenStack API • Diablo, Essex, Folsom, Grizzly, Havana, Icehouse, trunk • Blue Box • Cloudscaling • Crowbar • DreamHost • MetaCloud • Mirantis • Nebula • Piston • Rackspace Private Cloud
  32. 32. knife openstack Resources • knife openstack --help • docs.opscode.com/ plugin_knife_openstack.html • github.com/opscode/knife-openstack • tickets.opscode.com/browse/KNIFE/ component/
  33. 33. knife openstack 0.10.0 • Specify metadata during server create • Select network IDs to attach and bootstrap • Support availability zones • Use of names instead of only UUIDs
  34. 34. knife openstack Roadmap • more network and UUID cleanups • knife-hp/knife-rackspace consolidation • knife-cloud common base class • TravisCI for Chef-supported knife plugins
  35. 35. Test Kitchen
  36. 36. Test Kitchen • Integration tool for developing and testing infrastructure code and software on isolated target platforms • Integration test platform for your cookbooks on all the supported platforms with virtual machines • https://github.com/test-kitchen/kitchen-openstack
  37. 37. driver:! name: openstack! openstack_username: [YOUR OPENSTACK USERNAME]! openstack_api_key: [YOUR OPENSTACK API KEY]! openstack_auth_url: [YOUR OPENSTACK AUTH URL]! require_chef_omnibus: latest! image_ref: [SERVER IMAGE ID]! flavor_ref: [SERVER FLAVOR ID] Test Kitchen: kitchen.yml
  38. 38. Test Kitchen: OpenStack • Need blueprints for development • need a busser for Tempest • Possibly use RefStack for testing as well
  39. 39. Chef Metal • Chef recipes for deploying infrastructure • Libraries for repeatably creating machines and deployments with Chef primitives • Bootstrappers for many infrastructure types
  40. 40. Chef Metal: Providers • Cloud • Digital Ocean, EC2, Fog, OpenStack • Virtualization • Vagrant (VirtualBox, Fusion), VSphere • Containers • Docker & LXC • SSH • PXE in progress
  41. 41. machine 'mario' do! recipe 'postgresql'! recipe 'mydb'! tag 'mydb_master'! end! ! num_webservers = 1! ! 1.upto(num_webservers) do |i|! machine "luigi#{i}" do! recipe 'apache'! recipe 'mywebapp'! end! end Chef Metal: Example Recipe
  42. 42. Vagrant All-in-One Walkthrough
  43. 43. Setup • Instructions: http://bit.ly/ATLChef • ChefDK, Vagrant, Virtualbox installed
  44. 44. "The Plan" • Setup • Tools • Vagrantfile • Environment • Roles • Cookbooks • Dashboard • knife
  45. 45. Tools used • Bento •JEOS images •github.com/opscode/bento • Packer •image builder •packer.io • Chef Zero • Berkshelf
  46. 46. Vagrantfile • Vagrant plugins •vagrant-chef-zero •vagrant-omnibus • chef-client provider •environment = Vagrant-aio-nova •run_list = [“role[allinone-compute]”, “role[GLANCE]” ]
  47. 47. Environment • vagrant setup for all-in-one nova- network developer_mode = true • services each have attributes • network setup
  48. 48. Roles • allinone-compute • os-compute-single-controller • os-compute-worker
  49. 49. os-compute-single-controller • os-base • os-ops-database • openstack-ops-database::openstack-db • os-ops-messaging • os-identity • os-image • os-network • ...
  50. 50. os-compute-single-controller 2 • os-compute-setup • os-compute-conductor • os-compute-scheduler • os-compute-api • os-block-storage • os-compute-cert • os-compute-vncproxy • os-dashboard
  51. 51. os-compute-worker • os-base • openstack-compute::compute
  52. 52. Dashboard • https://localhost:8443 • admin/admin
  53. 53. chef_server_url 'http://10.10.6.135:4002'! node_name 'mray'! client_key '.chef/mray.pem'! knife[:openstack_username] = "admin"! knife[:openstack_password] = "admin"! knife[:openstack_tenant] = "admin"! knife with Vagrant
  54. 54. knife-openstack • Chef Zero creds •knife node list -c zero.rb • OpenStack creds •knife openstack -c zero.rb
  55. 55. Thanks! Justin Shepherd justin.shepherd@rackspace.com ! Matt Ray matt@opscode.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×