Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013

4,218 views

Published on

"VMware vCHS, Puppet, and Project Zombie" by Nicholas Weaver, Cloud Automation Architect, Hybrid Cloud Service, VMware.

Speaker Bio: Nicholas Weaver is the Cloud Automation Architect for VMware's vCloud Hybrid Service (vCHS) platform and the primary architect behind the vCHS automation framework (Project Zombie). He is also a co-creator of the Puppet Labs Razor project and many VMware-specific free tools. He previously worked in the CTO office for EMC, in the EMC field as a vSpecialist, and as a infrastructure engineer in financial, media, and retail companies. Nick loves software-driven control, hacking prototypes together, speaking at user groups, and demonstrating automation innovation to the masses. Nick can be found on Twitter and Github as @lynxbat.

  • Be the first to comment

VMware vCHS, Puppet, and Project Zombie - PuppetConf 2013

  1. 1. © 2011 VMware Inc. All rights reserved Nicholas Weaver - Cloud Automation Architect VMware vCloud Hybrid Service VMware vCHS, Puppet, and Project Zombie
  2. 2. 2 About me
  3. 3. 3 About Nick  Nicholas Weaver • VMware – Cloud Automation Architect, vCHS • (formerly) EMC – Office of the CTO, Global Alliances, vSpecialist, Demo-builder • (formerly) VMware/EMC Architect: Financial, Retail • Co-creator Project Razor • Twitter: @lynxbat • Blog: nickapedia.com • LinkedIn: www.linkedin.com/in/nicholasweaver
  4. 4. 4 What is vCloud Hybrid Service?
  5. 5. 5 vCloud Hybrid Service: Extending To The Cloud VMware vCloud Hybrid Service Your Data Center Any Application… No Changes Software-Defined Data Center VMware vSphere & vCloud Suite Existing & New Apps Seamless Networking Common management One Support call IaaS cloud owned and operated by VMware based on VMware software
  6. 6. 6 What do I do?
  7. 7. 7 Automation
  8. 8. 8
  9. 9. 9
  10. 10. 10
  11. 11. 11 Effort Evolution
  12. 12. 12 Why is automation important for vCHS?
  13. 13. 13 What is it really doing?
  14. 14. 14
  15. 15. 15
  16. 16. 16 The problem with success is…
  17. 17. 17
  18. 18. 18 Nothing stays the same
  19. 19. 19
  20. 20. 20
  21. 21. 21 We are not the first
  22. 22. 22
  23. 23. 23
  24. 24. 24
  25. 25. 25
  26. 26. 26 We need automation special sauce
  27. 27. 27 ?
  28. 28. 28 project zombie
  29. 29. 29 Key principles
  30. 30. 30 Scale
  31. 31. 31 Extensibility
  32. 32. 32 Simplicity
  33. 33. 33 Resiliency
  34. 34. 34
  35. 35. 35
  36. 36. 36 What is Project Zombie?
  37. 37. 37
  38. 38. 38
  39. 39. 39
  40. 40. 40
  41. 41. 41
  42. 42. 42 Rez
  43. 43. 43 Distributed Resource Management • Resources • State • Configuration • Location • Distributed Locking • Ownership • Event History
  44. 44. 44 Distributed Resource Management Razor Razor Storage Manager Storage Manager
  45. 45. 45 Engine
  46. 46. 46 Zombie Engine DSL - ZED -------------- 1 --------------- Init VPC - Engine - API Call ------------- 3 -------------- Install ESXi Engine Broker Plugin - Razor Inputs: - Razor IP - Blade - Hostname - Network Information Tasks: - Create Policy - Boot Blade ------------- 3 -------------- Prep TLM Engine Broker Plugin - Puppet vCenter Inputs: - TLM VCSA IP - TLM VCSA Creds - TLM DVS - Portgroup Name - Portgroup VLAN - Portgroup Settings - Ports: 128 - Load Balancing: IP Hash - Reset at Disconnect: Enabled Tasks: - Create PGs - d#p#v#-dmz-pg-## - d#p#v#-esx-pg-## - d#p#v#-pvt-pg-## ----------- 4 ------------ Deploy VPC VCSA Engine Broker Plugin - OVFTool Inputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - Network - VM Hardware Settings - Memory: 16384 Tasks: - Deploy VPC VCSA - d#p#v#-mgmt-vc0 - Update VM Hardware ----------- 4 ------------ Deploy VPC VSE Pub Pair Engine Broker Plugin - Puppet vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 1 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name - Type - Portgroup - Subnet Settings - Interface 1 - Name - Type - Portgroup - Subnet Settings Tasks: - Create VPC Pub Edge - d#p#v#-mgmt-vse-pub ----------- 4 ------------ Deploy VPC VSE Priv Pair Engine Broker Plugin - Puppet VMware-vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 2 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name: InterVPC - Type - Portgroup - Subnet Settings - Interface 1 - Name: DMZ - Type - Portgroup - Subnet Settings - Interface 2 - Name: PVT - Type - Portgroup - Subnet Settings - Interface 3 - Name: ESX - Type - Portgroup - Subnet Settings Tasks: - Create VPC Priv Edge - d#p#v#-mgmt-vse-priv ----------- 5 ------------ Configure VPC VSE Pub Pair Engine Broker Plugin - Puppet vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - DNS Settings - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default Route - Syslog Tasks: - Configure VPC VSE Pub Pair ----------- 5 ------------ Configure VPC VSE Priv Pair Engine Broker Plugin - Puppet VMware-vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings: - Syslog - DNS - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default Route Tasks: - Configure VPC VSE Priv Pair ----------- 7 ------------ Init VPC VCSA Engine Broker Plugin - Puppet VMware-VCSA Inputs: - Credentials - NTP Settings - Syslog Settings - DB Settings - Sizing: Large Tasks: - Init VPC VCSA ----------- 4 ------------ Deploy VPC VSM Engine Broker Plugin - OVFTool Inputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - Network Tasks: - Deploy VPC VSM - d#p#v#-mgmt-vsm0 ----------- 9 ------------ Configure VPC VSM Engine Broker Plugin - Puppet VMware-vShield Inputs: - VPC VSM IP - VPC VSM Creds - VPC VCSA IP - VPC VCSA Creds - NTP - Syslog Tasks: - Configure VPC VSM ----------- 4 ------------ Deploy VPC vCloud Engine Broker Plugin - ?? Inputs: - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Names - VM Settings - Network Tasks: - Deploy VPC vCloud Cells - Deploy VPC vCloud NFS - Deploy VPC vCloud DB -------------- 2 ----------------- Reserve VPC Engine Broker Plugin: - Rez Inputs: - VPC # ----------- 8 ------------ Configure VPC vCenter Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP Address - VPC VCSA Credentials - Licenses - vCenter - ESXi - vCloud Net & Sec - Retention Policies - Task: Enabled - Event: Enabled - Datacenter Name - Cluster Settings - Name - DRS Settings - EVC Settings? - DVS Configuration - Name: d#p#v# - Settings - Uplinks: 4 - MTU: 9000 - Enable NIOC - Portgroup settings - Name - VLAN - Settings -Tasks: - Configure VPC VCSA - Configure DVS ----------- 7 ------------ Init VPC VSM Engine Broker Plugin - ?? Inputs: - VPC VSM Network Settings Tasks: - Init VPC VSM ----------- 6 ------------ VPC VCSA Port Check Engine Broker Plugin - Util Inputs: - VPC VCSA IP Tasks: - VPC VCSA Port Check ----------- 6 ------------ VPC VSM Port Check Engine Broker Plugin - Util Inputs: - VPC VSM IP Tasks: - VPC VSM Port Check ----------- 6 ------------ Verify VPC ESXi Engine Broker Plugin - Puppet Util? Inputs: - VPC ESXi IP - VPC ESXii Creds Tasks: - Verify VPC ESXi ----------- 9 ------------ Add ESXi to VPC Datacenter Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC ESXi IP - VPC ESXii Creds - VPC VCSA IP - VPC VCSA Creds - VPC Datacenter Name Tasks: - Add ESXi to VPC Datacenter ----------- 10 ------------ Configure VPC ESXi Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - DNS Settings - NTP Settings - Syslog Settings - VMK Configuration - vMotion - Storage - FT - ??? Tasks: - Configure ESXi - Add to DVS - Add VMK Networking ----------- 11 ------------ Add Storage to VPC ESXi Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - iSCSI Settings - LUN Information Tasks: - Configure iSCSI - Add LUNs ------------- 3 --------------- Configure VNX VPC Storage Group Engine Broker Plugin: - Storage Controller Inputs: - VPC # - LUN IDs - ESXi iSCSI information Tasks: - Create VPC Storage Group - Configure VNX Host registrations - Add LUNs to VPC Storage Group ----------- 6 ------------ VPC vCloud Cell Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud Cell Port Check ----------- 6 ------------ VPC vCloud Cell Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud Cell Port Check ----------- 6 ------------ VPC vCloud NFS Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud NFS Port Check ----------- 6 ------------ VPC vCloud DB Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud DB Port Check ----------- 7 ------------ Configure VPC vCloud NFS Engine Broker Plugin - ?? Inputs: - VPC vCloud NFS IP - VPC vCloud NFS Creds - NFS Export Settings - ??? Tasks: - Config VPC vCloud NFS ----------- 7 ------------ Configure VPC vCloud DB Engine Broker Plugin - ?? Inputs: - VPC vCloud DB IP - VPC vCloud DB Creds - vCloud Database Config - ??? Tasks: - Configure VPC vCloud DB ----------- 8 ------------ Configure VPC vCloud Cell Engine Broker Plugin - ?? Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud NFS Config - VPC vCloud Installation Responses.properties - VPC vCloud Cell Cert - NTP Settings Tasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service ----------- 9 ------------ Configure VPC vCloud Cell Engine Broker Plugin - ?? Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud Installation Responses.properties - VPC vCloud NFS Config - VPC vCloud Cell Cert - NTP Settings Tasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service ----------- 12 ------------ Create VPC VM Storage Profile Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - Storage Profile Name Tasks: - Create VPC VM Storage Profile - Tag VPC Datastores ----------- 13 ------------ Add VPC vCenter to vCloud Engine Broker Plugin - Puppet VMware-vCloud Inputs: - VPC vCloud IP - VPC vCloud Creds - VPC VCSA IP - VPC VCSA Creds - ??? Tasks: - Add VPC vCenter to VPC vCloud ----------- 10 ------------ Configure VXLAN Engine Broker Plugin - Puppet VMware-vShield Inputs: - VPC VSM IP - VPC VSM Creds - Multicast Information - VPC DVS Information - VPC Cluster - ??? Tasks: - Configure VXLAN ----------- 11 ------------ Reconfigure VXLAN VMK Ports Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - VXLAN VMK Network Settings Tasks: - Configure VXLAN VMK Ports ----------- 14 ------------ Configure VPC vCloud Engine Broker Plugin - Puppet VMware-vCloud Inputs: - VPC vCloud IP - VPC vCloud Creds - Provider VDC Name - External Org Network Name - ??? Tasks: - Configure VPC vCloud ----------- 12 ------------ Configure VPC Cluster HA Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - VPC ESXi Hosts - VPC Cluster - HA Settings - Failure: percentage Tasks: - Add VPC ESXi hosts to Cluster - Configure VPC Cluster HA Settings ----------- 5 ------------ Prep VPC vCloud Cell Engine Broker Plugin - Puppet vCenter - ?? Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static Routes Tasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking ----------- 5 ------------ Prep VPC VCSA Engine Broker Plugin - ?? Inputs: - Credentials - Network Settings Tasks: - Configure Networking ----------- 5 ------------ Prep VPC vCloud Cell Engine Broker Plugin - Puppet vCenter - ?? Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static Routes Tasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking ----------- 5 ------------ Prep VPC VSM Engine Broker Plugin - ?? Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC VSM name - Network Settings Tasks: - PowerOn VM - Configure Networking
  47. 47. 47 Zombie Engine DSL - ZED Puppet Apply MCollective MCollective OVFTool Puppet Apply MCollective API Query
  48. 48. 48 Zombie Engine DSL - ZED Reserve Compute Install ESXi Add To Cluster Add Compute Reserve Compute Install ESXi Add To ClusterReplace Compute Un-Reserve Compute Remove From Cluster
  49. 49. 49 Distributed and location-aware execution Work to do Concurrent WorkConcurrent WorkConcurrent Work Concurrent WorkConcurrent WorkConcurrent Work P P P B B B BB Datacenter: Dallas Datacenter: Singapore Work to do
  50. 50. 50 Why we use Puppet • Greater investment in building modules than scripting • Much greater return on investment with evolution of the product • 13 in-house Puppet Modules so far (small example) • vCloud Director • vShield Networking • vSphere • We use a total of 47 modules for everything • We have Puppet modules for installing Zombie in Production, Integration, and Development (including Vagrant + Puppet use for laptops) • Project Zombie itself uses Puppet to do work, and is installed and upgraded using Puppet
  51. 51. 51 Why we use Puppet Puppet 1st Rule
  52. 52. 52 Cool things we do with Puppet • Engine can stage changes using MCollective to an environment • We can update that environment throughout job execution • We can apply the environment against a target at any time • We have the ability to load balance/scale both Engine execution and Puppet execution • Processes that would be complex to order in Puppet can be staged much easier with Zombie Engine + Puppet • ZED allows for iteration using configuration data into execution concurrency and queuing. Very cool layer on top of Puppet.
  53. 53. 53 Does it work?
  54. 54. 54
  55. 55. 55 Cloud Build A Maximum capacity per week: Man hours: 72 2 Time to delivery: 6 days
  56. 56. 56 Maximum capacity per week: Man hours: 1.5 336 x Zombie Engine instance count Time to delivery: 2.5 hours* Cloud Build A
  57. 57. 57 Details (per job): • ~120 individual tasks (plugin calls) • ~2700 discrete configuration points • ~1400 managed resources • Dynamically sized (pick the # of compute and storage) • Controls: vCloud Director, vCenter, ESXi, EMC VNX, Razor, vShield Manager, vShield Edge, Linux and Windows operating systems. Cloud Build A
  58. 58. 58 Cloud Build A
  59. 59. 59 So yeah… it works
  60. 60. 60 What’s next?
  61. 61. 61 Bring Your Own Licenses Bring Your Own VMs Bring Your Own Tools Customer Components VMware vCloud Hybrid Service Hybrid Service Delivery Components Infrastructure Hardware & Facilities Compute Storage Networking & Security OS Catalogs Application Catalogs Infrastructure Management Web Console vCloud API

×