Puppet, Hyper-V
And OpenStack
PETER POULIOT, CISSP
MICROSOFT N.E.R.D., CAMBRIDGE, MA
Bio
u  Peter Pouliot, CISSP
u  Email: ppouliot@microsoft.com
u  Education:
u  BA Philosophy - Rhode Island College
u  Certificate Windows NT Administration - Boston University CEC
u  Oracle DBA 8i – Oracle University
u  MS Telecommunication Security - Boston University MET College
u  Graduate Certificate Digital Forensics - Boston University MET College
u  Certifications:
u  CISSP, MCSE+I, MCT (NT4/2000), OCP (8i), Network+
u  9 years working on Datacenter/Network Infrastructure and Linux/Windows interoperability for
Ximian/Novell/SuSE.
u  May 2011 Deployed first Openstack/Hyper-V cloud while working in the Novell/Microsoft Joint
Interoperability Lab
u  March 2012 Started working for Microsoft to return Hyper-V support to Folsom release
u  Currently tasked with building a CI infrastructure to support Hyper-V in OpenStack
Agenda
u  Puppet at MS
u  Brief introduction to Hyper-V and OpenStack
u  Why Puppet?
u  A history of the OpenStack Hyper-V puppet modules
u  Challenges of Using Puppet on Windows.
u  State of Devops for Windows IT Pros
OpenStack/Hyper-V
u  Nova: Hyper-V can be used as a compute node in OpenStack
u  Python application running on Hyper-V
u  Publically Available MSI
u  MSI Includes Neutron/(formerly Quantum) agent for Hyper-V
u  Neutron
u  Currently has VLAN Support
u  OpenVSwitch for Hyper-V coming
u  Cinder: Windows can be used as a block storage node.
u  Cloudinit functionality for windows provided by Cloudbaseinit.
u  Public MSI available.
Puppet at MS
u  Currently using puppet to build the OpenStack CI Infrastructure for
Hyper-V
u  Puppet 3.2+ on Windows and Linux.
u  Running Puppetmaster/Passenger
u  Using Kickstart/Preseed/Unattend.xml/Cloudinit as entry for puppet
deployment and registration
u  Using both centralized and decentralized puppet.
Puppet on Windows?
u  Needed a way to configure Windows Hyper-V nodes consistently
and rapidly.
u  OpenStack deployments on Hyper-V using public binaries is
complicated.
u  Lots of Hyper-V tidbits need to be configured.
u  Lots of additional software requirements.
u  Manual process does not scale.
u  Wanted to help CERN deploy Hyper-V into their OpenStack
infrastructure.
Brief History of Puppet and
Openstack on Hyper-V
u  8 months ago Began automating deployment and configuration of
Hyper-V using puppet 2.7+ on windows server 2012.
u  Created a basic framework for deploying OpenStack nova using
public python binaries, with locally served files on Windows.
u  Included Windows configuration, vswitch creation and nova
configuration.
u  Deployment of additional software. (Git, 7zip, Python, python Modules)
u  After initial configuration moved to building supporting linux
deployment automation.
u  Development pace quickened and skill level increased.
Hyper-V/Puppet History Continued
Began automation of windows PE creation for windows pxe
automation, including integration with Linux PXE infrastructure.
Refactor of original Hyper-V/OpenStack module to automatically
retrieve all components with help from Octavian Ciuhandu from
Cloudbase Solutions.
Inclusion of deployment from MSI using Cloudbase Solutions prebuilt
MSI for Nova Compute on Hyper-V
A move to puppet 3.2+ on Windows 2012 and Hyper-V Server 2012
Major refactor including a Virtual Switch type/provider from Luis
Fernández Álvarez of CERN including dependency on Josh Cooper’s
Powershell Modules
Puppet Projects
Puppet-quartermaster: A pxe provisioning infrastructure for deploying linux (kickstart,
preseed, unattend.xml) based installations of linux and windows puppetized nodes.
https://github.com/ppouliot/puppet-quartermaster
Puppet-petools: Generates a x86_64 Windows PE image using the Automated
Deployment Kit. Includes retrieval and installation of the adk. Slipstreaming of virtio-
drivers for windows and other
https://github.com/ppouliot/puppet-petools
Puppet-openstack_hyper_v: hyper-v compute node configuration and openstack nova
deployment.
https://github.com/ppouliot/puppet-openstack_hyper_v
puppet-openstack_hyper_v
u  Installs a complete Python environment
u  Creates external virtual switches
u  Enables live migration
u  Enables the iScsi initiator service
u  Enables RDP Remote access
u  Configures NTP
u  Joins node to Domain
u  Installs Git, 7zip, and other dependancies
u  Installs OpenStack (Package, Public Binaries, Source)
puppet-quartermaster
u  Creates a basic PXE infrastructure
u  TFTP, HTTP, Samba, NFS
u  Squid-Proxy (package caching)
u  DNSMasq (Proxy DHCP)
u  Puppetmaster
u  Retrieves necessary pxe kernels and files, generates menus, and
necessary automation (kickstart, preseed, unattend.xml) for
deploying operating system
u  Includes as OS automation retrieval and installation of puppet and
registration with puppetmaster.
Puppet-petools
u  Downloads and Installs Microsoft’s Automated Deployment Kit
u  Creates a Windows PE x86_64 image
u  Mounts the Windows PE Image
u  Installs necessary Windows PE Packages
u  Retrieves and Installs KVM VirtIO drivers into WinPE Images
u  Installs additional Windows Drivers into winpe image.
u  Modifies startnet.cmd to map appropriate drives to launch a script
on a windows share.
u  Finalizes WinPE image and Copies all necessary files to allow for
WinPE to PXE onto a windows file share.
Challenges of Puppet on Windows
u  Windows isn’t Linux.
u  Lack of public modules for Windows.
u  No windows package management
u  Spaces in path naming
u  Reboots
u  Previously was no powershell provider for puppet/windows.
u  Move from using native .exe commands to the powershell
equivalent.
Windows IT Professionals and
Devops: A problem
u  Problem:
u  Devops is a new concept for the Windows IT Pro
u  Continuous Integration and Delivery are also new concepts for Windows
IT Pro
u  Skills need to updated
u  Enterprise shops usually have highly segregated skill sets.
u  Typical windows workflows are not automated or automation
How do we fix the problem?
u  Organize the Windows Puppet Community.
u  Share experiences
u  Share code
u  Create a rich catalog of public Puppet Modules for the Windows IT
Pro
u  Evangelism: Windows can be automated too
u  Education:
u  Remember, It’s only computer science!
More information
u  http://wiki.openstack.org/Hyper-V
u  IRC: Freenode: #openstack-hyper-v
u  Meetup: OpenStack Boston
u  Hyper-V MSIs for OpenStack:
u  http://www.cloudbase.it
u  Email: ppouliot@microsoft.com

Using Puppet for Deploying Hyper-V OpenStack Compute Nodes - PuppetConf 2013

  • 1.
    Puppet, Hyper-V And OpenStack PETERPOULIOT, CISSP MICROSOFT N.E.R.D., CAMBRIDGE, MA
  • 2.
    Bio u  Peter Pouliot,CISSP u  Email: ppouliot@microsoft.com u  Education: u  BA Philosophy - Rhode Island College u  Certificate Windows NT Administration - Boston University CEC u  Oracle DBA 8i – Oracle University u  MS Telecommunication Security - Boston University MET College u  Graduate Certificate Digital Forensics - Boston University MET College u  Certifications: u  CISSP, MCSE+I, MCT (NT4/2000), OCP (8i), Network+ u  9 years working on Datacenter/Network Infrastructure and Linux/Windows interoperability for Ximian/Novell/SuSE. u  May 2011 Deployed first Openstack/Hyper-V cloud while working in the Novell/Microsoft Joint Interoperability Lab u  March 2012 Started working for Microsoft to return Hyper-V support to Folsom release u  Currently tasked with building a CI infrastructure to support Hyper-V in OpenStack
  • 3.
    Agenda u  Puppet atMS u  Brief introduction to Hyper-V and OpenStack u  Why Puppet? u  A history of the OpenStack Hyper-V puppet modules u  Challenges of Using Puppet on Windows. u  State of Devops for Windows IT Pros
  • 4.
    OpenStack/Hyper-V u  Nova: Hyper-Vcan be used as a compute node in OpenStack u  Python application running on Hyper-V u  Publically Available MSI u  MSI Includes Neutron/(formerly Quantum) agent for Hyper-V u  Neutron u  Currently has VLAN Support u  OpenVSwitch for Hyper-V coming u  Cinder: Windows can be used as a block storage node. u  Cloudinit functionality for windows provided by Cloudbaseinit. u  Public MSI available.
  • 5.
    Puppet at MS u Currently using puppet to build the OpenStack CI Infrastructure for Hyper-V u  Puppet 3.2+ on Windows and Linux. u  Running Puppetmaster/Passenger u  Using Kickstart/Preseed/Unattend.xml/Cloudinit as entry for puppet deployment and registration u  Using both centralized and decentralized puppet.
  • 6.
    Puppet on Windows? u Needed a way to configure Windows Hyper-V nodes consistently and rapidly. u  OpenStack deployments on Hyper-V using public binaries is complicated. u  Lots of Hyper-V tidbits need to be configured. u  Lots of additional software requirements. u  Manual process does not scale. u  Wanted to help CERN deploy Hyper-V into their OpenStack infrastructure.
  • 7.
    Brief History ofPuppet and Openstack on Hyper-V u  8 months ago Began automating deployment and configuration of Hyper-V using puppet 2.7+ on windows server 2012. u  Created a basic framework for deploying OpenStack nova using public python binaries, with locally served files on Windows. u  Included Windows configuration, vswitch creation and nova configuration. u  Deployment of additional software. (Git, 7zip, Python, python Modules) u  After initial configuration moved to building supporting linux deployment automation. u  Development pace quickened and skill level increased.
  • 8.
    Hyper-V/Puppet History Continued Beganautomation of windows PE creation for windows pxe automation, including integration with Linux PXE infrastructure. Refactor of original Hyper-V/OpenStack module to automatically retrieve all components with help from Octavian Ciuhandu from Cloudbase Solutions. Inclusion of deployment from MSI using Cloudbase Solutions prebuilt MSI for Nova Compute on Hyper-V A move to puppet 3.2+ on Windows 2012 and Hyper-V Server 2012 Major refactor including a Virtual Switch type/provider from Luis Fernández Álvarez of CERN including dependency on Josh Cooper’s Powershell Modules
  • 9.
    Puppet Projects Puppet-quartermaster: Apxe provisioning infrastructure for deploying linux (kickstart, preseed, unattend.xml) based installations of linux and windows puppetized nodes. https://github.com/ppouliot/puppet-quartermaster Puppet-petools: Generates a x86_64 Windows PE image using the Automated Deployment Kit. Includes retrieval and installation of the adk. Slipstreaming of virtio- drivers for windows and other https://github.com/ppouliot/puppet-petools Puppet-openstack_hyper_v: hyper-v compute node configuration and openstack nova deployment. https://github.com/ppouliot/puppet-openstack_hyper_v
  • 10.
    puppet-openstack_hyper_v u  Installs acomplete Python environment u  Creates external virtual switches u  Enables live migration u  Enables the iScsi initiator service u  Enables RDP Remote access u  Configures NTP u  Joins node to Domain u  Installs Git, 7zip, and other dependancies u  Installs OpenStack (Package, Public Binaries, Source)
  • 11.
    puppet-quartermaster u  Creates abasic PXE infrastructure u  TFTP, HTTP, Samba, NFS u  Squid-Proxy (package caching) u  DNSMasq (Proxy DHCP) u  Puppetmaster u  Retrieves necessary pxe kernels and files, generates menus, and necessary automation (kickstart, preseed, unattend.xml) for deploying operating system u  Includes as OS automation retrieval and installation of puppet and registration with puppetmaster.
  • 12.
    Puppet-petools u  Downloads andInstalls Microsoft’s Automated Deployment Kit u  Creates a Windows PE x86_64 image u  Mounts the Windows PE Image u  Installs necessary Windows PE Packages u  Retrieves and Installs KVM VirtIO drivers into WinPE Images u  Installs additional Windows Drivers into winpe image. u  Modifies startnet.cmd to map appropriate drives to launch a script on a windows share. u  Finalizes WinPE image and Copies all necessary files to allow for WinPE to PXE onto a windows file share.
  • 13.
    Challenges of Puppeton Windows u  Windows isn’t Linux. u  Lack of public modules for Windows. u  No windows package management u  Spaces in path naming u  Reboots u  Previously was no powershell provider for puppet/windows. u  Move from using native .exe commands to the powershell equivalent.
  • 14.
    Windows IT Professionalsand Devops: A problem u  Problem: u  Devops is a new concept for the Windows IT Pro u  Continuous Integration and Delivery are also new concepts for Windows IT Pro u  Skills need to updated u  Enterprise shops usually have highly segregated skill sets. u  Typical windows workflows are not automated or automation
  • 15.
    How do wefix the problem? u  Organize the Windows Puppet Community. u  Share experiences u  Share code u  Create a rich catalog of public Puppet Modules for the Windows IT Pro u  Evangelism: Windows can be automated too u  Education: u  Remember, It’s only computer science!
  • 16.
    More information u  http://wiki.openstack.org/Hyper-V u IRC: Freenode: #openstack-hyper-v u  Meetup: OpenStack Boston u  Hyper-V MSIs for OpenStack: u  http://www.cloudbase.it u  Email: ppouliot@microsoft.com