Puppet and OpenStack in the Real World

               Kavit Munshi
         Chief Technical Officer

           What is Aptira?
           What is OpenStack?
           The Problem
           Aptira + Puppet +
            OpenStack
           Why Puppet?
What is Aptira?

    Aptira is a technology services
    provider that delivers:
    − Managed services
    − Bespoke solutions
    − Rich media solutions for digital
      agencies
    − Cloud computing infrastructure and
      consulting
What is OpenStack?

    OpenStack is free, open source Infrastructure
    as a Service (IaaS) project.
    −   Cloud Computing (nova)
    −   Object Store (swift)
    −   Block Storage (cinder)
    −   Network Virtualisation (quantum)
    −   Authentication (keystone)
    −   Image Catalog (glance)
What is OpenStack?

    Managed by the OpenStack Foundation
    −   5,600 individual members
    −   850 organisations
    −   87 countries
    −   $10 million in funding
    −   Independent home for OpenStack
    −   Technical Committee, Board of Directors, User
        Committee
What is OpenStack?

    One of the fastest growing open source
    projects in the world
    −   From 30,000 to 600,000 LOC in 2 years
    −   <100 people attended first OpenStack summit
    −   >1500 attended last two summits
    −   Rate of feature innovation is staggering
The Problem

    We deploy and maintain OpenStack on premise for our IaaS
    customers.

    We sell our ability to deploy and maintain OpenStack on
    location for private cloud customers.

    But there's a catch. Here's what we show our customers:
The Problem
This is what we show
our non-tech team




…and here's what I need
The Problem
The Problem

    Add some more ancillary services (MySQL, RabbitMQ, Nagios,
    iptables).

    Then triple it! (development, testing, production)

    Multiply by the number of private cloud customers.

    Need a solution that lets me express configurations in OO style:
    −   Reusable
    −   Templateable
    −   Inheritance

    Solution must scale to manage thousands of nodes
Puppet

    Puppet to the rescue!

    Puppet modules reusable, which makes each completed module a
    capability investment for the business.

    Native support for multiple environments and multiple sites.

    Couple with git (gitolite) for revision and environment management
    (http://puppetlabs.com/blog/git-workflow-and-puppet-environments/)

    Passenger plugin allows us to scale config management along with
    infrastructure to ensure performance.

    ~30 puppet modules written (we write our own)

    ~1,800 lines of puppet manifests

    Average ~50 lines of puppet per module
Puppet
Puppet
- Master-Agent architecture
- On the puppetmaster, each puppet module is created as
  follows
      - /etc/puppet/modules/<module_name>
      - Global variables are defined in site.pp
- The module has following components (each is a folder)
      - files (contains files needed for the module)
      - manifests (contains the init.pp file need to define
        the module)
      - templates (contains templates for the config files)
- The agent connects and checks what modules are
  applicable to it and syncs config accordingly
Puppet
Puppet
Aptira + Puppet + OpenStack

    Aptira infrastructure capability
    −   ~15,000 cores
    −   15TB RAM
    −   150TB HDD
    −   ~1000 nodes

    Our puppet infrastructure allows us to utilise and deploy this
    capability on demand from customers.

    CapEx becomes much more dynamic and efficient. We can
    service demands for huge amounts of power without owning
    huge amounts of infrastructure.

    We scale our IaaS as a Service!
Aptira + Puppet + OpenStack

    Facter fact developed to determine hardware type (or Virtual
    Machine hypervisor)
    −   Module to install and execute appropriate hardware
        management toolset (Dell OpenManage, HP ACU CLI,
        VMware Tools, etc)

    Augeas used to manage network interfaces (Debian style)

    We can also utilise public IaaS offerings, controlled by puppet.
    −   Bootstrap geographic presence without investing in
        physical infrasctructure.
Why Puppet?

    Aptira evaluates and chooses what we feel is
    best of breed.
    −   Self documenting
    −   Explicit dependency management (versus Chef)
    −   Enterprise version if required
    −   Large community base
    −   Tacit knowledge: staff have previous experience
        with puppet, many companies are using it.
Q&A

    http://www.aptira.com

    http://www.openstack.org

    http://www.puppetlabs.com

    Twitter:
    −   @Aptira
    −   @KavitAptira

OpenStack and Puppet

  • 1.
    Puppet and OpenStackin the Real World Kavit Munshi Chief Technical Officer  What is Aptira?  What is OpenStack?  The Problem  Aptira + Puppet + OpenStack  Why Puppet?
  • 2.
    What is Aptira?  Aptira is a technology services provider that delivers: − Managed services − Bespoke solutions − Rich media solutions for digital agencies − Cloud computing infrastructure and consulting
  • 3.
    What is OpenStack?  OpenStack is free, open source Infrastructure as a Service (IaaS) project. − Cloud Computing (nova) − Object Store (swift) − Block Storage (cinder) − Network Virtualisation (quantum) − Authentication (keystone) − Image Catalog (glance)
  • 4.
    What is OpenStack?  Managed by the OpenStack Foundation − 5,600 individual members − 850 organisations − 87 countries − $10 million in funding − Independent home for OpenStack − Technical Committee, Board of Directors, User Committee
  • 5.
    What is OpenStack?  One of the fastest growing open source projects in the world − From 30,000 to 600,000 LOC in 2 years − <100 people attended first OpenStack summit − >1500 attended last two summits − Rate of feature innovation is staggering
  • 6.
    The Problem  We deploy and maintain OpenStack on premise for our IaaS customers.  We sell our ability to deploy and maintain OpenStack on location for private cloud customers.  But there's a catch. Here's what we show our customers:
  • 7.
    The Problem This iswhat we show our non-tech team …and here's what I need
  • 8.
  • 9.
    The Problem  Add some more ancillary services (MySQL, RabbitMQ, Nagios, iptables).  Then triple it! (development, testing, production)  Multiply by the number of private cloud customers.  Need a solution that lets me express configurations in OO style: − Reusable − Templateable − Inheritance  Solution must scale to manage thousands of nodes
  • 10.
    Puppet  Puppet to the rescue!  Puppet modules reusable, which makes each completed module a capability investment for the business.  Native support for multiple environments and multiple sites.  Couple with git (gitolite) for revision and environment management (http://puppetlabs.com/blog/git-workflow-and-puppet-environments/)  Passenger plugin allows us to scale config management along with infrastructure to ensure performance.  ~30 puppet modules written (we write our own)  ~1,800 lines of puppet manifests  Average ~50 lines of puppet per module
  • 11.
  • 12.
    Puppet - Master-Agent architecture -On the puppetmaster, each puppet module is created as follows - /etc/puppet/modules/<module_name> - Global variables are defined in site.pp - The module has following components (each is a folder) - files (contains files needed for the module) - manifests (contains the init.pp file need to define the module) - templates (contains templates for the config files) - The agent connects and checks what modules are applicable to it and syncs config accordingly
  • 13.
  • 14.
  • 15.
    Aptira + Puppet+ OpenStack  Aptira infrastructure capability − ~15,000 cores − 15TB RAM − 150TB HDD − ~1000 nodes  Our puppet infrastructure allows us to utilise and deploy this capability on demand from customers.  CapEx becomes much more dynamic and efficient. We can service demands for huge amounts of power without owning huge amounts of infrastructure.  We scale our IaaS as a Service!
  • 16.
    Aptira + Puppet+ OpenStack  Facter fact developed to determine hardware type (or Virtual Machine hypervisor) − Module to install and execute appropriate hardware management toolset (Dell OpenManage, HP ACU CLI, VMware Tools, etc)  Augeas used to manage network interfaces (Debian style)  We can also utilise public IaaS offerings, controlled by puppet. − Bootstrap geographic presence without investing in physical infrasctructure.
  • 17.
    Why Puppet?  Aptira evaluates and chooses what we feel is best of breed. − Self documenting − Explicit dependency management (versus Chef) − Enterprise version if required − Large community base − Tacit knowledge: staff have previous experience with puppet, many companies are using it.
  • 18.
    Q&A  http://www.aptira.com  http://www.openstack.org  http://www.puppetlabs.com  Twitter: − @Aptira − @KavitAptira