Puppet Camp DC: Puppet for Everybody

873 views

Published on

Puppet Camp DC: "Puppet for Everybody" by Christopher Bowles, University of Texas at Austin

Published in: Technology, Spiritual
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
873
On SlideShare
0
From Embeds
0
Number of Embeds
122
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Intro: brief sketch about UT Austin
    “Going to give a brief overview about how we provide Puppet Enterprise as a service that enables DevOps at the University of Texas.”

  • Configuration Management is not just for sysadmins any more. People at every level of the organization have a vested interest in keeping their configurations verifiably correct.


    Development staff: Can use Puppet enterprise as a part of their deployment chain / to verify configuations / and to make minor changes
    Operational staff, naturally, use it to apply configurations using the trifecta (package / file / service)
    Management : Puppet Enterprise model is visible via the Puppet Console, can review reports, etc...

    However!
    These are very different use cases
    And very different levels of engagement with Puppet, the Puppet Enterprise console, and the Puppet RAL (Resource Abstraction Layer)

    Just as Puppet wraps the details of configurations in a layer of abstraction, we needed to abstract and smooth out Puppet’s internal works.
    So… “Puppet as a Service’!
  • The levels of expertise with Puppet in those different groups aligned along a continuum from Novice to Expert, with infinite levels between.
    We wanted a solution that would work for all of them.
  • Puppet Enterprise as the core.
    Bolted on three different aspects to the stock Puppet Enterprise software (Puppet server, Puppet Console, etc…)

    UT Puppet API –
    Code/Data Federation – Hierarchical Configuration -
  • A single entity - single puppet DNS name, single puppet console. One point of contact for all things puppet.
  • A standard API (in the loose sense). This includes all of the standards that we have set for our Puppet Modules, documentation.
    Puppet Console always sets the variables for the Puppet Modules via top level variables
    These variables are named in the same way
    Documentation is standardized

    With the combination of the right class and variable, the configurations are applied.


  • Hierarchical Configurations. Allows for Puppet Enterprise configurations to be completely modeled in Puppet Console node groups and inherited by child node groups and nodes. By providing Puppet as a Service, we empower other sysadmins to take advantage of the power of Puppet without necessarily having to know

    This:
    Allows for capturing of configuration data that cannot be automatically obtained (i..e. via facter). The “business logic” is encoded into Puppet
    Prevents duplication of configurations
    Allows for Sr Sysadmins to provide standard builds for Jr Sysadmins
  • Basic idea: Consumers have the ability to make changes to their own systems/services at the appropriate level of impact.

    Eliminating the bottleneck. Empowerment. Sysadmins are empowered to make changes and update configurations at the appropriate level of impact, via both Puppet Console standards and hiera backend federation.

    United States of Puppet
    Allows for general standards to be applied, i.e. Federal Government (Guarantees that any server attaching will get defaults)
    Allows for command-and-control of department/service/sub-group specific settings
    Implemented via:
    Puppet Console (very basic ACLs
    Code nomenclature
    via multiple source areas
  • All of our configurations are built from small, modular building blocks:
    Puppet Classes
    Variables
    Groups
    Nodes

    Let’s take a closer look at the details.
  • The three primary moving pieces are Classes, Variables and Groups.

    Classes – link the puppet node to the appropriate Puppet class. I.e. Provides the SSH class
    Variables – Sets top-level variables. Provides parameterization to the classes (if they are configured correctly – more on that later)
    Groups – Collections of classes and variables. Note that groups can be members of groups.
    Nodes – The actual server. Where the rubber hits the road. All the classes and variables from the groups are combined here via inheritance rules.
  • Let’s talk about nested group. Useful for Class and Variable inheritance.

    You can use nested groups to:
    Eliminate Configuration Duplication
    Override variables



  • Classes inheritance is immutable, so that classes can only be added and never taken away.

    NOTE: this is a good reason to be cautious when creating your hierarchy to ensure that you really want the classes to apply to ALL the nodes contained within the group.
    More on that…
  • Variables are a different beast. The child group or node inherits all of the variables, but in case of a conflict the child value wins.

    This is useful for two things:
    Broad default values can be applied at at very high-level group. These will be inherited by the child groups and finally applied to the nodes. At ahy point downstream, these values will show up in either the group or node. This adds visibility and makes it easy to override the variables via copy-pasta. Standards are important, here. More on that shortly.
    Beyond just All or node paramerization. It allows you to use Puppet Console groups to model the hierarchy within your organization in a way that is both intuitive and fairly straightforward. This is especially useful for variables that are based upon business process and are not easily codeable (i.e. SSH port and/or DNS name).

  • Let’s take a look at a practical example of this process.
  • This graph shows the inheritance properties in a real example. This is a common occurrence where you want to make sure that SSH is configured, but have various listenports configured in your organization. This is a good example of a variable that is based upon business rules and cannot be determined from the server itself.

    The SSH class and the listenport variable are assigned in the BASE group
    The DB group adds the mysql class and overrides the listenport variable.

    Everything comes together at the db-server node, which inherits all of the classes that have been assigned, but only the most recent listenport variable.


  • Resulting Set of Policy
    Same thing, from a Puppet console screenshot.
    Presenter note: Need to see if this is too small to see. Try viewing from the back of the room. Look into bluetooth presenter?
  • We found it useful to create a naming scheme for our hierarchical node groups. We add a prefix to each group name that gives the location in the hierarchy. This allows you to designate the hierarchy and find it in the flat list that puppet displays.

    group_foo = Department groups
    subgroup_foo = Service groups


    The generic example is on the left.
    Just as Puppet can model the configurations necessary for your servers through application of classes, the Puppet Enterprise Console can model your organization via groups. Creating a good hierarchy and placing configurations and variables in the proper level can provide major benefits in standardization and ease of Puppet management.

    You build your hierarchy by deciding how many groups you need. This is a balancing act that based upon the existing structures in your organization. I think that the rule of three applies well here. Default, Group, subgroup is a pretty good place to start.

    A specific example is on the right.
    Let’s say you have many different groups in your organization. One such group provides Database services, called “group_db”. Below that are subgroups that contain the MySQL and Oracle services.

    This is where we really chase the “infrastructure as code” concept. Ideally, there should be sufficient classes and variables contained within these three levels that you can take a newly provisioned node and attach it to one of the subgroups, run puppet and then you have a fully configured server. With these in place, you have configurations that are reproducible, scalable and can be stored in source control.
  • One note about naming. It’s important that your group names indicate their hierarchy, since the Puppet Enterprise console does not have a native way of displaying hierarchical data.

    Our standard is to have a single BASE group, but to have all other groups set off with a prefix that indicates their level (i.e. group, subgroup). Since the Console sorts the groups alphabetically, this ensures that they all show up together.
  • However, not all variables are simple. Some variables need to be expressed in complex data structures like arrays and hashes. They can also be arranged in hierarchical and federated group, just like the Puppet console variables.

    This involves creating a hierarchy in hiera.conf, as well as more console variables!


    Array: https://www.flickr.com/photos/ratatatratsy/5394750775/
    Hash Found nothing!
  • We found it useful to create a naming scheme for our hierarchical node groups. We add a prefix to each group name that gives the location in the hierarchy. This allows you to designate the hierarchy and find it in the flat list that puppet displays.

    group_foo = Department groups
    subgroup_foo = Service groups


    The generic example is on the left.
    Just as Puppet can model the configurations necessary for your servers through application of classes, the Puppet Enterprise Console can model your organization via groups. Creating a good hierarchy and placing configurations and variables in the proper level can provide major benefits in standardization and ease of Puppet management.

    You build your hierarchy by deciding how many groups you need. This is a balancing act that based upon the existing structures in your organization. I think that the rule of three applies well here. Default, Group, subgroup is a pretty good place to start.

    A specific example is on the right.
    Let’s say you have many different groups in your organization. One such group provides Database services, called “group_db”. Below that are subgroups that contain the MySQL and Oracle services.

    This is where we really chase the “infrastructure as code” concept. Ideally, there should be sufficient classes and variables contained within these three levels that you can take a newly provisioned node and attach it to one of the subgroups, run puppet and then you have a fully configured server. With these in place, you have configurations that are reproducible, scalable and can be stored in source control.
  • federated hierarchies are created in hiera in a similar manner to the way that we did it in the console with variables. Instead of nested groups, we use nested folders.

    The root above is your hiera data directory, usually /etc/puppetlabs/puppet/hieradata (configured in hiera.yaml file)

    TOP: Common hiera variables (aka keys) go here, usually in a common.yaml file, but totally dependent upon how hiera.yaml is configured
    Group(s): These map to large divisions of your enterprise/organization. Any way it makes sense for you. Any number of groups. These should map 1-1 with the node groups created earlier, which I will explain next.
    Subgroup(s): These map to smaller divisions of the groups above. Any number of subgroups or sub-sub-groups.
  • federated hierarchies are created in hiera in a similar manner to the way that we did it in the console with variables. Instead of nested groups, we use nested folders.

    The root above is your hiera data directory, usually /etc/puppetlabs/puppet/hieradata (configured in hiera.yaml file)

    TOP: Common hiera variables (aka keys) go here, usually in a common.yaml file, but totally dependent upon how hiera.yaml is configured
    Group(s): These map to large divisions of your enterprise/organization. Any way it makes sense for you. Any number of groups. These should map 1-1 with the node groups created earlier, which I will explain next.
    Subgroup(s): These map to smaller divisions of the groups above. Any number of subgroups or sub-sub-groups.
  • Mapping occurs via including the appropriate groups
    ----- Meeting Notes (6/6/14 14:58) -----
    Console Groups -> Variables
  • mysql-prod node inherits both locations, so hiera will search both locations
  • A key reason to have multiple hiera locations
    Can use hiera_hash function to combine keys from various hiera files
    Allows for complete configurations to be compiled from multiple small sources
    Great for +1 requirements
  • many-jointed modulepath in puppet.conf
    Modulepath=:/path/to/dept1:/path/to/dept2
    Each separate piece checked out to via puppetized VCSREPO runs
    Updates working copy only if newer
    Insert VCSREPO snippet here….
  • Wedge this into the code federation component... Might be a good place for a venn diagram

    Just as we used Namespaces for the node groups

    Need a visual aid for this?

    Next:
    - talk about hiera usage ( links to console)
    - Talk about code/hiera federation
    - Wrap it up!


  • Picture: https://www.flickr.com/photos/dunechaser/9312166108/in/photostream/

  • Next steps are to update our existing SVN source control to a full-fledged git workflow. We are planning on taking advantage of the Atlassian “Git essentials” package, which includes a lot of additional workflow pieces (Continuous Integration, Jira Integration)

    The key wins are:
    Instead of monolitihic repos, we get small agile repos (1/module). This enables a more agile workflow and tracking
    In addition, git excels at creating new branches. So, new development can occur on non-production branches, which will really speed up development
    One cool feature with git essentials is that it allows “per-branch” ACLs, which will allow us to lock down the Head/production branch for our Core modules, but enable easy collaboration for the rest of our organization

  • Using a git workflow opens up a lot of new possibilities:

    R10k push deployments instead of the existing pull deployments from SVN
    R10k can use Puppetfiles to create software-defined Puppet Environments (which is very cool)
    Automated testing
  • Wrapping it up
  • Puppet Camp DC: Puppet for Everybody

    1. 1. Puppet for Everybody! Federated and Hierarchical Puppet Enterprise Chris Bowles, Senior Systems Administrator University of Texas at Austin
    2. 2. Everybody? Absolutely! • Development • Operations • Management source: http://goo.gl/Mjr0dy
    3. 3. Continuum of Expertise Novice • Puppet Console • Variables Medium • Hiera Expert • Code
    4. 4. Puppet as a Service UT Puppet API (standards, culture) Code/Data Federation Puppet Enterprise Hierarchical Configs
    5. 5. Centrally Managed / Maintained Puppet as as Service DB group Web group Foo group
    6. 6. Standard Puppet API Puppet Console Configured Server! Class Console Variable (string) Hiera Variable (complex) Shared culture
    7. 7. Hierarchical Configurations Node-level Business Group(s) Defaults Enterprise Database Node Web • Shared area = business logic • All configs in groups • Plug and play
    8. 8. Federation (of control) • United States of Puppet mysql oracle Enforce Broad configs at “Federal Level” Set specific configs at “State Level” source: http://goo.gl/22neR7
    9. 9. Puppet as a Service (at UT) • Code or contentApps • i.e. ApacheServices • secure • standardized • configurable BASE Roles / Profiles
    10. 10. Building Blocks source: http://goo.gl/CHwab0
    11. 11. Puppet Console components • Classes • Variables • Group(s) • Nodes ssh $::ssh_listenport Database dbserver-01
    12. 12. Puppet Console Group Nesting! source: http://goo.gl/tUdl5U
    13. 13. Class Inheritance (immutable) BASE group assigns: ssh FOO group inherits: ssh assigns: syslog node Inherits: ssh, syslog
    14. 14. Variable Inheritance (child wins) Base group ssh_listenport = 22 Foo group ssh_listenport = 72 node1 ssh_listenport = 72 Foo2 group ssh_listenport = 99 node2 ssh_listenport = 99
    15. 15. All together now! source: http://goo.gl/K91CJA
    16. 16. BASE group Classes: ssh Variable: ssh_listenport => 22 DB group Classes: mysql Variables: ssh_listenport => 99 db-server (node) Classes: ssh , mysql Variables: ssh_port => 99 (from DB group) Classes: additive Variables Closest to the node wins
    17. 17. ENC node: RSOP Mysql class from DB group) Ssh class from ALL group Ssh_port override from DB group (2222) List of all the groups that the node belongs to. Hierarchy is implied via the Source attr. See ALL group.
    18. 18. Puppet Console: Building a federated hierarchy Subgroup Group Top BASE group_foo sub_bar group_db sub_mysql sub_oracle
    19. 19. What’s in a name? Puppet Console will display: (alphabetical) • BASE • group_db • group_foo • subgroup_bar • subgroup_mysql • subgroup_oracle
    20. 20. A Little more about Console Variables… class Console variable Config
    21. 21. Console Variables = top level Variables ($::foo) • init.pp • params.pp
    22. 22. One more level: Hiera source: http://goo.gl/3tfaVi
    23. 23. Hiera: used for complex variables key: value key2: value2 • Arrays • Hashes source: http://goo.gl/ge45I1
    24. 24. Hiera: Building a federated hierarchy Subgroup Group Top BASE group_foo sub_bar group_db sub_mysql sub_oracle
    25. 25. Creating hiera directory-based hierarchies with console variables Subgroup(s) • ./$group/$sub Group(s) • ./$group/ Top • ./ (no variable) $group $sub Broad to Specific
    26. 26. Hiera.yaml – rubber, meet road Top • ./common.yaml Group(s) • ./$group/common.yaml Subgroup(s) • ./$group/$sub/common.yaml ‘%{group}/%{sub}/common’ ‘%{group}/common’ ‘common’ Specific To Broad
    27. 27. Console Groups map to Hiera Nested Console Groups Variables (above the waterline) Hiera (below the waterline) BASE (no variable required) Foo group = foo Bar sub = bar ./ ./foo/ ./foo/bar/
    28. 28. Putting it together! Nested Console Group Variable Hiera (files based on hiera.yaml) DB group = db MySQL sub = mysql ./db/ ./db/mysql/ mysql node dept=db, svc= mysql BASE (no variables) ./ Hiera search path for mysql node: 1. ./common.yaml 2. ./db/common.yaml 3. ./db/mysql/common.yaml
    29. 29. Advanced Hiera Usage • “This data is exactly what I need… almost” • firewall, sudoers • +1 • Check out: – hiera_hash – hiera_array
    30. 30. Code/Data Federation Puppet Classes/Data Subgroup source Group source BASE source
    31. 31. Code Federation puppet.conf modulepath= /opt/puppet/modules/base: /opt/puppet/modules/group: /opt/puppet/modules/sub: BASE Repo Group Repo Subgroup Repo VCSREPO
    32. 32. Data Federation (via VCSREPO) • ./hieradata/ = ./common.yaml ./$group/common.yaml ./$group/$sub/common.yaml BASE Repo group Repo subgroup Repo VCSREPO
    33. 33. Puppet Class Namespaces • Namespaces prevent class collisions • ${group}_name • Examples: – base_ssh – db_ssh – mysql_ssh
    34. 34. A peek into the future… source: http://goo.gl/9GwKyQ
    35. 35. Git Workflow • Instead of this… • 1 git repo / module Core SVN repo (modules) Group SVN repo (modules) Head (production) branch Non-production branches (created as needed)
    36. 36. CI/CD • r10k push deployments (faster!) • Puppet Environments defined by code (Puppetfile) • Automated Testing Git repos r10k Puppet
    37. 37. Takeaways! • “Puppet as a Service” – Can provide hierarchical/federated configuration management as a service by building on top of Puppet Enterprise • Hierarchical Configs: building blocks that match organizational structure, plug and play • Federation: Empowering users at all levels of the enterprise
    38. 38. Thanks! Any Questions? • Note: slide deck available from PuppetLabs • Contact information: – Chris Bowles • Email: cbowles@austin.utexas.edu

    ×