SlideShare a Scribd company logo
Puppet control-repo

to the next level
(an opinionated big fat control-repo)
Alessandro Franceschi
@alvagante
A control-repo contains a Puppet environment
A magic word appeared somewhere sometime giving
shape to a common pattern
A single place containing all the our Puppet manifests,
modules and, generally, Hiera data
Each environment is a directory under

/etc/puppetlabs/code/environments/
The default environment is called production

/etc/puppetlabs/code/environments/production
Puppet environments are typically managed in git control-
repos.
Control repo components
• A minimal control-repo contains:

manifest directory for common manifests

modules directory for modules

hieradata directory for Hiera data, if used

Puppetfile with list of external modules to use

environment.conf to configure the environment
• Puppetfile is used by r10k or librarian-puppet to
deploy Puppet modules from external sources (git repo
or Modules Forge)
• Find an essential skeleton here:

https://github.com/puppetlabs/control-repo 

example42 big fat control-repo
An opinionated control-repo where more is better than less,
when you have the choice to remove, featuring:
A relevant amount of customisable profiles and tools with
sample Hiera data to configure common applications
Multiple configurable Vagrant environments
Integration with Docker for VM testing and images building
Integrations with Fabric and GitLab
Fits any setup: Puppet Enterprise, Foreman, standard OSS
Find it here:

https://github.com/example42/control-repo
3 + 1 ways to start
• Just start to play around
• Fork and customise for your project



• Start a new project from scratch,

cherry picking what you want to keep
• Just look around for inspiration or cherry picking
git clone https://github.com/example42/control-repo
cd control-repo

bin/setup_puppet.sh # Installs required gems and runs r10k
git clone https://github.com/example42/control-repo
cd control-repo
bin/git_setup_new_repo.sh

# vi files_to_adapt ; rm files to remove ; ... # Any change you may want
git commit -a -m "Repo based on https://github.com/example42/control-repo"

git remote add origin git@github.com:<yourname>/puppet-control-repo.git
git push -u origin --all
git clone https://github.com/<yourname>/control-repo
cd control-repo
git remote add upstream https://github.com/example42/control-repo

bin/setup_puppet.sh
Setup
The control repo requires Puppet 4, if not present:
bin/install_puppet.sh
If you already have r10k and possibly the other needed
gems (hiera-eyaml, deep_merge) then just run:
r10k puppetfile install -v
To install the above gems and then run r10k you can
use this script:
bin/setup_puppet.sh
Vagrant, Docker, Fabric are optional, suggested,
dependencies. You can install them, with:
bin/setup.sh
Install local git hooks with useful Puppet checks
bin/git_install_hooks.sh

vi .git/hooks/commit_hooks/config.cfg
Fabric integration
Optional integration with fabric.

Install it with any of these commands ;->
pip install fabric # If you have already pip installed

tp install fabric # If you have tp installed

bin/setup.sh # Use the main setup script
Tasks are defined in the *.py files on the base dir.
Show them with:
fab -l
The ones marked as [local] are run on the local host

The [remote] ones are supposed to be run on remote
nodes, specified as follows (or via the other ways to
set nodes where to run Fabric tasks):
fab -H node1,node2 puppet.install
demo
Explore
• Give a look to the following files and directories:


# The first manifest parsed by Puppet server

manifests/site.pp
# r10k Puppetfile and directory for public modules

Puppetfile
modules/
# Local tools and profiles for common use cases
site/profiles
site/tools
# Sample Hiera configuration file and data directory
hiera.yaml
hieradata/
# Directory with different Vagrant environments
vagrant/environments


# Support files to build docker images

docker/
# Blueprint directory for modules 

skeleton/
# Tools for various tasks (used by Fabric, used in devel/testing/ci phases
bin/
Test local code with Vagrant
• Multiple Vagrant environments


ls -l vagrant/environments/
• Each one customisable via config.yaml


cd vagrant/environments/ostest

vi config.yaml
• Local control-repo code and data can, by default,
be tested on Vagrant VMs both in puppet apply
and agent mode


cd vagrant/environments/puppetinfra

vagrant status

vagrant up dev-local-puppet-01

vagrant ssh dev-local-puppet-01

vm $ sudo su -

vm # /etc/puppetlabs/code/environments/production/bin/papply.sh
vm # puppet agent -t
Customising Vagrant environments
Single roles can be tested in relevant VMs:
fab vagrant.up:vm=dev-local-log-01
fab vagrant.provision:vm=dev-local-log-01
All Linux servers use this class of common resources:
# site/profile/manifests/base/linux.pp
# profile::base::linux common settings are in: hieradata/common.yaml
Each role may have specific Hiera settings in:
# hieradata/role/$role.yaml

# IE. For "log" specific Hiera data is in hieradata/role/log.yaml
Puppet Enterprise - Gitlab environment
Dedicated Vagrant environment with:

- Puppet Enterprise AIO is installed on a VM

- GitLab is installed on another VM

- A third VM with gitlab CI runner is added
The full setup involves:

- Configuration of Code Manager to automatically
deploy Puppet code on PE

- Configuration of GitLab

- Configuration of GitLab runners to run Puppet tests
Currently setup is not completely automated.

For details check vagrant/environments/pe_demo/README.md
Play with Docker
Test a role on a given OS:
fab docker.test_role:log,ubuntu-14.04

fab docker.test_role:git
# Available images: ubuntu-12.04, ubuntu-14.04, ubuntu-14.06

# centos-7, debian-7, debian-8, alpine-3.3
Build an image based on a role (with Rocker) (WIP):
fab docker.rocker_build_role:ansible,ubuntu-14.04
Build via tp multiple OS images based on a role (WIP):
fab docker.tp_build_role:ansible
Some design choices (all changeable)
• Profiles loaded via hiera_include
• Hiera driven noop mode
• Wide usage of Tiny Puppet in some profiles
• OS based base profile with class indirection
• Hiera eyaml backend. Hieradata in control-repo
• Separated public and site dirs. Site modules in
control-repo
• More is better than less (because you can always remove what you don't want)
Where to customise
• The control-repo is just the starting point for a
greenfield modern Puppet setup
• Define a way to set your nodes' roles and node
defining variables (the ones used in hiera.yaml)
• Select the public modules to use and add them to
Puppetfile
• Write local profiles in site/profile/manifests
• Review hiera.yaml logic and customise data in
hieradata/
• Customise your Vagrant environments
• Customise the skeleton to use for custom modules
Mostly welcomed contributions
• Use or have ideas from it and tell us what you think
• Feedback, critics and suggestions are really
needed to guide our evolutions
• If you have good tools or profiles, valid for general
use cases, PR them. More is better than less...
• Talk with us @example42 @alvagante
enough Puppet ramblings
Thank You
for all the fish
Alessandro Franceschi
@alvagante

More Related Content

What's hot

Puppet modules for Fun and Profit
Puppet modules for Fun and ProfitPuppet modules for Fun and Profit
Puppet modules for Fun and Profit
Alessandro Franceschi
 
Puppet @ Seat
Puppet @ SeatPuppet @ Seat
Puppet @ Seat
Alessandro Franceschi
 
Configuration Surgery with Augeas
Configuration Surgery with AugeasConfiguration Surgery with Augeas
Configuration Surgery with Augeas
Puppet
 
Puppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction KitPuppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction Kit
Alessandro Franceschi
 
Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點
William Yeh
 
Intro to-puppet
Intro to-puppetIntro to-puppet
Intro to-puppet
F.L. Jonathan Araña Cruz
 
Anatomy of a reusable module
Anatomy of a reusable moduleAnatomy of a reusable module
Anatomy of a reusable module
Alessandro Franceschi
 
Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.
Łukasz Proszek
 
Scalable Systems Management with Puppet
Scalable Systems Management with PuppetScalable Systems Management with Puppet
Scalable Systems Management with Puppet
Puppet
 
Developing IT infrastructures with Puppet
Developing IT infrastructures with PuppetDeveloping IT infrastructures with Puppet
Developing IT infrastructures with Puppet
Alessandro Franceschi
 
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Martin Etmajer
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 Edition
Joshua Thijssen
 
From SaltStack to Puppet and beyond...
From SaltStack to Puppet and beyond...From SaltStack to Puppet and beyond...
From SaltStack to Puppet and beyond...
Yury Bushmelev
 
Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...
Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...
Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...
Puppet
 
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Puppet
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
Puppet
 
Getting Started With Aura
Getting Started With AuraGetting Started With Aura
Getting Started With Aura
Chris Tankersley
 
DevOps(4) : Ansible(2) - (MOSG)
DevOps(4) : Ansible(2) - (MOSG)DevOps(4) : Ansible(2) - (MOSG)
DevOps(4) : Ansible(2) - (MOSG)
Soshi Nemoto
 
Ansible not only for Dummies
Ansible not only for DummiesAnsible not only for Dummies
Ansible not only for Dummies
Łukasz Proszek
 
Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)
Soshi Nemoto
 

What's hot (20)

Puppet modules for Fun and Profit
Puppet modules for Fun and ProfitPuppet modules for Fun and Profit
Puppet modules for Fun and Profit
 
Puppet @ Seat
Puppet @ SeatPuppet @ Seat
Puppet @ Seat
 
Configuration Surgery with Augeas
Configuration Surgery with AugeasConfiguration Surgery with Augeas
Configuration Surgery with Augeas
 
Puppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction KitPuppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction Kit
 
Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點Ansible 實戰:top down 觀點
Ansible 實戰:top down 觀點
 
Intro to-puppet
Intro to-puppetIntro to-puppet
Intro to-puppet
 
Anatomy of a reusable module
Anatomy of a reusable moduleAnatomy of a reusable module
Anatomy of a reusable module
 
Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.
 
Scalable Systems Management with Puppet
Scalable Systems Management with PuppetScalable Systems Management with Puppet
Scalable Systems Management with Puppet
 
Developing IT infrastructures with Puppet
Developing IT infrastructures with PuppetDeveloping IT infrastructures with Puppet
Developing IT infrastructures with Puppet
 
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 Edition
 
From SaltStack to Puppet and beyond...
From SaltStack to Puppet and beyond...From SaltStack to Puppet and beyond...
From SaltStack to Puppet and beyond...
 
Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...
Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...
Building and Testing from Scratch a Puppet Environment with Docker - PuppetCo...
 
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
 
Getting Started With Aura
Getting Started With AuraGetting Started With Aura
Getting Started With Aura
 
DevOps(4) : Ansible(2) - (MOSG)
DevOps(4) : Ansible(2) - (MOSG)DevOps(4) : Ansible(2) - (MOSG)
DevOps(4) : Ansible(2) - (MOSG)
 
Ansible not only for Dummies
Ansible not only for DummiesAnsible not only for Dummies
Ansible not only for Dummies
 
Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)
 

Similar to Puppet control-repo 
to the next level

Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationDockerizing a Symfony2 application
Dockerizing a Symfony2 application
Roman Rodomansky
 
Docker 102 - Immutable Infrastructure
Docker 102 - Immutable InfrastructureDocker 102 - Immutable Infrastructure
Docker 102 - Immutable Infrastructure
Adrian Otto
 
Docman - The swiss army knife for Drupal multisite docroot management and dep...
Docman - The swiss army knife for Drupal multisite docroot management and dep...Docman - The swiss army knife for Drupal multisite docroot management and dep...
Docman - The swiss army knife for Drupal multisite docroot management and dep...
Aleksey Tkachenko
 
02 Hadoop deployment and configuration
02 Hadoop deployment and configuration02 Hadoop deployment and configuration
02 Hadoop deployment and configuration
Subhas Kumar Ghosh
 
Improving your Drupal 8 development workflow DrupalCampLA
Improving your Drupal 8 development workflow DrupalCampLAImproving your Drupal 8 development workflow DrupalCampLA
Improving your Drupal 8 development workflow DrupalCampLA
Jesus Manuel Olivas
 
Build and deployment
Build and deploymentBuild and deployment
Build and deployment
WO Community
 
A Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy SystemA Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy System
adrian_nye
 
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios
 
Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.
Workhorse Computing
 
[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview
[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview
[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview
Leo Lorieri
 
Lean Drupal Repositories with Composer and Drush
Lean Drupal Repositories with Composer and DrushLean Drupal Repositories with Composer and Drush
Lean Drupal Repositories with Composer and Drush
Pantheon
 
Building a Drupal site with Git
Building a Drupal site with GitBuilding a Drupal site with Git
Building a Drupal site with Git
dirtytactics
 
Puppet camp london nov 2014 slides (1)
Puppet camp london nov 2014   slides (1)Puppet camp london nov 2014   slides (1)
Puppet camp london nov 2014 slides (1)
Puppet
 
Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
biicode
 
ActiveLAMP Process
ActiveLAMP ProcessActiveLAMP Process
ActiveLAMP Process
Tom Friedhof
 
Hands on Docker - Launch your own LEMP or LAMP stack - SunshinePHP
Hands on Docker - Launch your own LEMP or LAMP stack - SunshinePHPHands on Docker - Launch your own LEMP or LAMP stack - SunshinePHP
Hands on Docker - Launch your own LEMP or LAMP stack - SunshinePHP
Dana Luther
 
Puppet Camp Atlanta 2014: r10k Puppet Workflow
Puppet Camp Atlanta 2014: r10k Puppet WorkflowPuppet Camp Atlanta 2014: r10k Puppet Workflow
Puppet Camp Atlanta 2014: r10k Puppet Workflow
Puppet
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
Micael Gallego
 
Before & After Docker Init
Before & After Docker InitBefore & After Docker Init
Before & After Docker Init
Angel Borroy López
 
From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...
From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...
From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...
Docker, Inc.
 

Similar to Puppet control-repo 
to the next level (20)

Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationDockerizing a Symfony2 application
Dockerizing a Symfony2 application
 
Docker 102 - Immutable Infrastructure
Docker 102 - Immutable InfrastructureDocker 102 - Immutable Infrastructure
Docker 102 - Immutable Infrastructure
 
Docman - The swiss army knife for Drupal multisite docroot management and dep...
Docman - The swiss army knife for Drupal multisite docroot management and dep...Docman - The swiss army knife for Drupal multisite docroot management and dep...
Docman - The swiss army knife for Drupal multisite docroot management and dep...
 
02 Hadoop deployment and configuration
02 Hadoop deployment and configuration02 Hadoop deployment and configuration
02 Hadoop deployment and configuration
 
Improving your Drupal 8 development workflow DrupalCampLA
Improving your Drupal 8 development workflow DrupalCampLAImproving your Drupal 8 development workflow DrupalCampLA
Improving your Drupal 8 development workflow DrupalCampLA
 
Build and deployment
Build and deploymentBuild and deployment
Build and deployment
 
A Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy SystemA Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy System
 
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
 
Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.
 
[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview
[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview
[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview
 
Lean Drupal Repositories with Composer and Drush
Lean Drupal Repositories with Composer and DrushLean Drupal Repositories with Composer and Drush
Lean Drupal Repositories with Composer and Drush
 
Building a Drupal site with Git
Building a Drupal site with GitBuilding a Drupal site with Git
Building a Drupal site with Git
 
Puppet camp london nov 2014 slides (1)
Puppet camp london nov 2014   slides (1)Puppet camp london nov 2014   slides (1)
Puppet camp london nov 2014 slides (1)
 
Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
 
ActiveLAMP Process
ActiveLAMP ProcessActiveLAMP Process
ActiveLAMP Process
 
Hands on Docker - Launch your own LEMP or LAMP stack - SunshinePHP
Hands on Docker - Launch your own LEMP or LAMP stack - SunshinePHPHands on Docker - Launch your own LEMP or LAMP stack - SunshinePHP
Hands on Docker - Launch your own LEMP or LAMP stack - SunshinePHP
 
Puppet Camp Atlanta 2014: r10k Puppet Workflow
Puppet Camp Atlanta 2014: r10k Puppet WorkflowPuppet Camp Atlanta 2014: r10k Puppet Workflow
Puppet Camp Atlanta 2014: r10k Puppet Workflow
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
 
Before & After Docker Init
Before & After Docker InitBefore & After Docker Init
Before & After Docker Init
 
From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...
From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...
From Arm to Z: Building, Shipping, and Running a Multi-platform Docker Swarm ...
 

More from Alessandro Franceschi

Strategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoringStrategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoring
Alessandro Franceschi
 
DevOps - Evoluzione della specie - DevOps Heroes.pdf
DevOps - Evoluzione della specie - DevOps Heroes.pdfDevOps - Evoluzione della specie - DevOps Heroes.pdf
DevOps - Evoluzione della specie - DevOps Heroes.pdf
Alessandro Franceschi
 
Tiny Puppet Can Install Everything. Prove me wrong!
Tiny Puppet Can Install Everything. Prove me wrong!Tiny Puppet Can Install Everything. Prove me wrong!
Tiny Puppet Can Install Everything. Prove me wrong!
Alessandro Franceschi
 
ReUse Your (Puppet) Modules!
ReUse Your (Puppet) Modules!ReUse Your (Puppet) Modules!
ReUse Your (Puppet) Modules!
Alessandro Franceschi
 
Ten years of [Puppet] installations. What now?
Ten years of [Puppet] installations. What now?Ten years of [Puppet] installations. What now?
Ten years of [Puppet] installations. What now?
Alessandro Franceschi
 
Raise the bar! Reloaded
Raise the bar! ReloadedRaise the bar! Reloaded
Raise the bar! Reloaded
Alessandro Franceschi
 
Raise the bar!
Raise the bar!Raise the bar!
Raise the bar!
Alessandro Franceschi
 
Puppet modules: An Holistic Approach
Puppet modules: An Holistic ApproachPuppet modules: An Holistic Approach
Puppet modules: An Holistic Approach
Alessandro Franceschi
 
Spaghetti devops
Spaghetti devopsSpaghetti devops
Spaghetti devops
Alessandro Franceschi
 

More from Alessandro Franceschi (9)

Strategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoringStrategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoring
 
DevOps - Evoluzione della specie - DevOps Heroes.pdf
DevOps - Evoluzione della specie - DevOps Heroes.pdfDevOps - Evoluzione della specie - DevOps Heroes.pdf
DevOps - Evoluzione della specie - DevOps Heroes.pdf
 
Tiny Puppet Can Install Everything. Prove me wrong!
Tiny Puppet Can Install Everything. Prove me wrong!Tiny Puppet Can Install Everything. Prove me wrong!
Tiny Puppet Can Install Everything. Prove me wrong!
 
ReUse Your (Puppet) Modules!
ReUse Your (Puppet) Modules!ReUse Your (Puppet) Modules!
ReUse Your (Puppet) Modules!
 
Ten years of [Puppet] installations. What now?
Ten years of [Puppet] installations. What now?Ten years of [Puppet] installations. What now?
Ten years of [Puppet] installations. What now?
 
Raise the bar! Reloaded
Raise the bar! ReloadedRaise the bar! Reloaded
Raise the bar! Reloaded
 
Raise the bar!
Raise the bar!Raise the bar!
Raise the bar!
 
Puppet modules: An Holistic Approach
Puppet modules: An Holistic ApproachPuppet modules: An Holistic Approach
Puppet modules: An Holistic Approach
 
Spaghetti devops
Spaghetti devopsSpaghetti devops
Spaghetti devops
 

Recently uploaded

Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
davidjhones387
 
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
3a0sd7z3
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
xjq03c34
 
Bengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal BrandingBengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal Branding
Tarandeep Singh
 
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
APNIC
 
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
3a0sd7z3
 
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
rtunex8r
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
k4ncd0z
 
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
APNIC
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
Donato Onofri
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
Paul Walk
 
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
thezot
 

Recently uploaded (12)

Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
 
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
 
Bengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal BrandingBengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal Branding
 
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
 
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
 
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
 
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
 
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
 

Puppet control-repo 
to the next level

  • 1. Puppet control-repo
 to the next level (an opinionated big fat control-repo) Alessandro Franceschi @alvagante
  • 2. A control-repo contains a Puppet environment A magic word appeared somewhere sometime giving shape to a common pattern A single place containing all the our Puppet manifests, modules and, generally, Hiera data Each environment is a directory under
 /etc/puppetlabs/code/environments/ The default environment is called production
 /etc/puppetlabs/code/environments/production Puppet environments are typically managed in git control- repos.
  • 3. Control repo components • A minimal control-repo contains:
 manifest directory for common manifests
 modules directory for modules
 hieradata directory for Hiera data, if used
 Puppetfile with list of external modules to use
 environment.conf to configure the environment • Puppetfile is used by r10k or librarian-puppet to deploy Puppet modules from external sources (git repo or Modules Forge) • Find an essential skeleton here:
 https://github.com/puppetlabs/control-repo 

  • 4. example42 big fat control-repo An opinionated control-repo where more is better than less, when you have the choice to remove, featuring: A relevant amount of customisable profiles and tools with sample Hiera data to configure common applications Multiple configurable Vagrant environments Integration with Docker for VM testing and images building Integrations with Fabric and GitLab Fits any setup: Puppet Enterprise, Foreman, standard OSS Find it here:
 https://github.com/example42/control-repo
  • 5. 3 + 1 ways to start • Just start to play around • Fork and customise for your project
 
 • Start a new project from scratch,
 cherry picking what you want to keep • Just look around for inspiration or cherry picking git clone https://github.com/example42/control-repo cd control-repo
 bin/setup_puppet.sh # Installs required gems and runs r10k git clone https://github.com/example42/control-repo cd control-repo bin/git_setup_new_repo.sh
 # vi files_to_adapt ; rm files to remove ; ... # Any change you may want git commit -a -m "Repo based on https://github.com/example42/control-repo"
 git remote add origin git@github.com:<yourname>/puppet-control-repo.git git push -u origin --all git clone https://github.com/<yourname>/control-repo cd control-repo git remote add upstream https://github.com/example42/control-repo
 bin/setup_puppet.sh
  • 6. Setup The control repo requires Puppet 4, if not present: bin/install_puppet.sh If you already have r10k and possibly the other needed gems (hiera-eyaml, deep_merge) then just run: r10k puppetfile install -v To install the above gems and then run r10k you can use this script: bin/setup_puppet.sh Vagrant, Docker, Fabric are optional, suggested, dependencies. You can install them, with: bin/setup.sh Install local git hooks with useful Puppet checks bin/git_install_hooks.sh
 vi .git/hooks/commit_hooks/config.cfg
  • 7. Fabric integration Optional integration with fabric.
 Install it with any of these commands ;-> pip install fabric # If you have already pip installed
 tp install fabric # If you have tp installed
 bin/setup.sh # Use the main setup script Tasks are defined in the *.py files on the base dir. Show them with: fab -l The ones marked as [local] are run on the local host
 The [remote] ones are supposed to be run on remote nodes, specified as follows (or via the other ways to set nodes where to run Fabric tasks): fab -H node1,node2 puppet.install
  • 9. Explore • Give a look to the following files and directories: 
 # The first manifest parsed by Puppet server
 manifests/site.pp # r10k Puppetfile and directory for public modules
 Puppetfile modules/ # Local tools and profiles for common use cases site/profiles site/tools # Sample Hiera configuration file and data directory hiera.yaml hieradata/ # Directory with different Vagrant environments vagrant/environments 
 # Support files to build docker images
 docker/ # Blueprint directory for modules 
 skeleton/ # Tools for various tasks (used by Fabric, used in devel/testing/ci phases bin/
  • 10. Test local code with Vagrant • Multiple Vagrant environments 
 ls -l vagrant/environments/ • Each one customisable via config.yaml 
 cd vagrant/environments/ostest
 vi config.yaml • Local control-repo code and data can, by default, be tested on Vagrant VMs both in puppet apply and agent mode 
 cd vagrant/environments/puppetinfra
 vagrant status
 vagrant up dev-local-puppet-01
 vagrant ssh dev-local-puppet-01
 vm $ sudo su -
 vm # /etc/puppetlabs/code/environments/production/bin/papply.sh vm # puppet agent -t
  • 11. Customising Vagrant environments Single roles can be tested in relevant VMs: fab vagrant.up:vm=dev-local-log-01 fab vagrant.provision:vm=dev-local-log-01 All Linux servers use this class of common resources: # site/profile/manifests/base/linux.pp # profile::base::linux common settings are in: hieradata/common.yaml Each role may have specific Hiera settings in: # hieradata/role/$role.yaml
 # IE. For "log" specific Hiera data is in hieradata/role/log.yaml
  • 12. Puppet Enterprise - Gitlab environment Dedicated Vagrant environment with:
 - Puppet Enterprise AIO is installed on a VM
 - GitLab is installed on another VM
 - A third VM with gitlab CI runner is added The full setup involves:
 - Configuration of Code Manager to automatically deploy Puppet code on PE
 - Configuration of GitLab
 - Configuration of GitLab runners to run Puppet tests Currently setup is not completely automated.
 For details check vagrant/environments/pe_demo/README.md
  • 13. Play with Docker Test a role on a given OS: fab docker.test_role:log,ubuntu-14.04
 fab docker.test_role:git # Available images: ubuntu-12.04, ubuntu-14.04, ubuntu-14.06
 # centos-7, debian-7, debian-8, alpine-3.3 Build an image based on a role (with Rocker) (WIP): fab docker.rocker_build_role:ansible,ubuntu-14.04 Build via tp multiple OS images based on a role (WIP): fab docker.tp_build_role:ansible
  • 14. Some design choices (all changeable) • Profiles loaded via hiera_include • Hiera driven noop mode • Wide usage of Tiny Puppet in some profiles • OS based base profile with class indirection • Hiera eyaml backend. Hieradata in control-repo • Separated public and site dirs. Site modules in control-repo • More is better than less (because you can always remove what you don't want)
  • 15. Where to customise • The control-repo is just the starting point for a greenfield modern Puppet setup • Define a way to set your nodes' roles and node defining variables (the ones used in hiera.yaml) • Select the public modules to use and add them to Puppetfile • Write local profiles in site/profile/manifests • Review hiera.yaml logic and customise data in hieradata/ • Customise your Vagrant environments • Customise the skeleton to use for custom modules
  • 16. Mostly welcomed contributions • Use or have ideas from it and tell us what you think • Feedback, critics and suggestions are really needed to guide our evolutions • If you have good tools or profiles, valid for general use cases, PR them. More is better than less... • Talk with us @example42 @alvagante
  • 17. enough Puppet ramblings Thank You
for all the fish Alessandro Franceschi @alvagante