SlideShare a Scribd company logo

Our Puppet Story (Linuxtag 2014)

DECK36
DECK36
DECK36DECK36

In the past years it was our mission to manage development, testing and production environments for web projects with agile multi-team setups. Systems were often rather complex, with dozens of services involved. The infrastructure requirements changed frequently and as agile as the rest of project. And of course changes had to be tested and deployed continuously in a controlled and reproducible manner. A mission impossible without systematic configuration management and even with such a great tool like Puppet a continuous challenge. In this talk I will present our collection of useful tools, learnings and design patterns for Puppet. We will potentially come across topics like Vagrant, VeeWee, EC2, Docker, ENC, facter.d, git magic, Hiera, monitoring, autoregistration, rspec testing, MCollective, Puppet roles and profiles. This talk will not reinvent the wheel, but present some techniques that made us much more productive in our daily work and will hopefully help you in the future.

Our Puppet Story (Linuxtag 2014)

1 of 49
Download to read offline
Our Puppet Story
Martin Schütte
May 5 2014
About DECK36
• Small team of 7 engineers
• Longstanding expertise in designing, implementing and operating
complex web systems
• Developing own data intelligence-focused tools and web services
• Offering our expert knowledge in Automation & Operation,
Architecture & Engineering, Analytics & Data Logistics
Common Problem
“We hired someone. How can
we reproduce our dev
environment?”
Vagrant
Vagrant
Configuration tool for VMs and Provisioning.
“Local cloud”
• Self service
• Instant provisioning
• Cost efficient
• Elastic
• Pay per use
Ad

Recommended

Puppetizing Your Organization
Puppetizing Your OrganizationPuppetizing Your Organization
Puppetizing Your OrganizationRobert Nelson
 
Introduction to Puppet Scripting
Introduction to Puppet ScriptingIntroduction to Puppet Scripting
Introduction to Puppet ScriptingAchieve Internet
 
Javascript TDD with Jasmine, Karma, and Gulp
Javascript TDD with Jasmine, Karma, and GulpJavascript TDD with Jasmine, Karma, and Gulp
Javascript TDD with Jasmine, Karma, and GulpAll Things Open
 
Essential applications management with Tiny Puppet
Essential applications management with Tiny PuppetEssential applications management with Tiny Puppet
Essential applications management with Tiny PuppetAlessandro Franceschi
 
Puppet at Pinterest
Puppet at PinterestPuppet at Pinterest
Puppet at PinterestPuppet
 

More Related Content

What's hot

Puppet at GitHub / ChatOps
Puppet at GitHub / ChatOpsPuppet at GitHub / ChatOps
Puppet at GitHub / ChatOpsPuppet
 
Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)
Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)
Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)Tim Bunce
 
Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)DECK36
 
Writing Custom Puppet Types and Providers to Manage Web-Based Applications
Writing Custom Puppet Types and Providers to Manage Web-Based ApplicationsWriting Custom Puppet Types and Providers to Manage Web-Based Applications
Writing Custom Puppet Types and Providers to Manage Web-Based ApplicationsTim Cinel
 
Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...
Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...
Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...Puppet
 
Puppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction KitPuppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction KitAlessandro Franceschi
 
Learning Puppet Chapter 1
Learning Puppet Chapter 1Learning Puppet Chapter 1
Learning Puppet Chapter 1Vishal Biyani
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionJoshua Thijssen
 
Modules of the twenties
Modules of the twentiesModules of the twenties
Modules of the twentiesPuppet
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneSylvain Zimmer
 
Automation with Ansible and Containers
Automation with Ansible and ContainersAutomation with Ansible and Containers
Automation with Ansible and ContainersRodolfo Carvalho
 
Developing IT infrastructures with Puppet
Developing IT infrastructures with PuppetDeveloping IT infrastructures with Puppet
Developing IT infrastructures with PuppetAlessandro Franceschi
 
Puppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLabPuppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLabAlessandro Franceschi
 
Puppet Camp Charlotte 2015: Exporting Resources: There and Back Again
Puppet Camp Charlotte 2015: Exporting Resources: There and Back AgainPuppet Camp Charlotte 2015: Exporting Resources: There and Back Again
Puppet Camp Charlotte 2015: Exporting Resources: There and Back AgainPuppet
 
PyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsPyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsGraham Dumpleton
 
Nodejs Explained with Examples
Nodejs Explained with ExamplesNodejs Explained with Examples
Nodejs Explained with ExamplesGabriele Lana
 
Puppet - Configuration Management Made Eas(ier)
Puppet - Configuration Management Made Eas(ier)Puppet - Configuration Management Made Eas(ier)
Puppet - Configuration Management Made Eas(ier)Aaron Bernstein
 
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat Pôle Systematic Paris-Region
 
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Puppet
 

What's hot (20)

Puppet at GitHub / ChatOps
Puppet at GitHub / ChatOpsPuppet at GitHub / ChatOps
Puppet at GitHub / ChatOps
 
Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)
Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)
Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)
 
Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)
 
Writing Custom Puppet Types and Providers to Manage Web-Based Applications
Writing Custom Puppet Types and Providers to Manage Web-Based ApplicationsWriting Custom Puppet Types and Providers to Manage Web-Based Applications
Writing Custom Puppet Types and Providers to Manage Web-Based Applications
 
Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...
Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...
Scalable Cloud-Native Masterless Puppet, with PuppetDB and Bolt, Craig Watson...
 
Puppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction KitPuppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction Kit
 
Learning Puppet Chapter 1
Learning Puppet Chapter 1Learning Puppet Chapter 1
Learning Puppet Chapter 1
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 Edition
 
Modules of the twenties
Modules of the twentiesModules of the twenties
Modules of the twenties
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
 
Automation with Ansible and Containers
Automation with Ansible and ContainersAutomation with Ansible and Containers
Automation with Ansible and Containers
 
Developing IT infrastructures with Puppet
Developing IT infrastructures with PuppetDeveloping IT infrastructures with Puppet
Developing IT infrastructures with Puppet
 
Celery with python
Celery with pythonCelery with python
Celery with python
 
Puppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLabPuppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLab
 
Puppet Camp Charlotte 2015: Exporting Resources: There and Back Again
Puppet Camp Charlotte 2015: Exporting Resources: There and Back AgainPuppet Camp Charlotte 2015: Exporting Resources: There and Back Again
Puppet Camp Charlotte 2015: Exporting Resources: There and Back Again
 
PyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsPyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web Applications
 
Nodejs Explained with Examples
Nodejs Explained with ExamplesNodejs Explained with Examples
Nodejs Explained with Examples
 
Puppet - Configuration Management Made Eas(ier)
Puppet - Configuration Management Made Eas(ier)Puppet - Configuration Management Made Eas(ier)
Puppet - Configuration Management Made Eas(ier)
 
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
PyParis 2017 / Writing a C Python extension in 2017, Jean-Baptiste Aviat
 
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
 

Similar to Our Puppet Story (Linuxtag 2014)

Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)
Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)
Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)DECK36
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentationSuresh Kumar
 
V mware
V mwareV mware
V mwaredvmug1
 
Puppet Primer, Robbie Jerrom, Solution Architect VMware
Puppet Primer, Robbie Jerrom, Solution Architect VMwarePuppet Primer, Robbie Jerrom, Solution Architect VMware
Puppet Primer, Robbie Jerrom, Solution Architect VMwaresubtitle
 
Ansible: How to Get More Sleep and Require Less Coffee
Ansible: How to Get More Sleep and Require Less CoffeeAnsible: How to Get More Sleep and Require Less Coffee
Ansible: How to Get More Sleep and Require Less CoffeeSarah Z
 
Automating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps ApproachAutomating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps ApproachAkshaya Mahapatra
 
Using Puppet in Small Infrastructures
Using Puppet in Small InfrastructuresUsing Puppet in Small Infrastructures
Using Puppet in Small InfrastructuresRachel Andrew
 
Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Pavel Chunyayev
 
Kubernetes Boston — Custom High Availability of Kubernetes
Kubernetes Boston — Custom High Availability of KubernetesKubernetes Boston — Custom High Availability of Kubernetes
Kubernetes Boston — Custom High Availability of KubernetesMike Splain
 
Workflow story: Theory versus practice in Large Enterprises
Workflow story: Theory versus practice in Large EnterprisesWorkflow story: Theory versus practice in Large Enterprises
Workflow story: Theory versus practice in Large EnterprisesPuppet
 
Workflow story: Theory versus Practice in large enterprises by Marcin Piebiak
Workflow story: Theory versus Practice in large enterprises by Marcin PiebiakWorkflow story: Theory versus Practice in large enterprises by Marcin Piebiak
Workflow story: Theory versus Practice in large enterprises by Marcin PiebiakNETWAYS
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesHiroshi SHIBATA
 
Declarative Infrastructure Tools
Declarative Infrastructure Tools Declarative Infrastructure Tools
Declarative Infrastructure Tools Yulia Shcherbachova
 
Kubernetes - training micro-dragons without getting burnt
Kubernetes -  training micro-dragons without getting burntKubernetes -  training micro-dragons without getting burnt
Kubernetes - training micro-dragons without getting burntAmir Moghimi
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStackPuppet
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackke4qqq
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStackke4qqq
 
Postgres the hardway
Postgres the hardwayPostgres the hardway
Postgres the hardwayDave Pitts
 
Automating Complex Setups with Puppet
Automating Complex Setups with PuppetAutomating Complex Setups with Puppet
Automating Complex Setups with PuppetKris Buytaert
 

Similar to Our Puppet Story (Linuxtag 2014) (20)

Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)
Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)
Our Puppet Story – Patterns and Learnings (sage@guug, March 2014)
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
 
V mware
V mwareV mware
V mware
 
Puppet Primer, Robbie Jerrom, Solution Architect VMware
Puppet Primer, Robbie Jerrom, Solution Architect VMwarePuppet Primer, Robbie Jerrom, Solution Architect VMware
Puppet Primer, Robbie Jerrom, Solution Architect VMware
 
Ansible: How to Get More Sleep and Require Less Coffee
Ansible: How to Get More Sleep and Require Less CoffeeAnsible: How to Get More Sleep and Require Less Coffee
Ansible: How to Get More Sleep and Require Less Coffee
 
Automating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps ApproachAutomating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps Approach
 
Using Puppet in Small Infrastructures
Using Puppet in Small InfrastructuresUsing Puppet in Small Infrastructures
Using Puppet in Small Infrastructures
 
One-Man Ops
One-Man OpsOne-Man Ops
One-Man Ops
 
Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015
 
Kubernetes Boston — Custom High Availability of Kubernetes
Kubernetes Boston — Custom High Availability of KubernetesKubernetes Boston — Custom High Availability of Kubernetes
Kubernetes Boston — Custom High Availability of Kubernetes
 
Workflow story: Theory versus practice in Large Enterprises
Workflow story: Theory versus practice in Large EnterprisesWorkflow story: Theory versus practice in Large Enterprises
Workflow story: Theory versus practice in Large Enterprises
 
Workflow story: Theory versus Practice in large enterprises by Marcin Piebiak
Workflow story: Theory versus Practice in large enterprises by Marcin PiebiakWorkflow story: Theory versus Practice in large enterprises by Marcin Piebiak
Workflow story: Theory versus Practice in large enterprises by Marcin Piebiak
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 Minutes
 
Declarative Infrastructure Tools
Declarative Infrastructure Tools Declarative Infrastructure Tools
Declarative Infrastructure Tools
 
Kubernetes - training micro-dragons without getting burnt
Kubernetes -  training micro-dragons without getting burntKubernetes -  training micro-dragons without getting burnt
Kubernetes - training micro-dragons without getting burnt
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
 
Postgres the hardway
Postgres the hardwayPostgres the hardway
Postgres the hardway
 
Automating Complex Setups with Puppet
Automating Complex Setups with PuppetAutomating Complex Setups with Puppet
Automating Complex Setups with Puppet
 

Recently uploaded

Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Dynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringDynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringMassimo Talia
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfDomotica daVinci
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERNRonnelBaroc
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...shaiyuvasv
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfDomotica daVinci
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...ISPMAIndia
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfDomotica daVinci
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!XfilesPro
 
Q1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IP
Q1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IPQ1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IP
Q1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IPMemory Fabric Forum
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS Chicago
 
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxEvolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxKyle Willson
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVARobert McDermott
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersThousandEyes
 
Zi-Stick UBS Dongle ZIgbee from Aeotec manual
Zi-Stick UBS Dongle ZIgbee from  Aeotec manualZi-Stick UBS Dongle ZIgbee from  Aeotec manual
Zi-Stick UBS Dongle ZIgbee from Aeotec manualDomotica daVinci
 
Manual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveManual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveDomotica daVinci
 
2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch TuesdayIvanti
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Adrian Sanabria
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!KivenRaySarsaba
 
zigbee motion sensor user manual NAS-PD07B2.pdf
zigbee motion sensor user manual NAS-PD07B2.pdfzigbee motion sensor user manual NAS-PD07B2.pdf
zigbee motion sensor user manual NAS-PD07B2.pdfDomotica daVinci
 

Recently uploaded (20)

Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Dynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringDynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineering
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdf
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
 
Q1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IP
Q1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IPQ1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IP
Q1 Memory Fabric Forum: Building Fast and Secure Chips with CXL IP
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user group
 
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxEvolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVA
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for Partners
 
Zi-Stick UBS Dongle ZIgbee from Aeotec manual
Zi-Stick UBS Dongle ZIgbee from  Aeotec manualZi-Stick UBS Dongle ZIgbee from  Aeotec manual
Zi-Stick UBS Dongle ZIgbee from Aeotec manual
 
Manual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveManual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-Wave
 
2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch Tuesday
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!
 
zigbee motion sensor user manual NAS-PD07B2.pdf
zigbee motion sensor user manual NAS-PD07B2.pdfzigbee motion sensor user manual NAS-PD07B2.pdf
zigbee motion sensor user manual NAS-PD07B2.pdf
 

Our Puppet Story (Linuxtag 2014)

  • 1. Our Puppet Story Martin Schütte May 5 2014
  • 2. About DECK36 • Small team of 7 engineers • Longstanding expertise in designing, implementing and operating complex web systems • Developing own data intelligence-focused tools and web services • Offering our expert knowledge in Automation & Operation, Architecture & Engineering, Analytics & Data Logistics
  • 4. “We hired someone. How can we reproduce our dev environment?”
  • 6. Vagrant Configuration tool for VMs and Provisioning. “Local cloud” • Self service • Instant provisioning • Cost efficient • Elastic • Pay per use
  • 7. Vagrant VM Providers: • VirtualBox: “default”, works offline, ressource hungry • Docker: lightweight, requires Linux, good for testing • AWS EC2: remote VMs, good for automation (Jenkins) • 3rd party plugins for KVM, libvirt, ESXI, … Provisioning: • Shell script • Puppet, apply manifest or run agent • Chef, solo or client • Ansible playbooks • Docker containers
  • 8. “Synced folders are too slow.”
  • 9. Synced Folders Shared folders, mounted from host into guest. Options: • VirtualBox slow! • NFS often the best choice • SMB for Windows support • rsync new and promising
  • 10. “But our QA needs many VMs and their machines are slow.”
  • 11. vagrant-aws Vagrant.configure("2") do |config| config.vm.box = "dummy" config.vm.provider :aws do |aws, override| aws.access_key_id = "YOUR KEY" # ... region = "eu-west-1" aws.ami = "ami-20414854" aws.tags = { 'Role' => 'TestVM', 'Net' => 'Devnet' } end end
  • 12. “How can we configure all those VMs?”
  • 14. Puppet • Configuration Management • Declarative: Resources and Dependencies
  • 15. “How should we manage write access for multiple Ops/DevOps?”
  • 16. git workflows • use git! • use git hooks • use per-user environments for easy testing • repos for testing/production
  • 17. git hook: Syntax Check Git pre-commit hook with puppet-lint to syntax check Puppet, ERB templates, YAML files (http://github.com/gini/puppet-git-hooks) Example Output: $ git commit -m 'test' modules/graylog2/templates/server.conf.erb -:5: syntax error, unexpected $undefined ...rd_sha2 = "; _erbout.concat(( @ root_pwd_sha2 ).to_s); _erbo... ... ^ ERB syntax error in modules/graylog2/templates/server.conf.erb
  • 18. environments • per user env + production ⇒ easy testing with puppet agent -t --environment=user • two servers for testing/production Config File Environments: puppet.conf [mschuette] modulepath = $confdir/environments/mschuette/modules manifest = $confdir/environments/mschuette/manifests/site.pp pluginsync = true Directory Environments (Puppet >= 3.5.0): puppet.conf [main] environmentpath = $confdir/environments
  • 20. “But we cannot write and maintain all those modules.”
  • 22. “How do we use inventory and EC2 metadata in Puppet manifests?”
  • 23. Facter Gather information from system. • standard values • extensible via Puppet plugins Example Output: # facter -p architecture => i386 operatingsystem => CentOS operatingsystemrelease => 5.5 ... ipaddress => 172.16.182.129 ...
  • 24. stdlib facts.d • puppetlabs-stdlib reads facts from /etc/facter/facts.d • simple data inputs • e. g. ec2metadata, inventory lookup custom_facts.sh #! /bin/sh which ec2metadata >/dev/null 2>&1 || exit 1 echo "ec2_ami_id=$(ec2metadata --ami-id)" echo "ec2_instance_id=$(ec2metadata --instance-id)" echo "ec2_instance_type=$(ec2metadata --instance-type)" echo "ec2_public_ipv4=$(ec2metadata --public-ipv4)" echo "ec2_public_hostname=$(ec2metadata --public-hostname)"
  • 25. “There has to be a way to split modules and config parameters.”
  • 26. Hiera
  • 27. Hiera • banish top scope variables • use Hiera! • structure with roles & profiles
  • 28. Without Hiera (Puppet 2.x legacy code) node "mydevd+.vagrantup.com" inherits basenode-vagrant { $vmEnv = "development" include sysadmin include ntp if $::fqdn = "mydev01.vagrantup.com" { class { 'vpn': version => latest, ca_crt => '...', usr_crt => '...', usr_key => '...', } } else { class { 'vpn': version => "2.3.2-7~bpo70+1", ca_crt => '...', usr_crt => '...', usr_key => '...', } } # ... }
  • 29. Explicit Hiera Usage $vpn_version = hiera('vpn_version', 'latest') $vpn_ca_crt = hiera('vpn_ca_crt') $vpn_usr_crt = hiera('vpn_usr_crt') $vpn_usr_key = hiera('vpn_usr_key') class { 'vpn': version => $vpn_version, ca_crt => $vpn_ca_crt, usr_crt => $vpn_usr_crt, usr_key => $vpn_usr_key, }
  • 30. Hiera & Puppet 2.x compatibility class vpn($version = hiera('vpn::version', 'present'), $ca_crt = hiera('vpn::ca_crt'), $usr_crt = hiera('vpn::usr_crt'), $usr_key = hiera('vpn::usr_key')) { package { 'openvpn': ensure => $version; } # ... } class { 'vpn': } # or "include vpn"
  • 31. Puppet 3.x with Hiera site.pp hiera_include('include_classes', ['sysadmin']) node default { } profile_vpn.yaml include_classes: - ntp - vpn vpn::version: present vpn::ca_crt: ... vpn::usr_crt: ... vpn::usr_key: ...
  • 32. “Our modules and manifests grow too complex. How can we structure them?”
  • 33. Module Design Pattern: Roles & Profiles ..Resources...... Components: Resource modelling ...... Profiles: Implementation . Roles: Business Logic . Hiera: Data . Classifier from: Craig Dunn, Advanced Puppet Design
  • 34. “What other pitfalls will we encounter?”
  • 35. Puppet Problems • some tasks require two agent runs • apt-get upgrade and package dependencies • version mismatch between apt (or yum) and package • scoping and namespaces • exec is the new eval
  • 36. Namespace problems # this does not work, cf. #PUP-1073 package { 'memcached': ensure => present, provider => apt, } package { 'memcached': ensure => present, provider => gem, }
  • 37. exec tricks Both source and solution to a great many problems. You can do (and break) everything with exec and a shell script. But of course you should not.
  • 38. exec tricks # pkg name collision exec { 'npm install -g less': creates => '/usr/lib/node_modules/npm/node_modules/less', } # abuse puppet as cron, and hide the change exec { 'zabbix_update.sh': command => 'false', onlyif => "/opt/zabbix_update.sh $api_url && false", logoutput => on_failure, }
  • 39. “How can we monitor Puppet changes?”
  • 43. git hook: E-Mail Notification Git post-receive hook to notify team on push (http://git.kernel.org/cgit/git/git.git/tree/contrib/hooks/ post-receive-email?id=HEAD) Example E-Mail: - Log ---------------------------------------------- commit 5df04ee883b8de8a37bf0ac97eec068cd1f3a414 Author: N. N. <n.n@deck36.de> Date: Tue Jan 7 08:57:17 2014 +0000 fixed path to csync2 executable ---------------------------------------------------- Summary of changes: modules/user/files/etc/sudoers.d/support | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
  • 44. “How do we coordinate a cluster restart?”
  • 45. MCollective “multissh deluxe” AMQP client/server framework to • orchestrate actions • control puppet agents • run commands • query resources • … Alternatives: Ansible, serf, …
  • 46. “Why do we still manually configure DNS and monitoring?”
  • 47. Hooks to other systems • include in provisioning process • provide normative data as facts • register or update DNS name → e. g. Route 53 • register or update host in Zabbix monitoring → API
  • 48. Questions? class presentation { package { 'questions': ensure => 'answered', } } Links: • Vagrant • Puppet Language: Visual Index • Puppet Type Reference • Puppet Ask