Manageable Puppet
infrastructure
~April 2014 edition~
PuppetCampBerlin
Ger Apeldoorn - http://puppetspecialist.nl
1 / 44
Freelance PuppetConsultant
TrainerforPuppetLabs Benelux
Who's this?
2 / 44
Scope
Also... why this talk?
3 / 44
Common
pitfalls
4 / 44
Pitfalls
Cause & effect
Pitfalls
Lots of Workarounds
Unmaintainable codebase
Collaboration difficulties
5 / 44
Pitfalls
Cause & effect
Quick Wins
Fix your codebase!
Quick wins:
Move data to Hiera
Implement Code Review
Use Puppet-lint...
A Manageable Design
April 2014 edition
7 / 44
Requirements
Whadda we need
8 / 44
Our environment should be:
Easyto Use
Easyto Comprehend
Easyto Update
and...Safe
9 / 44
This stuff
isn't exactly
easy
10 / 44
But we cán make it safe and
manageable
11 / 44
Requirements
Easyto:
Use
Comprehend
Update
Safe
Safe
Useenvironments to test everything
Createahugetesting environment
Use...
Requirements
Easyto:
Use
Comprehend
Update
Safe
Manageable
Manageable
Keepaconsistent modulestructure
Using roles for abst...
Domains
Server Roles
All things data
Deployment &Workflow
14 / 44
Overview
Software Components
15 / 44
Software Components
Puppet Enterpriseor TheForeman
Hieraandhiera-eyaml (HierarchicalDatalookup)
Gerrit (Codereview system)...
Domain#1:
Server Roles
17 / 44
Alayer ofabstraction
18 / 44
How to do it?
Createroles module
root@puppet# puppet module generate gerapeldoorn-role
Createabase-roleto cover generic se...
How to do it? -Cont'd-
Put all requiredresources intheclasses
# modules/role/manifests/app.pp:
class role::app {
include a...
Domain#2:
All things Data
21 / 44
Hiera
Hierarchical data lookup tool
22 / 44
ConfiguredHierarchy:
#/etc/puppet/hiera.yaml:
:hierarchy:
- "%{::clientcert}"
- "%{::environment}"
- common
Node app01.aut...
Types of Hieradata
Regular values
# hiera/app01.autiplan.com.yaml
---
examplekey: value
24 / 44
Types of Hieradata
Arrays
# hiera/app01.autiplan.com.yaml
---
array: [ item1, item2, item3 ]
otherarray:
- item1
- item2
-...
Types of Hieradata
Hashes
# hiera/app01.autiplan.com.yaml
---
hash:
key1: value
key2: value
26 / 44
Types of Hieradata
Combinations
# hiera/app01.autiplan.com.yaml
---
hash:
key1: value
key2: value
key3:
- arrayvalue1
- ar...
Hiera-relatedfunctions
...and what to use them for
28 / 44
Whatdoes itdo?
Retrieves the first-found value in the
hierarchy. (top-down)
Whatto use itfor?
Basic variable-lookup.
Very ...
Whatdoes itdo?
Retrieves an array or hash value
in the hierarchy, concatinates all
found results
Whatto use itfor?
Combini...
Whatdoes itdo?
Includes all classes listed in the
array that is loaded from Hiera.
Takes elements from ALL
hierarchy level...
Whatdoes itdo?
Generates resources from a
HASH.
Whatto use itfor?
Generate any resource based on
data from Hiera.
Can also...
Databindings
Auto-loading of Hiera data for parameterized classes.
33 / 44
Whatdoes itdo?
Automatically loads class
parameters from Hiera.
Whatto use itfor?
Specify all class parameters in
Hiera.
U...
Putting it all together
Anything node-specific should be in Hiera!
35 / 44
APuppet Run:What calls what?
36 / 44
Domain#3:
Deployment & Workflow
37 / 44
Environments
Keeping the environmentalists happy
38 / 44
Environments
What is anenvironment?
Seperate modulepaths/site.pp.
Common environments: development, testing, production.
N...
Demo!
40 / 44
R10koverview
41 / 44
Final remarks
Keeppublic modules as-is,wherever possible
Create wrapper classes in company-module.
Create fork if needed, ...
Questions?
43 / 44
Freelance PuppetConsultant
TrainerforPuppetLabs Benelux
Thank you!
A howto of setting up this environment (and the workflo...
Upcoming SlideShare
Loading in …5
×

Puppet Camp Berlin 2014: Manageable puppet infrastructure

3,213 views

Published on

"Manageable puppet infrastructure" presented by Ger Apeldoorn, at Puppet Camp Berlin 2014

Published in: Software, Technology

Puppet Camp Berlin 2014: Manageable puppet infrastructure

  1. 1. Manageable Puppet infrastructure ~April 2014 edition~ PuppetCampBerlin Ger Apeldoorn - http://puppetspecialist.nl 1 / 44
  2. 2. Freelance PuppetConsultant TrainerforPuppetLabs Benelux Who's this? 2 / 44
  3. 3. Scope Also... why this talk? 3 / 44
  4. 4. Common pitfalls 4 / 44
  5. 5. Pitfalls Cause & effect Pitfalls Lots of Workarounds Unmaintainable codebase Collaboration difficulties 5 / 44
  6. 6. Pitfalls Cause & effect Quick Wins Fix your codebase! Quick wins: Move data to Hiera Implement Code Review Use Puppet-lint in a git-hook REFACTOR CONSTANTLY 6 / 44
  7. 7. A Manageable Design April 2014 edition 7 / 44
  8. 8. Requirements Whadda we need 8 / 44
  9. 9. Our environment should be: Easyto Use Easyto Comprehend Easyto Update and...Safe 9 / 44
  10. 10. This stuff isn't exactly easy 10 / 44
  11. 11. But we cán make it safe and manageable 11 / 44
  12. 12. Requirements Easyto: Use Comprehend Update Safe Safe Useenvironments to test everything Createahugetesting environment UseGit to promoteyour code 12 / 44
  13. 13. Requirements Easyto: Use Comprehend Update Safe Manageable Manageable Keepaconsistent modulestructure Using roles for abstraction Facilitatecollaboration 13 / 44
  14. 14. Domains Server Roles All things data Deployment &Workflow 14 / 44
  15. 15. Overview Software Components 15 / 44
  16. 16. Software Components Puppet Enterpriseor TheForeman Hieraandhiera-eyaml (HierarchicalDatalookup) Gerrit (Codereview system) Git (what else?) GitFlow, adaptedversionforGerrit R10K (Environment deployment tool) 16 / 44
  17. 17. Domain#1: Server Roles 17 / 44
  18. 18. Alayer ofabstraction 18 / 44
  19. 19. How to do it? Createroles module root@puppet# puppet module generate gerapeldoorn-role Createabase-roleto cover generic settings # modules/role/manifests/base.pp: class role::base { include users include ssh include motd ... 19 / 44
  20. 20. How to do it? -Cont'd- Put all requiredresources intheclasses # modules/role/manifests/app.pp: class role::app { include apache include tomcat apache::virtualhost { 'default': ... Includeroleinnodedefinition # site.pp: node 'app01.autiplan.com' { include role::base include role::app } 20 / 44
  21. 21. Domain#2: All things Data 21 / 44
  22. 22. Hiera Hierarchical data lookup tool 22 / 44
  23. 23. ConfiguredHierarchy: #/etc/puppet/hiera.yaml: :hierarchy: - "%{::clientcert}" - "%{::environment}" - common Node app01.autiplan.com: environment: testing Hieradata # hiera/app01.autiplan.com.yaml --- examplekey: value for app01.autiplan.com # hiera/testing.yaml --- examplekey: value for nodes in testing environment # hiera/common.yaml --- examplekey: value for all nodes It's all about Hierarchy What will bein$test? $test = hiera('examplekey') 23 / 44
  24. 24. Types of Hieradata Regular values # hiera/app01.autiplan.com.yaml --- examplekey: value 24 / 44
  25. 25. Types of Hieradata Arrays # hiera/app01.autiplan.com.yaml --- array: [ item1, item2, item3 ] otherarray: - item1 - item2 - item3 Note: Never use tabs in Hiera files! 25 / 44
  26. 26. Types of Hieradata Hashes # hiera/app01.autiplan.com.yaml --- hash: key1: value key2: value 26 / 44
  27. 27. Types of Hieradata Combinations # hiera/app01.autiplan.com.yaml --- hash: key1: value key2: value key3: - arrayvalue1 - arrayvalue2 key4: subhashkey1: value subhashkey2: value 27 / 44
  28. 28. Hiera-relatedfunctions ...and what to use them for 28 / 44
  29. 29. Whatdoes itdo? Retrieves the first-found value in the hierarchy. (top-down) Whatto use itfor? Basic variable-lookup. Very easy to create exceptions! Howto use it? $smarthost = hiera('smarthost') ExampleHieradata # hiera/mail.autiplan.com.yaml --- smarthost: smtp.myprovider.nl # hiera/testing.yaml --- smarthost: testsmtp.autiplan.com # hiera/common.yaml --- smarthost: mail.autiplan.com hiera('key' [,default_value]) 29 / 44
  30. 30. Whatdoes itdo? Retrieves an array or hash value in the hierarchy, concatinates all found results Whatto use itfor? Combining data from all hierarchy levels. Howto use it? $users = hiera_array('users') ExampleHieradata # hiera/app01.autiplan.com.yaml --- users: [ 'user1', 'user2' ] # hiera/testing.yaml --- users: [ 'testuser' ] # hiera/common.yaml --- users: [ 'user3', 'user4' ] hiera_array('key' [,default_value]) (andhiera_hash) 30 / 44
  31. 31. Whatdoes itdo? Includes all classes listed in the array that is loaded from Hiera. Takes elements from ALL hierarchy levels. Whatto use itfor? Lightweight ENC. Put all classes / roles in Hiera. Howto use it? node default { hiera_include('roles') } ExampleHieradata # hiera/web01.autiplan.com.yaml --- roles: - role::web # hiera/common.yaml --- roles: - role::base hiera_include('classes') 31 / 44
  32. 32. Whatdoes itdo? Generates resources from a HASH. Whatto use itfor? Generate any resource based on data from Hiera. Can also be used with hiera_hash to create resources from all levels! Howto use it? create_resources ('apache::vhost', hiera('vhosts', {})) ExampleHieradata # hiera/web01.autiplan.com.yaml --- vhosts: autiplan.com: alias: www.autiplan.com autiplan.dk: alias: www.autiplan.dk docroot: /var/www/html/autiplan.dk autiplan.nl: alias: www.autiplan.nl cdn.autiplan.com: port: 81 docroot: /var/www/html/cdn create_resources('type',HASH[,default_values]) 32 / 44
  33. 33. Databindings Auto-loading of Hiera data for parameterized classes. 33 / 44
  34. 34. Whatdoes itdo? Automatically loads class parameters from Hiera. Whatto use itfor? Specify all class parameters in Hiera. Use all hierarchical benefits for class parameters. Simplify the use of parameterized classes. Howto use it? include mysql::server ExampleHieradata # hiera/web01.autiplan.com.yaml --- mysql::server::root_password: m0ars3cr3t # hiera/common.yaml --- mysql::server::root_password: t0ps3cr3t mysql::server::package_name: mysql-server mysql::server::restart: true Data bindings 34 / 44
  35. 35. Putting it all together Anything node-specific should be in Hiera! 35 / 44
  36. 36. APuppet Run:What calls what? 36 / 44
  37. 37. Domain#3: Deployment & Workflow 37 / 44
  38. 38. Environments Keeping the environmentalists happy 38 / 44
  39. 39. Environments What is anenvironment? Seperate modulepaths/site.pp. Common environments: development, testing, production. Nodes request a specific environment. Why? Essential to prevent mistakes. NEVER edit code in production! The workflow helps us to 'promote' our code to production. 39 / 44
  40. 40. Demo! 40 / 44
  41. 41. R10koverview 41 / 44
  42. 42. Final remarks Keeppublic modules as-is,wherever possible Create wrapper classes in company-module. Create fork if needed, submit pull request for fixes. Add forked module (gitrepo) to Puppetfile. Thinkahead Always try to anticipate future applications. If it feels overly complicated, yer doin it wrong. Refactor! 42 / 44
  43. 43. Questions? 43 / 44
  44. 44. Freelance PuppetConsultant TrainerforPuppetLabs Benelux Thank you! A howto of setting up this environment (and the workflow!) is available on my blog: http://puppetspecialist.nl/mpi 44 / 44

×