Doing It Wrong with Puppet -

3,994 views
3,593 views

Published on

Presented at Puppet Camp Berlin 2014: "Doing It Wrong with Puppet" by Felix Frank, MPeXnetworks

Published in: Technology
1 Comment
4 Likes
Statistics
Notes
No Downloads
Views
Total views
3,994
On SlideShare
0
From Embeds
0
Number of Embeds
134
Actions
Shares
0
Downloads
33
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

Doing It Wrong with Puppet -

  1. 1. 1 / 33 Doing It Wrong with Puppet A small collection of anti-patterns, pitfalls and bad practices Felix Frank PuppetCamp Berlin 2014 April 11, 2014
  2. 2. 2 / 33 Bio Felix Frank 2004 – 2009 sysadmin and FOSS dev at DESY 2009 CS diploma at Tech Uni since 2009 Puppeteer at – we are a company that manages complex IT systems and services for business customers hosts a sizable server fleet relies on Puppet and git consequently
  3. 3. 3 / 33 Agenda Use boolean facts Expect C-like values for parameters Make excessive use of “if defined()” Use large numbers of execs Rely on dynamic scoping
  4. 4. 4 / 33 Use boolean values for facts On wanton ambiguity in your tool chain
  5. 5. 5 / 33 False is relative Consider this fact is virtual: true or false Broken manifest I if ! $::is virtual { include hardware monitoring } Broken manifest II if $::is virtual != false { include hardware monitoring } Stupid manifest if $::is virtual != "false" { include hardware monitoring }
  6. 6. 6 / 33 Some background True values in the puppet DSL true or any non-empty string Limitation in facter 1.x master agent fact value fact code ruby as String Correct way to implement such facts return the empty string for false
  7. 7. 7 / 33 It’s gonna be the future soon Facter 2 will allow boolean and other values widespread adoption quite far off still
  8. 8. 8 / 33 Up next Use boolean facts Expect C-like values for parameters Make excessive use of “if defined()” Use large numbers of execs Rely on dynamic scoping
  9. 9. 9 / 33 Expect C-like values for parameters Or: treating Puppet like a scripting language pt. 1
  10. 10. 10 / 33 The Perl trap The puppet user base . . . comprises lots of admins with *NIX backgrounds . . . also writes plenty of Shell and Perl scripts (also C) . . . and these languages have no pure boolean values True values e.g. in Puppet “foo”, any array, typically true in Perl “foo”, non-empty array, typically 1 False values e.g. in Puppet empty string, undef, false in Perl empty string/array/hash, typically 0
  11. 11. 11 / 33 Building confusing modules define server module($enabled=0) { $dir = "/etc/..." file { "$dir/$title.conf": ...} if $enabled == 1 { ...# take some action } } inevitable WTF moment server module { "foo": enabled => true } handled by documentation at best, and likely not until after the fact
  12. 12. 12 / 33 Up next Use boolean facts Expect C-like values for parameters Make excessive use of “if defined()” Use large numbers of execs Rely on dynamic scoping
  13. 13. 13 / 33 Make excessive use of “if defined()” A tale of borderline non-determinism
  14. 14. 14 / 33 A common problem Several modules will sometimes have to manage a common set or resources a subtree of /etc of mutual interest a package for required functionality etc. The naive implementation won’t work because Puppet doesn’t allow multiple declaration of the same resource class php { ... package { "imagemagick": ensure => present } } class tomcat { ... package { "imagemagick": ensure => present } }
  15. 15. 15 / 33 A common workaround Protect declarations with a function call class php { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => present } } } class tomcat { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => present } } }
  16. 16. 16 / 33 A possible issue with that There is no protection against contradiction class php { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => present } } } class graphicsmagick { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => absent } } }
  17. 17. 17 / 33 A more likely scenario It’s easy to lose metaparameters class php { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => present, require => File[...] } } } class tomcat { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => present, notify => Exec[...] } } }
  18. 18. 18 / 33 By the way. . . The latter issue can be worked around class php { ... if !defined(Package[imagemagick]) { package { "imagemagick": ensure => present, require => File[...] } } else { Package<| title == "imagemagick" |> { require +> File[...] } } }
  19. 19. 19 / 33 A word about stdlib puppetlabs-stdlib, a collection of helpful parser functions In theory, ensure resource() solves this more cleanly class php { ensure resource( ‘package’, ‘imagemagick’, { ensure => present } ) } avoids conflicts for basic properties more expressive power It cannot solve the whole problem though issue with metaparameters remains pertains to possible additional properties as well only slightly superior to if defined()
  20. 20. 20 / 33 The ideal(ized) solution Wrapper classes for shared dependencies class php { include imagemagick } class tomcat { include imagemagick } still won’t allow the easy handling of metaparameters etc. but you won’t even be tempted to try just require/subscribe/notify/. . . the class contradictions are not addressed but there is no sensible way to do that How is this better then? the manifest has clear, nonambiguous semantics parse order dependencies avoided, see final slides (virtual resources work too, but less flexibly)
  21. 21. 21 / 33 Up next Use boolean facts Expect C-like values for parameters Make excessive use of “if defined()” Use large numbers of execs Rely on dynamic scoping
  22. 22. 22 / 33 Use large numbers of execs Or: treating Puppet like a scripting language pt. 2
  23. 23. 23 / 33 Implementing a HOWTO in a manifest Setting up software often comprises editing files running scripts and programs . . . and often both of them in a set and mingled order it can be tempting to translate this verbatim exec { "curl http://... >/tmp/...": creates => "..." } -> exec { "unzip /tmp/...": creates => "/usr/local/..." } -> file { "/usr/local/.../etc/...": content => template(...) } -> exec { "/usr/local/...": ... } -> ...
  24. 24. 24 / 33 So what? Problems with this approach (likely among others) contradicts Puppet’s idea of resources the catalog becomes complex with items and relationships leads to plentiful error output in case of problems A more maintainable pattern consists of a monolithic, robust script to perform all setup either templated or with a managed config file a single exec resource to invoke it with precise condition(s) for when to run or better yet: create a deb or rpm package Also – a quick word on refreshonly nice antipattern: use it to run script after retrieving it prone for false positives and lost events
  25. 25. 25 / 33 So remember A small mnemonic
  26. 26. 26 / 33 Up next Use boolean facts Expect C-like values for parameters Make excessive use of “if defined()” Use large numbers of execs Rely on dynamic scoping
  27. 27. 27 / 33 Rely on dynamic scoping Or: how to jumble up your own manifest’s opinions . . . which is another bout with nondeterminism
  28. 28. 28 / 33 Brief review Dynamic scoping in Puppet 2.x mainly for variable values class foo { $limited = true include bar } class bar { if $limited { ... } } in Puppet 3.x only for resource defaults class foo { File { ensure => present } include bar }
  29. 29. 29 / 33 The jumble role::webserver apache tcpserver sysctl apache tcpserver sysctlsysctl include include include File { mode => 644 } thread optimization include include File { mode => 640 } thread optimization which default is in effect for sysctl? either, depending on parse order
  30. 30. 30 / 33 Mitigation? Idea: just take care that the parse order is correct only possible in very confined class structures scopes are generally too complex scopes of classes late in the chain change through unexpected factors
  31. 31. 31 / 33 Mixing things up scopes of classes late in the chain change through inclusion of more classes removal of one or more classes refactoring of manifests
  32. 32. 32 / 33 Conclusion Avoid! parameters and Hiera will get you there much safer You may want to move away from dynamic scopes anyway they will likely get deprecated and removed
  33. 33. 33 / 33 Thanks for your attention Image sources https://www.pinterest.com/pin/418553359088246576/ http://www.kulfoto.com/funny-pictures/17395/its-called- wireless-tech-and-its-the-future http://www.cacbasketball.com/b2-5v5-unification-finals- uhhh-ditka/ http://www.someecards.com/usercards/ viewcard/MjAxMy00MzdlNjAzZjE2MWRkMjk0 http://www.marketingpilgrim.com/2013/08/google-glass- update-like-having-an-admin-assistant-on-your- shoulder.html http://www.aboutbradsugars.com/tag/executive-coaching/ http://themetapicture.com/schrodingers-cat/ http://www.mrlovenstein.com/comic/50 http://funny-pics.co/photo/funny-cat-cheering-up-dog/
  34. 34. 34 / 33 We are hiring Always looking for techs who know their way around Puppet (or would like to) further the development of our homegrown infrastructure and tools will implement more technologies in our management ecosystem Visit us http://mpexnetworks.de/ueber-uns/jobs.html jobs@mpexnetworks.de
  35. 35. 35 / 33 Bonus content!
  36. 36. 36 / 33 Preferring new style class declaration the good thing about classes: they are singletons a class can be declared an arbitrary number of times Class parameterization a class with parameters must be one of a kind multiple declarations with different parameters just as contradictory as with resources (or more so) Additional fun declaration using include implies all parameters use their respective default value does not mix with new style class { } declaration mixing is allowed but only with all include statements before the class { } more parse order dependencies (yay!)

×