Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PuppetConf 2016: The Future of Testing Puppet Code – Gareth Rushgrove, Puppet

190 views

Published on

Here are the slides from Gareth Rushgrove's presentation called The Future of Testing Puppet Code. Watch the videos at https://www.youtube.com/playlist?list=PLV86BgbREluVjwwt-9UL8u2Uy8xnzpIqa

Published in: Technology
  • Be the first to comment

  • Be the first to like this

PuppetConf 2016: The Future of Testing Puppet Code – Gareth Rushgrove, Puppet

  1. 1. (without introducing more risk) The Future of Testing Puppet Code Puppet Gareth Rushgrove Who, What, Why, How
  2. 2. (without introducing more risk) @garethr
  3. 3. (without introducing more risk) Gareth Rushgrove
  4. 4. (without introducing more risk) This Talk What we’ll cover
  5. 5. - Why test Puppet anyway - Existing Puppet testing tools - Different users and usecases - Future challenges Gareth Rushgrove
  6. 6. Mainly a talk about problems, rather than solutions Gareth Rushgrove
  7. 7. Not a talk about how to test Puppet code. Lot of existing content on that subject Gareth Rushgrove
  8. 8. (without introducing more risk) Gareth Rushgrove PuppetConf 2014
  9. 9. (without introducing more risk) Gareth Rushgrove Survey of 300 Puppet users
  10. 10. (without introducing more risk) Gareth Rushgrove
  11. 11. (without introducing more risk) Why Test Puppet And a little history
  12. 12. Gareth Rushgrove
  13. 13. - You have lots of Puppet code - You’re targeting a wide range of OS’s - The Puppet language has more features Gareth Rushgrove
  14. 14. (without introducing more risk) Gareth Rushgrove Lots of code But how much is lots
  15. 15. Note that this just represents public code on GitHub. Lots of private Puppet code out there too Gareth Rushgrove
  16. 16. (without introducing more risk) Gareth Rushgrove How many .pp files on GitHub? SELECT COUNT(*) FROM [puppet.puppet_files] -- 489725
  17. 17. (without introducing more risk) Gareth Rushgrove How many lines of Puppet code? SELECT COUNT(line) total_lines FROM ( SELECT SPLIT(content, 'n') AS line FROM [puppet.puppet_content] ) -- 6697099
  18. 18. (without introducing more risk) Gareth Rushgrove How big are .pp files? SELECT QUANTILES(total_lines, 10) AS q FROM ( SELECT COUNT(line) total_lines FROM ( SELECT SPLIT(content, 'n') AS line, id FROM [puppet.puppet_content] ) GROUP BY id )
  19. 19. 50% of Puppet files are 20 lines or less. 90% of Puppet files are 100 lines or less Gareth Rushgrove
  20. 20. The largest Puppet file public on GitHub has 25,238 lines Gareth Rushgrove
  21. 21. (without introducing more risk) Gareth Rushgrove Mixed environments So many platforms
  22. 22. Gareth Rushgrove
  23. 23. A third of respondents run more than one OS on there development machine Gareth Rushgrove
  24. 24. Gareth Rushgrove
  25. 25. 60% of respondents run more than one OS in production Gareth Rushgrove
  26. 26. (without introducing more risk) Gareth Rushgrove The Puppet language Types, lambdas, iterators, functions
  27. 27. Gareth Rushgrove
  28. 28. (without introducing more risk) Existing Tools Providing some context
  29. 29. Gareth Rushgrove
  30. 30. (without introducing more risk) Core tools Common starting points Gareth Rushgrove
  31. 31. (without introducing more risk) Gareth Rushgrove puppet-lint
  32. 32. (without introducing more risk) Gareth Rushgrove Style conformance $ puppet-lint /etc/puppet/modules foo/manifests/bar.pp - ERROR: trailing whitespace found on li apache/manifests/server.pp - WARNING: variable not enclosed i ...
  33. 33. (without introducing more risk) Gareth Rushgrove rspec-puppet
  34. 34. (without introducing more risk) Gareth Rushgrove Unit tests for Puppet it { is_expected.to compile } it { is_expected.to contain_class('db') } it do is_expected.to contain_package('mysql-server') .with_ensure('present') end it { is_expected.to have_exec_resource_count(0) }
  35. 35. (without introducing more risk) Gareth Rushgrove puppet-syntax
  36. 36. (without introducing more risk) Gareth Rushgrove Popular Puppet gems SELECT package, COUNT(*) count FROM ( SELECT REGEXP_EXTRACT(line, r'gem ['|"](.*)['|"]') packag FROM ( SELECT SPLIT(content, 'n') line, id FROM [puppet.gemfile_contents] WHERE content CONTAINS 'gem' ) GROUP BY package, id ) GROUP BY 1 ORDER BY count DESC LIMIT 30;
  37. 37. (without introducing more risk) Gareth Rushgrove Popular gem results puppetlabs_spec_helper 1157 puppet 1142 rake 1135 puppet-lint 768 beaker-rspec 610 rspec-puppet 576 metadata-json-lint 566 beaker 440 puppet-blacksmith 425 serverspec 359
  38. 38. (without introducing more risk) Gareth Rushgrove puppetlabs_spec_helper
  39. 39. (without introducing more risk) Gareth Rushgrove Advanced tools Next steps
  40. 40. (without introducing more risk) Gareth Rushgrove beaker-rspec
  41. 41. (without introducing more risk) Gareth Rushgrove Serverspec
  42. 42. (without introducing more risk) Gareth Rushgrove rubocop
  43. 43. (without introducing more risk) Gareth Rushgrove Other tools Alternative approaches
  44. 44. (without introducing more risk) Gareth Rushgrove puppet-catalog-test
  45. 45. (without introducing more risk) Gareth Rushgrove Test catalogue compilation $ puppet-catalog-test -m modules -M site.pp [INFO] Using puppet 3.0.2 [PASSED] foo (compile time: 0.168182 seconds) [PASSED] default (compile time: 0.003451 seconds) ---------------------------------------- Compiled 2 catalogs in 0.1717 seconds (avg: 0.0858 seconds)
  46. 46. (without introducing more risk) Gareth Rushgrove Test Kitchen
  47. 47. (without introducing more risk) Different Users And different usecases
  48. 48. Gareth Rushgrove
  49. 49. Half of respondents regularly write tests for Puppet code Gareth Rushgrove
  50. 50. A quarter of respondents never write tests for Puppet code Gareth Rushgrove
  51. 51. The problem with a tool-centric world view Gareth Rushgrove
  52. 52. Understanding user needs means starting with users Gareth Rushgrove
  53. 53. Gareth Rushgrove - The Vox Pupuli folks - Lots of Puppet employees - You in-house Puppet expert - Custom functions The module authors
  54. 54. The regular Puppet users Gareth Rushgrove - Roles and profiles - Using Puppet weekly or more - PuppetDB, Puppet Server - Custom facts
  55. 55. The occasional Puppet user Gareth Rushgrove - Using Puppet as needed - Basic Puppet code - Hieradata
  56. 56. Today different types of Puppet users have a very different experience Gareth Rushgrove
  57. 57. Gareth Rushgrove
  58. 58. For those with no Ruby experience the average was 2 out of 5 Gareth Rushgrove
  59. 59. For those with a little Ruby experience the average was still only 2.5 out of 5 Gareth Rushgrove
  60. 60. Of the people who love the testing experience 33% are self-described power users of both Puppet and Ruby Gareth Rushgrove
  61. 61. Gareth Rushgrove
  62. 62. But more than 50% of respondents have little or no Ruby experience Gareth Rushgrove
  63. 63. (without introducing more risk) Challenges What we need for the future
  64. 64. (without introducing more risk) Gareth Rushgrove Improved getting started experience Helping a new generation of users
  65. 65. - Getting started testing documentation - Single entry point for users - Opinionated workflow Gareth Rushgrove
  66. 66. (without introducing more risk) Gareth Rushgrove Adopting acceptance testing Still too hard
  67. 67. 27 of the 67 Serverspec users are not using either Beaker or Test Kitchen. Gareth Rushgrove
  68. 68. Only 5 of the Beaker (without rspec) users are not also using beaker-rspec Gareth Rushgrove
  69. 69. 3 of the Test Kitchen users are also using some flavour of Beaker Gareth Rushgrove
  70. 70. So only 26% to 35% of respondents are using an acceptance testing tool with there Puppet code Gareth Rushgrove
  71. 71. (without introducing more risk) Gareth Rushgrove Experience matters Consistence interfaces
  72. 72. (without introducing more risk) Gareth Rushgrove Inconsistencies $ puppet-lint vs rake lint vs rake validate $ rake syntax vs puppet parser validate $ rspec vs rake spec $ BEAKER_destroy=no rake beaker vs beaker
  73. 73. We have a language style guide. What about other Puppet user interfaces? Gareth Rushgrove
  74. 74. (without introducing more risk) Gareth Rushgrove Ruby heritage Ruby as a development environment
  75. 75. 60% of respondents described themselves as Puppet power users Gareth Rushgrove
  76. 76. Only 12% of respondents described themselves as Ruby power users. A third of those work for Puppet! Gareth Rushgrove
  77. 77. - More help with using Ruby for testing? - Support for testing in other languages? - Native Puppet support for testing? - Containerised testing abstraction? Gareth Rushgrove
  78. 78. (without introducing more risk) Gareth Rushgrove jordan/spec
  79. 79. (without introducing more risk) Gareth Rushgrove Test Puppet with Puppet file { '/tmp/test': ensure => present, content => 'The file content', } assertion { 'that the file has the correct contents': subject => File['/tmp/test'], attribute => 'content', expectation => 'The file content', }
  80. 80. (without introducing more risk) Conclusions Where to go from here
  81. 81. Thanks to the community we have a collection of powerful testing tools in the Puppet ecosystem Gareth Rushgrove
  82. 82. But we don’t have consistent user interfaces or the opinionated workflows useful for beginners Gareth Rushgrove
  83. 83. Testing shouldn’t just be a power user feature Gareth Rushgrove
  84. 84. As a community we should talk about design more often Gareth Rushgrove
  85. 85. (without introducing more risk) Gareth Rushgrove https://goo.gl/forms/nchOp5hYdLLgmJ8z1
  86. 86. Gareth Rushgrove
  87. 87. (without introducing more risk) Questions? And thanks for listening

×