This document discusses Puppet::Tidy, a tool for formatting Puppet manifests. Puppet::Tidy works by parsing manifests line-by-line and applying transformations to lines that match certain criteria in order to standardize formatting. The talk describes how Puppet::Tidy currently works, examples of checks it performs, and ideas for future improvements such as incorporating more semantic knowledge and using a full parser.

1. Clean manifests with Puppet::Tidy
Jasper Lievisse Adriaanse
Engineering team, m:tier
Puppet Camp 2013 Gent
Jan. 31 - Feb. 1, 2013

2. Intro
m:tier
Puppet::Tidy
Future ideas
2 of 1

3. Who?
• Somewhat involved in open source...
◦ OpenBSD committer since 2006
◦ GNOME committer since 2011
◦ 35+ public repositories on GitHub
3 of 1

4. Who?
• Somewhat involved in open source...
◦ OpenBSD committer since 2006
◦ GNOME committer since 2011
◦ 35+ public repositories on GitHub
• Involved in m:tier since it’s founding in 2008
◦ Started using Puppet in 2009
3 of 1

5. Who?
cont.
Also wrote yasnippet-puppet-mode for Emacs
file<TAB> expands to:
file { "name":
owner => owner,
group => group,
mode => mode,
ensure => ensure,
require => require,
content => content,
source => source;
}
See: http://jasper.la/emacs.html
4 of 1

6. Why Puppet::Tidy?
• puppet-lint
• Easier to understand/expand/fix
5 of 1

7. Intro
m:tier
Puppet::Tidy
Future ideas
6 of 1

8. Puppet in m:tier
• One git repository
7 of 1

9. Puppet in m:tier
• One git repository
• ∼ 6k lines of Puppet code
7 of 1

10. Puppet in m:tier
• One git repository
• ∼ 6k lines of Puppet code
• Upto “Blue chip” customers
7 of 1

11. Puppet in m:tier
• One git repository
• ∼ 6k lines of Puppet code
• Upto “Blue chip” customers
• Three continents
7 of 1

12. Puppet in m:tier
• One git repository
• ∼ 6k lines of Puppet code
• Upto “Blue chip” customers
• Three continents
• mtier-puppet
7 of 1

13. Puppet in m:tier
cont.
(Semi-)automated bootstrap of OpenBSD laptops/servers
• LDAP
• Kerberos
• Users
• Packages
• Security updates
• .plocal
8 of 1

14. Puppet in m:tier
cont.
For more details:
puppetlabs.com/blog/
guest-post-a-puffy-in-the-corporate-aquarium-the-sequel/
9 of 1

15. Intro
m:tier
Puppet::Tidy
Future ideas
10 of 1

16. What is Puppet::Tidy?
• Formal definition:
◦ A re-formatter for Puppet manifests, working on syntactic level.
11 of 1

17. What is Puppet::Tidy?
• Formal definition:
◦ A re-formatter for Puppet manifests, working on syntactic level.
• Informal definition:
◦ A glorified bunch of regular expressions which make your Puppet code
look nice.
11 of 1

18. How does it work?
• Input is read line-by-line
12 of 1

19. How does it work?
• Input is read line-by-line
• One-pass transformation checks
12 of 1

20. How does it work?
• Input is read line-by-line
• One-pass transformation checks
◦ If the line matches criteria, it’s transformed
12 of 1

21. How does it work?
• Input is read line-by-line
• One-pass transformation checks
◦ If the line matches criteria, it’s transformed
• Output written to file, or reference passed back
12 of 1

22. Current checks
• Most common “errors”:
◦ expand tabs
13 of 1

23. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
13 of 1

24. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
◦ four digit mode
13 of 1

25. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
◦ four digit mode
◦ quoting
13 of 1

26. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
◦ four digit mode
◦ quoting
• attributes
13 of 1

27. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
◦ four digit mode
◦ quoting
• attributes
• titles
13 of 1

28. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
◦ four digit mode
◦ quoting
• attributes
• titles
• resource reference types
13 of 1

29. Current checks
• Most common “errors”:
◦ expand tabs
◦ comments
◦ four digit mode
◦ quoting
• attributes
• titles
• resource reference types
• strings
13 of 1

30. Using Puppet::Tidy
Standalone
#!/usr/bin/perl
use strict;
use Puppet::Tidy;
Puppet::Tidy::puppettidy();
14 of 1

31. Using Puppet::Tidy
Part of a larger whole
#!/usr/bin/perl
use strict;
use Puppet::Tidy;
my (@output, $source);
$source = << ’EOF’;
Exec[’$reboot’]
EOF
Puppet::Tidy::puppettidy
(source => $source, destination => @output);
15 of 1

32. Quick demo
input
Exec[’$reboot’] // What is it doing here?
file {
"/tmp/blah":
mode => 644
}
package { ’$openssh’:
ensure => present
}
16 of 1

33. Quick demo
output
Exec[$reboot] # What is it doing here?
file {
’/tmp/blah’:
mode => ’0644’
}
package { "$openssh":
ensure => present;
}
17 of 1

34. Installing Puppet::Tidy
• CPAN
18 of 1

35. Installing Puppet::Tidy
• CPAN
◦ $ cpan Puppet::Tidy
18 of 1

36. Installing Puppet::Tidy
• CPAN
◦ $ cpan Puppet::Tidy
• OpenBSD
18 of 1

37. Installing Puppet::Tidy
• CPAN
◦ $ cpan Puppet::Tidy
• OpenBSD
◦ $ pkg add p5-Puppet-Tidy
18 of 1

38. Intro
m:tier
Puppet::Tidy
Future ideas
19 of 1

39. Basic features
• Define checks to run
20 of 1

40. Basic features
• Define checks to run
• Moar checks!
20 of 1

41. Basic features
• Define checks to run
• Moar checks!
• Attribute alignment
20 of 1

42. Basic features
• Define checks to run
• Moar checks!
• Attribute alignment
• Output validation
20 of 1

43. Semantic knowledge
• Puppet::Tidy just works on syntactic level
21 of 1

44. Semantic knowledge
• Puppet::Tidy just works on syntactic level
• It should be smarter and know about blocks
21 of 1

45. Semantic knowledge
• Puppet::Tidy just works on syntactic level
• It should be smarter and know about blocks
◦ find the blocks
21 of 1

46. Semantic knowledge
• Puppet::Tidy just works on syntactic level
• It should be smarter and know about blocks
◦ find the blocks
◦ group the blocks
21 of 1

47. Semantic knowledge
• Puppet::Tidy just works on syntactic level
• It should be smarter and know about blocks
◦ find the blocks
◦ group the blocks
◦ and in the file bind them
21 of 1

48. Real parser
• Currently works on a line-by-line basis, but could use a real parser
22 of 1

49. Real parser
• Currently works on a line-by-line basis, but could use a real parser
• Anyone got experience with MARPA?
22 of 1

50. Questions || suggestions?
23 of 1

51. Thank you!
and thank to my employer m:tier for sponsoring the development
of Puppet::Tidy.
mail jasper@mtier.org
www jasper.la and www.mtier.org
twitter jasper la
github jasperla
CPAN search.cpan.org/~jasper
24 of 1