My talk from the pupet devops conference Berlin 2014 (http://code-your-config.com/). A low level tour of some terrible terrible puppet code, with advice on how to do it better (including showing off the awesome new each() construct you get in puppet 3.2)
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
"The worst code I ever wrote"
1.
2. The worst code I ever wrote (maybe)
Tomas Doran
@bobtfish
tdoran@yelp.com
15/11/2013
3. Key lessons
!
•‘Working’ != ‘done’
– I’m a terrible person
– Working now doesn’t mean you can maintain it
•Functions for reuse
– and my horrible propensity for inline templating
•exec {} abuse
– Lets fit an entire shell script in the command => “…”
•Looping
– Hacks you have probably used.
– Doing it sanely with puppet >= 3.2
tl;dl: This talk is about bad puppet code, and how and why to do it better. The worst code I ever wrote is a
lie ;)
We’re going to concentrate on 2 main topics - functions and looping, with a nod to exec abuse.
4. Managing ssh keys - gitolite
!
• Legacy user management code
–Old non-modular code
–Have to fit in
–Can’t change everything
• Built in sshkey {}
– Not suitable for our requirements
We’re rapidly updating our puppet codebase, but the user management code is one of the most complex
parts, so we wanted to be minimally invasive.. We needed to generate a very custom authorized keys files to
integrate gitolite authentication into our current model.
5. sshkey resource
The built in sshkey {} isn’t perfect for everyone.
Most of the solutions on forge aren’t generic
7. http://forge.puppetlabs.com/nightfly/ssh_keys
!
• Legacy user management code
!
–Old non-modular code
–Have to fit in
• Almost
– But not quite suitable for our requirements
Doing it myself as a concat {} seemed sane, especially given other people’s evidence..
8. Managing ssh keys - gitolite
!
• Legacy user management code
!
–Old non-modular code
–Have to fit in
–Can’t change everything
• Built in sshkey {}
– Not suitable for our requirements
!
• Hack what we want
with define
So, we’re gonna use a bunch of defines and concat {}, easy?
9. Inputs and outputs
!
gitolite::user { ‘tdoran’:
key_source => $key_source;
}
– Add in current user management code
– $keysource = puppet:///files/
users/tdoran/authorized_keys
/etc/authorized_keys.d/git.pub
– Use concat
– Multiple gitolite instances!
- API for creating a key we drop into our current user management path
- Eventual generated file, from a concat {} resource
- Note we can have multiple gitolite instances on one box, that makes everything much harder.
10. So how does it work?
First, get an array of key lines, with names prepended
$864
11. Split the key into an array
$864
We split the file of keys into an array of lines.
We remove comments, whitespace only lines etc
We capture the entire key passed in.
12. Split the key into an array
$864
We interpolate the username in front of that key, space separating them
13. Split the key into an array
$864
End up with an array of strings which are username and key joined by a space
We then abuse a define as a loop by calling it with the array of lines as $title
14. Prepend instance name
!
Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${username} ${restofkey}”]
– Use this as $name in a define to iterate
– ssh/authorized_keys.d/${instance}.pub
We then have an array of instances, and an array of users/keys.
Our array_prefix function makes the product of both lists.
Complexity due to iterating over two lists $key_lines and $instances
15. Prepend instance name
!
Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${username} ${restofkey}”]
– Use this as $name in a define to iterate
– ssh/authorized_keys.d/${instance}.pub
We then have an array of instances, and an array of users/keys.
Our array_prefix function makes the product of both lists.
Complexity due to iterating over two lists $key_lines and $instances
16. Prepend instance name
!
Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${username} ${restofkey}”]
– Use this as $name in a define to iterate
– ssh/authorized_keys.d/${instance}.pub
We then have an array of instances, and an array of users/keys.
Our array_prefix function makes the product of both lists.
Complexity due to iterating over two lists $key_lines and $instances
17. Prepend instance name
!
Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${username} ${restofkey}”]
– Use this as $name in a define to iterate
– ssh/authorized_keys.d/${instance}.pub
We then have an array of instances, and an array of users/keys.
Our array_prefix function makes the product of both lists.
Complexity due to iterating over two lists $key_lines and $instances
18. Prepend instance name
!
Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${username} ${restofkey}”]
– Use this as $name in a define to iterate
– ssh/authorized_keys.d/${instance}.pub
This is gross
We then have an array of instances, and an array of users/keys.
Our array_prefix function makes the product of both lists.
Complexity due to iterating over two lists $key_lines and $instances
19. My original next code
Stare in horror!
$864
This was hideous. Whilst looping without define abuse is hard, there’s just no excuse for this.
21. Sins
$864
Shift the user name off the array.
Lol, who knew you could do that?
Use the remaining data joined back up
LOL??
Hideous abuse of variables
22. Sins
Hideous abuse of interpolation
LOL??
LOL??
$864
So, here, I have a double quoted string containing erb, and I’m then using double quote (not erb)
interpolation.. Niiice.
30. I am a bad person
LOL?
$864
Someone stopped me ;)
31. Don’t abuse inline_template()
!
• Please?
– I’m a terrible person
– Working now doesn’t mean you can maintain it
• Functions for reuse
– Full power of ruby
– Composable
– There should be more libraries of functions
Do what I’m saying, not what I do :)
32. More sane
Except each user can have multiple keys, and we want to prevent any key being reused by multiple
users.
33. Issues with functions
!
• What functions do I even have?
– Hard to know what functions you have imported
– stdlib, builtins, ….your $modulepath here…
• What do these even do?
– Often need to read the code to determine
– Lots of functions packaged with modules - don’t do
this!
• Millions of twisty little functions, all alike
• Millions of twisty little functions, all different
– You know what this reminds me of?
37. a few too many..
The problem is, functions in a global namespace aren’t all that modular/composable.
You can call functions from functions - but dependency hell
38. we’re almost half way through….
Shall I stop now? :)
puppet problem is worse than this - what functions you have depends on what modules are in your codebase.
39. Get docs for project’s functions
I wrote a thing to help with this - this will document _your_ functions for you. gist at end of talk.
40. gitolite group file generation
!
• /etc/groups on system
!
• gitolite groups have a different
format:
@admins = tdoran joebloggs!
!
• Use exec {… add mess here … }
This is another great example of what not to do.
41. exec {} abuse
Syntax highlighting won’t save you now puny human!
$864
This. DO NOT DO THIS.
Please?
46. This madness actually worked
LOL?
!
•exec {} abuse
– Lets fit an entire shell script in the
command => “…”
– Don’t do it! :)
– Drop a perl/python/ruby script (or two)
instead, call them from the exec.
$864
– Think about writing a type/provider
– Lots of examples of parsedfile available
55. Hypocrisy avoidance
!
”There should be more libraries
of functions”
- Me, about 5m ago
– https://github.com/bobtfish/puppetsshkey_utilities
– https://github.com/bobtfish/puppet-better_file
– https://gist.github.com/bobtfish/7403811
– Whole module to follow (eventually)
56. That’s all folks
!
• Where was that code?
– https://github.com/bobtfish/puppet-sshkey_utilities
– https://github.com/bobtfish/puppet-better_file
• Did you say you were hiring?
– YES
– Hamburg.
– London.
– San Francisco.
• Questions?