The worst code I ever wrote (maybe)
Tomas Doran
@bobtfish
tdoran@yelp.com
15/11/2013
Key lessons
!

•‘Working’ != ‘done’
– I’m a terrible person
– Working now doesn’t mean you can maintain it

•Functions for...
Managing ssh keys - gitolite
!

• Legacy user management code
–Old non-modular code
–Have to fit in
–Can’t change everythi...
sshkey resource

The built in sshkey {} isn’t perfect for everyone.
Most of the solutions on forge aren’t generic
http://forge.puppetlabs.com/hetzeneckerl/ssh_authorized_key

These are the core issues.
But this code didn’t work for me.
http://forge.puppetlabs.com/nightfly/ssh_keys
!

• Legacy user management code
!

–Old non-modular code
–Have to fit in

•...
Managing ssh keys - gitolite
!

• Legacy user management code

!

–Old non-modular code
–Have to fit in
–Can’t change ever...
Inputs and outputs
!

gitolite::user { ‘tdoran’:
key_source => $key_source;
}
– Add in current user management code
– $key...
So how does it work?
First, get an array of key lines, with names prepended

$864
Split the key into an array

$864

We split the file of keys into an array of lines.
We remove comments, whitespace only l...
Split the key into an array

$864

We interpolate the username in front of that key, space separating them
Split the key into an array

$864

End up with an array of strings which are username and key joined by a space
We then ab...
Prepend instance name
!

Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${us...
Prepend instance name
!

Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${us...
Prepend instance name
!

Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${us...
Prepend instance name
!

Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${us...
Prepend instance name
!

Add user to every instance
– [“${username} ${restofkey}”]
– [“${instance} “]
– [“${instance} ${us...
My original next code
Stare in horror!

$864

This was hideous. Whilst looping without define abuse is hard, there’s just ...
Sins

LOL??

Hideous abuse of define - one $name for
each $key_lines

$864

We then split the line back up.
Sins

$864

Shift the user name off the array.
Lol, who knew you could do that?
Use the remaining data joined back up

LOL...
Sins
Hideous abuse of interpolation

LOL??
LOL??

$864

So, here, I have a double quoted string containing erb, and I’m th...
Sins
Hideous abuse of quoting

EVERY TYPE OF QUOTES!!!

LOL??

LOL??

LOL??

$864
Sins
Hideous abuse of quoting

$864

LOL??

LOL??
Sins
Hideous abuse of bash

LOL??
$864

ssh-keygen can’t be fed from stdin without disgusting trickery (modern bash specif...
Sins
So complex I’m tempting the template!

LOL??
$864

LOL??

And yes, it’s so complex I template the thing I’m interpola...
I am a bad person
LOL?

$864
I am a bad person
LOL?

$864
I am a bad person
LOL?

$864

Thankfully
I am a bad person
LOL?

$864

Someone stopped me ;)
Don’t abuse inline_template()
!

• Please?
– I’m a terrible person
– Working now doesn’t mean you can maintain it

• Funct...
More sane

Except each user can have multiple keys, and we want to prevent any key being reused by multiple
users.
Issues with functions
!

• What functions do I even have?
– Hard to know what functions you have imported
– stdlib, builti...
Everyone’s favorite language

If all you have is a hammer, everything looks like a nail
has a few functions

Unfortunately, there are no consistent naming conventions
arguably

And it’s hard to work out which function you actually want
a few too many..

The problem is, functions in a global namespace aren’t all that modular/composable.
You can call functio...
we’re almost half way through….

Shall I stop now? :)
puppet problem is worse than this - what functions you have depends ...
Get docs for project’s functions

I wrote a thing to help with this - this will document _your_ functions for you. gist at...
gitolite group file generation
!

• /etc/groups on system
!

• gitolite groups have a different
format:
@admins = tdoran j...
exec {} abuse
Syntax highlighting won’t save you now puny human!

$864

This. DO NOT DO THIS.
Please?
exec {} abuse
Syntax highlighting won’t save you now puny human!

$864
exec {} abuse
Syntax highlighting won’t save you now puny human!

$864
exec {} abuse
Syntax highlighting won’t save you now puny human!

$864

Arggh, the backslashes!
This madness actually worked
LOL?

$864
This madness actually worked
LOL?

!

•exec {} abuse

– Lets fit an entire shell script in the 

command => “…”
– Don’t do...
Which would you rather have to debug?

$864
OR THIS!?!

$864
Which one’s going to stay debugged?

$864
OR THIS?!?

$864
Looping sanely

!

This would work fine.
Need multiple instances :(
—parser future
—parser future

$864
—parser future
!

• Can’t do this at Yelp
(yet)

• Still using 2.7, because
$864
variable scope
Hypocrisy avoidance
!

”There should be more libraries
of functions”
- Me, about 5m ago

– https://github.com/bobtfish/pup...
That’s all folks
!

• Where was that code?
– https://github.com/bobtfish/puppet-sshkey_utilities
– https://github.com/bobt...
"The worst code I ever wrote"
Upcoming SlideShare
Loading in …5
×

"The worst code I ever wrote"

5,297 views

Published on

My talk from the pupet devops conference Berlin 2014 (http://code-your-config.com/). A low level tour of some terrible terrible puppet code, with advice on how to do it better (including showing off the awesome new each() construct you get in puppet 3.2)

Published in: Technology, Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
5,297
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

"The worst code I ever wrote"

  1. 1. The worst code I ever wrote (maybe) Tomas Doran @bobtfish tdoran@yelp.com 15/11/2013
  2. 2. Key lessons ! •‘Working’ != ‘done’ – I’m a terrible person – Working now doesn’t mean you can maintain it •Functions for reuse – and my horrible propensity for inline templating •exec {} abuse – Lets fit an entire shell script in the command => “…” •Looping – Hacks you have probably used. – Doing it sanely with puppet >= 3.2 tl;dl: This talk is about bad puppet code, and how and why to do it better. The worst code I ever wrote is a lie ;) We’re going to concentrate on 2 main topics - functions and looping, with a nod to exec abuse.
  3. 3. Managing ssh keys - gitolite ! • Legacy user management code –Old non-modular code –Have to fit in –Can’t change everything • Built in sshkey {} – Not suitable for our requirements We’re rapidly updating our puppet codebase, but the user management code is one of the most complex parts, so we wanted to be minimally invasive.. We needed to generate a very custom authorized keys files to integrate gitolite authentication into our current model.
  4. 4. sshkey resource The built in sshkey {} isn’t perfect for everyone. Most of the solutions on forge aren’t generic
  5. 5. http://forge.puppetlabs.com/hetzeneckerl/ssh_authorized_key These are the core issues. But this code didn’t work for me.
  6. 6. http://forge.puppetlabs.com/nightfly/ssh_keys ! • Legacy user management code ! –Old non-modular code –Have to fit in • Almost – But not quite suitable for our requirements Doing it myself as a concat {} seemed sane, especially given other people’s evidence..
  7. 7. Managing ssh keys - gitolite ! • Legacy user management code ! –Old non-modular code –Have to fit in –Can’t change everything • Built in sshkey {} – Not suitable for our requirements ! • Hack what we want with define So, we’re gonna use a bunch of defines and concat {}, easy?
  8. 8. Inputs and outputs ! gitolite::user { ‘tdoran’: key_source => $key_source; } – Add in current user management code – $keysource = puppet:///files/ users/tdoran/authorized_keys /etc/authorized_keys.d/git.pub – Use concat – Multiple gitolite instances! - API for creating a key we drop into our current user management path - Eventual generated file, from a concat {} resource - Note we can have multiple gitolite instances on one box, that makes everything much harder.
  9. 9. So how does it work? First, get an array of key lines, with names prepended $864
  10. 10. Split the key into an array $864 We split the file of keys into an array of lines. We remove comments, whitespace only lines etc We capture the entire key passed in.
  11. 11. Split the key into an array $864 We interpolate the username in front of that key, space separating them
  12. 12. Split the key into an array $864 End up with an array of strings which are username and key joined by a space We then abuse a define as a loop by calling it with the array of lines as $title
  13. 13. Prepend instance name ! Add user to every instance – [“${username} ${restofkey}”] – [“${instance} “] – [“${instance} ${username} ${restofkey}”] – Use this as $name in a define to iterate – ssh/authorized_keys.d/${instance}.pub We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances
  14. 14. Prepend instance name ! Add user to every instance – [“${username} ${restofkey}”] – [“${instance} “] – [“${instance} ${username} ${restofkey}”] – Use this as $name in a define to iterate – ssh/authorized_keys.d/${instance}.pub We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances
  15. 15. Prepend instance name ! Add user to every instance – [“${username} ${restofkey}”] – [“${instance} “] – [“${instance} ${username} ${restofkey}”] – Use this as $name in a define to iterate – ssh/authorized_keys.d/${instance}.pub We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances
  16. 16. Prepend instance name ! Add user to every instance – [“${username} ${restofkey}”] – [“${instance} “] – [“${instance} ${username} ${restofkey}”] – Use this as $name in a define to iterate – ssh/authorized_keys.d/${instance}.pub We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances
  17. 17. Prepend instance name ! Add user to every instance – [“${username} ${restofkey}”] – [“${instance} “] – [“${instance} ${username} ${restofkey}”] – Use this as $name in a define to iterate – ssh/authorized_keys.d/${instance}.pub This is gross We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances
  18. 18. My original next code Stare in horror! $864 This was hideous. Whilst looping without define abuse is hard, there’s just no excuse for this.
  19. 19. Sins LOL?? Hideous abuse of define - one $name for each $key_lines $864 We then split the line back up.
  20. 20. Sins $864 Shift the user name off the array. Lol, who knew you could do that? Use the remaining data joined back up LOL?? Hideous abuse of variables
  21. 21. Sins Hideous abuse of interpolation LOL?? LOL?? $864 So, here, I have a double quoted string containing erb, and I’m then using double quote (not erb) interpolation.. Niiice.
  22. 22. Sins Hideous abuse of quoting EVERY TYPE OF QUOTES!!! LOL?? LOL?? LOL?? $864
  23. 23. Sins Hideous abuse of quoting $864 LOL?? LOL??
  24. 24. Sins Hideous abuse of bash LOL?? $864 ssh-keygen can’t be fed from stdin without disgusting trickery (modern bash specific)
  25. 25. Sins So complex I’m tempting the template! LOL?? $864 LOL?? And yes, it’s so complex I template the thing I’m interpolating into a template..
  26. 26. I am a bad person LOL? $864
  27. 27. I am a bad person LOL? $864
  28. 28. I am a bad person LOL? $864 Thankfully
  29. 29. I am a bad person LOL? $864 Someone stopped me ;)
  30. 30. Don’t abuse inline_template() ! • Please? – I’m a terrible person – Working now doesn’t mean you can maintain it • Functions for reuse – Full power of ruby – Composable – There should be more libraries of functions Do what I’m saying, not what I do :)
  31. 31. More sane Except each user can have multiple keys, and we want to prevent any key being reused by multiple users.
  32. 32. Issues with functions ! • What functions do I even have? – Hard to know what functions you have imported – stdlib, builtins, ….your $modulepath here… • What do these even do? – Often need to read the code to determine – Lots of functions packaged with modules - don’t do this! • Millions of twisty little functions, all alike • Millions of twisty little functions, all different – You know what this reminds me of?
  33. 33. Everyone’s favorite language If all you have is a hammer, everything looks like a nail
  34. 34. has a few functions Unfortunately, there are no consistent naming conventions
  35. 35. arguably And it’s hard to work out which function you actually want
  36. 36. a few too many.. The problem is, functions in a global namespace aren’t all that modular/composable. You can call functions from functions - but dependency hell
  37. 37. we’re almost half way through…. Shall I stop now? :) puppet problem is worse than this - what functions you have depends on what modules are in your codebase.
  38. 38. Get docs for project’s functions I wrote a thing to help with this - this will document _your_ functions for you. gist at end of talk.
  39. 39. gitolite group file generation ! • /etc/groups on system ! • gitolite groups have a different format: @admins = tdoran joebloggs! ! • Use exec {… add mess here … } This is another great example of what not to do.
  40. 40. exec {} abuse Syntax highlighting won’t save you now puny human! $864 This. DO NOT DO THIS. Please?
  41. 41. exec {} abuse Syntax highlighting won’t save you now puny human! $864
  42. 42. exec {} abuse Syntax highlighting won’t save you now puny human! $864
  43. 43. exec {} abuse Syntax highlighting won’t save you now puny human! $864 Arggh, the backslashes!
  44. 44. This madness actually worked LOL? $864
  45. 45. This madness actually worked LOL? ! •exec {} abuse – Lets fit an entire shell script in the 
 command => “…” – Don’t do it! :) – Drop a perl/python/ruby script (or two) instead, call them from the exec. $864 – Think about writing a type/provider – Lots of examples of parsedfile available
  46. 46. Which would you rather have to debug? $864
  47. 47. OR THIS!?! $864
  48. 48. Which one’s going to stay debugged? $864
  49. 49. OR THIS?!? $864
  50. 50. Looping sanely ! This would work fine. Need multiple instances :(
  51. 51. —parser future
  52. 52. —parser future $864
  53. 53. —parser future ! • Can’t do this at Yelp (yet) • Still using 2.7, because $864 variable scope
  54. 54. Hypocrisy avoidance ! ”There should be more libraries of functions” - Me, about 5m ago – https://github.com/bobtfish/puppetsshkey_utilities – https://github.com/bobtfish/puppet-better_file – https://gist.github.com/bobtfish/7403811 – Whole module to follow (eventually)
  55. 55. That’s all folks ! • Where was that code? – https://github.com/bobtfish/puppet-sshkey_utilities – https://github.com/bobtfish/puppet-better_file • Did you say you were hiring? – YES – Hamburg. – London. – San Francisco. • Questions?

×