Sometimes Tools Matter       John E. Vincent  DevOpsDays Goteborg 2011
We all know about DevOps
We all know about DevOps     I R DEV!                I R OPS!
So whats the big deal?
“With XXXXX you are be able to do easilycommon task on your local or remote machine.The aim of XXXXX is to become the uniq...
Not Puppet                        Not Chef                   Not Capistrano   “With XXXXX you are be able to do easilycomm...
YANFT(yet another f*ing tool)
Were doing something wrong.    Something is missing.
Ill be over here with Capistrano, kthx                     You just need to write a Chef recipe and ….
You cant solve cultural issues with tools
You cant solve cultural issues with tools              or can you?
Some Issues●   Repository mismatch●   Different languages●   Volatile configuration●   Visibility●   Sensitive information...
Caveats and Comments●   No single tool is going to solve all your    problems – sorry.●   You may have already heard of so...
Impedance mismatches
Sample Shop●   Operations – Puppet, EC2●   Development – Maven, Spring●   War files●   Spring properties files●   Metric a...
Issues●   Operations repository isnt application repository●   Developers need to test locally during development●   Prope...
Know your Enemy
Know your Enemy           This sandwich kicked my ass.
Know your Enemy           This sandwich kicked my ass          because I didnt know my enemy
The competition isnt between “devops”tools. Its between devops and “shell scripts             covered in meat sauce”      ...
“shell scripts covered in meat sauce” ==                 inertia
Configuration
Configuration Competition●   XML files (unless you run bcfg2)●   Key/Value property files (.ini style files)●   YAML●   JS...
Configuration Champions●   ZooKeeper, Nesoi, Noah●   Moves volatile configuration outside of application●   Can be populat...
Packaging
Packaging Protagonists●   War files●   Lein, Maven/Ivy, Rubygems, Agner, CPAN, Pypi●   RVM●   Homebrew    Not always simpl...
Packaging Princes●   FPM, brew2deb, fpm-cookery●   Makes building OS packages “painless”●   OS packages have value (rollba...
Development Environments
Development Defeat●   Windows/OSX for development, Linux/Solaris    for production●   QA different than Production●   Diff...
Development Dreams●   Vagrant, Veewee, Whirr●   Provides “production” in a box●   Developers should be “deploying” locally...
Visibility
Visibility Villains●   uptime, *top, iostat, vmstat, sar*●   ssh●   tail●   Host Obsession Disorder    Remember who the co...
Visibility Victors - part 1●   Statsd, Graphite, Graylog2, Logstash●   ruby-metrics, codahale metrics for JVM●   “If it mo...
Visibility Victors - part 2●   Jenkins, Rundeck, MCollective●   Hosts dont matter, only services●   Sometimes you still ne...
The “other” stuff
Shitty Stuff●   Manual database migrations●   Rollbacks●   The myth of the sensitive
Smart Stuff●   Flyway, Liquibase, Rails migrations●   Consider NoSQL/Schema-less stores●   Roll forward. Never roll back.●...
There is, however, ONE tool that can solve almost every technological and cultural                problem
Just kidding!  (sort of)
This is the point where you realize Ive duped                     you
You are a tool
You are THE tool
A tale of two tools
This is Kelsey
Kelseys Problem●   Operations used Puppet with Cobbler as ENC●   Application configuration was in ERB templates    managed...
Kelseys Solution●   Programming motherf**er●   Learned some Java (in a week)●   Wrote some classes, built some jars●   Del...
This is Jordan
Jordans Problem●   Everyone has their own effing packaging format●   Distributions suck●   Debs suck to build●   RPMS suck...
Jordans Solution●   Hate Driven Development●   Wrote FPM (eFfing package manager)●   Alien done right●   Converts debs, rp...
Side note: If you ever say “I wonder ifsomeone has done X” stop at the “I” and go to          github.com/jordansissel
Real Talk●   Step outside your comfort zone●   Be a technologist not a specialist●   SysAdmins: Learn to write some code. ...
Go from this....
and this
to this
Thank You!
Keep in touch!●   @lusis - twitter●   lusis – github●   John Vincent –    linkedin●   blog.lusis.org●   john@lusis.org
Devopsdays se-2011
Upcoming SlideShare
Loading in...5
×

Devopsdays se-2011

483

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
483
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Devopsdays se-2011

  1. 1. Sometimes Tools Matter John E. Vincent DevOpsDays Goteborg 2011
  2. 2. We all know about DevOps
  3. 3. We all know about DevOps I R DEV! I R OPS!
  4. 4. So whats the big deal?
  5. 5. “With XXXXX you are be able to do easilycommon task on your local or remote machine.The aim of XXXXX is to become the unique anduniversal tool language that permit you to make your own brew, apt-get or yum package, same syntax for all your machine.”
  6. 6. Not Puppet Not Chef Not Capistrano “With XXXXX you are be able to do easilycommon task on your local or remote machine.The aim of XXXXX is to become the unique anduniversal tool language that permit you to make your own brew, apt-get or yum package, same syntax for all your machine.” Not Fabric Not DeployML Not CFengine
  7. 7. YANFT(yet another f*ing tool)
  8. 8. Were doing something wrong. Something is missing.
  9. 9. Ill be over here with Capistrano, kthx You just need to write a Chef recipe and ….
  10. 10. You cant solve cultural issues with tools
  11. 11. You cant solve cultural issues with tools or can you?
  12. 12. Some Issues● Repository mismatch● Different languages● Volatile configuration● Visibility● Sensitive information● Testability● Packaging
  13. 13. Caveats and Comments● No single tool is going to solve all your problems – sorry.● You may have already heard of some of these tools.● You will likely have to “mold” some of these tools to fit your environment● The tool itself is not the point, it is the end result● I dont have all the answers...
  14. 14. Impedance mismatches
  15. 15. Sample Shop● Operations – Puppet, EC2● Development – Maven, Spring● War files● Spring properties files● Metric assloads of XML● Database changes
  16. 16. Issues● Operations repository isnt application repository● Developers need to test locally during development● Properties files are artifacts● Some settings are environment specific● Some settings are “sensitive”● How do new settings get in CM?● Where do database changes fit?● What about the rest of the business?
  17. 17. Know your Enemy
  18. 18. Know your Enemy This sandwich kicked my ass.
  19. 19. Know your Enemy This sandwich kicked my ass because I didnt know my enemy
  20. 20. The competition isnt between “devops”tools. Its between devops and “shell scripts covered in meat sauce” (with apologies to @littleidea)
  21. 21. “shell scripts covered in meat sauce” == inertia
  22. 22. Configuration
  23. 23. Configuration Competition● XML files (unless you run bcfg2)● Key/Value property files (.ini style files)● YAML● JSON● Hard coded “stuff”● My-effing-SQL, Mongo-effing-DB All of these things are “simpler” to understand.
  24. 24. Configuration Champions● ZooKeeper, Nesoi, Noah● Moves volatile configuration outside of application● Can be populated by both operations and development*● Service discovery● No immediate need to learn a new DSL/Language● Can still control access*● Dont underestimate the power of environment variables● You still need “proper” configuration management
  25. 25. Packaging
  26. 26. Packaging Protagonists● War files● Lein, Maven/Ivy, Rubygems, Agner, CPAN, Pypi● RVM● Homebrew Not always simpler to understand but built into the community.
  27. 27. Packaging Princes● FPM, brew2deb, fpm-cookery● Makes building OS packages “painless”● OS packages have value (rollback, versioning, validation)● Need a single artifact that can be deployed in all environments
  28. 28. Development Environments
  29. 29. Development Defeat● Windows/OSX for development, Linux/Solaris for production● QA different than Production● Different versions of critical libraries, jars, gems whatever● Exploded war files vs. Packaged war files “Works on my machine”
  30. 30. Development Dreams● Vagrant, Veewee, Whirr● Provides “production” in a box● Developers should be “deploying” locally● Operations needs to make modules, manifests, cookbooks (whatever) flexible This is one of the hardest to accomplish
  31. 31. Visibility
  32. 32. Visibility Villains● uptime, *top, iostat, vmstat, sar*● ssh● tail● Host Obsession Disorder Remember who the competition is...
  33. 33. Visibility Victors - part 1● Statsd, Graphite, Graylog2, Logstash● ruby-metrics, codahale metrics for JVM● “If it moves, graph it”● Disk is “cheap”● Dashboards are the bomb. Become one with information radiators.● Logstash can pipe the shit out of EVERYTHING● You dont know what you dont need until you know what you have● Overcompensate initially (but dont over do it!)
  34. 34. Visibility Victors - part 2● Jenkins, Rundeck, MCollective● Hosts dont matter, only services● Sometimes you still need to get on a specific host....to reprovision it.● Wrap the access for auditing and accountability● DevOps is not about giving root access to developers● The “myth” of security.
  35. 35. The “other” stuff
  36. 36. Shitty Stuff● Manual database migrations● Rollbacks● The myth of the sensitive
  37. 37. Smart Stuff● Flyway, Liquibase, Rails migrations● Consider NoSQL/Schema-less stores● Roll forward. Never roll back.● You can never truly roll back anyway● “Sensitive” data is a myth (for some values of sensitive)
  38. 38. There is, however, ONE tool that can solve almost every technological and cultural problem
  39. 39. Just kidding! (sort of)
  40. 40. This is the point where you realize Ive duped you
  41. 41. You are a tool
  42. 42. You are THE tool
  43. 43. A tale of two tools
  44. 44. This is Kelsey
  45. 45. Kelseys Problem● Operations used Puppet with Cobbler as ENC● Application configuration was in ERB templates managed by Puppet● Cultural issues ruled out development actually managing the templates● System of Record was Cobbler
  46. 46. Kelseys Solution● Programming motherf**er● Learned some Java (in a week)● Wrote some classes, built some jars● Delivered an XML-RPC library that development could use to query Cobbler for information
  47. 47. This is Jordan
  48. 48. Jordans Problem● Everyone has their own effing packaging format● Distributions suck● Debs suck to build● RPMS suck moderately less to build
  49. 49. Jordans Solution● Hate Driven Development● Wrote FPM (eFfing package manager)● Alien done right● Converts debs, rpms, pypi, gems, compiled directories to native distro package format
  50. 50. Side note: If you ever say “I wonder ifsomeone has done X” stop at the “I” and go to github.com/jordansissel
  51. 51. Real Talk● Step outside your comfort zone● Be a technologist not a specialist● SysAdmins: Learn to write some code. Seriously.● Developers: Learn the OS. Seriously.● Everyone: Stop blaming other people● Software can help ease communication but only you can prevent forest fires.
  52. 52. Go from this....
  53. 53. and this
  54. 54. to this
  55. 55. Thank You!
  56. 56. Keep in touch!● @lusis - twitter● lusis – github● John Vincent – linkedin● blog.lusis.org● john@lusis.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×