Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Go away of I will replace you with a little Shell Script (english)

1,088 views

Published on

"Go away or I will replace you with a little Shell Script" (english edition of the GUUG FFG Devops Talk).

Published in: Internet
  • Hi there! Get Your Professional Job-Winning Resume Here - Check our website! http://bit.ly/resumpro
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Go away of I will replace you with a little Shell Script (english)

  1. 1. Go away or I will replace you with a very small Shell script Kristian Köhntopp Old Fart kris@syseleven.de “There is no such thing as a Devops Team”
  2. 2. „Old fart.“ 2 $FCE2
  3. 3. – Rajat Bhargava, http://devops.com/features/devops_and-enterprises/ „DevOps and Enterprises:
 It's a culture thing.“ 3 Influences
  4. 4. Economical influences: process and market maturity4 Pre-Dotcom: No Business-Model No BestPractices QualitativeBestPractices Metrics, improvementprocess outsourcingpossible unclear what actually has been sold Iterations Change
  5. 5. Technology influences: Scale-Up vs. Scale-Out 5
  6. 6. Around 2002:
 “Scale Up” doesn’t work.
  7. 7. “Scale Out” works just fine.
 Result: Moar Boxes!!1!one!!eleven!!!
  8. 8. –Kris, to anybody who doesn’t want to hear it anymore „Whenever you are using ssh you might as well open a ticket.“ 8
  9. 9. System administration, automated9 Branch Release Fix Merge
  10. 10. „Reproducible. 
 Parallelized. 
 Orchestrated.“ 10
  11. 11. What about the cultural wrapper? 11
  12. 12. de.alt.sysadmin.recovery Cat-5-O-Nine-Tails und Kekse von Antje Bendrich, Dezember 2008
  13. 13. “Canossa” Party 1999 http://www.42.org/~sec/Berichte/199907Canossa/bild09.jpg
  14. 14. Pre-DevOps (First Edition: 2001) via http://geek-whisperers.com/wp-content/uploads/2014/11/thomas-limoncelli.jpg
  15. 15. DevOps: Patrick Debois (2008, Belgium)
 Meanwhile in Amsterdam… Booking.com Office, 2008
  16. 16. ITIL vs. DevOps?
 
 Another
 Scale-Up vs. Scale-Out thingy… 16
  17. 17. Basic idea: Tear down this wall! Brandenburger Tor, 1989 - Foto: Thomas 'tlr' Roessler
  18. 18. 18 DevOps https://twitter.com/jezhumble/status/259368948859023362
  19. 19. A modern development environment19 • Automated Infrastructure • Shared Version Control • One Step Build and Deploy • Feature Flags, Graceful Degradation • Test and Measure all the things • Instant Communication nach John Allspaw (2009), http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr Systems Code Deploy Activation FailuredetectionFixing
  20. 20. Convergence in technology and culture20 • "Script all the things", "Centralize all monitoring", etc. pp. • "System administration built in at development stage." • Entwickler and Admins using the same tools • “Then let’s converge the teams."
 ➡ DevOps
  21. 21. So why are they still fighting? 21
  22. 22. Different success metrics for Operations Infrastructure
  23. 23. Feature-Development
  24. 24. Infrastructure-Development
  25. 25. “Booking Rules” 
 #1 If you break it, will you even notice?
 #2 If you break it, can you fix it? 
 These are rules dealing with fail! 25 Hack!
  26. 26. How did your patch change the worst case? http://www.instructables.com/id/Storing-garden-tools-with-style-aka-Zombiewall/
  27. 27. Fuck you, Nvidia! http://www.reddit.com/r/technology/comments/v5n9t/linus_to_nvidia_fuck_you/c51kp48
  28. 28. Infrastructure-thinking is hard to explain to outsiders. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=b0bc65729070b9cbdbb53ff042984a3c545a0e34
  29. 29. Strato loses several million hosted domains. http://www.computerwoche.de/a/defekter-emc-speicher-legt-strato-lahm,520152 14 Jahre her!
  30. 30. 25 million mail accounts offline. http://www.golem.de/0505/38057.html 10 Jahre her!
  31. 31. vs. "Never change a running system".
  32. 32. Not changing has costs.
  33. 33. Managing change… • From slides for an internal booking.com Dev Conf: »Some companies are creating elaborate change management processes in order to deal with change.
 
 That's not solving the change problem at all, it's just making change harder.« 33
  34. 34. Budgeting downtime •We have a budget for downtime. •A foreign concept, outside of Booking.
  35. 35. Budgeting downtime • "Validated corporate learning." • So we spend a number of € by not earning them. • What is the takeaway we got from that? • “Blameless Postmortem”
  36. 36. Careful Carelessness
  37. 37. Careful Carelessness • Skydiving: • Jumping out of an open plane door, because fun.
 • That's the visible part. • That's the "carelessness" part.
  38. 38. Careful Carelessness • Training • Team you can trust • Main, packed by you, checked by others • Reserve, packed by others, checked by you
  39. 39. In a way, that is what we do.
  40. 40. Careful Carelessness • If anything is wrong, at all:
 
 Don't jump!
  41. 41. Careful Carelessness • The goal is to survive a 4000 Meter drop rollout. • Every time. Every day. 50 times a day.
  42. 42. Survivability • Building systems in a way that 
 failure is survivable. • Because you can learn from failure, yours and those of others, • only if you manage to walk away from these failures, • and tell others the story.
  43. 43. Because we are learning from mistakes.
  44. 44. Managing change… • Focus on tests: • depending on fail survivability: • live (preferred) or • synthetic tests 44 Foto: Charly Kuehnast, Ampel: http://www.cleware-shop.de/USB-MiniTrafficLight-EN
  45. 45. Really? Really!
  46. 46. System Administration by Force… • Always shut systems down by pulling the plug… • … and even destroy production systems randomly. • Always test in production… • … and find methods to do so safely. 46
  47. 47. But doesn’t anybody think about the kittens?
  48. 48. –via Martin Seeger (NetUSE AG) „Nobody wants Backup.
 Everybody needs Restore.“ 48 Rebuild is just fine.
  49. 49. So why are they still fighting? 49
  50. 50. Monasca
  51. 51. How to win at Hipsterbingo… • »Monasca is an open-source multi-tenant, highly scalable, performant, fault-tolerant monitoring-as-a- service solution that integrates with OpenStack.
 
 It uses a REST API for high-speed metrics processing and querying and has a streaming alarm engine and notification engine.« 51 Bingo!
  52. 52. If you break it, can you fix it? • »Uses a number of underlying technologies:
 
 Apache Kafka, Apache Storm, Zookeeper, MySQL, Vagrant, Dropwizard, InfluxDB, Vertica.« 52
  53. 53. If you break it, can you fix it?53 Infrastructure success metric: system behavior in failure state.
  54. 54. Containerism…54
  55. 55. Not an isolated problem…55 Implementation tested on Devstack in VMware Fusion on a MacBook Air in St. Oberholz?
  56. 56. Cargo Culting…56 https://twitter.com/Manz/status/580160845587574784
  57. 57. „Computer science is the science of zeroes and ones. It’s not getting any more complicated than this.“ 57
  58. 58. So why is computer science so damn hard? • Nothing else plays abstractions Jenga and Tetris like CS: It’s all about layers.
 • Epsilon-Delta through 3 dozen layers, • in nonlinear systems, • with cascading dependencies. 58
  59. 59. – "Epigrams in Programming", Alan J. Perlis, Sep 1982, ACM SIGPLAN „55. A LISP programmer knows the value of everything, 
 but the cost of nothing.“ 59
  60. 60. –Ralf Ertzinger (cited from memory) „Haha. Go on, virtualize everything.
 But somebody will have to debug your matroshka networks eventually.“ 60
  61. 61. • Architect • Infrastructure Developer • System Administrator • Operator 61
  62. 62. 62 ?

×