Production is a bitch


Published on

Deploying to production at scale is a royal pain in the arse (but worth it) and these are some of my observations

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

  • Production is a bitch

    1. Production is a Bitch
    2. Some deployments got very well and wow we’re awesome and feel great and LETS PARTY!
    3. Some deployments go poorly and wreck weekends and make us feel bad and lets drown our sorrows
    4. We do both, but we’re learning
    5. Troublesome deployments usually fail in two ways
    6. Your code doesn’t work* * more on this later
    7. Your code doesn’t scale* * also more later
    8. Good deployments are usually based on two principles
    9. Developed with the right mindset
    10. Developed with good technique
    11. The mindset is the more important part
    12. “She was one in a million, yeah. So there’s five more, just in New South Wales” The Whitlams
    13. Always expect the worst and you’ll only get happy surprises
    14. Play the averages
    15. 24 Hours
    16. 332 Error Pages Served
    17. 1,690,671 Total Requests
    18. 0.019%
    19. your code doesn’t work
    20. “But it passes QA/CI/ Acceptance/whatever”
    21. Continuous Integration is a dress rehearsal
    22. “It’s some thing wrong with the framework or linux or ruby or something”
    23. You’ve probably just written something retarded
    24. ... except when you haven’t and there’s some obscure one in a million MySql bug
    25. your code doesn’t scale
    26. does it even matter?
    27. Average Response Time 139ms
    28. Maximum Response Time 87,997ms
    29. “using an RDBMS means your design is wrong” Ben Schwarz
    30. Bullshit.
    31. “crying yourself to sleep every night means your design is wrong”
    32. The correct answer to every question relating to maintenance, scalability, reliability, availability is:
    33. It depends.
    34. ...that being said, there are some very good guidelines and techniques that rarely go wrong
    35. Write noisy code Fail loudly and fast
    36. and make sure you’re listening
    37. Hoptoad Get Exceptional New Relic ExceptionNotifer
    38. Write unit tests for production and hook up alerting for failures
    39. Measure, then change
    40. New Relic is ace.
    41. ruby-prof railsbench Benchmark (stdlib) ab
    42. Write slow, clean code See what’s slow and how Make that fast
    43. @johnbarton