Production is a Bitch
Some deployments got very
   well and wow we’re
awesome and feel great and
      LETS PARTY!
Some deployments go
    poorly and wreck
weekends and make us feel
 bad and lets drown our
        sorrows
We do both,
but we’re learning
Troublesome deployments
  usually fail in two ways
Your code doesn’t work*

* more on this later
Your code doesn’t scale*

* also more later
Good deployments are
 usually based on two
       principles
Developed with the right
       mindset
Developed with good
     technique
The mindset is the more
    important part
“She was one in a million, yeah.
  So there’s five more, just in
      New South Wales”
The Whitlams
Always expect the worst
and you’ll only get happy
       surprises
Play the averages
24 Hours
332 Error Pages Served
1,690,671 Total Requests
0.019%
your code doesn’t work
“But it passes QA/CI/
Acceptance/whatever”
Continuous Integration is a
     dress rehearsal
“It’s some thing wrong with
 the framework or linux or
     ruby or something”
You’ve probably just
written something retarded
... except when you haven’t
 and there’s some obscure
 one in a million MySql bug
your code doesn’t scale
does it even matter?
Average Response Time

       139ms
Maximum Response Time

      87,997ms
“using an RDBMS means
 your design is wrong”

Ben Schwarz
Bullshit.
“crying yourself to sleep
every night means your
    design is wrong”
The correct answer to
every question relating to
 maintenance, scalability,
 reliability, availability is:
It depends.
...that being said, there are
some very good guidelines
and techniques that rarely
          go wrong
Write noisy code
Fail loudly and fast
and make sure you’re
     listening
Hoptoad
 Get Exceptional
   New Relic
ExceptionNotifer
Write unit tests for
production and hook up
  alerting for failures
Measure, then change
New Relic is ace.
ruby-prof
    railsbench
Benchmark (stdlib)
         ab
Write slow, clean code

See what’s slow and how

     Make that fast
http://envato.com
http://whoisjohnbarton.com

      @johnbarton
Upcoming SlideShare
Loading in...5
×

Production is a bitch

1,558

Published on

Deploying to production at scale is a royal pain in the arse (but worth it) and these are some of my observations

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,558
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide











































  • Production is a bitch

    1. 1. Production is a Bitch
    2. 2. Some deployments got very well and wow we’re awesome and feel great and LETS PARTY!
    3. 3. Some deployments go poorly and wreck weekends and make us feel bad and lets drown our sorrows
    4. 4. We do both, but we’re learning
    5. 5. Troublesome deployments usually fail in two ways
    6. 6. Your code doesn’t work* * more on this later
    7. 7. Your code doesn’t scale* * also more later
    8. 8. Good deployments are usually based on two principles
    9. 9. Developed with the right mindset
    10. 10. Developed with good technique
    11. 11. The mindset is the more important part
    12. 12. “She was one in a million, yeah. So there’s five more, just in New South Wales” The Whitlams
    13. 13. Always expect the worst and you’ll only get happy surprises
    14. 14. Play the averages
    15. 15. 24 Hours
    16. 16. 332 Error Pages Served
    17. 17. 1,690,671 Total Requests
    18. 18. 0.019%
    19. 19. your code doesn’t work
    20. 20. “But it passes QA/CI/ Acceptance/whatever”
    21. 21. Continuous Integration is a dress rehearsal
    22. 22. “It’s some thing wrong with the framework or linux or ruby or something”
    23. 23. You’ve probably just written something retarded
    24. 24. ... except when you haven’t and there’s some obscure one in a million MySql bug
    25. 25. your code doesn’t scale
    26. 26. does it even matter?
    27. 27. Average Response Time 139ms
    28. 28. Maximum Response Time 87,997ms
    29. 29. “using an RDBMS means your design is wrong” Ben Schwarz
    30. 30. Bullshit.
    31. 31. “crying yourself to sleep every night means your design is wrong”
    32. 32. The correct answer to every question relating to maintenance, scalability, reliability, availability is:
    33. 33. It depends.
    34. 34. ...that being said, there are some very good guidelines and techniques that rarely go wrong
    35. 35. Write noisy code Fail loudly and fast
    36. 36. and make sure you’re listening
    37. 37. Hoptoad Get Exceptional New Relic ExceptionNotifer
    38. 38. Write unit tests for production and hook up alerting for failures
    39. 39. Measure, then change
    40. 40. New Relic is ace.
    41. 41. ruby-prof railsbench Benchmark (stdlib) ab
    42. 42. Write slow, clean code See what’s slow and how Make that fast
    43. 43. http://envato.com http://whoisjohnbarton.com @johnbarton
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×