Failure Happens
F***, the F***ing thing is F***king F***ed*
            *Official WebOps term from Artur Bergman




      ...
This will be on the test:

 FAILURE HAPPENS!
25%

75%
25%

75%         Paranoid
25%   Pyromaniac



75%         Paranoid
Good
Book!
“multiple and unexpected
interactions of failures are
        inevitable”
                 -­‐Charles	
  Perrow
Failure Happens
define:
 Nines (roughly)
define:
 Nines (roughly)
   99%	 5256 min (3.5 days)
define:
 Nines (roughly)
   99%	 5256 min (3.5 days)
   99.9%	 528 min ( 8.8 hours )
define:
 Nines (roughly)
   99%	 5256 min (3.5 days)
   99.9%	 528 min ( 8.8 hours )
   99.99% 53 min
define:
 Nines (roughly)
   99%	 5256 min (3.5 days)
   99.9%	 528 min ( 8.8 hours )
   99.99% 53 min
   99.999% 5 min
define:
 Nines (roughly)
   99%	 5256 min (3.5 days)
   99.9%	 528 min ( 8.8 hours )
   99.99% 53 min
   99.999% 5 min
   9...
define:
 Nines (roughly)
   99%	 5256 min (3.5 days)
   99.9%	 528 min ( 8.8 hours )
   99.99% 53 min
   99.999% 5 min
   9...
Internet Routing... won’t.
;''-1(<"=/-)"3.1>0?-'"@'-':




!"#$$%"&'(')*)"+,-.,-/01,(   +/.01210*"345467"89:   #
#googlefail
YOU
Continuous Power...
       isn’t
365 Main SF
365 364.96 Main SF
Failure happens

 A single datacenter is the
 problem
 • Since they all fail at some point

 Recovery procedures after
 fa...
Truck 1, Rackspace 0
Geography is a
Single Point of Failure
+2304,$5%67"#,-8$1




 !"#$%#&'()(#*&+,&!"#$%&!'()* #%-#%*%,.&'(/,.#+%*&0+.1&-#%2+3&(/."4%*&(2&".&)%"*.&5678
!"#$%&''(   ...
Taser weilding robbers

C I Hosts' Chicago facility
robbed twice!

(the other two times were
merely "break-ins where thing...
Providers are
baskets too.
Failure Happens.
Anyone promising otherwise
 is either foolish or lying
          (or both).
Go Here!



  Jesse Robbins
jesse@oreilly.com
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Failure Happens Interop Nyc
Upcoming SlideShare
Loading in...5
×

Failure Happens Interop Nyc

963

Published on

Published in: Technology, Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
963
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Failure Happens Interop Nyc

  1. 1. Failure Happens F***, the F***ing thing is F***king F***ed* *Official WebOps term from Artur Bergman Jesse Robbins jesse@oreilly.com
  2. 2. This will be on the test: FAILURE HAPPENS!
  3. 3. 25% 75%
  4. 4. 25% 75% Paranoid
  5. 5. 25% Pyromaniac 75% Paranoid
  6. 6. Good Book!
  7. 7. “multiple and unexpected interactions of failures are inevitable” -­‐Charles  Perrow
  8. 8. Failure Happens
  9. 9. define: Nines (roughly)
  10. 10. define: Nines (roughly) 99% 5256 min (3.5 days)
  11. 11. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours )
  12. 12. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min
  13. 13. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min 99.999% 5 min
  14. 14. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min 99.999% 5 min 99.9999% 30 Seconds
  15. 15. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min 99.999% 5 min 99.9999% 30 Seconds 99.99999% 3 Seconds
  16. 16. Internet Routing... won’t.
  17. 17. ;''-1(<"=/-)"3.1>0?-'"@'-': !"#$$%"&'(')*)"+,-.,-/01,( +/.01210*"345467"89: #
  18. 18. #googlefail
  19. 19. YOU
  20. 20. Continuous Power... isn’t
  21. 21. 365 Main SF
  22. 22. 365 364.96 Main SF
  23. 23. Failure happens A single datacenter is the problem • Since they all fail at some point Recovery procedures after failure • Power was gone ~45 minutes • Most services took hours to come back • Some unnamed ones more than 12 hours
  24. 24. Truck 1, Rackspace 0
  25. 25. Geography is a Single Point of Failure
  26. 26. +2304,$5%67"#,-8$1 !"#$%#&'()(#*&+,&!"#$%&!'()* #%-#%*%,.&'(/,.#+%*&0+.1&-#%2+3&(/."4%*&(2&".&)%"*.&5678 !"#$%&''( +#,$-#$,%./-$0,1 )*
  27. 27. Taser weilding robbers C I Hosts' Chicago facility robbed twice! (the other two times were merely "break-ins where things were stolen")
  28. 28. Providers are baskets too.
  29. 29. Failure Happens. Anyone promising otherwise is either foolish or lying (or both).
  30. 30. Go Here! Jesse Robbins jesse@oreilly.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×