Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Failure Happens: CloudCamp Interop

1,950 views

Published on

Published in: Technology

Failure Happens: CloudCamp Interop

  1. 1. Failure Happens F***, the F***ing thing is F***king F***ed* *Official WebOps term from Artur Bergman jesse@oreilly.com
  2. 2. This will be on the test: FAILURE HAPPENS!
  3. 3. 25% 75%
  4. 4. 25% 75% Paranoid
  5. 5. 25% Pyromaniac 75% Paranoid
  6. 6. Good Book!
  7. 7. “multiple and unexpected interactions of failures are inevitable” -Charles Perrow
  8. 8. Failure Happens
  9. 9. define: Nines (roughly)
  10. 10. define: Nines (roughly) 99% 5256 min (3.5 days)
  11. 11. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours )
  12. 12. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min
  13. 13. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min 99.999% 5 min
  14. 14. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min 99.999% 5 min 99.9999% 30 Seconds
  15. 15. define: Nines (roughly) 99% 5256 min (3.5 days) 99.9% 528 min ( 8.8 hours ) 99.99% 53 min 99.999% 5 min 99.9999% 30 Seconds 99.99999% 3 Seconds
  16. 16. Internet Routing... won’t.
  17. 17. ;''-1(<quot;=/-)quot;3.1>0?-'quot;@'-': !quot;#$$%quot;&'(')*)quot;+,-.,-/01,( +/.01210*quot;345467quot;89: #
  18. 18. #googlefail
  19. 19. YOU
  20. 20. Continuous Power... isn’t
  21. 21. 365 Main SF
  22. 22. 365 364.96 Main SF
  23. 23. Failure happens A single datacenter is the problem • Since they all fail at some point Recovery procedures after failure • Power was gone ~45 minutes • Most services took hours to come back • Some unnamed ones more than 12 hours
  24. 24. Truck 1, Rackspace 0
  25. 25. Geography is a Single Point of Failure
  26. 26. +2304,$5%67quot;#,-8$1 !quot;#$%#&'()(#*&+,&!quot;#$%&!'()* #%-#%*%,.&'(/,.#+%*&0+.1&-#%2+3&(/.quot;4%*&(2&quot;.&)%quot;*.&5678 !quot;#$%&''( +#,$-#$,%./-$0,1 )*
  27. 27. Taser weilding robbers C I Hosts' Chicago facility robbed twice! (the other two times were merely quot;break-ins where things were stolenquot;)
  28. 28. Providers are baskets too.
  29. 29. Failure Happens. Anyone promising otherwise is either foolish or lying (or both).
  30. 30. Go Here! une 22-24, 2009 Jesse Robbins jesse@oreilly.com

×