More Related Content More from Jesse Robbins (7) Failure Happens Interop Nyc1. Failure Happens
F***, the F***ing thing is F***king F***ed*
*Official WebOps term from Artur Bergman
Jesse Robbins
jesse@oreilly.com
7. 25% Pyromaniac
75% Paranoid
17. define:
Nines (roughly)
99% 5256 min (3.5 days)
99.9% 528 min ( 8.8 hours )
99.99% 53 min
99.999% 5 min
99.9999% 30 Seconds
18. define:
Nines (roughly)
99% 5256 min (3.5 days)
99.9% 528 min ( 8.8 hours )
99.99% 53 min
99.999% 5 min
99.9999% 30 Seconds
99.99999% 3 Seconds
33. Failure happens
A single datacenter is the
problem
• Since they all fail at some point
Recovery procedures after
failure
• Power was gone ~45 minutes
• Most services took hours to come back
• Some unnamed ones more than 12 hours
41. Taser weilding robbers
C I Hosts' Chicago facility
robbed twice!
(the other two times were
merely "break-ins where things
were stolen")
47. Go Here!
Jesse Robbins
jesse@oreilly.com