Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
3. Bid Request
5. Rocket Fuel
6. Ad Served
1. Page Request
4. Bid & Ad
Simple View of Rocket Fuel
It’s a delicate equilibrium that is hard to maintain
Shield it with aggressive alerting, dashboards & canary monitoring
1st region server dies after
several hours of clogged RPC
Bad region moves to another region
server & soon kills it too!
Clients can go rouge, it can get as bad as a DoS attack
Protection via dynamic blacklists & size limit filters
» In absence of proxy: ‘The client is part of the cluster’ 
» Client must report availability error to calling application thread in short time span
» Follow circuit breaker pattern for read calls (Anecdote)
» ‘pseudo’ puts (local file) for write calls
 Blog post from Lars Hofhansl http://hadoop-hbase.blogspot.com/2012/09/hbase-client-timeouts.html