Scaling Twitter
Alex & Britt
(from Twitter)

Preconceived notions.

The story.

So, what happened?

What we did.

More boxes.
Not so many then. But now...
‣
Mongrels across 19 cores.
‣
Message processing/delivery across 16 cores.
‣
Jabber across 2 cores.
‣
MySQL on one big 8 core box.
‣
16GB+ for memcache across a bunch of
‣
machines.
‣ 32 cores total.

That’s a lot.

Why?
200 - 300 connections per second.
‣
Spiking at 800 connections per second.
‣
We’ve done 11,000 connections per second.
‣
MySQL has spiked to 2,400 queries per second.
‣
Alexa says we have a whole mess o’ traffic, and
‣
they’re not even seeing the API traffic.

memcache.

memcache.

memcache.

Cache the hell out of everything.

Denormalize.

find(:all,
:conditions => \"substring(
truncate(id,0),-2,1) =
#{@fugly_dist_idx}\",
:limit => 10)

Starling.

Community.

YOU CAN HAS QUESTIONS