Scaling Twitter - Railsconf 2007

Slide 1: Scaling Twitter Alex & Britt (from Twitter)

Slide 2: Preconceived notions.

Slide 3: The story.

Slide 4: So, what happened?

Slide 5: What we did.

Slide 6: More boxes. Not so many then. But now... ‣ Mongrels across 19 cores. ‣ Message processing/delivery across 16 cores. ‣ Jabber across 2 cores. ‣ MySQL on one big 8 core box. ‣ 16GB+ for memcache across a bunch of ‣ machines. ‣ 32 cores total.

Slide 7: That’s a lot.

Slide 8: Why? 200 - 300 connections per second. ‣ Spiking at 800 connections per second. ‣ We’ve done 11,000 connections per second. ‣ MySQL has spiked to 2,400 queries per second. ‣ Alexa says we have a whole mess o’ traffic, and ‣ they’re not even seeing the API traffic.

Slide 9: memcache.

Slide 10: memcache.

Slide 11: memcache.

Slide 12: Cache the hell out of everything.

Slide 13: Denormalize.

Slide 14: find(:all, :conditions => \"substring( truncate(id,0),-2,1) = #{@fugly_dist_idx}\", :limit => 10)

Slide 15: Starling.

Slide 16: Community.

Slide 17: YOU CAN HAS QUESTIONS