On October 23rd, 2014, we updated our
By continuing to use LinkedIn’s SlideShare service, you agree to the revised terms, so please take a few minutes to review them.
106deliveries 6000 deliveries per minute ≈ per millisecond
A System is only as strong as itsweakest link
80 MB/sec≈ 10 ms/seek
10 ms 100 seeksper seek ≈ per second
MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
How big are they? ≈ 1 tweet text = ≈ 140 characters 18 MB/min ≈ 200 bytes ≈ 26 GB/day Just tweet text!
Lots of deliveries
1 Digraph 2 Need to represent this 4 1 2 3 4 31 Matrix2 Naïve implementation is not scalable34
Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
200M registered users 2006 2008 2010 2011
Using the system 10B API calls 110,000 calls per day ≈ per second
REST API XML/JSON API over HTTPPoll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
Where do we want to be? Today - 200M people generate ~1600 TPSTomorrow - we want to support half the world and all its devices (right now, there are 6B people and 5B phones)