Twitter: Engineering for Real-Time (Stanford ACM 2011)

@rafﬁ Rafﬁ KrikorianGiving a talk at @StanfordACM.Want to know about engineering forreal-time?1 min ago via Twitter for iPhone Favorite Retweet Reply

@JordnJnkieJuice ✔Finally✈MF✈Famous✔EARTHQUAKE !!!!!1 min ago via Echofon Favorite Retweet Reply

300 miles to NYC

@remedyoakland RemedyStrawberry scones and rhubarbturnovers in the house!14 Apr via Facebook Favorite Retweet Reply

What’s a Tweet?

48 characters +1 for the birdWhat’s a Tweet?It’s a short message thats sent through 140 characters max

1 tweet

sizeof(1 tweet)

sizeof(1 tweet)= 140 characters

sizeof(1 tweet)≈ 200 bytes

= 140 characterssizeof(1 tweet) ≈ 200 bytes

@Astro_Soichi Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic Favorite Retweet Reply

AboutTweets a day

About230,000,000 Tweets a day

2.3E6 tweets day

2.3E6 tweets 2600 tweets ≈ day second

2.3E6 tweets 2600 tweets ≈ day second steady state

7,196TPS

Tokyo6,600 TPS

8,800 TPS

status/update

status/update HTTP 200 OK{

status/update HTTP 200 OK

Snowflakehttps://github.com/twitter/snowflake s

sRockdove

Storage {

status/update HTTP 200 OK{

HTTP 200 OK{ … "text":"Tweet!" …}

status/update HTTP 200 OK{ … "text":"Tweet!" …}

status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}

te FanoutK

te Flock FanoutK

te FanoutK

status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}

followed by following

@ladygagamother mons†er13.2 million followers@justinbieberJustin Bieber12.4 million followers@BarackObama44th President of the United States9.9 million followers@raffime!0.01 million followers

@ladygagamother mons†er13.2 million followers@raffime!0.01 million followers

@ladygaga mother mons†er 13.2 million followers @raffi me! 0.01 million followers10000x more followers than me

@ladygagamother mons†er13.2 million followers@raffime!0.01 million followers scaling for 107

Has...⇢ huge number of concurrent connections⇢ lots of I/O⇢ few persistent objects

Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasant, the cockis distinguished by darkgreen plumage on breast andmantle. The male has aniridescent violet neck, redbare facial skin and purplishgreen tail. The female issmaller than male, and has adull brown plumage withdark spots.

needS...⇢ ability to handle server workloads

needS...⇢ ability to handle server workloads⇢ flexibility in language

needS...⇢ ability to handle server workloads⇢ flexibility in language⇢ a real concurrency model

finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’

finagle |fəˈnāgəl| verb [ trans. ] informal needS... obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’

’S Infrastructure trades in...⇢ RAM & CPU⇢ bandwidth⇢ machine density

Switching to doesn’t implythat is a mistake

is aReal-Time Event-driven & Problem

What happens in the World happens on

Follow me atQuestions? twitter.com/raffi

http://paper.li	106
http://www.linkedin.com	25
http://a0.twimg.com	13
http://www.certipost.com	12
http://staging-assets.local.twitter.com	3
http://staging-assets.local.twitter.com	3
http://us-w1.rockmelt.com	3
http://us-w1.rockmelt.com	3

Twitter: Engineering for Real-Time (Stanford ACM 2011)

by Raffi Krikorian on Oct 02, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

8 Embeds 168

Statistics

Twitter: Engineering for Real-Time (Stanford ACM 2011) — Presentation Transcript