Twitter: Engineering for Real-Time (Stanford ACM 2011)

9,604 views
9,040 views

Published on

Published in: Technology

Twitter: Engineering for Real-Time (Stanford ACM 2011)

  1. @raffi Raffi KrikorianGiving a talk at @StanfordACM.Want to know about engineering forreal-time?1 min ago via Twitter for iPhone Favorite Retweet Reply
  2. @JordnJnkieJuice ✔Finally✈MF✈Famous✔EARTHQUAKE !!!!!1 min ago via Echofon Favorite Retweet Reply
  3. @usgs
  4. 300 miles to NYC
  5. @remedyoakland RemedyStrawberry scones and rhubarbturnovers in the house!14 Apr via Facebook Favorite Retweet Reply
  6. What’s a Tweet?
  7. 48 characters +1 for the birdWhat’s a Tweet?It’s a short message thats sent through 140 characters max
  8. 1 tweet
  9. sizeof(1 tweet)
  10. sizeof(1 tweet)= 140 characters
  11. sizeof(1 tweet)= 140 characters
  12. sizeof(1 tweet)= 140 characters
  13. sizeof(1 tweet)= 140 characters
  14. sizeof(1 tweet)≈ 200 bytes
  15. sizeof(1 tweet)≈ 200 bytes
  16. = 140 characterssizeof(1 tweet) ≈ 200 bytes
  17. = 140 characterssizeof(1 tweet) ≈ 200 bytes
  18. @Astro_Soichi Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic Favorite Retweet Reply
  19. @Astro_Soichi Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic Favorite Retweet Reply
  20. AboutTweets a day
  21. About230,000,000 Tweets a day
  22. 2.3E6 tweets day
  23. 2.3E6 tweets 2600 tweets ≈ day second
  24. 2.3E6 tweets 2600 tweets ≈ day second
  25. 2.3E6 tweets 2600 tweets ≈ day second
  26. 2.3E6 tweets 2600 tweets ≈ day second
  27. 2.3E6 tweets 2600 tweets ≈ day second steady state
  28. 7,196TPS
  29. Tokyo6,600 TPS
  30. 8,800 TPS
  31. status/update
  32. status/update
  33. status/update
  34. status/update HTTP 200 OK{
  35. status/update HTTP 200 OK{
  36. status/update HTTP 200 OK
  37. status/update HTTP 200 OK
  38. st
  39. s
  40. Snowflakehttps://github.com/twitter/snowflake s
  41. s
  42. s
  43. s
  44. s
  45. sRockdove
  46. s
  47. s
  48. t.co
  49. {
  50. {
  51. Storage {
  52. {
  53. status/update HTTP 200 OK{
  54. status/update HTTP 200 OK{
  55. status/update HTTP 200 OK{
  56. HTTP 200 OK{ … "text":"Tweet!" …}
  57. HTTP 200 OK{ … "text":"Tweet!" …}
  58. HTTP 200 OK{ … "text":"Tweet!" …}
  59. status/update HTTP 200 OK{ … "text":"Tweet!" …}
  60. status/update HTTP 200 OK{ … "text":"Tweet!" …}
  61. status/update HTTP 200 OK{ … "text":"Tweet!" …}
  62. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  63. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  64. te FanoutK
  65. te FanoutK
  66. te FanoutK
  67. te Flock FanoutK
  68. te FanoutK
  69. te FanoutK
  70. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  71. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  72. followed by following
  73. @ladygagamother mons†er13.2 million followers@justinbieberJustin Bieber12.4 million followers@BarackObama44th President of the United States9.9 million followers@raffime!0.01 million followers
  74. @ladygagamother mons†er13.2 million followers@raffime!0.01 million followers
  75. @ladygagamother mons†er13.2 million followers@raffime!0.01 million followers
  76. @ladygaga mother mons†er 13.2 million followers @raffi me! 0.01 million followers10000x more followers than me
  77. @ladygagamother mons†er13.2 million followers@raffime!0.01 million followers scaling for 107
  78. Has...⇢ huge number of concurrent connections⇢ lots of I/O⇢ few persistent objects
  79. Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasant, the cockis distinguished by darkgreen plumage on breast andmantle. The male has aniridescent violet neck, redbare facial skin and purplishgreen tail. The female issmaller than male, and has adull brown plumage withdark spots.
  80. Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasant, the cockis distinguished by darkgreen plumage on breast andmantle. The male has aniridescent violet neck, redbare facial skin and purplishgreen tail. The female issmaller than male, and has adull brown plumage withdark spots.
  81. needS...⇢ ability to handle server workloads
  82. needS...⇢ ability to handle server workloads⇢ flexibility in language
  83. needS...⇢ ability to handle server workloads⇢ flexibility in language
  84. needS...⇢ ability to handle server workloads⇢ flexibility in language
  85. needS...⇢ ability to handle server workloads⇢ flexibility in language
  86. needS...⇢ ability to handle server workloads⇢ flexibility in language
  87. needS...⇢ ability to handle server workloads⇢ flexibility in language⇢ a real concurrency model
  88. finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  89. finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  90. finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  91. finagle |fəˈnāgəl| verb [ trans. ] informal needS... obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  92. ’S Infrastructure trades in...⇢ RAM & CPU⇢ bandwidth⇢ machine density
  93. Switching to doesn’t implythat is a mistake
  94. is aReal-Time Event-driven & Problem
  95. What happens in the World happens on
  96. Follow me atQuestions? twitter.com/raffi

×