@raffi          Raffi KrikorianGiving a talk at @StanfordACM.Want to know about engineering forreal-time?1 min ago via Twitt...
@JordnJnkieJuice         ✔Finally✈MF✈Famous✔EARTHQUAKE !!!!!1 min ago via Echofon   Favorite   Retweet   Reply
@usgs
300 miles   to  NYC
@remedyoakland         RemedyStrawberry scones and rhubarbturnovers in the house!14 Apr via Facebook   Favorite   Retweet ...
What’s a Tweet?
48 characters                                              +1 for the birdWhat’s a Tweet?It’s a short message thats sent t...
1 tweet
sizeof(1 tweet)
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)≈ 200 bytes
sizeof(1 tweet)≈ 200 bytes
= 140 characterssizeof(1 tweet)                ≈ 200 bytes
= 140 characterssizeof(1 tweet)                ≈ 200 bytes
@Astro_Soichi          Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic       Favorite   Retweet   Reply
@Astro_Soichi          Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic       Favorite   Retweet   Reply
AboutTweets a day
About230,000,000  Tweets a day
2.3E6 tweets    day
2.3E6 tweets 2600 tweets            ≈    day        second
2.3E6 tweets 2600 tweets            ≈    day        second
2.3E6 tweets 2600 tweets            ≈    day        second
2.3E6 tweets 2600 tweets            ≈    day        second
2.3E6 tweets 2600 tweets            ≈    day        second                       steady state
7,196TPS
Tokyo6,600 TPS
8,800 TPS
status/update
status/update
status/update
status/update    HTTP 200 OK{
status/update    HTTP 200 OK{
status/update HTTP 200 OK
status/update HTTP 200 OK
st
s
Snowflakehttps://github.com/twitter/snowflake                                       s
s
s
s
s
sRockdove
s
s
t.co
{
{
Storage          {
{
status/update    HTTP 200 OK{
status/update    HTTP 200 OK{
status/update    HTTP 200 OK{
HTTP 200 OK{    …    "text":"Tweet!"    …}
HTTP 200 OK{    …    "text":"Tweet!"    …}
HTTP 200 OK{    …    "text":"Tweet!"    …}
status/update    HTTP 200 OK{    …    "text":"Tweet!"    …}
status/update    HTTP 200 OK{    …    "text":"Tweet!"    …}
status/update    HTTP 200 OK{    …    "text":"Tweet!"    …}
status/update                      Fanout    HTTP 200 OK{    …    "text":"Tweet!"    …}
status/update                      Fanout    HTTP 200 OK{    …    "text":"Tweet!"    …}
te     FanoutK
te     FanoutK
te     FanoutK
te              Flock     FanoutK
te     FanoutK
te     FanoutK
status/update                      Fanout    HTTP 200 OK{    …    "text":"Tweet!"    …}
status/update                      Fanout    HTTP 200 OK{    …    "text":"Tweet!"    …}
followed by   following
@ladygagamother mons†er13.2 million followers@justinbieberJustin Bieber12.4 million followers@BarackObama44th President of...
@ladygagamother mons†er13.2 million followers@raffime!0.01 million followers
@ladygagamother mons†er13.2 million followers@raffime!0.01 million followers
@ladygaga       mother mons†er       13.2 million followers       @raffi       me!       0.01 million followers10000x more...
@ladygagamother mons†er13.2 million followers@raffime!0.01 million followers  scaling for            107
Has...⇢ huge number of concurrent connections⇢ lots of I/O⇢ few persistent objects
Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasa...
Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasa...
needS...⇢ ability to handle server workloads
needS...⇢ ability to handle server workloads⇢ flexibility in language
needS...⇢ ability to handle server workloads⇢ flexibility in language
needS...⇢ ability to handle server workloads⇢ flexibility in language
needS...⇢ ability to handle server workloads⇢ flexibility in language
needS...⇢ ability to handle server workloads⇢ flexibility in language
needS...⇢ ability to handle server workloads⇢ flexibility in language⇢ a real concurrency model
finagle |fəˈnāgəl|          needS...                                       verb [ trans. ] informal                       ...
finagle |fəˈnāgəl|          needS...                                       verb [ trans. ] informal                       ...
finagle |fəˈnāgəl|          needS...                                       verb [ trans. ] informal                       ...
finagle |fəˈnāgəl|                                       verb [ trans. ] informal          needS...                     ob...
’S Infrastructure trades in...⇢ RAM & CPU⇢ bandwidth⇢ machine density
Switching to      doesn’t implythat     is a mistake
is aReal-Time Event-driven         &         Problem
What happens in the    World    happens on
Follow me atQuestions?   twitter.com/raffi
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
Upcoming SlideShare
Loading in...5
×

Twitter: Engineering for Real-Time (Stanford ACM 2011)

5,297

Published on

Published in: Technology
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,297
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
18
Embeds 0
No embeds

No notes for slide

Twitter: Engineering for Real-Time (Stanford ACM 2011)

  1. 1. @raffi Raffi KrikorianGiving a talk at @StanfordACM.Want to know about engineering forreal-time?1 min ago via Twitter for iPhone Favorite Retweet Reply
  2. 2. @JordnJnkieJuice ✔Finally✈MF✈Famous✔EARTHQUAKE !!!!!1 min ago via Echofon Favorite Retweet Reply
  3. 3. @usgs
  4. 4. 300 miles to NYC
  5. 5. @remedyoakland RemedyStrawberry scones and rhubarbturnovers in the house!14 Apr via Facebook Favorite Retweet Reply
  6. 6. What’s a Tweet?
  7. 7. 48 characters +1 for the birdWhat’s a Tweet?It’s a short message thats sent through 140 characters max
  8. 8. 1 tweet
  9. 9. sizeof(1 tweet)
  10. 10. sizeof(1 tweet)= 140 characters
  11. 11. sizeof(1 tweet)= 140 characters
  12. 12. sizeof(1 tweet)= 140 characters
  13. 13. sizeof(1 tweet)= 140 characters
  14. 14. sizeof(1 tweet)≈ 200 bytes
  15. 15. sizeof(1 tweet)≈ 200 bytes
  16. 16. = 140 characterssizeof(1 tweet) ≈ 200 bytes
  17. 17. = 140 characterssizeof(1 tweet) ≈ 200 bytes
  18. 18. @Astro_Soichi Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic Favorite Retweet Reply
  19. 19. @Astro_Soichi Soichi Noguchihttp://twitpic.com/17dfqb -8 Mar 10 via TwitPic Favorite Retweet Reply
  20. 20. AboutTweets a day
  21. 21. About230,000,000 Tweets a day
  22. 22. 2.3E6 tweets day
  23. 23. 2.3E6 tweets 2600 tweets ≈ day second
  24. 24. 2.3E6 tweets 2600 tweets ≈ day second
  25. 25. 2.3E6 tweets 2600 tweets ≈ day second
  26. 26. 2.3E6 tweets 2600 tweets ≈ day second
  27. 27. 2.3E6 tweets 2600 tweets ≈ day second steady state
  28. 28. 7,196TPS
  29. 29. Tokyo6,600 TPS
  30. 30. 8,800 TPS
  31. 31. status/update
  32. 32. status/update
  33. 33. status/update
  34. 34. status/update HTTP 200 OK{
  35. 35. status/update HTTP 200 OK{
  36. 36. status/update HTTP 200 OK
  37. 37. status/update HTTP 200 OK
  38. 38. st
  39. 39. s
  40. 40. Snowflakehttps://github.com/twitter/snowflake s
  41. 41. s
  42. 42. s
  43. 43. s
  44. 44. s
  45. 45. sRockdove
  46. 46. s
  47. 47. s
  48. 48. t.co
  49. 49. {
  50. 50. {
  51. 51. Storage {
  52. 52. {
  53. 53. status/update HTTP 200 OK{
  54. 54. status/update HTTP 200 OK{
  55. 55. status/update HTTP 200 OK{
  56. 56. HTTP 200 OK{ … "text":"Tweet!" …}
  57. 57. HTTP 200 OK{ … "text":"Tweet!" …}
  58. 58. HTTP 200 OK{ … "text":"Tweet!" …}
  59. 59. status/update HTTP 200 OK{ … "text":"Tweet!" …}
  60. 60. status/update HTTP 200 OK{ … "text":"Tweet!" …}
  61. 61. status/update HTTP 200 OK{ … "text":"Tweet!" …}
  62. 62. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  63. 63. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  64. 64. te FanoutK
  65. 65. te FanoutK
  66. 66. te FanoutK
  67. 67. te Flock FanoutK
  68. 68. te FanoutK
  69. 69. te FanoutK
  70. 70. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  71. 71. status/update Fanout HTTP 200 OK{ … "text":"Tweet!" …}
  72. 72. followed by following
  73. 73. @ladygagamother mons†er13.2 million followers@justinbieberJustin Bieber12.4 million followers@BarackObama44th President of the United States9.9 million followers@raffime!0.01 million followers
  74. 74. @ladygagamother mons†er13.2 million followers@raffime!0.01 million followers
  75. 75. @ladygagamother mons†er13.2 million followers@raffime!0.01 million followers
  76. 76. @ladygaga mother mons†er 13.2 million followers @raffi me! 0.01 million followers10000x more followers than me
  77. 77. @ladygagamother mons†er13.2 million followers@raffime!0.01 million followers scaling for 107
  78. 78. Has...⇢ huge number of concurrent connections⇢ lots of I/O⇢ few persistent objects
  79. 79. Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasant, the cockis distinguished by darkgreen plumage on breast andmantle. The male has aniridescent violet neck, redbare facial skin and purplishgreen tail. The female issmaller than male, and has adull brown plumage withdark spots.
  80. 80. Gosai KijiPhasianus versicolor alsoknown as Japanese Pheasantis a bird of the lowlands.Closely related to theCommon Pheasant, the cockis distinguished by darkgreen plumage on breast andmantle. The male has aniridescent violet neck, redbare facial skin and purplishgreen tail. The female issmaller than male, and has adull brown plumage withdark spots.
  81. 81. needS...⇢ ability to handle server workloads
  82. 82. needS...⇢ ability to handle server workloads⇢ flexibility in language
  83. 83. needS...⇢ ability to handle server workloads⇢ flexibility in language
  84. 84. needS...⇢ ability to handle server workloads⇢ flexibility in language
  85. 85. needS...⇢ ability to handle server workloads⇢ flexibility in language
  86. 86. needS...⇢ ability to handle server workloads⇢ flexibility in language
  87. 87. needS...⇢ ability to handle server workloads⇢ flexibility in language⇢ a real concurrency model
  88. 88. finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  89. 89. finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  90. 90. finagle |fəˈnāgəl| needS... verb [ trans. ] informal obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  91. 91. finagle |fəˈnāgəl| verb [ trans. ] informal needS... obtain (something) by devious or dishonest means : Ted attended all the football games he could finagle tickets for. • [ intrans. ] act in a devious or dishonest manner : they wrangled and finagled over the fine points.⇢ ability to handle server workloads DERIVATIVES⇢ flexibility in language finagler |fəˈnāg(ə)lər| noun⇢ a real concurrency model ORIGIN 1920s (originally U.S.): from dialect fainaigue [cheat] ; perhaps from Old French fornier ‘deny.’
  92. 92. ’S Infrastructure trades in...⇢ RAM & CPU⇢ bandwidth⇢ machine density
  93. 93. Switching to doesn’t implythat is a mistake
  94. 94. is aReal-Time Event-driven & Problem
  95. 95. What happens in the World happens on
  96. 96. Follow me atQuestions? twitter.com/raffi

×