Twitter for CS10 @ Berkeley (Spring 2011)

1,697 views
1,599 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,697
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Twitter for CS10 @ Berkeley (Spring 2011)

  1. 1. UC Berkeley - CS10
  2. 2. Giv ing a talk at Ca l’s CS10 - TheBeauty and Joy of Computing.inst. eecs.berkeley.e du/~cs10/sp1116 Mar via Twitter for iPhone from UC Berkeley 277 Cory Hall Berkeley, CA View Tweets at this place
  3. 3. Follow me! twitter.com/raffi or send a SMS to 40404 follow raffi
  4. 4. To start... What’s a Tweet?
  5. 5. Twitter isthe nature of news.
  6. 6. “Rudimentary communicationamong individuals in real timeallows many to move together asone” - @biz
  7. 7. One WordApril 10, 2008 4:33:23 PM
  8. 8. One Word April 10, 2008 4:33:23 PM And it’s overApril 11, 20081:27:26 PM
  9. 9. Super bowl February 2011the Grammys February 2011
  10. 10. TunisiaJanuary 2011 Egypt January 2011
  11. 11. http://twitter.com/#!/toptweets/status/12483108178
  12. 12. http://twitter.com/#!/Emergency_In_SF/status/29440739442
  13. 13. http://twitter.com/#!/i80chains/status/9726084734
  14. 14. http://twitter.com/#!/remedyoakland/status/29002198672
  15. 15. http://twitter.com/#!/AlbionsOven/status/10015063036
  16. 16. How does Twitter work?
  17. 17. Twitter is just three things:Tweets,relationships between those threeAnd the users, anD Timelines
  18. 18. A Tweet is a simple thing 140 characters
  19. 19. What’s in a tweet? { “id” : 787167620, “text” : “Free”, “date” : “2008-04-1 1 13:27:26” “user” : “jamesbuck” }* *not quite, but close!
  20. 20. Where do they go? Following Followed by
  21. 21. Tweets multiply
  22. 22. TWEETs MULTIPLY
  23. 23. HOME TIMELINE [27886578700, 27856404785, 27727644786, 27679412331, 27594550593, 27583976236, 27580246968, 27576503025, 27545926564, 27545450496, 27539864336...
  24. 24. Is that it? ... YES! ... sorta ...
  25. 25. things get more complicated at scaleAnd scaling “real-time” is hard
  26. 26. How many Tweets are there?
  27. 27. How many Tweets are there? 140M! off the chart
  28. 28. 140M tweets 1600 tweets per day ≈ per second
  29. 29. 140M tweets 1B tweets 1600 tweets per day ≈ per second week
  30. 30. Tweets multiply 8,080 people need to see my tweet
  31. 31. @ladygagamother mons†er6.8 million followers@justinbieberJustin Bieber5.7 million followers@BarackObama44th President of the United States5.6 million followers@raffime!5.1 thousand followers
  32. 32. @ladygagamother mons†er8.8 million followers@raffime!8.0 thousand followers differs by10 scaling 10 63
  33. 33. 106deliveries 6000 deliveries per minute ≈ per millisecond
  34. 34. A System is only as strong as itsweakest link
  35. 35. 80 MB/sec≈ 10 ms/seek
  36. 36. 10 ms 100 seeksper seek ≈ per second
  37. 37. MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  38. 38. How big are they? ≈ 1 tweet text = ≈ 140 characters 18 MB/min ≈ 200 bytes ≈ 26 GB/day Just tweet text!
  39. 39. Lots of deliveries
  40. 40. 1 Digraph 2 Need to represent this 4 1 2 3 4 31 Matrix2 Naïve implementation is not scalable34
  41. 41. Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  42. 42. 200M registered users 2006 2008 2010 2011
  43. 43. Using the system 10B API calls 110,000 calls per day ≈ per second
  44. 44. REST API XML/JSON API over HTTPPoll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
  45. 45. Latency200ms100ms 0ms
  46. 46. Where do we want to be? Today - 200M people generate ~1600 TPSTomorrow - we want to support half the world and all its devices (right now, there are 6B people and 5B phones)
  47. 47. Follow me atQuestions? twitter.com/raffi

×