Twitter for CS10 @ Berkeley (Spring 2011)

  • 1,220 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,220
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. UC Berkeley - CS10
  • 2. Giv ing a talk at Ca l’s CS10 - TheBeauty and Joy of Computing.inst. eecs.berkeley.e du/~cs10/sp1116 Mar via Twitter for iPhone from UC Berkeley 277 Cory Hall Berkeley, CA View Tweets at this place
  • 3. Follow me! twitter.com/raffi or send a SMS to 40404 follow raffi
  • 4. To start... What’s a Tweet?
  • 5. Twitter isthe nature of news.
  • 6. “Rudimentary communicationamong individuals in real timeallows many to move together asone” - @biz
  • 7. One WordApril 10, 2008 4:33:23 PM
  • 8. One Word April 10, 2008 4:33:23 PM And it’s overApril 11, 20081:27:26 PM
  • 9. Super bowl February 2011the Grammys February 2011
  • 10. TunisiaJanuary 2011 Egypt January 2011
  • 11. http://twitter.com/#!/toptweets/status/12483108178
  • 12. http://twitter.com/#!/Emergency_In_SF/status/29440739442
  • 13. http://twitter.com/#!/i80chains/status/9726084734
  • 14. http://twitter.com/#!/remedyoakland/status/29002198672
  • 15. http://twitter.com/#!/AlbionsOven/status/10015063036
  • 16. How does Twitter work?
  • 17. Twitter is just three things:Tweets,relationships between those threeAnd the users, anD Timelines
  • 18. A Tweet is a simple thing 140 characters
  • 19. What’s in a tweet? { “id” : 787167620, “text” : “Free”, “date” : “2008-04-1 1 13:27:26” “user” : “jamesbuck” }* *not quite, but close!
  • 20. Where do they go? Following Followed by
  • 21. Tweets multiply
  • 22. TWEETs MULTIPLY
  • 23. HOME TIMELINE [27886578700, 27856404785, 27727644786, 27679412331, 27594550593, 27583976236, 27580246968, 27576503025, 27545926564, 27545450496, 27539864336...
  • 24. Is that it? ... YES! ... sorta ...
  • 25. things get more complicated at scaleAnd scaling “real-time” is hard
  • 26. How many Tweets are there?
  • 27. How many Tweets are there? 140M! off the chart
  • 28. 140M tweets 1600 tweets per day ≈ per second
  • 29. 140M tweets 1B tweets 1600 tweets per day ≈ per second week
  • 30. Tweets multiply 8,080 people need to see my tweet
  • 31. @ladygagamother mons†er6.8 million followers@justinbieberJustin Bieber5.7 million followers@BarackObama44th President of the United States5.6 million followers@raffime!5.1 thousand followers
  • 32. @ladygagamother mons†er8.8 million followers@raffime!8.0 thousand followers differs by10 scaling 10 63
  • 33. 106deliveries 6000 deliveries per minute ≈ per millisecond
  • 34. A System is only as strong as itsweakest link
  • 35. 80 MB/sec≈ 10 ms/seek
  • 36. 10 ms 100 seeksper seek ≈ per second
  • 37. MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  • 38. How big are they? ≈ 1 tweet text = ≈ 140 characters 18 MB/min ≈ 200 bytes ≈ 26 GB/day Just tweet text!
  • 39. Lots of deliveries
  • 40. 1 Digraph 2 Need to represent this 4 1 2 3 4 31 Matrix2 Naïve implementation is not scalable34
  • 41. Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  • 42. 200M registered users 2006 2008 2010 2011
  • 43. Using the system 10B API calls 110,000 calls per day ≈ per second
  • 44. REST API XML/JSON API over HTTPPoll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
  • 45. Latency200ms100ms 0ms
  • 46. Where do we want to be? Today - 200M people generate ~1600 TPSTomorrow - we want to support half the world and all its devices (right now, there are 6B people and 5B phones)
  • 47. Follow me atQuestions? twitter.com/raffi