Twitter - Guest Lecture UC Berkeley CS10 Fall 2010

  • 2,491 views
Uploaded on

I was invited to give a guest lecture at Cal's CS10 ("The Beauty and Joy of Computing"). These are the slides on Twitter.

I was invited to give a guest lecture at Cal's CS10 ("The Beauty and Joy of Computing"). These are the slides on Twitter.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,491
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. UC Berkeley - CS10
  • 2. Giv ing a talk at Ca l’s CS10 - The Beauty and Joy of Computing. inst. eecs.berkeley.e du/~cs10/fa10 20 Oct via Twitter for iPhone from UC Berkeley 155 Donner Lab Berkeley, CA View Tweets at this place
  • 3. To start... What’s a Tweet?
  • 4. Follow me! twitter.com/raffi or send a SMS to 40404 follow raffi
  • 5. “Rudimentary communication among individuals in real time allows many to move together as one” - @biz
  • 6. One Word April 10, 2008 4:33:23 PM
  • 7. One Word April 10, 2008 4:33:23 PM And it’s over April 11, 2008 1:27:26 PM
  • 8. Haiti January 2010 Chile February 2010
  • 9. MTV VMAs September 2010 Emmys August 2010
  • 10. The humble beginnings
  • 11. How does Twitter work?
  • 12. Twitter is just A “struct”, a graph, & a LIST
  • 13. A Tweet is a simple thing 140 characters
  • 14. What’s The data structure of a tweet? { “id ” : 787167620, “text” : “Free”, “ date” : “2008-04 -11 13:27:26” “user” : “jamesbuck” }* *not quite, but something like this
  • 15. Where do they go? Following Followed by
  • 16. Tweets multiply!
  • 17. It’s a huge graph
  • 18. Viewing Tweets
  • 19. Home Timeline [27886578700, 27856404785, 27727644786, 27679412331, 27594550593, 27583976236, 27580246968, 27576503025, 27545926564, 27545450496, 27539864336...
  • 20. Is that it? ... YES! ... sorta ...
  • 21. things get more complicated at scale
  • 22. And scaling real-time is hard
  • 23. How many Tweets are there?
  • 24. How many Tweets are there? 90M!
  • 25. 90M tweets per day ≈ 1000 tweets per second
  • 26. Lots of deliveries
  • 27. Tweets multiply 5,162 people need to see my tweet
  • 28. @ladygaga mother mons†er 6.8 million followers @justinbieber Justin Bieber 5.7 million followers @BarakObama 44th President of the United States 5.6 million followers @raffi me! 5.1 thousand followers
  • 29. @ladygaga mother mons†er 6.8 x 106 followers @raffi me! 5.1 x 103 followers differs by 10 3
  • 30. @ladygaga mother mons†er 6.8 x 106 followers scaling 10 6
  • 31. 6000 106 deliveries per minute = deliveries per millisecond
  • 32. A System is only as strong as its weakest link
  • 33. 80 MB per sec ≈ 10 ms per seek
  • 34. 10 ms per seek = 100 seeks per second
  • 35. MySQL Can’t generate IDs fast enough Centralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  • 36. How big are they? 1 tweet text = 140 characters ≈ 200 bytes
  • 37. 1000 tweets per second ≈ 195 KB/sec ≈ 11 MB/min ≈ 16 GB/day Just tweet text!
  • 38. Lots of deliveries
  • 39. 1 Digraph 2 Need to represent this 4 1 2 3 4 3 1 Matrix 2 Naïve implementation is not scalable 3 4
  • 40. Photo used under Creative Commons from jurvetson Distributed graph database flockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  • 41. 160M registered users 2006 2008 2010
  • 42. Using the system 6B API calls per day ≈ 70,000 calls per second
  • 43. Where do we want to be? Today - 160M people generate ~1000 TPS Tomorrow - we want to support half the world and all its devices (right now, there are 6B people and 5B phones)
  • 44. Follow me at Questions? twitter.com/raffi