• Save
Twitter for CS10 @ Berkeley (Spring 2011)
Upcoming SlideShare
Loading in...5
×
 

Twitter for CS10 @ Berkeley (Spring 2011)

on

  • 1,473 views

 

Statistics

Views

Total Views
1,473
Views on SlideShare
1,471
Embed Views
2

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 2

http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Twitter for CS10 @ Berkeley (Spring 2011) Twitter for CS10 @ Berkeley (Spring 2011) Presentation Transcript

  • UC Berkeley - CS10
  • Giv ing a talk at Ca l’s CS10 - TheBeauty and Joy of Computing.inst. eecs.berkeley.e du/~cs10/sp1116 Mar via Twitter for iPhone from UC Berkeley 277 Cory Hall Berkeley, CA View Tweets at this place
  • Follow me! twitter.com/raffi or send a SMS to 40404 follow raffi
  • To start... What’s a Tweet?
  • Twitter isthe nature of news.
  • “Rudimentary communicationamong individuals in real timeallows many to move together asone” - @biz
  • One WordApril 10, 2008 4:33:23 PM
  • One Word April 10, 2008 4:33:23 PM And it’s overApril 11, 20081:27:26 PM
  • Super bowl February 2011the Grammys February 2011
  • TunisiaJanuary 2011 Egypt January 2011
  • http://twitter.com/#!/toptweets/status/12483108178
  • http://twitter.com/#!/Emergency_In_SF/status/29440739442
  • http://twitter.com/#!/i80chains/status/9726084734
  • http://twitter.com/#!/remedyoakland/status/29002198672
  • http://twitter.com/#!/AlbionsOven/status/10015063036
  • How does Twitter work?
  • Twitter is just three things:Tweets,relationships between those threeAnd the users, anD Timelines
  • A Tweet is a simple thing 140 characters
  • What’s in a tweet? { “id” : 787167620, “text” : “Free”, “date” : “2008-04-1 1 13:27:26” “user” : “jamesbuck” }* *not quite, but close!
  • Where do they go? Following Followed by
  • Tweets multiply
  • TWEETs MULTIPLY
  • HOME TIMELINE [27886578700, 27856404785, 27727644786, 27679412331, 27594550593, 27583976236, 27580246968, 27576503025, 27545926564, 27545450496, 27539864336...
  • Is that it? ... YES! ... sorta ...
  • things get more complicated at scaleAnd scaling “real-time” is hard
  • How many Tweets are there?
  • How many Tweets are there? 140M! off the chart
  • 140M tweets 1600 tweets per day ≈ per second
  • 140M tweets 1B tweets 1600 tweets per day ≈ per second week
  • Tweets multiply 8,080 people need to see my tweet
  • @ladygagamother mons†er6.8 million followers@justinbieberJustin Bieber5.7 million followers@BarackObama44th President of the United States5.6 million followers@raffime!5.1 thousand followers
  • @ladygagamother mons†er8.8 million followers@raffime!8.0 thousand followers differs by10 scaling 10 63
  • 106deliveries 6000 deliveries per minute ≈ per millisecond
  • A System is only as strong as itsweakest link
  • 80 MB/sec≈ 10 ms/seek
  • 10 ms 100 seeksper seek ≈ per second
  • MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  • How big are they? ≈ 1 tweet text = ≈ 140 characters 18 MB/min ≈ 200 bytes ≈ 26 GB/day Just tweet text!
  • Lots of deliveries
  • 1 Digraph 2 Need to represent this 4 1 2 3 4 31 Matrix2 Naïve implementation is not scalable34
  • Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  • 200M registered users 2006 2008 2010 2011
  • Using the system 10B API calls 110,000 calls per day ≈ per second
  • REST API XML/JSON API over HTTPPoll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
  • Latency200ms100ms 0ms
  • Where do we want to be? Today - 200M people generate ~1600 TPSTomorrow - we want to support half the world and all its devices (right now, there are 6B people and 5B phones)
  • Follow me atQuestions? twitter.com/raffi