• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Twitter for CS10 @ Berkeley (Spring 2011)
 

Twitter for CS10 @ Berkeley (Spring 2011)

on

  • 1,446 views

 

Statistics

Views

Total Views
1,446
Views on SlideShare
1,444
Embed Views
2

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 2

http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Twitter for CS10 @ Berkeley (Spring 2011) Twitter for CS10 @ Berkeley (Spring 2011) Presentation Transcript

    • UC Berkeley - CS10
    • Giv ing a talk at Ca l’s CS10 - TheBeauty and Joy of Computing.inst. eecs.berkeley.e du/~cs10/sp1116 Mar via Twitter for iPhone from UC Berkeley 277 Cory Hall Berkeley, CA View Tweets at this place
    • Follow me! twitter.com/raffi or send a SMS to 40404 follow raffi
    • To start... What’s a Tweet?
    • Twitter isthe nature of news.
    • “Rudimentary communicationamong individuals in real timeallows many to move together asone” - @biz
    • One WordApril 10, 2008 4:33:23 PM
    • One Word April 10, 2008 4:33:23 PM And it’s overApril 11, 20081:27:26 PM
    • Super bowl February 2011the Grammys February 2011
    • TunisiaJanuary 2011 Egypt January 2011
    • http://twitter.com/#!/toptweets/status/12483108178
    • http://twitter.com/#!/Emergency_In_SF/status/29440739442
    • http://twitter.com/#!/i80chains/status/9726084734
    • http://twitter.com/#!/remedyoakland/status/29002198672
    • http://twitter.com/#!/AlbionsOven/status/10015063036
    • How does Twitter work?
    • Twitter is just three things:Tweets,relationships between those threeAnd the users, anD Timelines
    • A Tweet is a simple thing 140 characters
    • What’s in a tweet? { “id” : 787167620, “text” : “Free”, “date” : “2008-04-1 1 13:27:26” “user” : “jamesbuck” }* *not quite, but close!
    • Where do they go? Following Followed by
    • Tweets multiply
    • TWEETs MULTIPLY
    • HOME TIMELINE [27886578700, 27856404785, 27727644786, 27679412331, 27594550593, 27583976236, 27580246968, 27576503025, 27545926564, 27545450496, 27539864336...
    • Is that it? ... YES! ... sorta ...
    • things get more complicated at scaleAnd scaling “real-time” is hard
    • How many Tweets are there?
    • How many Tweets are there? 140M! off the chart
    • 140M tweets 1600 tweets per day ≈ per second
    • 140M tweets 1B tweets 1600 tweets per day ≈ per second week
    • Tweets multiply 8,080 people need to see my tweet
    • @ladygagamother mons†er6.8 million followers@justinbieberJustin Bieber5.7 million followers@BarackObama44th President of the United States5.6 million followers@raffime!5.1 thousand followers
    • @ladygagamother mons†er8.8 million followers@raffime!8.0 thousand followers differs by10 scaling 10 63
    • 106deliveries 6000 deliveries per minute ≈ per millisecond
    • A System is only as strong as itsweakest link
    • 80 MB/sec≈ 10 ms/seek
    • 10 ms 100 seeksper seek ≈ per second
    • MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
    • How big are they? ≈ 1 tweet text = ≈ 140 characters 18 MB/min ≈ 200 bytes ≈ 26 GB/day Just tweet text!
    • Lots of deliveries
    • 1 Digraph 2 Need to represent this 4 1 2 3 4 31 Matrix2 Naïve implementation is not scalable34
    • Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
    • 200M registered users 2006 2008 2010 2011
    • Using the system 10B API calls 110,000 calls per day ≈ per second
    • REST API XML/JSON API over HTTPPoll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
    • Latency200ms100ms 0ms
    • Where do we want to be? Today - 200M people generate ~1600 TPSTomorrow - we want to support half the world and all its devices (right now, there are 6B people and 5B phones)
    • Follow me atQuestions? twitter.com/raffi