Twitter by the Numbers (Columbia University)

by the #s with @raffi

Giv ing a @twitter t alk at ColumbiaUni versity talking a bout Twitter’sNumbers!22 Feb via Twitter for iPhone ty from Mudd Building at Columbia Universi 500 West 120th Street New York, New York View Tweets at this place

http://twitter.com/#!/toptweets/status/12483108178

http://twitter.com/#!/Emergency_In_SF/status/29440739442

http://twitter.com/#!/i80chains/status/9726084734

http://twitter.com/#!/remedyoakland/status/29002198672

http://twitter.com/#!/AlbionsOven/status/10015063036

What’s a Tweet?It’s a short message thats sent through 140 characters

How many are there?

How many are there? 110M!

110M tweets 1200 tweets per day ≈ per second

How big are they? 1 tweet text = 140 characters ≈ 200 bytes

1200 tweets per ≈ 230 KB/sec second ≈ 14 MB/min ≈ 19 GB/day Just tweet text!

MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake

ampura mons from ch used under Creative Com Photo1 TB generated 10 TB generated per day per day

10 TBper day in total ≈ 120 MB per sec 80 MB = per sec Photo used u n der Creative C ommons from Mac Users G uide

Where do they go? Followed by Following Asymmetric Digraph

1 Digraph 2 Need to represent this 4 1 2 3 4 31 Matrix2 Naïve implementation is not scalable34

200M registered users 2006 2008 2010 2011

Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb

@ladygagamother mons†er8.3 million followers@justinbieberJustin Bieber7.5 million followers@BarakObama44th President of the United States6.7 million followers@raffime!0.007 million followers

How do they get out? 10B API calls 100,000 calls per day ≈ per second

REST API XML/JSON API over HTTPPoll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets

Latency200ms100ms 0ms

752%in 2008

1358% in 2009

Where do we want to be? Today - 200M people generate ~1200 TPSTomorrow - we want to support half the world and all its devices (5B phones and 6B people)

Real challenges in front of us Real time Indexing, search, and analytics Relevance systems Graph databases Storage Scalability and efficiency

Follow me atQuestions? twitter.com/raffi

http://www.twitter-howto.com	4159
http://www.freshnetworks.com	1244
http://twitter-howto.com	634
http://michalpalasz.pl	380
http://www.futurelab.net	68
http://zubeta.com	65
http://www.business2community.com	31
http://www.elevensocial.net	28
http://www.visionwt.com	28
http://michaelbatistich.com	22
http://wasabipear.wordpress.com	21
http://www.litmanlive.co.uk	21
http://www.bartvandecasteel.nl	21
http://feeds2.feedburner.com	18
http://www.litmanlive.me	17
http://translate.googleusercontent.com	13
http://dot19.com	12
http://flavors.me	10
http://www.slidescene.com	10
http://www.treffiletti.com	8
http://www.dot19.com	7
http://feeds.feedburner.com	7
http://www.baqmar.be	7
http://tjerkeilander.tumblr.com	6
http://paper.li	6
http://posterous.com	5
http://toby.visionwt.rupert.vwt	5
http://uat2012.visionwt.com	4
https://twitter.com	3
http://webcache.googleusercontent.com	3
https://p.yammer.com	3
http://twitter.com	3
http://www.futurelab.be	2
http://edicolaeuropea.blogspot.com	2
http://localhost	2
http://www.futurelab.be	2
http://iloubnan.info	1
http://litmanlive.tumblr.com	1
http://sites.agorian.com	1
http://web.archive.org	1
http://static.slidesharecdn.com	1
http://dhanpalani.posterous.com	1
http://www.hanrss.com	1
http://www.google.com	1
http://www.planetnography.com	1
http://slidescene.com	1
http://futurelab.net	1
http://74.6.238.254	1
http://conversation.ecairn.com	1
https://s7-us2.ixquick-proxy.com	1

Twitter by the Numbers (Columbia University)

by Raffi Krikorian on Mar 13, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

50 Embeds 6,891

Statistics

Twitter by the Numbers (Columbia University) — Presentation Transcript