SlideShare a Scribd company logo
1 of 26
by the #s
witter by the
    ing a t alk entitled “T
Giv
                   @twit  terU a t Cal.
Number    s” with
Lots o f ##s!                      ne
                 Tw itter for iPho
1m inute ago via
                     on
Retweet ed by 1 pers
What’s a Tweet?

It’s a short message that's sent through




                                           140 characters
How many are there?
How many are there?   70M
                      60M




                            Today! *




                             *off   the chart
op_oh
                                 Commons from lo
                    der Creative
       Photo used un




47M served                                               70M tweets
  per day                                                  per day
70M tweets
   per day   =   800 tweets
                 per second
How big are they?


1 tweet text   =    140
                    characters
               ≈    200 bytes
800 tweets
per second       ≈       160 KB/sec
                 ≈ 9 MB/min
                 ≈ 12 GB/day
      Just tweet text!
MySQL
         Can’t generate IDs fast enough
Centralized and a single point of failure




                                    snowflake
                                    Highly available and uncoordinated (10kqps)
                                    Compatible with the           ecosystem
                                    http://github.com/twitter/snowflake
ampura
                                    Commons from ch
                       der Creative
          Photo used un




1 TB generated                                               8 TB generated
    per day                                                     per day
8 TB
per day
in total
                                                              ≈   100 MB
                                                                  per sec


  Photo used u
               nder Creative C
                              ommons from
                                            Mac Users G
                                                       uide
                                                              =   80 MB
                                                                  per sec
Where do they go?
              Followed by
  Following




                            Asymmetric Digraph
Tweets multiply
1

                   Digraph                           2
            Need to represent this

                                                         4
    1   2      3     4                    3
1




                             Matrix
2




                             Naïve implementation is not scalable
3
4
150M registered users




     2006     2008      2010
Photo used under Creative Commons from jurvetson




          Distributed graph database

flockdb   High rate of CRUD operations
          Complex set arithmetic queries
          http://github.com/twitter/flockdb
@ladygaga
mother mons†er
6.1 million followers

@BarakObama
44th President of the United States
5.3 million followers

@justinbieber
Justin Bieber
5.1 million followers

@raffi
me!
4.1 thousand followers
How do they get out?


6B API calls
    per day    ≈   70,000 calls
                   per second
REST API
         XML/JSON API over HTTP
Poll-based system / pseudo real-time




                               hosebird
                               Streaming API
                               Long poll HTTP
                               Near real-time delivery of Tweets
752%
in 2008
1358%
 in 2009
Where do we want to be?

           Today - 150M people generate ~1000 TPS

Tomorrow - we want to support half the world and all its devices

                  (5B phones and 6B people)
Real challenges in front of us
                 Real time

       Indexing, search, and analytics

             Relevance systems

              Graph databases

                   Storage

          Scalability and efficiency
Questions?   Follow me at
             twitter.com/raffi

More Related Content

What's hot

Omnichannel B2B Architecture
Omnichannel B2B ArchitectureOmnichannel B2B Architecture
Omnichannel B2B ArchitectureDivante
 
Presentation poa
Presentation poaPresentation poa
Presentation poajbenozillo
 
Grofers Business Model By Rakesh Panghal
Grofers Business Model By Rakesh Panghal Grofers Business Model By Rakesh Panghal
Grofers Business Model By Rakesh Panghal Rakesh Panghal
 
Superbrain
SuperbrainSuperbrain
Superbraintdkhar
 
Importance of Yoga in The Corporate Sector
Importance of Yoga in The Corporate SectorImportance of Yoga in The Corporate Sector
Importance of Yoga in The Corporate SectorYogita Mate
 
Importance of yoga for students
Importance of yoga for studentsImportance of yoga for students
Importance of yoga for studentsOakridg Int
 
Tudo que quiser é seu Neville Goddard (1).pdf
Tudo que quiser é seu Neville Goddard (1).pdfTudo que quiser é seu Neville Goddard (1).pdf
Tudo que quiser é seu Neville Goddard (1).pdfMCarvalho5
 
10 Reasons Why You Should Play Golf
10 Reasons Why You Should Play Golf10 Reasons Why You Should Play Golf
10 Reasons Why You Should Play GolfScullivan
 
ML @ Instacart: Improving the quality of On-demand Grocery
ML @ Instacart: Improving the quality of On-demand GroceryML @ Instacart: Improving the quality of On-demand Grocery
ML @ Instacart: Improving the quality of On-demand GrocerySharath Rao
 
Yoga
YogaYoga
YogaSam
 
Yoga for Back Pain
Yoga for Back PainYoga for Back Pain
Yoga for Back PainSatwa Yoga
 
Plan marketing reseau ALLIANCE IN MOTION GLOBAL
Plan marketing reseau ALLIANCE IN MOTION GLOBAL Plan marketing reseau ALLIANCE IN MOTION GLOBAL
Plan marketing reseau ALLIANCE IN MOTION GLOBAL GontrandToh
 

What's hot (18)

Yoga and lifestyle
Yoga and lifestyleYoga and lifestyle
Yoga and lifestyle
 
Omnichannel B2B Architecture
Omnichannel B2B ArchitectureOmnichannel B2B Architecture
Omnichannel B2B Architecture
 
Presentation poa
Presentation poaPresentation poa
Presentation poa
 
Grofers Business Model By Rakesh Panghal
Grofers Business Model By Rakesh Panghal Grofers Business Model By Rakesh Panghal
Grofers Business Model By Rakesh Panghal
 
Superbrain
SuperbrainSuperbrain
Superbrain
 
Kangen Water India
Kangen Water IndiaKangen Water India
Kangen Water India
 
Grofers (Business Model)
Grofers (Business Model)Grofers (Business Model)
Grofers (Business Model)
 
Importance of Yoga in The Corporate Sector
Importance of Yoga in The Corporate SectorImportance of Yoga in The Corporate Sector
Importance of Yoga in The Corporate Sector
 
Importance of yoga for students
Importance of yoga for studentsImportance of yoga for students
Importance of yoga for students
 
Online shopping
Online shoppingOnline shopping
Online shopping
 
Tudo que quiser é seu Neville Goddard (1).pdf
Tudo que quiser é seu Neville Goddard (1).pdfTudo que quiser é seu Neville Goddard (1).pdf
Tudo que quiser é seu Neville Goddard (1).pdf
 
10 Reasons Why You Should Play Golf
10 Reasons Why You Should Play Golf10 Reasons Why You Should Play Golf
10 Reasons Why You Should Play Golf
 
ML @ Instacart: Improving the quality of On-demand Grocery
ML @ Instacart: Improving the quality of On-demand GroceryML @ Instacart: Improving the quality of On-demand Grocery
ML @ Instacart: Improving the quality of On-demand Grocery
 
Why you should save water?
Why you should save water?Why you should save water?
Why you should save water?
 
Yoga
YogaYoga
Yoga
 
Yoga for Back Pain
Yoga for Back PainYoga for Back Pain
Yoga for Back Pain
 
Plan marketing reseau ALLIANCE IN MOTION GLOBAL
Plan marketing reseau ALLIANCE IN MOTION GLOBAL Plan marketing reseau ALLIANCE IN MOTION GLOBAL
Plan marketing reseau ALLIANCE IN MOTION GLOBAL
 
More store
More storeMore store
More store
 

Viewers also liked

What we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can helpWhat we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can helpKen Mugrage
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Rich Heimann
 
Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon Raffi Krikorian
 
Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Raffi Krikorian
 
Aula 5 - Redes Sociais
Aula 5 - Redes SociaisAula 5 - Redes Sociais
Aula 5 - Redes SociaisRebeca Ferrari
 
Sistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplosSistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplosAricelio Souza
 
Scaling Twitter
Scaling TwitterScaling Twitter
Scaling TwitterBlaine
 
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17spark-project
 
Fitbit presentation
Fitbit presentationFitbit presentation
Fitbit presentationjryan39
 
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)We Are Social Singapore
 
Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureP. Taylor Goetz
 
BlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephBlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephSage Weil
 
Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)We Are Social Singapore
 

Viewers also liked (20)

The Future of Check ins
The Future of Check insThe Future of Check ins
The Future of Check ins
 
Hacking Conway's Law
Hacking Conway's LawHacking Conway's Law
Hacking Conway's Law
 
Digital in 2017 Global Overview
Digital in 2017 Global OverviewDigital in 2017 Global Overview
Digital in 2017 Global Overview
 
What we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can helpWhat we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can help
 
Scalding @ Coursera
Scalding @ CourseraScalding @ Coursera
Scalding @ Coursera
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
 
Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon
 
Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)
 
Aula 5 - Redes Sociais
Aula 5 - Redes SociaisAula 5 - Redes Sociais
Aula 5 - Redes Sociais
 
Sistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplosSistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplos
 
Scaling Twitter
Scaling TwitterScaling Twitter
Scaling Twitter
 
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17
 
Fitbit presentation
Fitbit presentationFitbit presentation
Fitbit presentation
 
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
 
Digital in 2017: South America
Digital in 2017: South AmericaDigital in 2017: South America
Digital in 2017: South America
 
Presentation social network
Presentation social networkPresentation social network
Presentation social network
 
Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm Architecture
 
BlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephBlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for Ceph
 
Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)
 
Big data and Hadoop
Big data and HadoopBig data and Hadoop
Big data and Hadoop
 

Similar to Twitter by the Numbers - A Look at Tweet Volume, Users, APIs and More

Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Raffi Krikorian
 
Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Raffi Krikorian
 
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Raffi Krikorian
 
Maintaining reliability in an unreliable world
Maintaining reliability in an unreliable worldMaintaining reliability in an unreliable world
Maintaining reliability in an unreliable worldJeremy Edberg
 
From incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successFrom incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successStartupfest
 
The Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet ScaleThe Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet Scalegoodfriday
 
Why all payments innovations are rubbish
Why all payments innovations are rubbishWhy all payments innovations are rubbish
Why all payments innovations are rubbishChris Skinner
 
The History and Possible Futures of the Internet
The History and Possible Futures of the InternetThe History and Possible Futures of the Internet
The History and Possible Futures of the InternetLarry Smarr
 
Big Data vs Data Warehousing
Big Data vs Data WarehousingBig Data vs Data Warehousing
Big Data vs Data WarehousingThomas Kejser
 
2012: The End of the World?
2012: The End of the World?2012: The End of the World?
2012: The End of the World?Saumil Shah
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb, Inc.
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Masa Nakatsu
 
The Creativity Machine
The Creativity MachineThe Creativity Machine
The Creativity MachineSal
 
Maq Software Live On Cutting Edge Dream Spark Yatra
Maq Software  Live On Cutting Edge  Dream Spark YatraMaq Software  Live On Cutting Edge  Dream Spark Yatra
Maq Software Live On Cutting Edge Dream Spark YatraMAQ Software
 
Riak at Posterous
Riak at PosterousRiak at Posterous
Riak at Posterouscapotej
 

Similar to Twitter by the Numbers - A Look at Tweet Volume, Users, APIs and More (20)

Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)
 
Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)
 
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
 
500Startups @ Twitter
500Startups @ Twitter500Startups @ Twitter
500Startups @ Twitter
 
Maintaining reliability in an unreliable world
Maintaining reliability in an unreliable worldMaintaining reliability in an unreliable world
Maintaining reliability in an unreliable world
 
From incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successFrom incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator success
 
The Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet ScaleThe Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet Scale
 
Why all payments innovations are rubbish
Why all payments innovations are rubbishWhy all payments innovations are rubbish
Why all payments innovations are rubbish
 
Colorado leadership v4
Colorado leadership v4Colorado leadership v4
Colorado leadership v4
 
The History and Possible Futures of the Internet
The History and Possible Futures of the InternetThe History and Possible Futures of the Internet
The History and Possible Futures of the Internet
 
Big Data vs Data Warehousing
Big Data vs Data WarehousingBig Data vs Data Warehousing
Big Data vs Data Warehousing
 
Ngn abridged oct2010
Ngn abridged oct2010Ngn abridged oct2010
Ngn abridged oct2010
 
2012: The End of the World?
2012: The End of the World?2012: The End of the World?
2012: The End of the World?
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?
 
The Creativity Machine
The Creativity MachineThe Creativity Machine
The Creativity Machine
 
IPv6 Matrix Project
IPv6 Matrix ProjectIPv6 Matrix Project
IPv6 Matrix Project
 
IPv6 Matrix Project - general presentation
IPv6 Matrix Project - general presentationIPv6 Matrix Project - general presentation
IPv6 Matrix Project - general presentation
 
Maq Software Live On Cutting Edge Dream Spark Yatra
Maq Software  Live On Cutting Edge  Dream Spark YatraMaq Software  Live On Cutting Edge  Dream Spark Yatra
Maq Software Live On Cutting Edge Dream Spark Yatra
 
Riak at Posterous
Riak at PosterousRiak at Posterous
Riak at Posterous
 

More from Raffi Krikorian

Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)Raffi Krikorian
 
Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Raffi Krikorian
 
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Raffi Krikorian
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time WebRaffi Krikorian
 
Developing for @twitterapi #hack4health
Developing for @twitterapi #hack4healthDeveloping for @twitterapi #hack4health
Developing for @twitterapi #hack4healthRaffi Krikorian
 
Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Raffi Krikorian
 
How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010Raffi Krikorian
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapiRaffi Krikorian
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ ChirpRaffi Krikorian
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time GeostreamsRaffi Krikorian
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"Raffi Krikorian
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Raffi Krikorian
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorRaffi Krikorian
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingRaffi Krikorian
 

More from Raffi Krikorian (20)

Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
 
Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)
 
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
 
#rtgeo (Where 2.0 2011)
#rtgeo (Where 2.0 2011)#rtgeo (Where 2.0 2011)
#rtgeo (Where 2.0 2011)
 
Users and Geo
Users and GeoUsers and Geo
Users and Geo
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time Web
 
Developing for @twitterapi #hack4health
Developing for @twitterapi #hack4healthDeveloping for @twitterapi #hack4health
Developing for @twitterapi #hack4health
 
Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)
 
How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapi
 
Twitter API Annotations
Twitter API AnnotationsTwitter API Annotations
Twitter API Annotations
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp
 
Energy / Tweet
Energy / TweetEnergy / Tweet
Energy / Tweet
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time Geostreams
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"
 
What's happening here?
What's happening here?What's happening here?
What's happening here?
 
WattzOn @ ETech 2009
WattzOn @ ETech 2009WattzOn @ ETech 2009
WattzOn @ ETech 2009
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth Simulator
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warming
 

Recently uploaded

Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationDianaGray10
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Tecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for RotogravureTecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for RotogravureAntonio de Llamas
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?Rustici Software
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Memoori
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Tetracrom printing process for packaging with CMYK+
Tetracrom printing process for packaging with CMYK+Tetracrom printing process for packaging with CMYK+
Tetracrom printing process for packaging with CMYK+Antonio de Llamas
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfHCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfROWELL MARQUINA
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxmprakaash5
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationBuild Intuit
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 

Recently uploaded (20)

Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automation
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Tecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for RotogravureTecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for Rotogravure
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Tetracrom printing process for packaging with CMYK+
Tetracrom printing process for packaging with CMYK+Tetracrom printing process for packaging with CMYK+
Tetracrom printing process for packaging with CMYK+
 
BoSEU24 | Bill Thompson | Talk From Another Century
BoSEU24 | Bill Thompson | Talk From Another CenturyBoSEU24 | Bill Thompson | Talk From Another Century
BoSEU24 | Bill Thompson | Talk From Another Century
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfHCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptx
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientation
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 

Twitter by the Numbers - A Look at Tweet Volume, Users, APIs and More

  • 2. witter by the ing a t alk entitled “T Giv @twit terU a t Cal. Number s” with Lots o f ##s! ne Tw itter for iPho 1m inute ago via on Retweet ed by 1 pers
  • 3. What’s a Tweet? It’s a short message that's sent through 140 characters
  • 4. How many are there?
  • 5. How many are there? 70M 60M Today! * *off the chart
  • 6. op_oh Commons from lo der Creative Photo used un 47M served 70M tweets per day per day
  • 7. 70M tweets per day = 800 tweets per second
  • 8.
  • 9. How big are they? 1 tweet text = 140 characters ≈ 200 bytes
  • 10. 800 tweets per second ≈ 160 KB/sec ≈ 9 MB/min ≈ 12 GB/day Just tweet text!
  • 11. MySQL Can’t generate IDs fast enough Centralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  • 12. ampura Commons from ch der Creative Photo used un 1 TB generated 8 TB generated per day per day
  • 13. 8 TB per day in total ≈ 100 MB per sec Photo used u nder Creative C ommons from Mac Users G uide = 80 MB per sec
  • 14. Where do they go? Followed by Following Asymmetric Digraph
  • 16. 1 Digraph 2 Need to represent this 4 1 2 3 4 3 1 Matrix 2 Naïve implementation is not scalable 3 4
  • 17. 150M registered users 2006 2008 2010
  • 18. Photo used under Creative Commons from jurvetson Distributed graph database flockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  • 19. @ladygaga mother mons†er 6.1 million followers @BarakObama 44th President of the United States 5.3 million followers @justinbieber Justin Bieber 5.1 million followers @raffi me! 4.1 thousand followers
  • 20. How do they get out? 6B API calls per day ≈ 70,000 calls per second
  • 21. REST API XML/JSON API over HTTP Poll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
  • 24. Where do we want to be? Today - 150M people generate ~1000 TPS Tomorrow - we want to support half the world and all its devices (5B phones and 6B people)
  • 25. Real challenges in front of us Real time Indexing, search, and analytics Relevance systems Graph databases Storage Scalability and efficiency
  • 26. Questions? Follow me at twitter.com/raffi