500Startups @ Twitter


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

500Startups @ Twitter

  1. 1. 500Startups
  2. 2. Giving a talk about ou r APIs and oursc ale to the visitin g 500Startupsclass.28 Mar via Twitter for iPhone from Twitter 795 Folsom Street San Francisco, CA View Tweets at this place
  3. 3. >750K client applications +Officially Owned and Operated
  4. 4. Twitter is just three things:Tweets,relationships between those threeAnd the users, anD Timelines
  5. 5. What is the TwitterAPI?REST API- provides basic Twitter functionality- Read / Write (Tweet, Follow, DM, etc.)Search API- real-time search indexStreaming API- HTTP long poll connection- Tweets in real-time
  6. 6. Using the system 10B API calls 115,000 calls per day ≈ per second
  7. 7. Three main objectsStatus Objects - the Tweet (text, author, and metadata)User objects - username, screen name, avatarTimelines - orderings of Tweets
  8. 8. Quick note on timelinesuser - a the tweets a given user has authoredhome - the main timeline you would see in a clientMentions - all the tweets that @mention a user
  9. 9. Causing changeTweeting - a POST to status/updateFollowing - a POST to friendships/createDM-ing - a POST to direct_messages/new
  10. 10. AuthenticationOAuth 1.0a- signature based requests- user driven access to the API- client differentiated access to the APIApplications don’t have passwords- applications store tokens for their users- users can change passwords, but tokens still workOAuth 2.0- coming soon!
  11. 11. Limits350 OAuth calls / user / hour / IP- authenticated calls goes against calling user- unauthenticated calls goes against calling IP“Natural” limits- limits on number of Tweets / DMs sent- limits on number of follows / unfollows a dayStatus Limits- can’t send “duplicate” tweets
  12. 12. Streaming APIPersistent connections- get pushed a tweet, in real-time, that matches your predicate- “push” version of search- read-onlyUser streams / Site streams- re-create the client experience using streams- great for “client” experiences
  13. 13. Latency200ms100ms 0ms
  14. 14. Streaming API’s Track and followTrack- watch a particular keyword- up to 200 can be sentFollow- get all the tweets (RTs, etc.) from a particular user- up to 400 can be issued
  15. 15. Tools of the Tradedev.twitter.com- documentation centraltwurl- OAuth enabled version of curl- allows you to manually test authenticated and unauthenticated RESTTwitter for Mac- built in “developer console”
  16. 16. Is that it? ... YES! ... sorta ...
  17. 17. things get more complicated at scaleAnd scaling “real-time” is hard
  18. 18. How many Tweets are there?
  19. 19. How many Tweets are there? 140M! off the chart
  20. 20. 140M tweets 1600 tweets per day ≈ per second
  21. 21. Tweets multiply 8,080 people need to see my tweet
  22. 22. @ladygagamother mons†er8.9 million followers@justinbieberJustin Bieber8.3 million followers@BarackObama44th President of the United States7.1 million followers@raffime!8.5 thousand followers
  23. 23. @ladygaga mother mons†er 8.9 million followers @raffi me! 8.5 thousand followers 30,000 tweets1 tweet from ≈ 10 scaling for delivered1000x more followers @ladygaga 6than me per second
  24. 24. A System is only as strong as itsweakest link
  25. 25. 200M registered users 2006 2008 2010 2011
  26. 26. Lots of deliveries
  27. 27. MySQL Can’t generate IDs fast enoughCentralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  28. 28. Photo used under Creative Commons from jurvetson Distributed graph databaseflockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  29. 29. Follow me atQuestions? twitter.com/raffi