How Twitter Works (Arsen Kostenko Technology Stream)
•6 likes•1,509 views
Report
Share
Download to read offline
Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneurs and startuppers. Annually it takes place at the beginning of October in Lviv at Arena Lviv stadium. In 2016 the conference gathered more than 1800 participants and over 100 speakers from companies like Microsoft, Philips, Twitter, UBER and IBM. More details about the conference at itarena.lviv.ua.
Recommended
More Related Content
What's hot
What's hot(20)
Viewers also liked
Viewers also liked(20)
Similar to How Twitter Works (Arsen Kostenko Technology Stream)
Similar to How Twitter Works (Arsen Kostenko Technology Stream) (20)
More from IT Arena
More from IT Arena(20)
Recently uploaded
Recently uploaded(20)
How Twitter Works (Arsen Kostenko Technology Stream)
- 1. HOW TWITTER WORKS The Challenge
- 2. … and more! How Twitter Works
- 3. Tweets How Twitter Works Just 140 characters???
- 5. #oscars
- 6. #worldcup
- 8. Twitter Architecture - How Twitter Works Tweets author mention actions card counts
- 9. Twitter Architecture - How Twitter Works Let’s Draw Twitter’s Architecture! clients routing API logic storage
- 10. Twitter Architecture - How Twitter Works Twitter Architecture: The Old Days (~2010) clients routing logic storage MySQL Cluster TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients API The Monorail (Ruby) The Monorail (Ruby) The Monorail (Ruby)
- 11. Twitter Architecture - How Twitter Works Twitter Architecture: The Old Days (~2010)
- 12. Twitter Architecture - How Twitter Works Twitter Architecture: Microservices clients routing logic storage Macaw-SwiftMacaw-Swift User StoreGizmoduck (User Service) Web API TFE (Twitter Front End) Tweetypie (Tweet Service) Timeline Service Social Graph Service Tweet Store Macaw-Tweets Search Service Direct Messages Service Macaw-Users Macaw-Search Macaw-DM Graph Store Tweet Graph Store Direct Messages Store TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients Distributed Key-Value Store API Macaw-SwiftMacaw-SwiftTweet API Macaw-SwiftMacaw-SwiftUsers API Macaw-SwiftMacaw-SwiftSearch API Macaw-SwiftMacaw-SwiftMessages API Macaw-SwiftMacaw-SwiftTweet Service Tweetypie (Tweet Service) Macaw-SwiftMacaw-SwiftUser Service Tweetypie (Tweet Service) Macaw-SwiftMacaw-SwiftTimeline Service Tweetypie (Tweet Service) Macaw-SwiftMacaw-SwiftSocial Graph Service Tweetypie (Tweet Service) Macaw-SwiftMacaw-SwiftSearch Service Tweetypie (Tweet Service) Macaw-SwiftMacaw-SwiftDirect Messages Service Macaw-Users Macaw-SwiftMacaw-SwiftTimeline API
- 13. Twitter Architecture - How Twitter Works Twitter Architecture: Microservices 10-100x performance hundreds of million tweets/day
- 16. Twitter Architecture - How Twitter Works Twitter Clients clients routing logic storage Manhattan (Distributed DB) Gizmoduck (User Service) Web API Tweetypie (Tweet Service) Timeline Service Social Graph Service TBird (Tweet Store) Tweets API Search Service Direct Messages Service Users API Search API Messages API Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) TFE (Twitter Front End) HTTPS TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients Manhattan (Distributed DB) API Timeline API
- 17. Twitter Architecture - How Twitter Works Twitter Front End (TFE) clients routing logic storage Manhattan (Distributed DB) Gizmoduck (User Service) TFE (Twitter Front End) Tweetypie (Tweet Service) Timeline Service Social Graph Service TBird (Tweet Store) Search Service Direct Messages Service Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) HTTPS HTTP TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients Manhattan (Distributed DB) API Web API Tweets API Users API Search API Messages API Timeline API
- 18. Twitter Architecture - How Twitter Works Presentation Layer Services clients routing logic storage Manhattan (Distributed DB) Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service TBird (Tweet Store) Search Service Direct Messages Service Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) HTTP TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients HTTPS Web API Tweets API Users API Search API Messages API TFE (Twitter Front End) Thrift Manhattan (Distributed DB) API Timeline API
- 19. Twitter Architecture - How Twitter Works Application Logic Services clients routing logic storage Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) HTTP TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients HTTPS TFE (Twitter Front End) Thrift Web API Tweets API Users API Search API Messages API Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service Direct Messages Service Manhattan (Distributed DB) API Timeline API
- 20. Twitter Architecture - How Twitter Works Storage Services clients routing logic storage HTTP TweetDeck Twitter for iOS Twitter for Android twitter.com Twitter API Clients HTTPS TFE (Twitter Front End) Thrift Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service Direct Messages Service Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) API Web API Tweets API Users API Search API Messages API Timeline API
- 21. Twitter Architecture - How Twitter Works Write Path Example: Tweet clients routing logic storage HTTP TweetDeck Twitter for iOS Twitter for Android Twitter API Clients HTTPS TFE (Twitter Front End) Thrift Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) twitter.com API Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service Direct Messages Service Web API Tweets API Users API Search API Messages API Timeline API
- 22. Twitter Architecture - How Twitter Works Write Path Example: Tweet clients routing logic storage HTTP TweetDeck Twitter for iOS Twitter API Clients HTTPS Thrift Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) TFE (Twitter Front End) API Twitter for Android twitter.com Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service Direct Messages Service Web API Tweets API Users API Search API Messages API Timeline API
- 23. Twitter Architecture - How Twitter Works Write Path Example: Tweet clients routing logic storage HTTP TweetDeck Twitter for iOS Twitter API Clients HTTPS Thrift Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) TFE (Twitter Front End) API Twitter for Android twitter.com Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service Direct Messages Service Web API Tweets API Users API Search API Messages API Timeline API
- 24. Twitter Architecture - How Twitter Works Write Path Example: Tweet clients routing logic storage HTTP TweetDeck Twitter for iOS Twitter API Clients HTTPS Thrift Direct Messages Service Manhattan (Distributed DB) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) TFE (Twitter Front End) Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service API Twitter for Android twitter.com TBird (Tweet Store) Web API Tweets API Users API Search API Messages API Timeline API
- 25. Twitter Architecture - How Twitter Works Write Path Example: Tweet clients routing logic storage HTTP TweetDeck Twitter for iOS Twitter API Clients HTTPS Thrift Direct Messages Service Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) TFE (Twitter Front End) Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service API Twitter for Android twitter.com Web API Tweets API Users API Search API Messages API Timeline API
- 26. Twitter Architecture - How Twitter Works Building a Timeline Consider user timelines for three different Twitter users
- 27. Twitter Architecture - How Twitter Works Building a Timeline If you follow these three users... time
- 28. Twitter Architecture - How Twitter Works Building a Timeline … we can build your Twitter timeline as a list of tweets in reverse chronological order: time Although we can also rely on other pieces of information to create tweet timelines that are more relevant for our users.
- 29. Twitter Architecture - How Twitter Works Building a Timeline clients routing logic storage TweetDeck Twitter for iOS Twitter for Android Twitter API Clients twitter.com TFE (Twitter Front End) Gizmoduck (User Service) Tweetypie (Tweet Service) Timeline Service Social Graph Service Search Service Direct Messages Service Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Manhattan (Distributed DB) API Web API Tweets API Users API Search API Messages API Timeline API
- 30. Twitter Architecture - How Twitter Works Building a Timeline logic storage Tweetypie (Tweet Service) Timeline Servicec Manhattan (Distributed DB) TBird (Tweet Store) Flock (Graph Store) TFlock (Tweet Graph Store) DMBird (Direct Msg Store) Haplo (Redis cluster) API Web API Tweets API Users API Search API Messages API Timeline API Fanout Service
- 31. TWITTER SERVICE STACK How Twitter Works
- 32. Service Stack Most services at Twitter run on the following stack: Hardware Twitter Service Stack - How Twitter Works Distributed Resource Manager Job Scheduler Runtime RPC Framework Operating System Network Server REST Framework Hardware Mesos Aurora Scala + JVM Finagle Linux TwitterServer Finatra
- 33. Mesos and Aurora Distributed resource manager Twitter Service Stack - How Twitter Works Hardware OS Hypervisor OS Hardware OS Mesos Job Job Job Job Job Job OSJob OSJob OSJob OSJob Job scheduler
- 34. Scala, Finagle and Finatra Twitter Service Stack - How Twitter Works An object-oriented, functional programming language Compiles to Java bytecode Used by most production services at Twitter An extensible RPC system for high-concurrency clients and servers Supports HTTP, Thrift, and other protocols A framework for fast and testable HTTP services
- 35. #oneteam Mike Cvet @mikecvet Moses Nakamura @mnnakamura Saša Gargenta @sasa Anna Sulkina @asulkina Ricardo Cervera-Navarro @ricardoc John Zeman @zeman Julie Dilger @JulieDil Nisha Antony @nantony345
- 36. Questions?