Designing a Scalable Twitter - Patterns for Designing Scalable Real-Time Web Applications

Designing a Scalable Twitter - Patterns for Designing Scalable Real-Time Web Applications

• 1.
  Designing a scalabletwitterNati Shalom,CTO & Founder GigaSpacesJohn D. MitchellMad Scientist of Friendster.
• 2.
  About GigaSpaces TechnologiesEnablingapplications to run a distributed cluster as if it was a single machine…75+ Cloud CustomersAmong Top 50 Cloud Vendors300+ Direct Customers22
• 3.
  Introduction – Whytwitter as an exampleEverybody knows twitter Twitter exposes really difficult scaling challenges Twitter can serve as a generic example for any real-time/social web applicationIf you can built a scalable twitter your already half way through.
• 4.
  Why twitter isdifferent then other web apps?Twitter makes many to many relationship first class citizenTwitter behaves like a living organismEvery post gets to number of followers Every user follows number of users FollowersFollowingUserUser
• 5.
  Scaling challenges 1- Fluctuating growth Every user network (Following/Followers) grows constantly Traffic tends to fluctuate randomlyFollowersFollowingUserUser
• 6.
  Scaling challenges 2– The crowd effectRe-tweet can lead to a perfect stormEvery Scaling has a physical limitFollowersUser
• 7.
  Designing a scalabletwitter
• 8.
  Approach 1 – Use share nothing web scaling approachApplicationApplicationUse load balancer and stateless web server to load balance the web traffic
• 9.
  Grow by addingmore web containersRead ServiceRead ServiceWebData BasePublish ServicePublish ServiceUsersLoad BalancerWeb
• 10.
  Reader/Publisher Servicenamespace MicroBlog.Services{ public interface IReaderService { ICollection<Post> GetUserPosts(String userID);ICollection<Post> GetUserPosts(String userID, DateTime fromDate); }}namespace MicroBlog.Services{ public interface IPublisherService { void PublishPost(Post post); }}
• 11.
  UsersApproach 1 scalabilitychallengeApplicationApplicationApplicationApplicationRead ServiceRead ServiceRead ServiceRead ServiceThe database becomes the bottleneckWebData BasePublish ServicePublish ServiceWebPublish ServicePublish ServiceLoad BalancerWeb WebIISWeb
• 12.
  Reader – DatabaseImplementationpublic ICollection<Post> GetUserPosts(string userID, DateTime fromDate) {// Create command:IDbCommand dbCommand = _dbConnection.CreateCommand(); dbCommand.CommandText = String.Format("SELECT * FROM Post WHERE UserID='{0}' AND PostedOn > {1}", userID, fromDate);// Execute command:IDataReader dataReader = dbCommand.ExecuteReader();// Translate results from db records to .NET objects:List<Post> result = ReadPostsFromDataReader(dataReader);// Return results:return result;}
• 13.
  Eliminate database bottlenecks– a questions of alternativesSolving the database bottleneck requires changeDifference approaches offers various degree of changeDatabase clusters- read-only databasesRequires significant data access design changesDoes not improve performance In Memory DataGrid – Change at the application data not the databaseMemcache – Small change/Small valuefairly limited not transactional, doesn’t support SQL query, non clusteredGoogle App Engine Persistency (On top of Big Table)Provides standard JPA access (Smaller change)Many model limitationsDoes not provide database integration into existing modelApplication Partitioning (several different databases for the same application)Requires significant data access design changesNeed to synchronize between the databases for transactional integrity
• 14.
  The NOSQL movementDistributedkey/value store (#NOSQL) – New hot trend, influenced by Google, Amazon..) Use Distributed commodity HW then expensive HWDesigned for massive scaleExamplesFIlesystem based implementationAmazon Dynamo/SimpleDBGoogle Big TableCasandraMemory based implementationGigaSpacesGemstoneIBM extremeScaleJBoss infinispan Oracle CoherenceDid you know?Disk failure/year - 3% vs. 0.5 - 0.9% estimated
• 15.
  No correlation betweenfailure rate and disk type - SCSI, SATA, or fiber channel
• 16.
  Lower temperatures areassociated with higher failure ratesNOSQL common principlesAssume that failure is inevitable Disk, machine, network will failDon’t avoid it (through costly HW) - cope with itReduce the impact of failure by:Keeping multiple replicas Distributing the data (partitioning)CAP TheoremRelax some of the consistency constrains – eventual consistencyUse Map/Reduce to handle aggregationParallel queryExecute the query close to the dataAvailable as Filesystem or In-Memory implementation
• 17.
  In-Memory vs Filesystembased approachesIn-memory Pros:Complementary – the database can be kept unchanged Designed for real time performance (Not limited to disk I/O)Enable execution of the code with the data (stored procedure) ConsMemory is more expensive then file-system (Cost of per GB storage)File-SystemPros:Can manage terra bytes of dataLow cost per GB storageConsFairly radical change -> requires complete re-writeBest of both worldsMemory based storage for real time accessFilesystem for long-term data
• 18.
  Choosing the rightsolution for our twitter appPerformance analysis:Twitter allows 150 req/hour per userFor 1M users that means 40,000 req/secCapacity The actual data that needs to be accessed in real-time is only the window of time between interactions (Last 10/30 min should be more then sufficient)The rest of the data can be stored in file-system storage (For users who just logged in)Assuming a ratio of 20% writes and the size per post is 128 bytes:Data produced = 40k*20% * 128 = 1Mb/SecWe can keep a buffer of an hour in less then 5GSolution: Use In Memory Data Grid for the real-time access and files-ystem storage for long term, search and other analytics requirements
• 19.
  Scaling twitter usingSpaceBased Architecture (SBA)
• 20.
  What is SpaceBased Architecture (SBA)Processing unit cloud: Scale through Partitioning
• 21.
  Virtualized middlewareWhat isa space:Elegant – 4 API
• 22.
  Solves:
• 23.
  Data sharing
• 24.
  Messaging
• 25.
  Workflow