Large scale searchPatterns for dealing with large-scale search systems
overview•How to provide a scalable platform forboth users and data•Issues introduced by a scalable platform•Patterns for dealing with the issues
Big dataAs data volumes increase they can prove too large for any one server to manage
Big data: PartitioningData can be partitioned into suitably sized“shards” and placed on different servers
Partitioning: divide and conquer ? ? ? ?Each user’s search queries all shards in paralleland combines results to provide fast responses
Big Search Loads ? ? ? ? ? ? ? ? ? ?However, as user volumes increase, the loads on each shard server can become too great
Replication ? ? ? ? ? ? ? ? ? ?To spread the load of many simultaneous users, indexes need to be replicated
Scaling SummaryPartitioning Replicationcoping with data volumes coping with user volumes (and providing redundancy in the event of failure)
Issues So far so good - but a scalable system withmany servers raises the concerns of balancing Consistency and Availability...
Consistency vs availabilityServers ! Content Freshness As the number of required servers increases, there is an increased probability that a server will fail or lag when adding new content.
Consistency vs availabilityServers earliest cross-server latest available consistent content content ???? These potential inconsistencies across servers introduces a dilemma - search the latest available content or older, consistent content?
Consistency vs availability Consistency Availability FULL Shard Managed sticky user Every man for Consistency Consistency InConsistency sessions himselfWhat follows is a number of architectural patterns, each of which will make atrade-off between the consistency and availability of content being searched
PatternsConsistency Availability Full Consistency All servers are designed to coordinate together when applying batches of Description new content. If any one server fails to apply updates, all servers abandon this batch of updates. •All users of the system see the same version of content i.e. the same point in time. Pros •Complex distributed transaction software is required to coordinate updates. Cons •Any failure on a server delays the visibility of new content on all servers.
PatternsConsistency Availability shard consistency Within each “shard” replica servers strive to maintain identical copies of Description the same content. New content additions are coordinated within each shard with any failure of a replica server aborting additions to that shard. •Update failures are isolated to impacting availability of new content on a single shard. Pros •All users see the same (potentially uneven) content. •Complex distributed transaction software may be required to coordinate updates. Cons •Any failure delays the visibility of new content in that shard. •Shards may be “uneven” in the points-in-time they represent
PatternsConsistency Availability managed inconsistency All servers apply updates independently, with an agreed tolerance for “drift” between the freshness of content held on servers. When this Description threshold is reached the servers with the newest content halt updates until the drift gap is closed (this may require removing a failing replica server from active service). •New content is continually made available to users until pre-deﬁned Pros tolerances for failures are exceeded •Different users may see different results depending on which (almost) replica the load balancer chooses to service their queries Cons •Individual users hitting the refresh button may also see different results as a result of non-exact replica servers •Shards may be “uneven” in the points-in-time they represent
PatternsConsistency Availability sticky user sessions All servers are allowed to update independently. The load balancer is conﬁgured to route a user’s searches to the same choice of replica server Description in each shard whenever possible to hide any temporary drift between replicas. •New content is continually made available to users. Pros •Individual users should not experience a “step back in time” when repeating the same query due to inconsistent replicas. •Different users may see different results depending on which (almost) Cons replica the load balancer chooses to service the query. •Shards may be “uneven” in the points-in-time they represent.
PatternsConsistency Availability every man for himself Description All servers are allowed to update independently. Pros •New content is continually made available to users •Different users may see different results depending on which (almost) replica the load balancer chooses to service the query. Cons •Individual users hitting the refresh button may also see different results as a result of non-exact replica servers
considerations when selecting a pattern•Pick an acceptable user experience as a starting point •“I always expect to be acting on the latest available information” •“I need all results to represent the same point in time” •“I expect hitting the refresh button to take me forward in time, never back” •“I expect to always see the same content as my colleagues”•Recognise not all user requirements are realisable so rank them byimportance.•Pick a pattern that works best for the selected requirements •Consider mixing some patterns e.g. “Managed Inconsistency” with “Sticky user sessions” seems a good compromise between maintaining (perceived) consistency and content availability •Consider different strategies for different user groups e.g. VIP users will always see the guaranteed-latest content.