Sharded Solr setup with master

An example of a sharded, replicated, load balanced SOLR setup

Solr Host Configuration
Documents indexed into manually selected master.
Solr Master 1
Solr Master 2
Documents For indexing

replication Replication duplicates indexes to two slaves each
Master 1
Slave 1a
Slave 1b
Master 2
Slave 2a
Slave 2b

Each shard host has two webapps a co-ordinator and a shard
Master 1
Master 2
Slave 1a
Slave 1b
Slave 2a
Slave 2b
co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard

All search requests come in via a load balancer
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

Load balancer has three VIPs, public search and two shard VIPs
Master 1
Master 2
Public VIP Shard 1 VIP Shard 2 VIP
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

Public requests balanced between co-ordinator webapps
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

Co-ordinator forwards requests to all shard VIPs
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

(it will then aggregates the results to return to the user)
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

Shard VIP balances between each slave for that shard
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

Masters need replication too, so...
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b

We replicate to backup masters replication replication
Master 1
Master 2
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b
Master 2
backup
Master 1
backup

Upon failure of master2, we switch to the backup replication
Master 1
Master 2
disabled
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b
Master 2
Master 1
backup

And start indexing to the new master replication indexing
Master 1
Master 2
disabled
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b
Master 2
Master 1
backup
documents

And establish the old master, once functioning, as a new backup replication replication
Master 1
Master 2
backup
load balancer
Public
Slave 1a
Slave 1b
Slave 2a
Slave 2b
Master 2
Master 1
backup

Conclusion
Load balanced
Replicated
Redundant
Sharded
Scalable
Slave replication has been in production >2yrs
Master replication still a theory

Sharded Solr setup with master

by Sourcesense

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 7

Statistics