<ul>An example of a sharded, replicated, load balanced SOLR setup </ul>
<ul>Solr Host Configuration </ul>Documents indexed into manually selected master. <ul>Solr Master 1 </ul><ul>Solr Master 2...
<ul>Solr Host Configuration </ul>replication Replication duplicates indexes to two slaves each <ul>Master 1 </ul><ul>Slave...
<ul>Solr Host Configuration </ul>Each shard host has two webapps a co-ordinator and a shard <ul>Master 1 </ul><ul>Master 2...
<ul>Solr Host Configuration </ul>All search requests come in via a load balancer <ul>Master 1 </ul><ul>Master 2 </ul><ul>l...
<ul>Solr Host Configuration </ul>Load balancer has three VIPs, public search and two shard VIPs <ul>Master 1 </ul><ul>Mast...
<ul>Solr Host Configuration </ul><ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer ...
<ul>Solr Host Configuration </ul>Public requests balanced between co-ordinator webapps <ul>Master 1 </ul><ul>Master 2 </ul...
<ul>Solr Host Configuration </ul>Co-ordinator forwards requests to all shard VIPs <ul>Master 1 </ul><ul>Master 2 </ul>Publ...
<ul>Solr Host Configuration </ul>(it will then aggregates the results to return to the user) <ul>Master 1 </ul><ul>Master ...
<ul>Solr Host Configuration </ul>Shard VIP balances between each slave for that shard <ul>Master 1 </ul><ul>Master 2 </ul>...
<ul>Solr Host Configuration </ul>Masters need replication too, so... <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard ...
<ul>Solr Host Configuration </ul>We replicate to backup masters replication replication <ul>Master 1 </ul><ul>Master 2 </u...
<ul>Solr Host Configuration </ul>Upon failure of master2, we switch to the backup replication <ul>Master 1 </ul><ul><li>Ma...
disabled </li></ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public  <ul>Slave 1a </ul><ul>Slave 1b </ul><...
<ul>Solr Host Configuration </ul>And start indexing to the new master replication indexing <ul>Master 1 </ul><ul>Master 2 ...
<ul>Solr Host Configuration </ul>And establish the old master, once functioning, as a new backup replication replication <...
Upcoming SlideShare
Loading in …5
×

Sharded Solr setup with master

5,936 views

Published on

An example of a sharded, replicated, load balanced SOLR setup

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,936
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
0
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Sharded Solr setup with master

  1. 1. <ul>An example of a sharded, replicated, load balanced SOLR setup </ul>
  2. 2. <ul>Solr Host Configuration </ul>Documents indexed into manually selected master. <ul>Solr Master 1 </ul><ul>Solr Master 2 </ul>Documents For indexing
  3. 3. <ul>Solr Host Configuration </ul>replication Replication duplicates indexes to two slaves each <ul>Master 1 </ul><ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Master 2 </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>
  4. 4. <ul>Solr Host Configuration </ul>Each shard host has two webapps a co-ordinator and a shard <ul>Master 1 </ul><ul>Master 2 </ul><ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  5. 5. <ul>Solr Host Configuration </ul>All search requests come in via a load balancer <ul>Master 1 </ul><ul>Master 2 </ul><ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  6. 6. <ul>Solr Host Configuration </ul>Load balancer has three VIPs, public search and two shard VIPs <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  7. 7. <ul>Solr Host Configuration </ul><ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  8. 8. <ul>Solr Host Configuration </ul>Public requests balanced between co-ordinator webapps <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  9. 9. <ul>Solr Host Configuration </ul>Co-ordinator forwards requests to all shard VIPs <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  10. 10. <ul>Solr Host Configuration </ul>(it will then aggregates the results to return to the user) <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  11. 11. <ul>Solr Host Configuration </ul>Shard VIP balances between each slave for that shard <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  12. 12. <ul>Solr Host Configuration </ul>Masters need replication too, so... <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard
  13. 13. <ul>Solr Host Configuration </ul>We replicate to backup masters replication replication <ul>Master 1 </ul><ul>Master 2 </ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard <ul>Master 2 <li>backup </li></ul><ul>Master 1 <li>backup </li></ul>
  14. 14. <ul>Solr Host Configuration </ul>Upon failure of master2, we switch to the backup replication <ul>Master 1 </ul><ul><li>Master 2
  15. 15. disabled </li></ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard <ul>Master 2 </ul><ul>Master 1 <li>backup </li></ul>
  16. 16. <ul>Solr Host Configuration </ul>And start indexing to the new master replication indexing <ul>Master 1 </ul><ul>Master 2 <li>disabled </li></ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard <ul>Master 2 </ul><ul>Master 1 <li>backup </li></ul>documents
  17. 17. <ul>Solr Host Configuration </ul>And establish the old master, once functioning, as a new backup replication replication <ul>Master 1 </ul><ul>Master 2 <li>backup </li></ul>Public VIP Shard 1 VIP Shard 2 VIP <ul>load balancer </ul>Public <ul>Slave 1a </ul><ul>Slave 1b </ul><ul>Slave 2a </ul><ul>Slave 2b </ul>co-ordinator shard co-ordinator shard co-ordinator shard co-ordinator shard <ul>Master 2 </ul><ul>Master 1 <li>backup </li></ul>
  18. 18. <ul>Conclusion </ul><ul><li>Load balanced
  19. 19. Replicated
  20. 20. Redundant
  21. 21. Sharded
  22. 22. Scalable
  23. 23. Slave replication has been in production >2yrs
  24. 24. Master replication still a theory </li></ul>

×