Download as PDF, PPTX
Uploaded byMark Miller
SolrCloud Failover and Testing
This document discusses SolrCloud failover and testing. It provides an overview of how SolrCloud uses ZooKeeper to elect an overseer node to monitor cluster state and automatically create a new replica on an available node when one goes down, allowing failover capability. It also discusses challenges with distributed testing and recommends focusing more on backfilling tests when changing code, fixing frequently failing tests, and adding more unit tests to improve Solr's testing culture.
SolrCloud Failover and Testing
- 1. SolrCloud Failover and Testing MarkMiller (Cloudera)
- 2. Mark Miller Lucene Committer,Solr Committer. Works for Cloudera. A lot of work on SolrCloud.
- 3. At Cloudera… We arebuilding an Enterprise Data Hub Search is a part of that. Solr is our search engine.
- 5. Solr On HDFS Performanceis good. It can be even better. A shared filesystem has advantages.
- 6.
- 7. Limitation We replicate via bothSolr and HDFS. Replicating with just one has huge tradeoffs. We are working on better tradeoffs.
- 8. autoAddReplicas A new percollection option. When a replica goes down, it is replaced on a node that is still up. A shared filesystem as well means all replicas can go down and you can still automatically failover. + -
- 9. How Does itWork? SolrCloud elects a fault tolerant, single node to be an Overseer. The Overseer monitors the cluster state in ZooKeeper. Creates a new SolrCore on a machine that is up when necessary to replace ‘downed’ replicas.
- 10. Let’s Do ADemo!
- 11. SolrCloud Testing Let’s talkabout tests.
- 12. SolrCloud Tests We dida straw man implementation of SolrCloud first. We did the same for tests. We favored integration tests over unit tests. We did not make enough tests.
- 13. Distributed Tests Are hard. Fora variety of reasons. The Lucene / Solr testing framework hurts in order to help.
- 14. The Lucene /Solr Test Framework Randomized Testing. Rule Enforcement. The Jenkins Cluster.
- 15. Mocks We avoided doingthem early - too much churn. They can be dangerous to future contributors / refactoring. Some of the early mocking that did get in is a little painful. We need them for good unit tests.
- 16. Testing Culture Lucene hasA+ testing culture. In many cases, it’s easier for Lucene. Solr has a C testing culture. Solr needs to get better.
- 17. Prescription? More focus onback filling tests when adding features or changing code. More focus on fixing frequently failing tests. More focus on unit tests.
- 18.