Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SolrCloud Failover and Testing


Published on

  • Be the first to comment

SolrCloud Failover and Testing

  1. 1. SolrCloud Failover and Testing Mark Miller (Cloudera)
  2. 2. Mark Miller Lucene Committer, Solr Committer. Works for Cloudera. A lot of work on SolrCloud.
  3. 3. At Cloudera… We are building an Enterprise Data Hub Search is a part of that. Solr is our search engine.
  4. 4. Solr On HDFS Performance is good. It can be even better. A shared filesystem has advantages.
  5. 5. SolrCloud Reminder
  6. 6. Limitation We replicate via both Solr and HDFS. Replicating with just one has huge tradeoffs. We are working on better tradeoffs.
  7. 7. autoAddReplicas A new per collection option. When a replica goes down, it is replaced on a node that is still up. A shared filesystem as well means all replicas can go down and you can still automatically failover. + -
  8. 8. How Does it Work? SolrCloud elects a fault tolerant, single node to be an Overseer. The Overseer monitors the cluster state in ZooKeeper. Creates a new SolrCore on a machine that is up when necessary to replace ‘downed’ replicas.
  9. 9. Let’s Do A Demo!
  10. 10. SolrCloud Testing Let’s talk about tests.
  11. 11. SolrCloud Tests We did a straw man implementation of SolrCloud first. We did the same for tests. We favored integration tests over unit tests. We did not make enough tests.
  12. 12. Distributed Tests Are hard. For a variety of reasons. The Lucene / Solr testing framework hurts in order to help.
  13. 13. The Lucene / Solr Test Framework Randomized Testing. Rule Enforcement. The Jenkins Cluster.
  14. 14. Mocks We avoided doing them early - too much churn. They can be dangerous to future contributors / refactoring. Some of the early mocking that did get in is a little painful. We need them for good unit tests.
  15. 15. Testing Culture Lucene has A+ testing culture. In many cases, it’s easier for Lucene. Solr has a C testing culture. Solr needs to get better.
  16. 16. Prescription? More focus on back filling tests when adding features or changing code. More focus on fixing frequently failing tests. More focus on unit tests.
  17. 17. The End @heismark Thank You.