![Tomás Fernández Löbbe Plugtree LLC [email_address] October 7th 2010](https://image.slidesharecdn.com/solrmeter-lucenerevolution-110722152421-phpapp02/85/solrmeter-lightning-talk-lucene-revolution-2010-1-320.jpg)
Recommended
More Related Content
Similar to Test Solr Installations with SolrMeter
Similar to Test Solr Installations with SolrMeter (20)
Recently uploaded
Recently uploaded (20)
Test Solr Installations with SolrMeter
- 1. Tomás Fernández Löbbe Plugtree LLC [email_address] October 7th 2010
- 3. What is SolrMeter? An open source stress test tool specifically designed to test Solr installations.
- 4. Technically Speaking A standalone Java Swing application that will emulate N users executing queries and updateing documents to your Solr server. Java 6 Swing Guice 2.0 SolrJ Maven 2 JUnit JFreeChart
- 12. The SolrMeter project Google Code Google Groups Apache License 2.0 Contributions
- 15. Questions?
- 16. Thank you!
Editor's Notes
- Hello, I’m Tomás and I’ll be talking about SolrMeter.
- This graphic shows pretty much how the process of toning Solr should be, cyclic, right? The default configuration should be just a start point, but then you have to modify the different variables according to your context and see/analyze how the server responds by testing it. You will do many different kind of tests and definitely stress test will be on your list. You need to be sure before going live with your system that it will support the real load from your users. It also becomes very important in Solr to do stress tests because some of the configuration variables need many cycles of trial and error to find the best solution. The Merge factor for example, we know that setting a bigger value would help indexing and a lower value would help searches, but what is the best number for your specific context? I’s kind of difficult to know from the scratch, its easier to try one value and see how the system responds, right? This is one of many examples, like, caches, JVM memory assigned to the web container, etc.
- What is SolrMeter? It’s basically a tool that will help you tune your Solr installation by doing stress test easier. It’s specifically designed for Solr and it’s open source, under the Apache License 2.0. Nada técnico Basicamente una herramienta que te va a ayudar a tunear la instalación de Solr y asegurarte que la misma soportará la carga de usuarios reales, ejecutando cualquiera de las operaciones básicas al motor de búsqueda. Licencia Apche 2.0, con todo lo que eso implica.
- Nothing new here, is like a JMeter, but specifically to Solr, it should be easier to configure. I’s build with Java and all this frameworks/libraries here described. It uses SolrJ to connect to Solr, Google Guice for dependency injection, Maven2, Swing for the Graphic Interface and JFreeChart for the charts.
- First thing to do is to create a queries file. This is just a text file with one query per line. The more queries you add and the more realistic they are, the better. These are the exact terms that are going to be added to the “q” parameter of the Solr query, you can use phrases, specify field, you could specify boosts, boolean operators, etc. One important thing is that you have to be carefull with the number of queries you add, if you use too few queries, they will be all cached and the overall results of the test are going to be wrong.
- Second, create a Filter Queries file. This is pretty much the same as the last one, but with the terms that are going to be added on a fq parameter. You can also leave blanc spaces to specify that sometimes the query is executed with no filters. Ofcourse, yo can also NOT use filter queries at all.
- Third, create the fields file, with the fields you want to use for faceting. Depending on you application, here you might put all the fields you are going to have on your real application for faceting or at least many of them. Again, the closer it is the test to the real application, the better, am I right? You can also not use faceting at all if you don’t need to.
- Locate the files you have just created. Right now you have to specify the exact location of the text files, we are about to put some file chooser or something more user friendrly here. Th same with other options that could be a checkbox or something like that instead of plain text.
- Specify the URL where you want to send your queries. Of course, you need to have all the necessary permissions for an external application running where you are running SolrMeter to query Solr.
- Just hit the start button of the query panel and the queries are going to start executing. How are they going to be executed? Well, up to date we have (at least on trunk) two modes of execution one is one thread executing all the queries one after the other, the other mode is N threds, executing only one query per minute. We try to keep everything that’s possible pluggable, and this “execution modes” are not the exception. We are going to add more execution modes in he future, but anyone could implement their own and use it if you download the code.
- There are a couple of charts available, and there are a couple more of statistics, like mode, median, mean, last minute mean, etc. There is also the log of all exceptions, last executed queries, etc. The same way the queries are simulated, you can also simulate updates. The file for updates is a litle more complicated becouse you have to put one document per line, specifying field:value pairs. Al the required fields must be present. Some of the statistics are just for queries, but, for example this one, the “Time Line” statistic will show also the updates, commits and optimize. If Solr doesn’t use autocommit, you can tell SolrMeter to commit your changes every some amount of documents or every some time (Like Solr does). The Updates uses an executor, just like queries and you can choose between the existing ones or create your own. Optimize, on the other hand, are not executed automatically, you can choose when to optimize your index with the “Optmize Now” button.
- The project is hosted at google code and we manage everything from there, wiki, issues, and ofcourse, the source code. We also have a google group for mailing list and as I said before, it’s licensed unther the Apache License 2.0. SolrMeter is a young project and it has a hole lot of work to do. Every new feature we add, we think in 10 more for the future. Any feedback, idea, or any way you can or want to contribute to the project is welcome.
- Hablar de cómo queremos que evolucione SolrMeter, de la versión 0.2.0, de engancharlo con mas request handlers y finalmente el tema de ayudar en la configuración. We are now building the 0.2.0 version. This version objective is to have a much more pluggable architecture, allow an easy way of extending the tool. That’s whay we added Guice as dependency injection framework. There are also lot’s of usability features for this version, some of them are already implemented and some of them aren’t yet. The first thing we want to add in the future, is more integration with Solr Request Handlers, be able to know all the statistics of Solr from SolrMeter during the test. This means, for example, being able to see a graph of how the caches are behaveing without needing to go to the Solr admin page and press F5. Cover as many cases of Solr installations… right now, not everithing that can be done by solr can be tested with SolrMeter, we want that gap to be every day a litle bit smaller. Another thing we think would be intresting to add is some kind of rule bases suggestion, like something that could suggest you changes on your configuration to improve performance, etc. But anyway, we have to go step by step. There are many other ideas and you can propose your own, just check the issues page of the project’s site.
- Here are some of the links for contact, my email and the company web page.
- [ver tiempo, si no hay tiempo saltear esto] Any Questions?