Develop a solr request handler plugin

10,427 views

Published on

In this set of slides we give a step by step tutorial on how to develop a fully functional solr request handler plugin. Additionally we provide links to full source code which can be used as a template to rapidly start creating your own request handler.

Published in: Technology
4 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total views
10,427
On SlideShare
0
From Embeds
0
Number of Embeds
1,378
Actions
Shares
0
Downloads
136
Comments
4
Likes
6
Embeds 0
No embeds

No notes for slide

Develop a solr request handler plugin

  1. 1. andrew.janowczyk@searchbox.com
  2. 2. Solr is◦ Blazing fast open source enterprise search platform◦ Lucene-based Search Server◦ Written in Java◦ Has REST-like HTTP/XML and JSON APIs◦ Extensive plugin architecturehttp://lucene.apache.org/solr/
  3. 3.  Allows for the development of plugins whichprovide advanced operations Types of plugins:◦ RequestHandlers Uses url parameters and returns own response◦ SearchComponents Responses are embedded in other responses (such as/select)◦ ProcessFactory Response is stored into a field along with thedocument during index time
  4. 4.  A quick tutorial on how to program aRequestHandler to◦ Be initialized◦ Parse configuration file arguments◦ Do something useful, (counts some words in query)◦ Format and return response We’ll name our plugin “DemoPlugin” andshow how to stick it into the solrconfig.xmlfor loading
  5. 5.  In the next slide, we’ll specify a list of variablescalled “words”, and each list subtype is a string“word” We want to load these specific words and thencount them on all subsequent queries. Ex: config file has “body”, “fish”, “dog” Query is: dog body body body fish fish fish fishorange Result should be:◦ body=3.0◦ fish=4.0◦ dog=1.0
  6. 6. <requestHandler name=“/newendpoint"class="com.searchbox.DemoPlugin"><lst name=“words"><str name=“word">body</str><str name=“word">fish</str><str name=“word">dog</str></lst></requestHandler>Variables will be loaded from this sectionduring the init method discussed later
  7. 7.  We can see that we’re asking for Solr to loadcom.searchbox.DemoPlugin. This will be theoutput of our project in .jar file format Copy the .jar file to the lib directory in theSolr installation so that Solr can find it. That’s it!
  8. 8. package com.searchbox;import java.util.HashMap;import java.util.List;import org.apache.solr.common.SolrException;import org.apache.solr.common.params.CommonParams;import org.apache.solr.common.params.SolrParams;import org.apache.solr.common.util.NamedList;import org.apache.solr.common.util.SimpleOrderedMap;import org.apache.solr.handler.RequestHandlerBase;import org.apache.solr.request.SolrQueryRequest;import org.apache.solr.response.SolrQueryResponse;import org.apache.solr.search.*;import org.slf4j.Logger;import org.slf4j.LoggerFactory;public class DemoPlugin extends RequestHandlerBase {private static Logger LOGGER = LoggerFactory.getLogger(DemoPlugin.class);volatile long numRequests;volatile long totalTime;volatile long numErrors;List<String> words;
  9. 9.  Initialization is called when the plugin is firstloaded This most commonly occurs when Solr isstarted up At this point we can load things from file(models, serialized objects, etc) Have access to the variables set insolrconfig.xml
  10. 10.  We have selected to pass a list called “words”and have also provided the list “fish”, ”body”,”cat” of words we’d like to count. During initialization we need to load this listfrom solrconfig.xml and store it locally
  11. 11. @Overridepublic void init(NamedList params) {words= (NamedList)params.get(“words”)).getAll(“word”);if (words.isEmpty()) {throw newSolrException(SolrException.ErrorCode.SERVER_ERROR,"Need to specify at least one word in requestHandler config!");}}super.init(params); //pass the rest of the init up}Notice that we’ve loaded the list “words” andthen all of its attributes called “word” and putthem into the class level variable words.
  12. 12. @Overridepublic void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws Exception{numRequests++;long startTime = System.currentTimeMillis();try {HashMap<String, Double> counts = new HashMap<String, Double>();SolrParams params = req.getParams();String q = params.get(CommonParams.Q); //get the q param from urlfor (String string : q.split(" ")) {if (words.contains(string)) {Double oldcount = counts.containsKey(string) ? counts.get(string) : 0;counts.put(string, oldcount + 1);}}• We start off by keeping track in a volatile variable the number of requests we’ve seen (for use laterin statistics), and we’d like to know how long the process takes so we note the time.• Next we initialize our local variable which will contain our word counts• Next we get the “q” parameter from the URL which was sent to us• We do a very silly split by space to break it into words, and iterate through each of the words. If theword is in our “words” variable, we keep a running total of the number of times it appears
  13. 13. NamedList<Double> results = new NamedList<Double>();for (String word : words) {results.add(word, counts.get(word));}rsp.add("results", results);} catch (Exception e) {numErrors++;LOGGER.error(e.getMessage());} finally {totalTime += System.currentTimeMillis() - startTime;}}• Now that we’ve looked at all of the strings, and our process is done we need to return the results.• We create a namedlist of type double to hold the counts, and then iterate through our words adding themto the response• Finally, we add our result list to the Solr response variable rsp• We also see the other end of the catch statement, which is used to collect error counts and print the errorto the Solr logger• Finally we add the time it took to the total time
  14. 14. @Overridepublic String getDescription() {return "Searchbox DemoPlugin";}@Overridepublic String getVersion() {return "1.0";}@Overridepublic String getSource() {return "http://www.searchbox.com";}@Overridepublic NamedList<Object> getStatistics() {NamedList all = new SimpleOrderedMap<Object>();all.add("requests", "" + numRequests);all.add("errors", "" + numErrors);all.add("totalTime(ms)", "" + totalTime);return all;}• In order to have a production grade plugin, users expect to see certain pieces of informationavailable in their Solr admin panel• Description, version and source are just Strings• We see getStatistics() actually uses the volatile variables we were keeping track of before, sticksthem into another named list and returns them. These appear under the statistics panel in Solr.• That’s it!
  15. 15. http://192.168.56.101:8983/solr/core_name/newendpoint?q=dog%20body%20body%20body%20fish%20fish%20fish%20fish%20orange<response><lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><lst name="results"><double name="body">3.0</double><double name="fish">4.0</double><double name="dog">1.0</double></lst></response>
  16. 16. • Because we’ve overridden thegetStatistics() method, we can get real-time stats from the admin panel!
  17. 17. Happy Developing!Full Source Code available at:http://www.searchbox.com/developing-a-request-handler-for-solr

×