Mastering solr


Published on

Mastering Solr will give an introduction to the more advanced use of Solr. It offers a starting point to go beyond the possibilies offered by the Solr integration modules.

The following subjects will be covered:

- Getting Solr running locally
- Integration modules: Search API vs ApacheSolr
- Configuring an index
- Search relevancy: basic boosting of search results.
- The anatomy of a request to Solr
- Hooks for altering/adding parameters to the request to Solr
- Search relevancy: advanced date boosting by boost functions
- Search relevancy: advanced boosting by boost queries

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mastering solr

  1. 1. Mastering solr Jur de Vries
  2. 2. Who am I?Developer/architect at TriquantaTrainer at Wizzlern
  3. 3. Use caseMarket placeAdvertisementsAdjust relevancyPaid boosting of addsOf course we use Drupal and Apache Solr
  4. 4. Running Solr LocallyDownload latest version (3.6)Be sure to download distribution (not src)Unpack solrGo to example directoryRun java -jar start.jar
  5. 5. Drupal: which contrib?2 Possibilities Apachesolr search Search api with solr backend
  6. 6. Apache solr searchStreghts: Supported by Acquia Easy to set up MatureWeaknesses Integration with views (still in dev)
  7. 7. Search ApiStrengths Flexible Indexes all entities Excellent views integration Related fields are easy to add to indexWeaknesses Not supported (yet) by Acquia Solr backend has some issues
  8. 8. Drupal: which contrib?Apachesolr search integration Quick setup AcquiaSearch API Exportable configuaration Views integration Index all entitiesDepends on your needs
  9. 9. Basic use of search apiCreate serverCreate index Select fields to index Define data alterations Define processorsStart indexing
  10. 10. Field typesInteger, date, booleanString or fulltext? Fulltext will get processed! Tokenize Stopwords Ignore case String is as is
  11. 11. DemoRun solrCopy schema.xml and solrconfig.xml (!)Create serverCreate indexCreate view ads Ad filter exposed: search
  12. 12. Advanced use of Search apiThis talk is about Solr, not about search APIUnderstand Solr first!Many resources on the webWatch screencasts etc
  13. 13. Mastering SolrMastering solr is understanding solrWhat happens after a Drupal module?Lets have a look at the request
  14. 14. Solr requestLook at solr logParameters: start rows q (query) qf (query fields) fl (fields) fq (filter query)
  15. 15. Field namesitem_id, idt_.., ss_.., → why?Solr has to know how to handle fieldsField api: field names differDynamic field names: tell solr field type!
  16. 16. Schema.xmlDefines field types and fieldsThe real tweaking starts here!Lets have a look! dynamicField field type analyzersCopyfield
  17. 17. What can you do in schema.xml?Synonyms (is disabled by default)Stopwords (and, or, etc)StemmingProper multilingual handling
  18. 18. Browse the schemaSolr offers schema browsingGo to: http://localhost:8983/solr/admin
  19. 19. Search relevancyTypes of boosting: Field level boost Boost function Boost query (QueryElevation)
  20. 20. Boost parametersField level boosting: qf qf:t_body^20 score in field is multiplied by 20Boost function: bf bf:product(fieldname, 2) result of function is added to scoreBoost query: bqboost (only for edismax) like bf but multiplication
  21. 21. Lets boost titleField level boost is incorporated in Search API...But, where are the numbers in the request???Search api solr forgot to add them!There is a patch :-)But lets do it another way...
  22. 22. Debugging SolrLets add &echoParams=all to the request...Where do all these parameters come from?Solrconfig.xml!!!Among other things: request handlerLets look at the dismax request handler
  23. 23. Solrconfig.xml(Default) Request handler: Default parameters Add Spellcheck Tweak all kinds of search behavior! Lets add default search fields with boost
  24. 24. Boost functionMathematical functions on field valuesAvailable functions: sum(x,y): x + y product(x,y): x * y scale(x, minTarget, maxTarget) recip(x, m, a, b): x / (m * a + b) ms(): time → ms(NOW/DAY, created) Many more!
  25. 25. Boost dateWe need ms(): big values!Linear? To much differenceRecip!recip(x,1,1000,1000)if x 1000: half1 year: 3.1e10recip(ms(NOW/YEAR?, created),1,3.1e10,3.1e10)bf=recip(ms(NOW/YEAR?, created),1,3.1e10,3.1e10)^3Use a graphing tool!
  26. 26. Boost queriesDo a query like fq:Boost adds: content_type:add bq=content_type:add bq=(content_type)^20
  27. 27. Debugging relevancyWe know how to boostHow can finetuning be done?solr has the solutions: add debugQuery=on
  28. 28. debugQuery=onnormal source
  29. 29. RelevancyChoose your boosting methodsTry in your browserFinetuning: debugQuery=on, sourceAdd parameters to solrconfig.xmlOr...
  30. 30. Add parameters in codeusehook_search_api_solr_query_alter(array &$call_args, SearchApiQueryInterface $query)$call_args[params][bq] = (t_title:foo)^20$call_args[params][bf][] = b_promote
  31. 31. Override solr service classIn Search API: define server classextend solr service classOnly change key methodsIts all about passing parameters!
  32. 32. ConclusionTweak indexing in schema.xml Stopwords MultilingualTweak searching in solrconfig.xmlTweak searching by passing variablesThis is only an introduction!
  33. 33. Questions?
  34. 34. Feedback & follow-up: