Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Apache Solr                                               - search for everyone!http://www.flickr.com/photos/malikdhadha/
• Co-founder and R&D Director at Integrasco• Founder and developer of Notpod• Leader of javaBin Sørlandet• Programmer and ...
A global leader in social intelligence
What is search?http://www.flickr.com/photos/denverjeffrey/5133538450/
http://www.flickr.com/photos/somegeekintn/3709203268/
This is Apache Solr• Open Source enterprise search server from  Apache• Built on Apache Lucene• Offers additional features...
First, a little history...
• Started out as a in-house CNET project for adding  search functionality to the CNET website in 2004
• Started out as a in-house CNET project for adding  search functionality to the CNET website.• Donated to Apache Software...
• Started out as a in-house CNET project for adding  search functionality to the CNET website.• Donated to Apache Software...
• Since version 3.1 (March 2011), Solr and Lucene are  now sharing the same codebase.                         +
• Since version 3.1 (March 2011), Solr and Lucene are  now sharing the same codebase.• Meaning sharing of features and fix...
wget http://apache.uib.no/lucene/solr/3.6.1/apache-solr-3.6.1.tgztar xvf apache-solr-3.6.1.tgzcd apache-solr-3.6.1/example...
...and we’re up!
cd exampledocs/./post.sh ipod_other.xml
The obvious part – full text searchinghttp://www.flickr.com/photos/49889874@N05/6877840735/
• q=yourquery• Example:  q=android AND ios&rows=100
Don’t worry - it’s not just XML!
The Schemahttp://www.flickr.com/photos/14804582@N08/2111269218/
Key elements of schema.xml• Unique identifer• Default search field• Types• Fields and dynamic fields• Copy fields
Solr configurationhttp://www.flickr.com/photos/esetianto/4099842490/
Key elements of solrconfig.xml• Settings for your search index• Warm-up routines• Cache settings• Replication• Update chain
Featureshttp://xkcd.com/619/
Facets
Facets
Facets
Just add this to your URL:• facet=true&facet.field=field• Example:  facet=true&facet.field=language
Facet queries
Facet queries&facet=true&facet.query=price:[* TO 100]&facet.query=price:[100 TO 200]&facet.query=price:[200 TO 300]&facet....
Now you want to drill down!http://www.flickr.com/photos/kk/4712925031/
Filter queries
Filter queries
Filter queries
Just add this to your URL:• fq=field:value• Example:  fq=source:facebook.com
Produce «word clouds»
•TermsComponent•TermVectorComponent
TermVectorComponent    Term vector information aggregator
Scalabilityhttp://www.flickr.com/photos/dickyfeng/3249837481/
•Sharding•Replication
Index sharding strategy   Solr         Solr          Solr           Solr          Solrinstance 1   instance 2    instance ...
Index sharding strategy   Solr         Solr          Solr           Solr              Solrinstance 1   instance 2    insta...
Index sharding strategy   Solr         Solr          Solr           Solr              Solrinstance 1   instance 2    insta...
Just add this to your URL:• shards=shard1,shard2• Example: q=android&shards=solr1.node.com/s olr,solr2.node.com/solr,solr3...
ReplicationIndexer      Master             Slave      android   Search
Replication configuration
Integration of Solrhttp://www.flickr.com/photos/certified_su/229016531/
Solr has support for many different            languages•   Ruby•   PHP•   Java•   Scala•   Python•   .NET•   Perl•   Java...
Tips & Gotcha’sOr; how to avoid the sinkholes!http://www.flickr.com/photos/67165210@N00/4661419386/
«What data do your  clients need?»
«Figure out what kind of  searches you will be         doing»
«Spend a siginficant    amount of timedesigning schema.xml»
«Add dynamic fields for ALL your field types»
«Do not use Solr as your primary data store!»
«The 20 million mark»
But most importantly...Don’t panic!
http://www.flickr.com/photos/11304375@N07/2046228644
http://www.flickr.com/photos/davidw/2201099990/
Thank you!http://www.jeremiahblatz.com/personal/pics/Australia_Travel_Pictures_2009/day12/164_Sunrise_Great_Barrier_Reef.h...
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Apache Solr - search for everyone!
Upcoming SlideShare
Loading in …5
×

Apache Solr - search for everyone!

2,283 views

Published on

Talk presented at Baksia meet up i Oslo on November 23rd 2011.

Published in: Technology
  • Be the first to comment

Apache Solr - search for everyone!

  1. 1. Apache Solr - search for everyone!http://www.flickr.com/photos/malikdhadha/
  2. 2. • Co-founder and R&D Director at Integrasco• Founder and developer of Notpod• Leader of javaBin Sørlandet• Programmer and Open Source enthusiast Jaran Nilsen twitter.com/jarannilsen
  3. 3. A global leader in social intelligence
  4. 4. What is search?http://www.flickr.com/photos/denverjeffrey/5133538450/
  5. 5. http://www.flickr.com/photos/somegeekintn/3709203268/
  6. 6. This is Apache Solr• Open Source enterprise search server from Apache• Built on Apache Lucene• Offers additional features to those of Lucene
  7. 7. First, a little history...
  8. 8. • Started out as a in-house CNET project for adding search functionality to the CNET website in 2004
  9. 9. • Started out as a in-house CNET project for adding search functionality to the CNET website.• Donated to Apache Software Foundation in 2006
  10. 10. • Started out as a in-house CNET project for adding search functionality to the CNET website.• Donated to Apache Software Foundation in 2006• Graduated from incubation status in 2007
  11. 11. • Since version 3.1 (March 2011), Solr and Lucene are now sharing the same codebase. +
  12. 12. • Since version 3.1 (March 2011), Solr and Lucene are now sharing the same codebase.• Meaning sharing of features and fixes between the projects at a much higher rate +
  13. 13. wget http://apache.uib.no/lucene/solr/3.6.1/apache-solr-3.6.1.tgztar xvf apache-solr-3.6.1.tgzcd apache-solr-3.6.1/example/java -jar start.jar 4 small steps...
  14. 14. ...and we’re up!
  15. 15. cd exampledocs/./post.sh ipod_other.xml
  16. 16. The obvious part – full text searchinghttp://www.flickr.com/photos/49889874@N05/6877840735/
  17. 17. • q=yourquery• Example: q=android AND ios&rows=100
  18. 18. Don’t worry - it’s not just XML!
  19. 19. The Schemahttp://www.flickr.com/photos/14804582@N08/2111269218/
  20. 20. Key elements of schema.xml• Unique identifer• Default search field• Types• Fields and dynamic fields• Copy fields
  21. 21. Solr configurationhttp://www.flickr.com/photos/esetianto/4099842490/
  22. 22. Key elements of solrconfig.xml• Settings for your search index• Warm-up routines• Cache settings• Replication• Update chain
  23. 23. Featureshttp://xkcd.com/619/
  24. 24. Facets
  25. 25. Facets
  26. 26. Facets
  27. 27. Just add this to your URL:• facet=true&facet.field=field• Example: facet=true&facet.field=language
  28. 28. Facet queries
  29. 29. Facet queries&facet=true&facet.query=price:[* TO 100]&facet.query=price:[100 TO 200]&facet.query=price:[200 TO 300]&facet.query=price:[300 TO 400]&facet.query=price:[400 TO 500]&facet.query=price:[500 TO *]
  30. 30. Now you want to drill down!http://www.flickr.com/photos/kk/4712925031/
  31. 31. Filter queries
  32. 32. Filter queries
  33. 33. Filter queries
  34. 34. Just add this to your URL:• fq=field:value• Example: fq=source:facebook.com
  35. 35. Produce «word clouds»
  36. 36. •TermsComponent•TermVectorComponent
  37. 37. TermVectorComponent Term vector information aggregator
  38. 38. Scalabilityhttp://www.flickr.com/photos/dickyfeng/3249837481/
  39. 39. •Sharding•Replication
  40. 40. Index sharding strategy Solr Solr Solr Solr Solrinstance 1 instance 2 instance 3 instance 4 instance N
  41. 41. Index sharding strategy Solr Solr Solr Solr Solrinstance 1 instance 2 instance 3 instance 4 instance N ipod OR iphone Search
  42. 42. Index sharding strategy Solr Solr Solr Solr Solrinstance 1 instance 2 instance 3 instance 4 instance N ipod OR iphone Search
  43. 43. Just add this to your URL:• shards=shard1,shard2• Example: q=android&shards=solr1.node.com/s olr,solr2.node.com/solr,solr3.node.co m/solr
  44. 44. ReplicationIndexer Master Slave android Search
  45. 45. Replication configuration
  46. 46. Integration of Solrhttp://www.flickr.com/photos/certified_su/229016531/
  47. 47. Solr has support for many different languages• Ruby• PHP• Java• Scala• Python• .NET• Perl• JavaScript
  48. 48. Tips & Gotcha’sOr; how to avoid the sinkholes!http://www.flickr.com/photos/67165210@N00/4661419386/
  49. 49. «What data do your clients need?»
  50. 50. «Figure out what kind of searches you will be doing»
  51. 51. «Spend a siginficant amount of timedesigning schema.xml»
  52. 52. «Add dynamic fields for ALL your field types»
  53. 53. «Do not use Solr as your primary data store!»
  54. 54. «The 20 million mark»
  55. 55. But most importantly...Don’t panic!
  56. 56. http://www.flickr.com/photos/11304375@N07/2046228644
  57. 57. http://www.flickr.com/photos/davidw/2201099990/
  58. 58. Thank you!http://www.jeremiahblatz.com/personal/pics/Australia_Travel_Pictures_2009/day12/164_Sunrise_Great_Barrier_Reef.html

×