Optimizing Solr for Performance  & User Experience
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Optimizing Solr for Performance & User Experience

  • 10,487 views
Uploaded on

Using analytics to understand customer search behavior, we used a strategy pattern to return the right search strategy for each on-site search for a big boost in sales.

Using analytics to understand customer search behavior, we used a strategy pattern to return the right search strategy for each on-site search for a big boost in sales.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
10,487
On Slideshare
10,459
From Embeds
28
Number of Embeds
2

Actions

Shares
Downloads
61
Comments
0
Likes
3

Embeds 28

http://www.slideshare.net 26
http://www.twylah.com 2

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Optimizing Solr for Performance
    & User Experience
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 2.
    • Largest selection of sheet music
    • 3. 12 year old online business
    • 4. 600k+ SKUs
    • 5. 1.5M Songs
    • 6. Faceted navigation with Solr
    • 7. On-site search with Solr
    • 8. Lots of needles in a huge haystack
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll : @briandoll
  • 9.
    • 15 VMWare app server instances
    • 10. 2k app requests per minute
    • 11. MySQL database
    • 12. 5 memcache instances
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 13.
    • 1k Solr requests per minute
    • 14. ~100ms average response time
    • 15. 3 Solr hosts on VMWare
    • 16. 50% CPU load, 2GB memory each
    • 17. 1.00 Apdex Score
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 18. We love that the structure of your data is part of your Solr environment (schema.xml)
    And the way you query against that data is part of your app
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 19. on-site search is not as varied as we had thought
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 20. Surprisingly, lots of people search
    like this:
    Piano
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 21. What if we applied a strategy pattern to our search queries?
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 22. “Instrument” searches could get their own strategy.
    “Artist” searches could get their own.

    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 23. Make sure your default search is working well first!
    Subtle changes to our algorithm provided a 32% increase in per-search value, with some phrases seeing an increase of over 3000%!
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 24. We construct various strategies:
    ...
    DEFAULT_SEARCH_HASH = {
    :id_codes => { :search_type => :phrase, :boost => 1024, :slop => 2 },
    :bucket_proper_names => { :search_type => :phrase, :boost => 500, :slop => 2 },
    :bucket_titles => { :search_type => :phrase, :boost => 450, :slop => 100 },
    :bucket_songs => { :search_type => :phrase, :boost => 64, :slop => 5 },
    :bucket_primary_cats => { :search_type => :phrase, :boost => 32, :slop => 100},
    :bucket_secondary_cats => { :search_type => :phrase, :boost => 16, :slop => 100 },
    :bucket_binding => { :search_type => :phrase, :boost => 2, :slop => 100},
    :bucket_lead_time => { :search_type => :phrase, :boost => 1, :slop => 100},
    :bucket_blurbs => { :search_type => :phrase, :boost => 0.05, :slop => 7 },
    :text => { :search_type => :term, :boost => 0.005 }
    }
    ...
    def self.popular_instrument_strategy(search_phrase)
    params = {:instrument_text => {:search_type => :phrase, :boost => 600, :slop => 2 } }
    Strategy.new("default-instrument", search_phrase, params , :top_selling)
    end
    ...
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 25. And then we have an algorithm to help determine which strategy we should apply to an incoming search request:
    [redacted]
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 26. Implementing these strategies provided an 86% increase in per-search value.
    We can now fine-tune individual categories of searches, as well as specific phrases.
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 27. Rule of thumb:
    Don't try to please everybody. Please the people who make you money.
    Tuning your search algorithm is an endless game, but if you focus on pleasing customer X, it'll cost you.
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • 28. In order to make all this happen, we relied heavily on two tools:
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll