Optimizing Solr for Performance  & User Experience
Upcoming SlideShare
Loading in...5
×
 

Optimizing Solr for Performance & User Experience

on

  • 10,294 views

Using analytics to understand customer search behavior, we used a strategy pattern to return the right search strategy for each on-site search for a big boost in sales.

Using analytics to understand customer search behavior, we used a strategy pattern to return the right search strategy for each on-site search for a big boost in sales.

Statistics

Views

Total Views
10,294
Views on SlideShare
10,266
Embed Views
28

Actions

Likes
3
Downloads
59
Comments
0

2 Embeds 28

http://www.slideshare.net 26
http://www.twylah.com 2

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Optimizing Solr for Performance  & User Experience Optimizing Solr for Performance & User Experience Presentation Transcript

  • Optimizing Solr for Performance
    & User Experience
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
    • Largest selection of sheet music
    • 12 year old online business
    • 600k+ SKUs
    • 1.5M Songs
    • Faceted navigation with Solr
    • On-site search with Solr
    • Lots of needles in a huge haystack
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll : @briandoll
    • 15 VMWare app server instances
    • 2k app requests per minute
    • MySQL database
    • 5 memcache instances
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
    • 1k Solr requests per minute
    • ~100ms average response time
    • 3 Solr hosts on VMWare
    • 50% CPU load, 2GB memory each
    • 1.00 Apdex Score
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • We love that the structure of your data is part of your Solr environment (schema.xml)
    And the way you query against that data is part of your app
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • on-site search is not as varied as we had thought
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • Surprisingly, lots of people search
    like this:
    Piano
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • What if we applied a strategy pattern to our search queries?
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • “Instrument” searches could get their own strategy.
    “Artist” searches could get their own.

    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • Make sure your default search is working well first!
    Subtle changes to our algorithm provided a 32% increase in per-search value, with some phrases seeing an increase of over 3000%!
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • We construct various strategies:
    ...
    DEFAULT_SEARCH_HASH = {
    :id_codes => { :search_type => :phrase, :boost => 1024, :slop => 2 },
    :bucket_proper_names => { :search_type => :phrase, :boost => 500, :slop => 2 },
    :bucket_titles => { :search_type => :phrase, :boost => 450, :slop => 100 },
    :bucket_songs => { :search_type => :phrase, :boost => 64, :slop => 5 },
    :bucket_primary_cats => { :search_type => :phrase, :boost => 32, :slop => 100},
    :bucket_secondary_cats => { :search_type => :phrase, :boost => 16, :slop => 100 },
    :bucket_binding => { :search_type => :phrase, :boost => 2, :slop => 100},
    :bucket_lead_time => { :search_type => :phrase, :boost => 1, :slop => 100},
    :bucket_blurbs => { :search_type => :phrase, :boost => 0.05, :slop => 7 },
    :text => { :search_type => :term, :boost => 0.005 }
    }
    ...
    def self.popular_instrument_strategy(search_phrase)
    params = {:instrument_text => {:search_type => :phrase, :boost => 600, :slop => 2 } }
    Strategy.new("default-instrument", search_phrase, params , :top_selling)
    end
    ...
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • And then we have an algorithm to help determine which strategy we should apply to an incoming search request:
    [redacted]
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • Implementing these strategies provided an 86% increase in per-search value.
    We can now fine-tune individual categories of searches, as well as specific phrases.
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • Rule of thumb:
    Don't try to please everybody. Please the people who make you money.
    Tuning your search algorithm is an endless game, but if you focus on pleasing customer X, it'll cost you.
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll
  • In order to make all this happen, we relied heavily on two tools:
    SFBay Apache Lucene/Solr Meetup - Jan 2010 Brian Doll :@briandoll