Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Elasticsearch for Westcoast

2,233 views

Published on

How Flax helped Westcoast, the UK's largest privately owned IT distributor, implement Elasticsearch for their website

Published in: Software
  • Be the first to comment

  • Be the first to like this

Elasticsearch for Westcoast

  1. 1. Charlie Hull, Managing Director, Flax Nick Gushlow, Systems Architect, Westcoast Elasticsearch London Meetup charlie@flax.co.uk www.flax.co.uk/blog +44 (0) 8700 118334 Twitter: @FlaxSearch Search is never Simple Elasticsearch for Westcoast
  2. 2. Building open source search applications since 2001 Independent, honest advice and analysis Expert design & development, Apache Solr committers Test-driven relevancy and performance tuning Custom training & mentoring for your staff Flexible support up to 24/7/365 with SLAs Come to our Meetups!
  3. 3. Charlie Hull, Managing Director & co-founder of Flax Nick Gushlow, Systems Architect at Westcoast Who are we? @FlaxSearch
  4. 4. Why Westcoast needed a new search engine The source data & the plan The trouble with.... Building an admin panel for search The search goes live Lessons learned What we'll cover today @FlaxSearch
  5. 5. Largest privately owned IT distributor in UK & Ireland £1.5 billion turnover Apple, HP, Lenovo, Microsoft, Samsung, Toshiba Includes XMA / QC Supplies and Viglen Who are Westcoast? @FlaxSearch
  6. 6. Old SQL based search not accurate enough 8000 searches per day, 90% SKU based You searched iPad …. did you actually want an iPad? Customers used Google / competitors to find part numbers, Static traffic numbers – 3500 user per day 7am to 7pm Increase web revenue further, currently £40m Why a new search engine? @FlaxSearch
  7. 7. Business approved a project to implement a change to ‘improve search’ Google Search Appliance SLI Apptus Fredhopper Elasticsearch Time for a change @FlaxSearch
  8. 8. Live pricing XML data sheets Business user management interface Synonyms / Exclusions Boosts Search vs Search vs Search Requirements @FlaxSearch
  9. 9. 0.5m products Nested data (attributes) Supplied as XML, one file per product The source data @FlaxSearch
  10. 10. 0.5m products Nested data (attributes) Supplied as XML, one file per product BUT! – Live Pricing API will restrict results at search time – Different for every end customer – Based on hard to explain business rules The source data @FlaxSearch
  11. 11. Elasticsearch Java client Custom Java indexer (Dropwizard) Search application (Dropwizard) Admin panels (AngularJS) Agile process The plan @FlaxSearch
  12. 12. First, do your search Send 5000 results to legacy pricing API Merge the pricing information with search results Now build your facets (including on price) Hang on, doesn't Elasticsearch do facets for you? The trouble with facets @FlaxSearch
  13. 13. Front end systems built by third party – Solution: Search app with JSON API (defined by them) – Encrypted JSON for use during sessions More trouble with facets @FlaxSearch
  14. 14. Front end systems built by third party – Solution: Search app with JSON API (defined by them) – Encrypted JSON for use during sessions Data for all the facets must be supplied to the UI – Full result counts for applied facets need to be returned, in the order they were applied – Solution: lots of searches More trouble with facets @FlaxSearch
  15. 15. Front end systems built by third party – Solution: Search app with JSON API (defined by them) – Encrypted JSON for use during sessions Data for all the facets must be supplied to the UI – Full result counts for applied facets need to be returned, in the order they were applied – Solution: lots of searches Custom facets for some customers – Solution: an index of facet definitions More trouble with facets @FlaxSearch
  16. 16. Boost for individual items – Easy! Define in the source data Term boosts – e.g. some Macbooks over other Macbooks – Harder – but still defined in source data The trouble with boosting @FlaxSearch
  17. 17. A great way to run search projects! ...unless not everyone can do Agile The trouble with Agile @FlaxSearch
  18. 18. Allows Westcoast to adjust – Synonyms / Exclusions – Remove items from index – Test searches – Test synonyms then push to live • Synonyms are index side as default query is AND Built in AngularJS Building an Admin panel @FlaxSearch
  19. 19. @FlaxSearch
  20. 20. @FlaxSearch
  21. 21. @FlaxSearch
  22. 22. A single node for Elasticsearch A single node for index & search applications Ultimately mirrored for failover Query load very low (1 QPS) – But this may change! Business hours support by Flax The search goes live @FlaxSearch
  23. 23. @FlaxSearch
  24. 24. @FlaxSearch
  25. 25. Elasticsearch results were good Business maintenance, large, boring, never ending work Changing customer behaviour is slow Search results over 30% faster on average Time savings for sales staff Post Live @FlaxSearch
  26. 26. Integrating with legacy systems is hard Business rules can be hard to understand & harder to explain Not everything can be done with search If you want to do Agile, make sure everyone else can Lessons learned @FlaxSearch
  27. 27. Thankyou! Any questions? charlie@flax.co.uk www.flax.co.uk/blog +44 (0) 8700 118334 Twitter: @FlaxSearch @FlaxSearch
  28. 28. Plug 3rd & 4th February 2016, Cambridge UK Open source search for Bioinformatics Free event near Cambridge on Wellcome Genome Campus covering both Solr & Elasticsearch, talks & hands-on workshops http://www.ebi.ac.uk/pdbe/about/events @FlaxSearch
  29. 29. Plug #2 20th March 2016, Padua, Italy First International Workshop on Recent Trends in News Information Retrieval One-day workshop as part of the European Conference on Information Retrieval (ECIR 2016) – submission deadline end of January http://research.signalmedia.co/newsir16/index.html (including a free test dataset of 1m news articles!) @FlaxSearch

×