Haystack 2019 - Architectural considerations on search relevancy in the context of e-commerce - Johannes Peter
![11
Towards a search taxonomy
Official (visible) categories
Virtual (invisible) categories
Define category by query:
match( awesomemobiles[0-9]{1,2}[rs]{0,1} )](https://image.slidesharecdn.com/2019-04-22haystackcharlottesvillejohannespeter-190517230201/75/haystack-2019-architectural-considerations-on-search-relevancy-in-the-context-of-ecommerce-johannes-peter-11-2048.jpg?cb=1671894848)
Haystack 2019 - Architectural considerations on search relevancy in the context of e-commerce - Johannes Peter
With an increasing amount of relevancy factors, relevancy fine-tuning becomes more complex as changing the impact of factors produces increasingly more unintended side effects. In recent years, there has been a lot of discussion about how learning algorithms can replace manual relevancy fine-tuning in order to manage this complexity. However, discussions about the challenge of relevancy should additionally consider architectural aspects. Especially microservice-based architectures provide many ways to encapsulate and to separate complexities of search solutions, which facilitates optimizing the search as well as locating and fixing problems. Generally, relevancy factors can be assigned to three different groups, each handled at a different stage of the search request processing. The first group contains contextual factors that depend on certain characteristics of a query, such as query-related boosts lifting up top-sellers for queries or category-related boosts to distinguish products from their accessories. Such contextual factors can be handled as a step of the preprocessing of queries. The respective boosting information can simply be appended to the query before it is actually sent to the search engine. Ideally, the normalization of the query is done beforehand. The second group contains factors that are considered for all queries in more or less the same way, e. g. a ranking function basing on keyword occurrences, product topicality or sales in total. Factors related to this group can be handled directly by configuring the search engine. The third group contains situational factors. For instance, a certain product might be a good match for a certain query in general, but for situational circumstances it should not appear among the top five products (e. g. because it is out of stock). Such situational factors can be handled by resorting result sets, after they were returned by the search engine. The handling of the different factors within successive stages of search request processing will be discussed from an architectural perspective. Implications for applying learning algorithms and the implementation of a personalized search will be considered.
Recommended
More Related Content
What's hot
What's hot(20)
Similar to Haystack 2019 - Architectural considerations on search relevancy in the context of e-commerce - Johannes Peter
Similar to Haystack 2019 - Architectural considerations on search relevancy in the context of e-commerce - Johannes Peter(20)
![[Webinar] Getting to Insights Faster: A Framework for Agile Big Data](https://cdn.slidesharecdn.com/ss_thumbnails/agile-big-datav2-131122164332-phpapp01-thumbnail.jpg?width=768&fit=bounds)
More from OpenSource Connections
More from OpenSource Connections(20)
Recently uploaded
Recently uploaded(20)
![[DSC Europe 23] Branka Panic - Peace in the age of artificial intelligence.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/dsceurope23brankapanic-peaceintheageofartificialintelligence-231204184932-31f96125-thumbnail.jpg?width=768&fit=bounds)
Haystack 2019 - Architectural considerations on search relevancy in the context of e-commerce - Johannes Peter
- 1. 1 Architectural considerations on search relevance in the context of e-commerce Johannes Peter Architect / Engineer @ MediaMarktSaturn Retail Group Haystack, 2019 The Search Relevance Conference Charlottesville, Virginia
- 2. 2 Who we are
- 3. 3 Who we are
- 4. 4 Where we are • Growth à New ways • Old search: Commercial tool stack • Scaling business requires ⁃ Scaling of load ⁃ Scaling of quality
- 5. 5 Google BigQuery Tools & Technologies Headless CMS Kubernetes Cluster (GKE) Services & APIs Elasticsearch Apache NiFi Data APIs
- 6. 6 Apache NiFiGoogle BigQuery Search Data Flow Key-Value Store Elasticsearch Search Analytics API Headless CMS Data APIs User Signals Search Feature API Data Aggregations Extracted Feature Data Search Mnmt. Content Lookups Index data
- 7. 7 Search API Search API Query Standardization Consumer Elasticsearch Redirects Campaign Rules gc4 washing machines gamingconsole 4 washing machine /path/washing_machine contains(gamingconsole) boost(sku1)
- 8. 8 Query mapping & ranking Query 1 Query 2 … gamingconsole 4 gc4 Stopwords gamingconsole 4 gc4 Synonyms gamingconsole 4 gc 4 gc4 Shingles gamingconsole 4 gc 4 gamingconsole4 gc4 gc4 • Which query variant is the best? ⁃ Number of matches? ⁃ Conversion rate?
- 9. 9 Signal aggregation Queries Clicks / Sales Relation awesomemobile 4 awesomemobile 3r sku1 : category1 sku2 : category2 sku3 : category1 sku4 : category1 +awesomemobile -case : category1 awesomemobile 4 case awesomemobile 3r case sku5 : category4 sku6 : category4 sku7 : category3 sku8 : category4 +awesomemobile +case : category4
- 10. 10 Search API Search API Query Standardization Consumer Elasticsearch Redirects Campaign Rules Contextual Information gc4 awesomemobile 4 case contains(case AND awesomemobile) boost(category4)
- 11. 11 Towards a search taxonomy Official (visible) categories Virtual (invisible) categories Define category by query: match( awesomemobiles[0-9]{1,2}[rs]{0,1} )
- 12. 12 Search relevance architecture Search API … Elasticsearch Contextual information Facet re-ranking Cosmetic product re-ranking Regular Search Re-ranking
- 13. 13 • Benefits - Flexible auto-generation of boosting rules – continuous improvement without changing or re- training of the basic model - Reduced complexity of basic relevance model - Ambiguity of a query is determined on the basis of user behavior • Limitations - Variance related to interactions between categories and basic factors cannot be explained • Potential - The query index can be the basis for various additional features (e. g. query relaxation) Conclusion