Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

idalab seminar #7 Next generation product search at Zalando using deep learning

295 views

Published on

At Zalando, the in house search engine currently matches full text queries with products through a cascaded architecture. Each step in the cascade processes the input in a specific way, for example, locating mentions of brands, spell checking, and disambiguation. Finally the preprocessed input is used to filter the product attributes (such as color=red, brand=Nike). This architecture has multiple drawbacks, such as fragility, limited scalability and extensibility. Duncan and his colleagues are working on replacing this cascade with a single step end-2-end deep learning architecture which involves no textual preprocessing and directly filters image content as well as product meta-data. In this talk, Duncan described the components involved in such a system, as well as potential advantages and disadvantages.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

idalab seminar #7 Next generation product search at Zalando using deep learning

  1. 1. Duncan Blythe Agency for Data Science Machine learning & AI Mathematical modelling Data strategy Next generation product search at Zalando using deep learning idalab seminar #7 | January 26th 2018
  2. 2. End-2-End Search Idalab Seminar Duncan Blythe Jan 2018
  3. 3. About Me Duncan Blythe Research Scientist @ Zalando Research M.Math.Phil in Mathematics/ Philosophy @ Oxford Ph.D. & M.Sc. in Machine Learning/ Computational Neuroscience @ TU Berlin Postdoc in Biomedical Engineering @ DZNE Bonn (Helmholtz)
  4. 4. Zalando ● Multinational fashion e-commerce company ● Operates in 14 countries in the EU ● > 23 million active customers ● More than 14,000 employees ● > 1,900 tech-employees ● More than 100 data scientists/ research engineers etc.. ● Large data scientific component
  5. 5. Zalando
  6. 6. Zalando
  7. 7. Zalando Research ● 15 Researchers with PhDs ● Backgrounds CS/ ML/ Physics/ Math/ Linguistics ● 4 subteams ● RL, Vision, NLP, Retrieval ● We are growing/ hiring!
  8. 8. NLP Team @ Zalando Research Roland Vollgraf Alan Akbik Duncan Blythe Leonidas Lefakis
  9. 9. NLP Team @ Zalando Research Credit to Han Xiao (now @ ten cent) for the core work
  10. 10. Outline 1. How do many product search systems work? 2. Why do we need an end-to-end product search system? 3. What data do we need to build end-to-end search? 4. End-to-end retrieval model 5. Discussion
  11. 11. Classical full-text retrieval framework Query String/Symbolic representation Product String/Symbolic representation Matching offline Parsing Indexing
  12. 12. Classic product search system with filter query Indexing { “brand”: “Miss Selfridge”, “category”: “Umhängetasche”, “color”: “red”, ... } Message Queue Structured string index Filter query *Animation brand="nike" AND color="orange"
  13. 13. Parsing a full-text query to a filter query { "brand": "Miss Selfridge", "category": "Umhängetasche", "color": "red", ... } Structured string index { "color": "red", "category": "shirt" } Matching IndexingParsing
  14. 14. Query understanding as a pipeline (ideal) tokenize lemmatize spell-correct recognize named-entity disambiguate Filter queryquery-builder recognize synonym & acronym User query normalize Parsing "Jeckk Wolfskin BluEjackets" brand="Jack Wolfskin" AND category="coat" AND color="blue" jecck wolfskin bluejackets jeckk+wolfksin+blue+jackets “jeckk wolfksin”+blue+jacket “jack wolfskin”+blue+jacket “jack wolfskin”+blue+jacket “jack wolfskin”+blue+coat ?“jack wolfskin”?+blue+coat ?jack+wolf-skin?+blue+coat
  15. 15. Query understanding in practice tokenize lemmatize spell-correct recognize named-entity disambiguate Filter queryquery-builder Full text query normalize Queryparsing recognize synonym & acronym "Jeckk Wolfskin BluEjackets" brand="Jack Wolfskin" AND category="coat" AND color="blue"
  16. 16. Pros & cons of a pipeline system Upside: intuitive, modular, many off-the-shelf packages, easy to collaborate ● Fragile ● Complicated dependency ● Not straightforward to improve overall search experience ● Difficult to scale out on other languages
  17. 17. Problems of a symbolic-based system ● Limited interpretability; ● Hard-coding rules to enable acronyms, synonyms, etc. Can't scale to different appdomains; ● No matter how well the intention is, the overall system will turn into a set of heuristics. Upside: easy to implement, efficient, very well-studied
  18. 18. Question 1: If finding the right article is the final goal, then why should we even care about pipeline components?
  19. 19. Question 2: How can we associate “petite” with “for the smaller man” without hard-coding for each language?
  20. 20. “End-2-End” product search Product search system ● Trained as mapping from queries to products ● Fully data-driven ● All components trained with deep learning
  21. 21. “End-2-End” product search Question 1: If finding the right article is the final goal, then why should we even care about spell-checking? Question 2: How can we associate “fur mamas” with “Schwangerschaftsmode” without hard-coding on each domain? eliminate components in the pipeline find better representation for query and product An end-to-end product search system with deep learning more robust easier to maintain more scalable simpler architecture smarter
  22. 22. Classical system vs end-to-end product search system Query Symbolic representation Product Symbolic representation ① indexing② parsing ③ matching offline Query Latent representation Product Latent representation matching offline deep learning deep learning
  23. 23. Text ↔ product data sources
  24. 24. Three types of data sources ● Click through data ● Crowdsourcing annotations ● Customer reviews User-generated content Product
  25. 25. Extracting Query↔Product mapping from message queue receive-query: "denim shirt" search-result user type in search-box see search result page retrieval-search-result click a product click-through: SKU00000-001 retrieve-reco -result "denimshirt" Message Queue Time Time { query: "denim shirt" skus: ["SKU00000-001", "SKU00000-002"] } see PDP search-result PDP PDP click-through: SKU0000-002 click on reco
  26. 26. Example of Query → Product map {"query":"ananas", "skus":[ {"id":"CE321D0HP-A11","freq":371}, {"id":"RL651E02D-F11","freq":273}, {"id":"EV411AA0K-T11","freq":243}, {"id":"L1211E001-A11","freq":208}, {"id":"ES121D0ON-C11","freq":180}, ... {"id":"TO226K009-I11","freq":2}, {"id":"BH523F01J-A11","freq":2}, {"id":"MOC83C00C-J11","freq":1}, {"id":"MOC83C001-J11","freq":1}, {"id":"HG223F04A-A11","freq":1}]}
  27. 27. {"sku":"CZ621C04O-G11", "queries":[ {"text":"chi+chi+london","freq":998}, {"text":"abendkleid","freq":403}, {"text":"ballkleid","freq":394}, {"text":"cocktailkleid","freq":134}, {"text":"kleid","freq":125}, {"text":"kleider","freq":118}, {"text":"abendkleider","freq":79}, {"text":"abendkleid+lang","freq":58}, {"text":"kleid+lang","freq":46}, {"text":"abiballkleid","freq":46}, {"text":"chi+chi","freq":43}, {"text":"lange+kleider","freq":40}, {"text":"ballkleider","freq":36} Example of Product → Query map {"text":"ballkkeid+lang","freq":1}, {"text":"ball+kleid","freq":1}, {"text":"abschlusskleid+leng","freq":1}, {"text":"abschlussballkleider","freq":1}, {"text":"abschluss+kleider+rot","freq":1}, {"text":"abenkleid","freq":1}, {"text":"abendskleid","freq":1}, {"text":"abendkleider+in+lang","freq":1}, {"text":"abendkleider+abendkleider","freq":1}, {"text":"abendkleid+damen","freq":1}, {"text":"abendkleid+chi+chi+london","freq":1}, {"text":"abendkleid+/ballkleid","freq":1}, {"text":"abend+kleid","freq":1}]}
  28. 28. Encoder-Matcher Architecture (q, ) ...(u, ) (y, ) RNN RNN RNN... character-embedding query-encoder attribute-encoder matcher {brand: "Nike", color: "olive"} YES/NO image-encoder
  29. 29. Component parts Query encoder Image encoder MatcherAttribute Encoder ● Recurrent neural network ● LSTM/ GRU etc.. ● Character ● No preprocessing ● Language flag ● Convolutional neural network ● VGG/ ResNet/ AlexNet ● Multiple views? ● Pretraining methods ● Data augmentation ● Textual embedding ● word2vec-esque ● Pretraining fashion corpora ● “Objective function” ● Ranking/ classification ● Nonlinear/ linear ● E.g. cosine similarity ● Test time considerations
  30. 30. Convolutional image encoders Layer1 Layer2 Layer3 Layern ...
  31. 31. Convolutional image encoders: fDNA Sebastian Heinz Christian Bracher Nearest neighbours: Product map:
  32. 32. Query encoders "denim shirt" <UK> d e n i m _ s h i r t RNN RNN RNN RNN RNN RNN RNN RNN RNN RNN RNN RNN
  33. 33. Alignment objective We want this to be large:
  34. 34. Demo
  35. 35. Code Review
  36. 36. Discussion: pros/cons of an end-to-end product search Query Symbolic representation Product Symbolic representation ① indexing② parsing ③ matching offline Query Latent representation Product Latent representation matching offline deep learning deep learning Classic Deep learning based End2End Scalable, maintainable, data-driven Need a lot of data, comp. resources
  37. 37. Thanks for your attention!

×