This document discusses the challenges of search relevancy and provides examples of techniques used to improve search results, including: 1) It explains concepts like precision and recall that are used to evaluate search results and the difficulty of determining what constitutes a "good" result. 2) It provides examples of how search can go wrong when queries are ambiguous and shows different types of search queries people commonly use. 3) It describes techniques like DSSM and C-DSSM that use deep learning to build robust representations of queries and documents to improve relevancy.

1. The dark art of search relevancy

2. Hi, I’m Eddie.
@ejlbell

3. We collect the world of
fashion into a customisable
shopping experience.

4. 90 thousand
3,800,000
items scraped per day.
items updated / hour

5. 5

6. Why is search hard?
6

7. True NegativesFalse Negatives
Selected Elements
True
Positives
False
Positives
Relevant Elements
Precision Recall
How many selected
items are relevant?
How many selected
items are relevant?

8. BM25
8

9. When it goes wrong…
“Little Black Dress”

10. When it goes wrong…
“Red Valentino”

11. BCBGMAXZRIA
11

13. Dress

14. Dress Dress Shirt

15. Dress Dress Shirt Shirt Dress

16. What is a good result?

17. Clickthrough Data

18. Crowdsource Relevance

19. Ordinal
“Light Pink Heels”

20. Pairwise
“Blue Trainers”

21. Pairwise
“Blue Trainers”

22. Search Term Example
1.) designer + category “hermes sandal”
2.) designer + colour + category “burberry black boots”
3.) designer + fabric + category “chloe leather top”
4.) color + category “gray hoodie”
5.) designer + type “asos bag”
What are people actually searching?

23. DSSM

24. DSSM with 1D convolution
and max pooling
C-DSSM

25. CSSM Results
Search Result Score
Download office Excel 0.54
Word office online 0.50
Apartment office hours 0.33
Internation office berklely 0.27
“Microsoft Office”
Search Result Score
Car body kits 0.70
Auto body parts 0.55
Calculate body fat 0.22
Forcefield body armour 0.17
“Car body shop”

26. Computing the
results
Store hidden layer representation in postgres
Rank by cosine distance
Speed up search with ANN
Random projection trees
Calculate query representation at run time
Docker, Django-rest, chef, empire, auto-scaling

27. Images?

28. Images Train 8 layer CNN
Swap out soft max layer
A soft max layer for each label of interest
60 Million parameters
Build robust learned representations
Represents products as a 4096 element vector

29. Image Classifier
Sub-category Score
Male, clothing, suits, 3 piece suits 0.72263
Male, clothing, suits, 2 piece suits 0.12102
Male, clothing, jackets, formal jackets 0.10818
Male, clothing, coats, trench coats 0.02396
Male, clothing, jackets, casual jackets 0.00949
Colors
Blue 0.99397
Gray 0.00481
Black 0.00071

33. Search is fun but search is hard
Conclusion
33

34. thank you