1. 1
Improving Relevancy Through
Query Expansion
Beyond The
Search Engine

2. 2
What is Ibotta?
“Make Every
Purchase Rewarding”
Users browse and unlock offers they
like. Then take a picture of their receipt
and instantly get cash back.

3. 3
Ibotta’s Search Challenge
Relatively small corpus that rotates every few weeks
Many dead end searches
Many missed opportunity for related content

4. 4
Search Team
Epoch
Figuring out Elasticsearch
Building a Taxonomy
W
ord Embeddings
Query Expansion
Mar ‘18 May ‘18 Aug ‘18Apr ‘18 Oct ‘18

5. 5
Search V0

6. 6
Cons:
● No sorting beyond tf*idf
● Some results are obviously
bad
● Missing lots of related
content
Pros:
● Handles partial matching with
ngram analyzer
Search V0

7. Directive:
“Make search Better”
7

8. 8
User Intent
Food Staples
milk
bread
cereal
yogurt
cheese
eggs
coffee
Categories
dairy
laundry
food
meat
produce
grocery
Brands
Colgate
Loreal
Dove
Coke
Kleenex

9. Taxonomy?
9

10. 10
Taxonomy
Existing categories were too broad
ML team had aggregated a rich taxonomy around millions of
products
We used machine learning to add categories where missing and
correct bad labels.

11. 11
Taxonomy
“Ball Park Beef Franks” => “meat, sausage, hot dog”
“Tide” => “cleaning supplies, detergent, laundry”
“Silk” => “milk-alternative, soy milk”

12. 12

13. 13
Taxonomy
More fine-grained categorization of content
Provides more categories for users to hit on with search terms
Divorced search results from “navigational categories”

14. Can we use synonyms?
14

15. 15
Synonyms
hot dog
red hot
sausage
hamburger?
buns? ketchup?

16. 16

17. 17
Word Embeddings

18. 18
Word Embeddings

19. 19

20. 20

21. 21
“BLT” => “cluster_1”
“Sandwich” => “cluster_1”
“Burger” => “cluster_1”
“Pizza” => “cluster_0”
“Calzone” => “cluster_0”
“Spaghetti” => “cluster_0”
ES Synonyms

22. 22
Word Embeddings
Input: internal product descriptions and metadata
Algorithm: Word2Vec
Cluster words using K-means based on their vectors with three
levels of granularity
Upload as Synonyms file to ES

23. 23
Word Embeddings
Cons:
Unsupervised learning produced some weird results
Doesn’t handle arbitrary input
Too much recall
Pros:
Easy to hack in using ES synonyms feature
Lots of online examples and tutorials

24. 24

25. 25
<<< um… WAT?

26. What if we could predict
what the user is thinking?
26

27. 27
Query Expansion

28. 28
Category Prediction
Utilizes AWS BlazingText on SageMaker
Supervised model using text documents with explicit
category labels
Makes up to three predictions that align with our
taxonomy hierarchy

29. 29
.Hotdog
Sandwiches
Italian food
Noodles

30. 30

31. 31

32. 32

33. 33

34. 34

35. 35

36. Okay, now what?
36

37. 37
Continued Work
More ML predictions
Retailers likely to carry product
Associated brands
Known Entity Detection
Personalization based on user behavior
Machine Comprehension for question searches

38. 38
Take Aways
Start Tracking ASAP
Understand how your users expect search to work
Every application has different search needs
Keep it simple; Measure every change

39. 39
Any Questions?

40. 40
Ibotta is Hiring =)
https://www.ibotta.com/careers