Embeddings are a great way to represent information, but in production is still not a commodity. But you can still find ways to use them with lambda architecture.

1. Haystack / MICES / Berlin Buzzwords
June 10, 2020
WalkingAroundYourNearest
Neighbors
with lambda architecture
Elias Nema

2. Dense vectors (embeddings) are the new oil
But sometimes refinement costs are too high
Because dense vectors are dense
Storing them and searching through them might put an
excessive load on your search engine
Haiku
Walkingaroundyournearestneighbors

3. Content is generated by users
Millions of items are added and removed daily
As soon as item is posted we want to produce recommendations for it
As well as recommend this item to the other items from the catalog
Usecase:Recommendations@OLXGroup
Online Marketplace
25+ countries
350M+ MAU

4. Constraints
• We have embeddings for our items. But,
• It’s not feasible to store these item embeddings in our search engine (many reasons).
• Though we still would like to use them for recommendations.
So, which options do we have?

5. Embeddingspace
Item 1
Item 2
Item 3
Item 4

6. New item
Embeddingspace

7. Hey neighbor!
Embeddingspace

8. Embeddingspace
Closest
2nd
3rd
4th

9. Embeddingspace
Closest
2nd
3rd

10. Embeddingspace
Closest
2nd
3rd
2nd
3rd
4th

11. Flow
1st 2nd
1st: Neighbors
for the new item
2nd: Update
neighborhood
All items cache
New item has recommendations as
well as also being recommended to
the other items

12. Flow
Application to calculate
nearest neighbors
(faiss+sqlite)
Fast cache
Stream of
item updates
Daily batch job to
avoid drifting

13. Application to calculate
nearest neighbors
(faiss+sqlite)
Fast cache
Stream of
item updates
Daily batch job to
avoid drifting
Speed layer
Batch layer
Serving
layer
Lambdaarchitecture

14. Conclusion
•Embeddings are a great way to represent information, but in production are still not
a commodity.
•You can still find ways to use them in production e.g. with lambda architecture.
•Of course, this approach has limited reranking capabilities.
• … or you can use Sphinx (avito.ru does a brute force vector ranking for up to 5M
documents in <10ms - sorry, in Russian).
Elias Nema
eliasnema
eliasnema.com