OLX is an online classifieds company with over 200 million monthly active users across 30+ countries. It receives around 10 million new image uploads per day across its various marketplaces. The speaker works on Apollo, OLX's machine learning system for analyzing images. Apollo uses computer vision models to extract metadata like categories and image quality from uploaded photos. The models are trained on Amazon SageMaker but served using Kubernetes due to cost and operational advantages over SageMaker hosting. One challenge is detecting duplicate images, as 30% of images and 10% of ads are duplicates. Hashing algorithms are used to identify duplicates while being resilient to minor image changes.

1. Infrastructure for Image Models
at OLX
Alexey Grigorev
21.05.2019

2. About me
● Originally from Russia, live in Berlin
● Background: Databases
● Started as Software Engineer
● Masters in TU Berlin
● Data Science since 2013
● Book: Mastering Java for Data Science
● Kaggler in the past
● Now: Data Scientist at OLX
“ML Engineer”

3. OLX Group

4. OLX
● Tech hubs:
○ Poznan, Lisbon, Berlin
○ Buenos Aires, Delhi
● 30+ countries
● 200M MAU
● 10M sellers & 30M buyers
● 30M new listings per month
● Live listings:
○ OLX.PL (18M listings)
○ OLX.UA (12.5M listings)
○ OLX.IN (3M listings)

5. Apollo at OLX
Image uploads per day
● eu-west-1: up to 6.5 m
○ PL: 3 mln
○ UA: 1 mln
● ap-southeast-1: 2.2 m
○ IN: 0.7 mln
● eu-central-1: 1.6 m
● us-west-1: 55k
● total: ~10 m per day

6. Apollo X - the metadata service
● Data about images:
○ Category: what is on the image
○ Image quality: is the image good
● How to extract metadata?
● With machine learning!

7. Training
s3
SageMaker
we
Model

8. Serving
s3
event
listener
Apollo X
Category
Quality
tf-serving
mms
user
redis mysql
ObjectCreated:Put
Apollo bucket
Batch
Batch
AWS + Kubernetes

9. Sagemaker vs Self-hosted
Sagemaker
● Sagemaker is great for model training and testing in prod
● But it’s expensive for serving millions of requests
● Hard to monitor
● Hard to make Ops happy
Self-hosted (Kubernetes)
● Cheaper
● Fits existing infra
● Metrics, Logs, Alerts
● SREs know how to deal with issues

10. Image quality

11. Cover image

12. Forbidden items

13. Duplicates
● Duplicates is a problem
● 30% of images are duplicates
● 10% of ads are duplicates

14. Image hashes
● MD5 - cryptographic hash
● Perceptive hashes:
○ DHash, PHash, WHash (imagehash library)
● Take image
● Change pixel, resize, re-compress
● Different MD5, same phash

15. s3
ObjectCreate
ES
hashes
Image index
ObjectDelete
IngestorIngestor
Hash calculation
Image Index

16. Image Index
● Lambdas are super easy to scale
Images per hour
114 prs

17. Contact information
● http://alexeygrigorev.com & contact@alexeygrigorev.com
● https://github.com/alexeygrigorev
● https://www.linkedin.com/in/agrigorev

18. OLX is hiring
● https://www.olxgroup.com/search/engineering/all-locations/all-brands