Codementor - Data Science at OLX

1. Data Science at OLX
Alexey Grigorev
21 April 2022
Developer Growth Summit 2022

2. Plan
● What is OLX
● Data Science at OLX
● Moderation system
● Recommender system
● Way of working
● Expectations from data scientists

3. Source: https://www.dvnovosti.ru/eao/2019/08/24/103683/

4. Seller

5. Seller
Icons: Bananas, Apple

6. Buyer Seller
Icons: Bananas, Apple

7. Buyer Seller
Icons: Bananas, Apple
💶
💶
💶

8. Buyer Seller
Icons: Bananas, Apple
💶
💶 💶

9. Buyer Seller
Icons: Bananas, Apple
Marketplace

10. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

12. olxgroup.com

14. Plan
● What is OLX
● Data Science at OLX
● Moderation system
● Recommender system
● Way of working
● Expectations from data scientists

15. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

16. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

17. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

18. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

19. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

20. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

21. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳
● Smart ranking
● Reducing null-searches
● Query categorization
● Spell checking

22. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳
● Smart ranking
● Reducing null-searches
● Query categorization
● Spell checking
● Collaborative filtering
● Item2Vec

23. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

24. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳
● Image quality
● Listing quality
● Deal prediction
● Price prediction

25. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

26. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳
● NSFW detection
● Forbidden items
● Fraud detection
● Duplicate detection
● Chat moderation

27. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳

28. Buyer Seller
Online
Marketplace
Icons: Bananas, Apple
Website
💳
● User segmentation
● Bid optimization

29. Plan
● What is OLX
● Data Science at OLX
● Moderation system
● Recommender system
● Way of working
● Expectations from data scientists

30. Such description. So much text
https://olx.com
OLX

31. Such description. So
much text
Such description. So
much text
Such description. So
much text
https://olx.com
OLX

32. https://olx.com
OLX

33. https://olx.com
OLX

34. Problems:
● Illegal items
● NSFW content
● Duplicates
● Spam
● Fraud

35. Content moderation
Such description
So much text

36. Content moderation
ML
Such description
So much text
Automatic
moderation system

37. Content moderation
ML
Such description
So much text
Accept
Reject
Automatic
moderation system

38. Content moderation
ML
Such description
So much text
Accept
Reject
Moderation queue
MP
Automatic
moderation system
Moderation panel
Moderators

39. Content moderation
ML
Such description
So much text
Accept
Reject
Moderation queue
MP
Automatic
moderation system
Moderation panel
Accept
Reject
Moderators

40. ML
Such description
So much text
Accept
Reject
Moderation queue
Automatic
moderation system
Duplicate
detection
Forbidden
items
Other ML
models

41. ML
Such description
So much text
MP
Automatic
moderation system
Moderation panel
Accept
Reject
Moderators
s3
ES
Duplicate
detection
system
Hashes
Accept
Reject
Moderation queue

42. ML
Such description
So much text
MP
Automatic
moderation system
Moderation panel
Accept
Reject
Moderators
s3
ES
Duplicate
detection
system
Hashes
Accept
Reject
Moderation queue
Index listings & images

43. ML
Such description
So much text
MP
Automatic
moderation system
Moderation panel
Accept
Reject
Moderators
s3
ES
Duplicate
detection
system
Hashes
Accept
Reject
Moderation queue
Detect duplicates

44. ML
Such description
So much text
MP
Automatic
moderation system
Moderation panel
Accept
Reject
Moderators
s3
ES
Duplicate
detection
system
Hashes
Accept
Reject
Moderation queue
Moderate duplicates

45. ML
Such description
So much text
MP
Automatic
moderation system
Moderation panel
Accept
Reject
Moderators
s3
ES
Duplicate
detection
system
Hashes
Accept
Reject
Moderation queue
Collect feedback

46. https://tech.olx.com/a-two-step-framework-for-duplicate-detection-fbbe4c905480
https://tech.olx.com/detecting-image-duplicates-at-olx-scale-7f59e4b6aef4

47. Plan
● What is OLX
● Data Science at OLX
● Moderation system
● Recommender system
● Way of working
● Expectations from data scientists

49. Collaborative filtering
A
Users
Items
≈ U V
x

50. Collaborative filtering
Users
Items
≈ x
Item
vector
User
vector

51. d2
d1
󰬋
🧦
🧸

52. d2
d1
󰬋
🧦
🧸

54. Collaborative filtering
Icons: srip
similarity

55. Neural networks (“item2vec”)
Icons: srip
Neural network
Neural network
similarity

56. https://habr.com/ru/company/avito/blog/491942/
Deep Learning Recommender Systems - Cristian Martinez, Ilia Ivanov (video)

57. Plan
● What is OLX
● Data Science at OLX
● Moderation system
● Recommender system
● Way of working
● Expectations from data scientists

58. A project like this is very complex
We need a team (or multiple teams) to make it work: it’s a joined effort of many
people working together

59. Roles in teams
● Product Manager (PM)
● Engineering Manager (EM)
● Software Engineers
○ Backend Engineers (BE)
○ Data Engineers (DE)
○ ML Engineer (MLE)
○ Site Reliability Engineers (SRE)
○ Frontend Engineers (FE)
○ Mobile Engineers
● Product Analysts (PA)
● Data Scientists (DS)

60. Team A
Team B
Team C
Product
PM
PM
PM
Head of
Product
PA
PA
Head of
Analytics
DS
DS
DS
Manager
Data Tech
EM
EM
EM
Head of
Engineering
BE
DE
BE
FE
BE SRE
FE SRE
FE
Matrix structure

61. Feature teams
● A cross-functional team with experts in different areas
● All work together on one feature/product
● All have the same goal!
● Anyone can work on anything, as long as it helps achieve the goal
PA DS DE BE SRE
EM
PM

62. Goal setting
● OKRs, set quarterly
● Great alignment tool: other teams know what you’re doing
● Whatever team is doing, should be in line with their OKRs
Example:
● O
○ Catch more fraudsters
● KRs
○ Precision of model A improves from 30% to 60% while staying at the same recall level
○ Model B is tested in 5 key markets

63. Plan
● What is OLX
● Data Science at OLX
● Moderation system
● Recommender system
● Way of working
● Expectations from data scientists

64. Product
Definition
Data
Processing
Modeling Evaluation Production Customers
Data
Collection
Stages

65. DATA SCIENTIST
DATA SCIENTIST
DATA SCIENTIST
Product
Definition
Data
Processing
Modeling Evaluation Production Customers
Data
Collection
Stages
Focus on modelling and evaluation, a bit
on production

66. DATA SCIENTIST
DATA SCIENTIST
DATA SCIENTIST
Product
Definition
Data
Processing
Modeling Evaluation Production Customers
Data
Collection
Stages

67. Interested in working with us?
● We have open positions

68. Links
OLX:
● OLX tech blog: tech.olx.com
● Jobs at OLX: joinolx.com
Me:
● alexeygrigorev.com
● DataTalks.Club
● LinkedIn: agrigorev
● Twitter: @Al_Grigor