MLOps at OLX
โข
0 likesโข147 views
This document discusses MLOps at OLX, including: - The main areas of data science work at OLX like search, recommendations, fraud detection, and content moderation. - How OLX uses teams structured by both feature areas and roles to collaborate on projects. - A maturity model for MLOps with levels from no MLOps to fully automated processes. - How OLX has improved from siloed work to cross-functional teams and adding more automation to model creation, release, and application integration over time.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MLOps at OLX
Similar to MLOps at OLX (20)
More from Alexey Grigorev
More from Alexey Grigorev (20)
Recently uploaded
Recently uploaded (20)
MLOps at OLX
- 1. MLOps at OLX Alexey Grigorev Principal Data Scientist at OLX Group Founder at DataTalks.Club
- 2. MLOps at OLX Alexey Grigorev Principal Data Scientist at OLX Group Founder at DataTalks.Club Productionizing ML at OLX
- 5. Plan โ Data Science at OLX โ Way of Working โ MLOps maturity model โ Improvisation
- 6. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization
- 7. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization โ Smart ranking โ Reducing null-searches โ Query categorization โ Spell checking
- 8. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization โ Collaborative filtering โ Item2Vec
- 9. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization โ NSFW detection โ Forbidden items โ Fraud detection โ Duplicate detection โ Chat moderation
- 10. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization โ Image quality โ Listing quality โ Deal prediction
- 11. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization โ User segmentation โ Bid optimization
- 13. Data Science at OLX Main areas: โ Search โ Recommendations โ Trust & Safety โ Seller Experience โ Monetization
- 14. Such description. So much text https://olx.com OLX
- 15. Such description. So much text Such description. So much text Such description. So much text https://olx.com OLX
- 19. Problems: โ Illegal items โ NSFW content โ Duplicates โ Spam โ Fraud
- 20. Content moderation ML Such description So much text Accept Reject Moderation queue MP Automatic moderation system Moderation panel Accept Reject Moderators
- 21. ML Such description So much text Accept Reject Moderation queue Automatic moderation system Duplicate detection Forbidden items Other ML models
- 22. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue
- 23. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Index listings & images
- 24. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Detect duplicates
- 25. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Moderate duplicates
- 26. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Collect feedback
- 29. Plan โ Data Science at OLX โ Way of Working โ MLOps maturity model โ Improvisation
- 30. A project like this is very complex We need a team (or multiple teams) to make it work: itโs a joined effort of many people working together
- 31. Roles in teams โ Product Manager (PM) โ Engineering Manager (EM) โ Software Engineers โ Backend Engineers (BE) โ Data Engineers (DE) โ ML Engineer (MLE) โ Site Reliability Engineers (SRE) โ Frontend Engineers (FE) โ Mobile Engineers โ Product Analysts (PA) โ Data Scientists (DS)
- 32. Team A Team B Team C Product PM PM PM Head of Product PA PA Head of Analytics DS DS DS Manager Data Tech EM EM EM Head of Engineering BE DE BE FE BE SRE FE SRE FE Matrix structure
- 33. Feature teams โ A cross-functional team with experts in different areas โ All work together on one feature/product โ All have the same goal! โ Anyone can work on anything, as long as it helps achieve the goal PA DS DE BE SRE EM PM
- 34. Goal setting โ OKRs, set quarterly โ Great alignment tool: other teams know what youโre doing โ Whatever team is doing, should be in line with their OKRs Example: โ O โ Catch more fraudsters โ KRs โ Precision of model A improves from 30% to 60% while staying at the same recall level โ Model B is tested in 5 key markets
- 35. Plan โ Data Science at OLX โ Way of Working โ MLOps maturity model โ Improvisation
- 36. MLOps Maturity Levels โ Level 0: No MLOps โ Level 1: DevOps but no MLOps โ Level 2: Automated training โ Level 3: Automated model deployment โ Level 4: Full MLOps automation https://docs.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model
- 37. People โ Siloed lone data scientists โ Siloed teams โ Cross-functional teams
- 38. Model creation โ Training โ Laptop โ AWS Batch โ AWS Sagemaker โ Experiment tracking โ Central MLFlow server โ Version controlling โ Code โ always โ Data โ rarely โ Models โ rarely
- 39. Model release โ Manual release โ for PoCs and less mature teams โ Automatic release via CI/CD (gitlab) โ for the rest โ Metric-based automated retraining/release โ rarely โ No handover to SWE โ DS/team own the full cycle
- 40. Application integration โ Unit and integration tests โ always โ Rely on software engineers to integrate to OLX backend โ A/B tests โ often
- 41. Plan โ Data Science at OLX โ Way of Working โ MLOps maturity model โ Improvisation
- 42. Iโm happy to talk more about โ Processes โ Our data platform โ Experimentation โ Model deployment โ And other things!