Automatic Image Cropping - A journey from a Master Thesis to Production
•
1 like•176 views
The slides from my talk at ML Conference 2019 Berlin
Recommended
More Related Content
Similar to Automatic Image Cropping - A journey from a Master Thesis to Production
Similar to Automatic Image Cropping - A journey from a Master Thesis to Production (20)
More from Alexey Grigorev
More from Alexey Grigorev (20)
Recently uploaded
Recently uploaded (20)
Automatic Image Cropping - A journey from a Master Thesis to Production
- 1. A journey from a Master Thesis to Production Alexey Grigorev 11/12/2019 Automatic Image Cropping
- 2. ● Aishwarya Manjunath Shetty (Naspers, TU/e) ○ The master thesis student who did all the work ● Selçuk Sandıkcı (Naspers) ● Dmitri Jarnikov (Naspers, TU/e) ● Sandor Akszenovics (OLX Group) ● Carmine Paolino (OLX Group) Credit
- 3. ● Software Engineer in the past ● Masters in BI ● Doing ML for 6 years, now Data Scientist at OLX Group About me
- 4. Plan ● Cropping in online classifieds ● Saliency detection ● Neural networks for saliency detection ● Experiments ● Deployment
- 10. Hypothesis ● Properly cropped image ⇒ ● Better visibility in home feed and search results ⇒ ● Better overall listing quality ⇒ ● Better overall listing performance Properly cropped image ⇒ Good listing performance
- 11. Plan ● Cropping in online classifieds ● Saliency detection ● Neural networks for saliency detection ● Experiments ● Deployment
- 13. Saliency Detection 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.8 1 0.8 0 0 0 0.8 0.9 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 Pixels P(pixel is salient)
- 14. Image cropping
- 15. Background removal (let’s pretend it’s good)
- 16. Plan ● Cropping in online classifieds ● Saliency detection ● Neural networks for saliency detection ● Experiments ● Deployment
- 19. CNN → FC Replace with Conv Layer
- 20. Fully Convolutional Networks for Semantic Segmentation https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf FC network
- 21. FC network Fully Convolutional Networks for Semantic Segmentation https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf
- 22. There’s some spatial info as well
- 23. Deeply supervised salient object detection with short connections https://arxiv.org/abs/1611.04849 DSS network
- 24. Deeply supervised salient object detection with short connections https://arxiv.org/abs/1611.04849 DSS network CNN (e.g. VGG)
- 25. Deeply supervised salient object detection with short connections https://arxiv.org/abs/1611.04849 DSS network
- 26. Deeply supervised salient object detection with short connections https://arxiv.org/abs/1611.04849 DSS network Deep supervision
- 27. Deeply supervised salient object detection with short connections https://arxiv.org/abs/1611.04849 DSS network Short connections
- 28. Deeply supervised salient object detection with short connections https://arxiv.org/abs/1611.04849 DSS network Short connections
- 30. DSS Examples
- 31. DSS Examples
- 32. DSS Examples
- 33. DSS Examples
- 34. Plan ● Cropping in online classifieds ● Saliency detection ● Neural networks for saliency detection ● Experiments ● Deployment
- 35. Data driven! Plan: ● Use OLX.PT for the experiment ● Pick 200 images from each category ● Manually evaluate them ● Select the best one ● Go live there!
- 36. Experiments
- 37. Experiments
- 39. Evaluation results ● Good: 90% and above ○ Animals (dogs, horses, cats); cars, motorcycles; shoes ● 80%-90% ○ Electronics, cellphones, clothing ● Bad: 50% or worse ○ Real estate (offices, shops, garages, apartments)
- 40. Selecting the category ● Cars: too important ● Animals: unclear business opportunity ● Fashion: low liquidity, good cropping quality ⇒ Fashion
- 41. Plan ● Cropping in online classifieds ● Saliency detection ● Neural networks for saliency detection ● Experiments ● Deployment
- 43. Cropping service Cropper 204 No Content
- 44. Engagement rings
- 45. Engagement rings
- 71. Summary ● Proper cropping ⇒ Good performance
- 72. Summary ● Proper cropping ⇒ Good performance ● Saliency detection can be used for cropping
- 73. Summary ● Proper cropping ⇒ Good performance ● Saliency detection can be used for cropping ● DSS improve FC networks with: ○ Deep supervision: refining the location info at every granularity ○ Short connections: helping earlier levels with info from later
- 74. Summary ● Proper cropping ⇒ Good performance ● Saliency detection can be used for cropping ● DSS improve FC networks with: ○ Deep supervision: refining the location info at every granularity ○ Short connections: helping earlier levels with info from later ● Be data driven: select the best performing and most impactful category
- 75. Summary ● Proper cropping ⇒ Good performance ● Saliency detection can be used for cropping ● DSS improve FC networks with: ○ Deep supervision: refining the location info at every granularity ○ Short connections: helping earlier levels with info from later ● Be data driven: select the best performing and most impactful category ● Slack is a great way to sneak peak and validate the performance
- 76. Summary ● Proper cropping ⇒ Good performance ● Saliency detection can be used for cropping ● DSS improve FC networks with: ○ Deep supervision: refining the location info at every granularity ○ Short connections: helping earlier levels with info from later ● Be data driven: select the best performing and most impactful category ● Slack is a great way to sneak peak and validate the performance ● Kubernetes takes care of scaling
- 77. Summary ● Proper cropping ⇒ Good performance ● Saliency detection can be used for cropping ● DSS improve FC networks with: ○ Deep supervision: refining the location info at every granularity ○ Short connections: helping earlier levels with info from later ● Be data driven: select the best performing and most impactful category ● Slack is a great way to sneak peak and validate the performance ● Kubernetes takes care of scaling ● Let it crash - and let the infra restart the service
- 78. * * work in progress
- 79. Contact information ● http://alexeygrigorev.com & contact@alexeygrigorev.com ● https://github.com/alexeygrigorev ● https://www.linkedin.com/in/agrigorev ● https://www.slideshare.net/AlexeyGrigorev
- 80. Feedback (and link to the slides!) https://forms.gle/QUU46Pf48iStVhWL8
- 81. Questions?