The document discusses an automated background removal solution for e-commerce fashion images, highlighting the challenges of manual processing and the implementation of a deep learning pipeline. It outlines the dataset of 32,000 images, the modeling process using the U2-Net architecture, and impressive evaluation results with intersection over union (IoU) scores averaging over 99%. The presentation concludes with a focus on the effectiveness of the model and addresses production verification challenges.

1. Automated
background removal
IN SCOPE OF E-COMMERCE FASHION IMAGE
PROCESSING

2. Simona Stolnicu
Data Scientist @ Levi9
Romania
Olexander Miroshynchenko
Machine Learning Engineer
@ GlobalLogic
Speakers

3. Agenda
▪ Intro Wehkamp
▪ Problem definition
▪ Input data
▪ Modeling
▪ Results
▪ Examples
▪ Q&A

4. >400.000
different
products
>500.000
daily
visitors
640 million
sales
19/20
11 million
sent
packages
>950
colleagues
65%
of all customers
shop mobile
72%
of our customer
are female
Over 2.000 brands
C&A // Vingino // Hunkemöller // Mango // Tommy Hilfiger // Scotch & Soda // ONLY
HK Living // House Doctor // Woood // Bloomingville // Zuiver // whkmp’s own
Wehkamp facts
Focus on: fashion // living // beauty / baby & kids

5. - Thousands of product photos are processed in the local photo studio.
- Background of these images is extremely various
- Removing background by hand is time and human resources consuming
- The solution is an automated pipeline using deep learning.
Problem definition
Given: dataset images ~ 32.000 fashion images and masks
Goal: generate mask for image in order to delete a background
5

6. 6
Pipeline overview

7. 7
Dataset
- Input data type: image – mask
- 32 000 images clustered in 6 categories for a balanced
dataset (for 5 categories we used PCA + Kmeans)
- Total Training samples: 25 600 pair images
- Total Validation samples: 6 400 pair images

8. 8
Raw image and
image with applied
mask
Our goal

9. 9
● Analysis
● Cleaning
● Clustering on image product types (PCA + Kmeans)
● Long pants
● Shorts
● Short-sleeved tops/dresses
● Long-sleeved tops/dresses
● Beachwear/sportswear/accessories,
● White color products (done manually)
● Resizing images to 320x320
● Augmentation
● Original image
● Vertical flip
● Cropping
● Combination of images
Data processing

10. 10
The network architecture is inspired from the paper: U2-Net:
Going Deeper with Nested U-Structure for Salient Object Detection
(Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan,
Osmar R. Zaiane and Martin Jagersand University of Alberta, Canada).
We trained the network in environment using Horovod which
allowed to put in place a distributed training with Pytorch. During
the each epoch, distinct batches are trained on multiple workers and
the results are merged by averaging the parameters between
workers.
Modeling

11. 11
Characteristics:
- Parameters setting (Adam optimizer, dynamic learning
rate, batch_size=10*GPUs, epochs=30)
- Batch learning technique
- Time: 9h using 20 workers with Horovod on small
instance cluster type
- Evaluation methods : Intersection over Union (IoU)
metric
- TensorBoard metrics: losses, learning rate, IoU
metric/epoch for validation dataset
Modeling

12. 12
Evaluation metric: Intersection over Union (IoU)
Truth label Predicted label
Modeling

13. 13
Truth label Predicted label
Modeling
Good
Poor Excellent
IoU: 0.4036 IoU: 0.7330 IoU: 0.9857
IoU =
Area of Intersection
Area of Union

14. 14
Truth label Predicted label
= 0.9899
Modeling
IoU =
Area of Intersection
Area of Union

15. 15
Average validation performance: 99.435%
● Over 99.32% of images with 95% performance
score
● Over 98.72% of images with 97% performance
score
● Over 93.14% of images with 99% performance
Image Cluster IoU score (%)
Long pants 99.428
Shorts 99.617
Short-sleeve tops,
dresses
99.502
Long-sleeve tops,
dresses
99.588
Beachwear,
sportswear,
nightwear,
accessories
98.815
White color clothes 98.868
Results

16. 16
Real image Hand made mask ML mask
Results
99.297%
Score

17. 17
Real image Hand made mask ML mask
Results
99.508%
Score

18. 18
Real image Hand made mask ML mask
Results
99.417%
Score

19. 19
Real image Hand made mask ML mask
Results
88.189%
Score

20. 20
Real image Hand made mask ML mask
Results
99.695%
Score

21. 21
Real image Hand made mask ML mask
Results
97.875%
Score

22. 22
Real image Hand made mask ML mask
Results
99.828%
Score

23. 23
Real image Hand made mask ML mask
Results
81.311%
Score

24. 24
Real image Hand made mask ML mask
Results
98.104%
Score

25. 25
Real image Hand made mask ML mask
Results
99.742%
Score

26. 26
Real image Hand made mask ML mask
Results
99.590%
Score

27. 27
Real image Hand made mask ML mask
Results
77.020%
Score

28. 28
● Dataset cleaning and clustering
● Finding a suitable architecture for the network (previous trials with MaskRCNN (93.7%) and BASNet (94.4%)
proposals).
● Horovod parallelization of GPUs when computing additional metrics for evaluating epoch results on
validation data
● Finding appropriate augmentation to solve various cases where the model had weaknesses
● Prediction mask when the background has the same shade as the product
● Mask verification on production
Challenges

29. THANKS FOR YOUR ATTENTION!

30. Q&A section

31. Simona Stolnicu
Data Scientist @ Levi9
Romania
Olexander Miroshynchenko
Machine Learning Engineer
@ GlobalLogic
Speakers