1. CutMix is a simple data augmentation technique that improves image classification performance by mixing patches of images and their labels during training. 2. It works by replacing image patches from training examples with patches from other random images, and mixing the ground truth labels proportionally. 3. Experiments show that CutMix helps models focus on less discriminative features, improves classification accuracy, enhances object localization ability, and increases robustness to adversarial examples and out-of-distribution inputs.

1. CutMix
Regularization Strategy to Train Strong Classifiers with Localizable Features

2. Summary
1. CutMix is extremely simple, requiring
only 20 lines of code.
2. The image on the right shows what
CutMix is and how effective it is.
3. CutMix was performed in conjunction
with other standard data augmentations.

3. The Code

4. Introduction
• Data Augmentation is necessary to prevent the Network from focusing on
only a few features and overfitting to them.
• Regional dropout strategies have proven effective on improving
classification and localization tasks by focusing the model on less
discriminative features of images.
• However, they are inefficient as they zero-out or fill regions with random
noise, a serious problem for data-hungry CNNs.

5. CutMix: Method
• Instead of simply removing pixels replace
the removed regions with a patch from
another image.
• The ground truth labels are also mixed
proportionally to the number of pixels of
combined images.

6. Advantages
• There is no uninformative pixel during training, making training efficient.
• Attends to non-discriminative parts of objects.
• Enhance localization ability by identifying the object from a partial view.
• Robust to adversarial attacks and Out of Distribution (OOD) examples.
• The training and budgets remains the same. (No extra computation!)

7. Algorithm
• Let 𝑥 denote a training image and 𝑦 denote its label.
• Generate 𝑥, 𝑦 by combining 𝑥 𝐴, 𝑦 𝐴 and 𝑥 𝐵, 𝑦 𝐵 .
• 𝑥 = 𝑀 ⊙ 𝑥 𝐴 + 1 − 𝑀 ⊙ 𝑥 𝐵
• 𝑦 = 𝜆𝑦 𝐴 + 1 − 𝜆 𝑦 𝐵
• 𝜆 ∼ 𝐵𝑒𝑡𝑎 𝛼, 𝛼 , 𝛼 = 1 for all experiments. 𝜆 is the mixing ratio.
• 𝑀 ∈ {0,1} is a binary mask whose area is determined by 𝜆.
• ⊙ indicates element-wise multiplication.

8. Mask Generation
• Generate a bounding box 𝐵 = (𝑟𝑥, 𝑟𝑦, 𝑟𝑤, 𝑟ℎ) for the mask.
• 𝑟𝑥~𝑈𝑛𝑖𝑓 0, 𝑊 , 𝑟𝑤 = 𝑊 1 − 𝜆, 𝑊 = 𝑖𝑚𝑎𝑔𝑒 𝑤𝑖𝑑𝑡ℎ
• 𝑟𝑦~𝑈𝑛𝑖𝑓 0, 𝐻 , 𝑟ℎ = 𝐻 1 − 𝜆, 𝐻 = 𝑖𝑚𝑎𝑔𝑒 ℎ𝑒𝑖𝑔ℎ𝑡
• Boundary box is unaware of image size and may not be entirely on the image.
• Fill the mask 𝑀 with 0’s inside 𝐵 and with 1’s outside.
• The cropped area ratio should be 1 − 𝜆.
• Rounding error and boundary box overflow may cause the true 𝜆 to be different.

9. Results

10. Comparison with Other Methods
One can see that CutMix learns to identify separate parts of
the image by analyzing the CAM visualizations.
Other methods either cannot do this (Mixup) or are
inefficient (Cutout).

11. Performance on Classification

12. Resistance to Overfitting

13. Weakly Supervised Object Localization
• Weakly Supervised Object
Localization (WSOL) is a task
where the aim is to localize the
object when only the class label
is available.

14. Transfer Learning of CutMix Models

15. Ablation Studies
• Ablation study using different values of
𝛼 on the beta distribution shows that 𝛼 =
1 has the best results.
• Ablation study with using CutMix at
different layers shows that using CutMix
on the input layer produces the best
results.
• Different implementations of CutMix are
all inferior to the proposed
implementation.

16. Robustness
• Deep Learning models are prone to overconfident predictions.
• This leads them to be vulnerable to adversarial attacks.
• They are also prone to giving incorrect answers when the test data is
from a different distribution from the training data.

17. Robustness to Image Manipulation

18. Robustness to Different Images

19. Conclusion
• CutMix is a simple yet effective regularization technique that
improves performance on a wide array of tasks.
• By forcing the model to focus on non-discriminative parts of an image
efficiently, it both improves performance and reduces overconfident
predictions.

20. Final Thoughts
• Perhaps using CutMix in GANs will reduce mode collapse and
improve performance.
• Image Segmentation may also benefit from CutMix.