Presentation for Deep Learning Tech Talks/Idea Pitching for Lets Stop Wildfires Hackathon (https://www.meetup.com/AI-for-Mankind/events/265983290/)

1. lbumentations: fast and lbumentations: fast and
flexible image augmentationsflexible image augmentations
Sr. Computer Vision Engineer at Lyft
Ph.D. In Physics
Vladimir Iglovikov
Kaggle GrandMaster

2. Computer Vision TasksComputer Vision Tasks
Input Output

3. Need for the dataNeed for the data

4. Problems with the labeled dataProblems with the labeled data
Expensive to collect
Hard to label
Legal or privacy problems

5. AugmentationsAugmentations

6. AugmentationsAugmentations
1. Synthetically increase dataset.

7. AugmentationsAugmentations
1. Synthetically increase dataset.
2. Make network invariant to transforms (rotations,
brightness, flips, etc).

8. AugmentationsAugmentations
1. Synthetically increase dataset.
2. Make network invariant to transforms (rotations,
brightness, flips, etc).
3. Work as a regularizer.

9. AugmentationsAugmentations

10. Augmentations vs HeavierAugmentations vs Heavier
architecturearchitecture

11. lbumentationslbumentations
Used in almost all winning
solutions to CV problems at Kaggle.
Used in academia: 32+ citations
Used in industry

12. Who works on Albumentations?Who works on Albumentations?
Alexander Buslaev
Mapbox, Belarus
Kaggle Master
Vladimir Iglovikov
Lyft, USA
Kaggle Grandmaster
Alex Parinov
X5, Russia
Kaggle Master
Eugene Khvedchenia
Jumio, Ukraine
Kaggle Grandmaster
Mikhail Druzhinin
Simicon, Russia
and many contributors at Github

13. CPU becomes bottleneck => we want fast transforms

14. We want different targets to beWe want different targets to be
transformed in synctransformed in sync
Original Transformed
import albumentation as A
strong = albu.Compose([
A.HorizontalFlip(p=0.5),
A.RandomSizedCrop((700, 900), 600, 600),
A.RGBShift(p=0.35),
A.Blur(blur_limit=11, p=0.31415),
A.RandomBrightness(p=0.2019),
A.CLAHE(p=0.666),
], p=0.1984)
1
2
3
4
5
6
7
8
9
10
1. image
2. bounding boxes
3. mask
4. keypoints

15. Pixel-level transformsPixel-level transforms
38+ transforms
Original RandomRain
RandomSnow RandomFog

16. Spatial TranformsSpatial Tranforms
Support masks, bounding boxes, keypoints
33+ transforms
,
,
,
,
,
,
RandomCrop
RandomResizedCrop
HorizontalFlip
ElasticTransform
ShiftScaleRotate
CropNonEmptyMaskIfExists
Lambda

17. Key pointsKey points
,
,
,
,
,
RandomCrop
RandomResizedCrop
HorizontalFlip
RandomScale
ShiftScaleRotate
CropNonEmptyMaskIfExists
Lambda

18. Multiple targetsMultiple targets
We want to apply the
same transform to:
N images
M masks
K bounding boxes
L keypoints
simultaneously.
import albumentations as A
aug = A.Compose(transformations,
p=0.42,
additional_targets={
'image1': 'image',
...
'imageN': 'image',
'bboxes1': 'bboxes',
...
'bboxesM': 'bboxes',
'keypoints1': 'keypoints',
...
'keypointsK': 'keypoints',
'mask1': 'mask',
...
'maskL': 'mask'
})
transformed = aug(image=img1, image2=img1, ....)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

19. transform = A.Compose([
A.RandomCrop(768, 768),
A.OneOf([
A.RGBShift(),
A.HueSaturationValue()
]),
])
1
2
3
4
5
6
7
# to yaml
A.save(transform, '/tmp/transform.yml', data_format='yaml')
# to json
A.save(transform, '/tmp/transform.json')
# to python dictionary
transform_dict = A.to_dict(transform)
1
2
3
4
5
6
7
8
Serialization / DeserializationSerialization / Deserialization

20. Serialization / DeserializationSerialization / Deserialization
transform = A.Compose([
A.RandomCrop(768, 768),
A.OneOf([
A.RGBShift(),
A.HueSaturationValue()
]),
])
1
2
3
4
5
6
7
# to yaml
A.save(transform, 'transform.yml', data_format='yaml')
# to json
A.save(transform, 'transform.json')
# to python dictionary
transform_dict = A.to_dict(transform)
1
2
3
4
5
6
7
8
# Load from yaml
loaded_transform = A.load('transform.yml', data_format='yaml')
# Load from json
loaded_transform = A.load('transform.json')
# Load from dictionary
loaded_transform = A.from_dict(transform_dict)
1
2
3
4
5
6
7
8

21. DocumentationDocumentation
https://github.com/albu/albumentations/tree/master/notebooks https://albumentations.readthedocs.io

22. Deep Learning CompetitionsDeep Learning Competitions
https://github.com/albu/albumentations/blob/master/docs/hall_of_fame.rst

23. Topcoder Urban3D ChallengeTopcoder Urban3D Challenge
Image Mask
Instance Segmentation
1st place
Alexander Buslaev
Augmentations:
OpticalDistortion, Grid, Flip, ShiftScaleRotate, Transpose,
HueSaturationValue

24. Topcoder: Spacenet 3 challengeTopcoder: Spacenet 3 challenge
Semantic Segmentation
1st place
Alexander Buslaev
Augmentations:
Flip, ShiftScaleRotate, Transpose, HueSaturationValue

25. KKaggle: Carvana Image Masking Challengeaggle: Carvana Image Masking Challenge
Semantic Segmentation
1st place
Vladimir Iglovikov,
Alexander Buslaev,
Artem Sanakoyeu
Augmentations:
HorizontalFlip, ShiftScaleRotate, RandomBrightness,
HueSaturationValue
Input Output

26. KKaggle: Data Science Bowl 2018aggle: Data Science Bowl 2018
Instance Segmentation
1st place
Alexander Buslaev,
Selim Seferbekov
Viktor Durnov
Augmentations:
CLAHE, IAASharpen, IAAEmboss, IAAAdditiveGaussianNoise, ToGray, InvertImg,
RandomRotate90, Flip, Transpose, MotionBlur, MedianBlur, Blur, RandomContrast,
RandomBrightness, ShiftScaleRotate, OpticalDistortion, GridDistortion, ElasticTransform,
IAAPerspective, IAAPiecewiseAffine, HueSaturationValue, ChannelShuffle
Input Output

27. KKaggle: Inclusive Imagesaggle: Inclusive Images
Multilabel classification
1st place
Pavel Ostyakov
Augmentations:
RandomRotate90, Flip, Transpose, GaussNoise, MedianBlur,
ShiftScaleRotate, RandomBrightness, HueSaturationValue.
Adapting Convolutional Neural Networks for Geographical Domain Shift, https://arxiv.org/abs/1901.06345

28. KKaggle: TGS Salt Identification Challengeaggle: TGS Salt Identification Challenge
Multilabel classification
1st place
Eugene Babakhin,
phalanx
Augmentations:
HorizontalFlip, RandomBrightness, RandomContrast,
ShiftScaleRotate.

29. KKaggle: SIIM-ACR Pneumothoraxaggle: SIIM-ACR Pneumothorax
SegmentationSegmentation
Semantic Segmentation
1st place
Aimoldin Anuar
Augmentations:
HorizontalFlip, RandomContrast, RandomGamma,
RandomBrightness, ElasticTransform, GridDistortion,
OpticalDistortion, ShiftScaleRotate

30. KKaggle: iMaterialist (Fashion) CVPR 2019: FGVC6aggle: iMaterialist (Fashion) CVPR 2019: FGVC6
Instance Segmentation
1st place
Miras Amir
Augmentations:
HorizontalFlip, CutOut, RandomBrightnessContrast,
JpegCompression
Input Output

31. Future plansFuture plans
AutoAugment
Transforms on GPU
3D images and LIdar point clouds
Extra tools for tuning augmentations

32. ContributorsContributors

33. Thank you!Thank you!
LinkedIn:
Twitter:
https://www.linkedin.com/in/iglovikov/
https://twitter.com/viglovikov