Population Based Augmentation (PBA) is a new technique for efficiently learning data augmentation policies that is 1000x faster than previous methods like AutoAugment. PBA uses an evolutionary algorithm and random search to discover adaptive augmentation schedules. It learns a CIFAR-10 policy in just 5 GPU hours, compared to the 5,000 GPU hours required for AutoAugment. The learned PBA schedules apply stronger augmentations over the course of training.

1. K U L A
1000x Faster Data Augmentation
Population Bas ed Augmentation :
Effic ie n t L e a r n in g o f Au g me n ta tio n Po lic y Sc h e d u le s
b y Daniel Ho, Eric Liang, Richard Liaw Jun 7, 2019 (ICML)

2. K U L A
Effect of Population Based Augmentation applied to images, which differs at
different percentages into training.

3. K U L A
Data Augmentation
Most models only use basic data augmentation strategies.

4. K U L A
Augmentation with AutoAugment
Source : AutoAugment
Learns operations to apply with certain probability and magnitude.

5. K U L A
What’s the catch?
AutoAugment is too computationally expensive to learn.
PBA algorithm uses 1000x less compute.
Source : Population Based Training

6. K U L A
Population Based Augmentation
PBA learns CIFAR augmentation policy in 5 Titan XP GPU hours.
AutoAugment learns in 5,000 GPU hours.
CIFAR-10
Source : Population Based Training

7. K U L A
Population Based Augmentation
PBA learns CIFAR augmentation policy in 5 Titan XP GPU hours.
AutoAugment learns in 5,000 GPU hours.
Table 2. *CIFAR-100 models are trained with the policies learned on CIFAR-10 data.
Source : Population Based Training

8. K U L A
How is the augmentation schedule
learned?
Hyperparameter search using a mix of evolutionary algorithms and random search
to discover adaptive augmentation policy schedules quickly.
Source : Population Based Training

9. K U L A
Policy Search Space
Algorithm 1 : The PBA augmentation policy template, the parameters of which are optimized by PBT. The parameter vector is a vector of
(op, prob, mag) tuples. There are two instances of each op in the vector, and this parameter cannot be changed. PBT learns a schedule
for the prob and mag parameters during the course of training a population of child models.
Source : Population Based Training

10. K U L A
Exploration function used in PBA Implementation
Algorithm 2 : The PBA explore function. Probability parameters have possible values from 0% to 100% in increments of 10%, and
magnitude parameters have values from 0 to 9 inclusive.
Source : Population Based Training

11. K U L A
Learned Augmentation Policy Schedules
Effect of Population Based Augmentation applied to images showing stronger augmentations
as training progresses.
Source : Population Based Training

12. K U L A
Thank you!
Population Based Augmentation
Code : https://github.com/arcelien/pba