Data Orchestration Summit www.alluxio.io/data-orchestration-summit-2019 November 7, 2019 tf.data: TensorFlow Input Pipeline Speakers: Jiri Simsa, Google

1. tf.data: TensorFlow Input Pipeline
https://www.tensorflow.org/guide/data
https://www.tensorflow.org/guide/data_performance
1
Jiri Simsa

2. 2
Why input pipeline API?

3. 3
Why input pipeline API?
- data might not fit into memory
- data might require (randomized) pre-processing
- efficiently utilize hardware
- decouple loading + pre-processing from distribution

4. tf.data: TensorFlow Input Pipeline
4
Extract:
- read data from memory / storage
- parse file format
Transform:
- text vectorization
- image transformations
- video temporal sampling
- shuffling, batching, …
Load:
- transfer data to the accelerator
time
flops
CPU
accelerators

5. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
model = ...
model.fit(dataset, epochs=10)
55

6. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
model = ...
model.fit(dataset, epochs=10)
66
reads data from storage

7. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
model = ...
model.fit(dataset, epochs=10)
77
applies user-defined preprocessing

8. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
model = ...
model.fit(dataset, epochs=10)
88
batches data for training efficiency

9. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
model = ...
model.fit(dataset, epochs=10)
99
training APIs natively support tf.data

10. 10
Input Pipeline Performance

11. Software Pipelining
11

12. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
model = ...
model.fit(dataset, epochs=10)
121212

13. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
dataset = dataset.prefetch(buffer_size=X)
model = ...
model.fit(dataset, epochs=10)
131313

14. Parallel Transformation
14

15. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess)
dataset = dataset.batch(batch_size=32)
dataset = dataset.prefetch(buffer_size=X)
model = ...
model.fit(dataset, epochs=10)
151515

16. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess, num_parallel_calls=Y)
dataset = dataset.batch(batch_size=32)
dataset = dataset.prefetch(buffer_size=X)
model = ...
model.fit(dataset, epochs=10)
161616

17. Parallel Extraction
17

18. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord")
dataset = dataset.map(preprocess, num_parallel_calls=Y)
dataset = dataset.batch(batch_size=32)
dataset = dataset.prefetch(buffer_size=X)
model = ...
model.fit(dataset, epochs=10)
181818

19. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord", num_parallel_readers=Z)
dataset = dataset.map(preprocess, num_parallel_calls=Y)
dataset = dataset.batch(batch_size=32)
dataset = dataset.prefetch(buffer_size=X)
model = ...
model.fit(dataset, epochs=10)
191919

20. import tensorflow as tf
def preprocess(record):
...
dataset = tf.data.TFRecordDataset(".../*.tfrecord", num_parallel_readers=Z)
dataset = dataset.map(preprocess, num_parallel_calls=Y)
dataset = dataset.batch(batch_size=32)
dataset = dataset.prefetch(buffer_size=X)
model = ...
model.fit(dataset, epochs=10)
20
tf.data.experimental.AUTOTUNE
2020

21. tf.data Options
21
- determinism
- statistics
- optimizations (autotuning, fusion, vectorization, parallelization, ...)
- threading (private thread pool, intra op parallelism)

22. tf.data Options
22
- determinism
- statistics
- optimizations (autotuning, fusion, vectorization, parallelization, ...)
- threading (private thread pool, intra op parallelism)
dataset = ...
options = tf.data.Options()
options.experimental_optimization.map_parallelization = True
dataset = dataset.with_options(options)

23. TFDS: TensorFlow Datasets
23
- https://www.tensorflow.org/datasets/datasets
- canned datasets ready to be used

24. TFDS: TensorFlow Datasets
24
- https://www.tensorflow.org/datasets/datasets
- canned datasets ready to be used
import tensorflow as tf
import tensorflow_datasets as tfds
# See available datasets
print(tfds.list_builders())
# Construct a tf.data.Dataset
dataset = tfds.load(name="mnist", split=tfds.Split.TRAIN)
# Customize your input pipeline
dataset = dataset.shuffle(1024).batch(32)
for features in dataset.take(1):
image, label = features["image"], features["label"]

25. Thank You!