The document provides an overview of using TensorFlow and Spark for constructing and distributing neural networks, highlighting the complexity of defining neural network hyperparameters. It discusses the implementation of distributed TensorFlow, allowing the execution of TensorFlow graphs across clusters and jobs. The conclusion emphasizes the significance of distributed programming and the availability of other libraries compatible with Spark.

1. Tensorflow
Multithreading to construct neural networks
Maxence Queyrel
June 15, 2016
Quinten / UPMC

2. Table of contents
1. Introduction : Neural networks hyperparameters
2. Tensorflow with Spark
3. Distributed Tensorflow
4. Conclusion
1

3. Introduction : Neural networks
hyperparameters

4. Introduction : Neural networks hyperparameters
• Neural networks are complex models : They have a lot of
hyperparameters
• It is hard to define a neural network structure/graph
• Need to test several structures : execution time could be very
important
2

5. Introduction : Example of hyperparameters
- Number of layers
- Number of neurons for each layer
- Activation function
- Learning rate
- Number of iterations
- dropout probability
- Number of filters
- Filter’s dimensionality
- Convolution step



Only for Convolutional neural networks
3

6. Tensorflow with Spark

7. Tensorflow with Spark : The tools you need ?
TensorFlow
Spark
A cluster
4

8. Tensorflow with Spark : How does it work ?
5

9. Tensorflow with Spark : How does it work ?
6

10. Tensorflow with Spark : How does it work ?
7

11. Tensorflow with Spark : How does it work ?
8

12. Tensorflow with Spark : How does it work ?
9

13. Tensorflow with Spark : How does it work ?
10

14. Tensorflow with Spark : Example
11

15. Tensorflow with Spark : efficiency
Input : Iris DataSet from scikit learn.
This benchmark has been made on 45 NN for each execution.
12

16. Distributed Tensorflow

17. Distributed Tensorflow : What is it ?
This is a new option in TensorFlow 0.8.0, it allows to :
• Run a TensorFlow graph on a cluster
• Split the graph in several jobs
• Jobs can contain several tasks
13

18. Distributed Tensorflow : How does it work ?
14

19. Distributed Tensorflow : How does it work ?
15

20. Distributed Tensorflow : How does it work ?
16

21. Distributed Tensorflow : How does it work ?
17

22. Distributed Tensorflow : How does it work ?
18

23. Distributed Tensorflow : How does it work ?
19

24. Distributed Tensorflow : How to use it ?
• We have to launch our program by command line with the
correct arguments
• To get the arguments, we need to define some TensorFlow Flags
in the code
20

25. Distributed Tensorflow : Example
Python code
Shell code
21

26. Conclusion

27. Conclusion
• Importance of including distributed programming
• Spark : It exists other libraries running on spark like Theano or
Caffe
• Distributed TensorFlow : Development of libraries to replicate
models
22

28. Thank you for the intention
22

29. Questions ?
22