1) Introduction to neural networks hyperparameters 2) TensorFlow with Spark 3) Distributed TensorFlow 4) Conclusion

1. Tensorflow
Multithreading to construct neural networks
Maxence Queyrel
June 15, 2016
Quinten / UPMC

2. Table of contents
1. Introduction : Neural networks hyperparameters
2. Tensorflow with Spark
3. Distributed Tensorflow
4. Conclusion
1

3. Introduction : Neural networks
hyperparameters

4. Introduction : Neural networks hyperparameters
• Neural networks are complex models : They have a lot of
hyperparameters
• It is hard to define a neural network structure/graph
• Need to test several structures : execution time could be very
important
2

5. Introduction : Example of hyperparameters
- Number of layers
- Number of neurons for each layer
- Activation function
- Learning rate
- Number of iterations
- dropout probability
- Number of filters
- Filter’s dimensionality
- Convolution step



Only for Convolutional neural networks
3

6. Tensorflow with Spark

7. Tensorflow with Spark : The tools you need ?
TensorFlow
Spark
A cluster
4

8. Tensorflow with Spark : How does it work ?
5

9. Tensorflow with Spark : How does it work ?
6

10. Tensorflow with Spark : How does it work ?
7

11. Tensorflow with Spark : How does it work ?
8

12. Tensorflow with Spark : How does it work ?
9

13. Tensorflow with Spark : How does it work ?
10

14. Tensorflow with Spark : Example
11

15. Tensorflow with Spark : efficiency
Input : Iris DataSet from scikit learn.
This benchmark has been made on 45 NN for each execution.
12

16. Distributed Tensorflow

17. Distributed Tensorflow : What is it ?
This is a new option in TensorFlow 0.8.0, it allows to :
• Run a TensorFlow graph on a cluster
• Split the graph in several jobs
• Jobs can contain several tasks
13

18. Distributed Tensorflow : How does it work ?
14

19. Distributed Tensorflow : How does it work ?
15

20. Distributed Tensorflow : How does it work ?
16

21. Distributed Tensorflow : How does it work ?
17

22. Distributed Tensorflow : How does it work ?
18

23. Distributed Tensorflow : How does it work ?
19

24. Distributed Tensorflow : How to use it ?
• We have to launch our program by command line with the
correct arguments
• To get the arguments, we need to define some TensorFlow Flags
in the code
20

25. Distributed Tensorflow : Example
Python code
Shell code
21

26. Conclusion

27. Conclusion
• Importance of including distributed programming
• Spark : It exists other libraries running on spark like Theano or
Caffe
• Distributed TensorFlow : Development of libraries to replicate
models
22

28. Thank you for the intention
22

29. Questions ?
22