The trend of neural networks has been attracted a huge community of researchers and practitioners. However, not all of the upfront runners are masters of deep learning and the colorful frameworks could be confusing, especially for the newcomers. In this presentation, I demystified the mystery of the leading frameworks of deep learning and provided a guideline on how to choose the most suitable option.

1. Deep learning
frameworks
Navid Kalaei
Shiraz University of Technology

2. “AI is the new
electricity”
Andrew NG, co-founder of Google
Brain

3. Deep Learning Is Booming!

4. Major Players
● Tensorfelow
● PyTorch
● Caffe2
● Sonet
● Keras
● MXNet
● Gloun
● CNTK
● Chainer
● DL4J
● ONNX

6. TensorFlow
● The most popular framework! (high job postings/great
resources)
● Developed by Google
● Used in major Google products like Gmail and Translate
● The most active framework on Github

7. TensorFlow
● Official support for Python
● Experimental support for C++, and Go
● Community support for C#, and Julia
● Reach detailed documentation
● TensorBoard: monitoring and visualizing models
● TensorFlow Serving: serve your model in production at
scale including distributed training

8. TensorBoard
TensorBoard provides the
visualization and tooling needed for
machine learning experimentation
TensorFlow’s Visualization
Toolkit

9. TensorBoard
● Track and visualize metrics such as loss and accuracy
● Visualize the model graph (ops and layers)
● View histograms of weights, biases, or other tensors as
they change over time
● Project embeddings to a lower dimensional space
● Display images, text, and audio data
● Profile TensorFlow programs

10. TensorBoard

11. Tensorflow
Serving
TensorFlow Serving is a flexible,
high-performance serving system
for machine learning models

12. TensorFlow Serving
● Designed for production environments
● TensorFlow Serving makes it easy to deploy new
algorithms and experiments, while keeping the same
server architecture and APIs.
● TensorFlow Serving provides out-of-the-box integration
with TensorFlow models, but can be easily extended to
serve other types of models and data.

13. TensorFlow Serving

14. TensorFlow Lite
Deploy machine learning models
on mobile and IoT devices
TensorFlow Lite is an open source
deep learning framework for
on-device inference.

15. TensorFlow Lite

16. Cons of TensorFlow
● Pretty low-level
● Requires a tons of boilerplate coding
● By default, creates static computation graphs at compile
time
● However, with eager execution, dynamic computation
graphs are available

18. PyTorch
● Created by Facebook
● A native define by run framework
● Used by Stanford NLP Group, Twitter, and Salesforce

19. PyTorch
● Is like more traditional programming
● Despite build and run TensorFlow, PyTorch creates
computational graph on each iteration. After the
iteration the memory is freed.
● Great in debugging: pdb, PyCharm, and common python
debuggers
● Mobile and large scale production via Caffe2

20. PyTorch
● Declarative data parallelism
● Many pretrained models
● Modular parts
● Distributed training

21. Cons of PyTorch
Lack of model serving
Lack of native monitors and visualizations
Could be connected to TensorBoard

23. Sonnet
● Created by DeepMind
● Built on top of TensorFlow
● Designed to provide simple, composable abstractions for
machine learning research
● Creates native python objects, then attach them to graph
computation. This makes modularity easier.

25. Keras
● The minimalist Python based library
● The best learning tool for beginners
● Can be run on top of TensorFlow and Microsoft CNTK
● Support for huge Neural Network types
● Suitable for prototyping
● Very readable
● Built-in support for multiple GPUs
● Can be converted to TensorFlow and be trained on
Google Cloud

26. Cons of Keras
● Very high-level -> not that customizable
● Constrained to TensorFlow and CNTK
● Less functionality than lower level liberaries

28. mxnet
● Amazon’s framework
● Adapted by AWS
● Native support for a huge variety of programming
languages
● Designed for scale linearly with number of processors
● High-performance imperative API
● Has the simplicity of Keras and dynamically of PyTorch
● Allows hybridization: Declarative like TensorFlow,
imperative like PyTorch, switch in between by Gluon

29. MXNet Highlights

30. MXNet: Mixed Programming Paradigm

32. CNTK
● Developed by Microsoft
● Supports Python, C++, C#, Java
● Supports for CNNs and RNNs
● Used in Skype, Xbox, and Cortana
● Easily develop products for speech and image problems
● Integrates with Apache Spark
● Very well integration with Azure
● Handles passing sequences like sentences better than
others

33. Cons of CNTK
● No conventional open source licence
● Mostly Windows developers

35. Chainer
● Created by preferred.jp a japanese startup
● Similar to PyTorch is define-by-run imperative API
● But difficult to debug
● Community is relatively small
● Supported by giants like IBM and Intel
● Run on multiple GPUs with little effort
● Main use cases: speech recognition, machine translation,
sentiment analysis

38. DL4J
● Mainly for Java and Scala
● Supports a huge variety of neural networks
● Is made for enterprise scale
● Works with Apache hadoop and Spark on distributed
CPUs and GPUs
● Great documentation
● Good for developing full-stacked Java pipeline which
includes Android devices

39. Cons of DL4J
● Java is not popular among machine learning projects
● Hard to integrate with other machine learning libraries

41. Core ML
● Not a framework to build models
● Helps bring existing models created with other
frameworks to Apple devices

43. ONNX
● The open ecosystem for interchangeable AI models
● Created in a partnership between Microsoft and
Facebook
● Train a model in one tool stack, and deploy it with the
other

44. Factors to
consider
● Model architecture
● Programming language
● Training device
● Target device
● Scale of the product
● Project deadline

45. Conclusion
● A beginner -> Kera
● Production on Google Cloud -> TensorFlow
● A researcher -> PyTorch but also try Sonnet
● Production on AWS -> mxnet
● Production on Azure -> CNTK
● A Java developer -> DL4J
● Some spare time -> Chainer
● Need to interpret models between frameworks -> ONNX
● An IOS developer -> Core ML