Apache MXNet is an open-source deep learning framework that supports distributed GPU training and a variety of programming languages. Its architecture includes a runtime dependency engine, a storage allocator, and optimized data loading for large datasets. The framework is part of an ecosystem that includes various tools for machine learning, computer vision, and data processing.

1. What Is Apache MXNet ?
● A deep learning framework
● Open source Apache 2.0 license
● Supports distributed gpu cluster training/deployment
● Of deep neural networks
● It supports a variety of language bindings
● Supports hybridize for increased speed/optimization
● Supports near linear scaling on gpu / host clusters
● Provides support for the Horovod framework

2. MXNet Language Bindings
● MXNet has a Python based API
● MXNet also supports the following language bindings
– Scala
– Julia
– Clojure
– Java
– C++
– R
– Perl

3. MXNet Related Terms
Horovod
MMS
DGL
ONNX
Hyperparameter
D2l.ai
KVStore
DMLC
A distributed deep learning framework from Uber
MXNet Model Server
Deep Graph Library
Open Neural Network Exchange
A parameter whose value is used to control the learning process
A jupyter notebook based deep learning book for Mxnet ++
Key-value store interface used by MXNet
Distributed (Deep) Machine Learning Community - GitHub

4. MXNet Eco System
Coach RL
Deep Graph
GluonFR
InsightFace
Keras-MXNet
MXBoard
MXFusion
MXNet Model
Optuna
Sockeye
A Python reinforcement learning framework
DGL is a Python pkg for deep learning on graphs
A community driven toolkit for face detection and recognition
A face detection and recognition repository
A back end of high level API Keras
Logging API's for TensorBoard visualisation
A modular deep probabilistic programming library
A flexible tool for serving models exported from Mxnet
A hyperparameter optimization framework
A sequence to sequence framework for neural translation

5. MXNet Eco System
TensorLY
TVM
Xfer
GluonCV
GluonNLP
GluonTS
A high level API for tensor methods
An open deep learning stack for GPU's, CPU's etc
A library for the transfer of knowledge in deep nets
A computer vision toolkit with a rich model zoo
Deep learning models for natural language processing
A toolkit for probabilistic time series modelling

6. MXNet User Community

7. MXNet Architecture

8. MXNet Architecture
● Runtime Dependency Engine
– Schedules and executes the operations
– According to their read/write dependency
● Storage Allocator
– Efficiently allocates and recycles memory blocks
– On host (CPU) and devices (GPUs)
● Resource Manager
– Manages global resources, such as
– The random number generator and temporal space
● NDArray
– Dynamic, asynchronous n-dimensional arrays

9. MXNet Architecture
● Symbolic Execution
– Static symbolic graph executor, which provides
– Efficient symbolic graph execution and optimization
● Operator
– Operators that define static forward/gradient calc (backprop)
● SimpleOp
– Operators that extend NDArray operators and
– Symbolic operators in a unified fashion
● Symbol Construction
– Symbolic construction, which provides a way to construct
– A computation graph (net configuration)

10. MXNet Architecture
● KVStore
– Key-value store interface for efficient parameter synchronization
● Data Loading(IO)
– Efficient distributed data loading and augmentation

11. MXNet Data Loading
● For large data sets data loading is optimized in MXNet
● Data format
– Uses dmlc-core’s binary recordIO implementation
● Data Loading
– Reduced IO cost by utilizing the threaded iterator
– Provided by dmlc-core
● Interface design
– Write MXNet data iterators in just a few lines of Python

12. MXNet Dependency Engine
● Helps to parallelize computation across devices
● Helps to synchronize computation when
– We introduce multi-threading
● A run time dependency schedule graph is created
● The graph is then used to
– Optimize processing
– Optimize memory use
– Aid parallelism when using
● GPU / CPU clusters
● For deep learning memory use
– Usage during training > during prediction

13. MXNet Forward Vs Backward Graph

14. Available Books
● See “Big Data Made Easy”
– Apress Jan 2015
●
See “Mastering Apache Spark”
– Packt Oct 2015
●
See “Complete Guide to Open Source Big Data Stack
– “Apress Jan 2018”
● Find the author on Amazon
– www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
●
Connect on LinkedIn
– www.linkedin.com/in/mike-frampton-38563020

15. Connect
● Feel free to connect on LinkedIn
– www.linkedin.com/in/mike-frampton-38563020
● See my open source blog at
– open-source-systems.blogspot.com/
● I am always interested in
– New technology
– Opportunities
– Technology based issues
– Big data integration