Presentation by Johan Vos at Data Science Leuven Meetup on 27/06 https://www.meetup.com/Data-Science-Leuven/

1. Distributed Deep Learning Using Java on the
Client and in the Cloud
Johan Vos, Gluon, @johanvos
https://gluonhq.com

2. What we will discuss
• Brief introduction in deep learning and how to do that in
Java using deeplearning4j.
• Client-side deep learning (on mobile) helps with privacy,
security and improves the model in the Cloud.
• Leverage Java deep learning algorithms in cross-
platform, performant and beautiful Java mobile apps.

3. My history
• MSc engineering, KULeuven, 1995
• PhD Applied Physics, TUDelft, 2000
• Data processing and visualization
• Involves
– Sensors, small devices
– HPC
– Visualisation
• Java

4. When science shops become mature, they turn into Java
shops
• Java works on large enterprise systems, with many
libraries available
• Java works with small devices and sensors
• Java has AI libraries
• Java has a REPL: Jshell
• IDE’s
• Mature language
• Security is a cornerstone

5. It’s similar

6. It’s similar
crack
cat

7. It’s similar
crack
cat
algebra
algebra

8. It’s similar
crack
detailed cat
algebra
algebra
Quantum computing
Software 2.0

9. Deep Learning
• A class of machine learning algorithms
• Can be supervised and unsupervised
• Raw input data (e.g. images, text, numbers) traverses a
number of layers, where the last layer typically is a
classification
• Resources: https://deeplearning4j.org/
deeplearningforbeginners.html

10. Deep Learning

11. Deeplearning4j
• https://deeplearning4j.org
• Java framework providing Deep Learning functionality
• Implementations use native mathematical libraries and
GPU (if available)
• Created by SkyMind
• Open-source at github

12. Learning about digits
• Mnist dataset
• 60,000 labeled training
images
• 10,000 labeled test images
• Supervised learning

13. About the code
• All code shown in this presentations is in
https://github.com/gluonhq/gluon-samples/tree/master/mnist
• Java standalone application to train model and make
predictions
• Server using Payara on Oracle Cloud
• Mobile app sending requests to cloud and making local
predictions

14. Train and evaluate network
• Retrieve mnist data (70,000 labeled images)
• Create a 2-layer neural network
• Train network with mnist data
• Evaluate quality (how much % of test images fails?)
• Make prediction with own data

15. Creating a Model for mnist
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
.layer(0, new DenseLayer.Builder()
.nIn(height * width)
.nOut(100)
.activation(Activation.RELU)
.weightInit(WeightInit.XAVIER)
.build())
.layer(1, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
.nIn(100)
.nOut(outputNum)
.activation(Activation.SOFTMAX)
.weightInit(WeightInit.XAVIER)
.build())
.backprop(true)
.setInputType(InputType.convolutional(height, width, channels))
.build();

16. Train Model
==================Confusion Matrix=========================
0 1 2 3 4 5 6 7 8 9
---------------------------------------------------
961 0 2 2 0 6 5 1 3 0 | 0 = 0
0 1113 2 2 0 1 4 2 11 0 | 1 = 1
8 8 908 18 9 3 14 14 45 5 | 2 = 2
2 1 15 929 0 29 2 12 14 6 | 3 = 3
1 4 5 1 917 0 9 2 6 37 | 4 = 4
9 4 3 26 9 794 10 5 24 8 | 5 = 5
11 3 4 1 15 19 899 2 4 0 | 6 = 6
2 11 23 7 5 1 0 948 3 28 | 7 = 7
7 6 6 22 8 16 10 10 881 8 | 8 = 8
10 7 1 12 40 11 0 21 4 903 | 9 = 9

17. Server setup
• Runs in every cloud or on premise
• 12 million Java developers can do this

18. Mobile client
• Java app, using JavaFX for UI
• Allow camera to take pictures, or select pictures from
gallery
• Send image to Cloud
• Cloud returns prediction
• Show prediction in the app
• If incorrect, ask user to provide correct result
• Send corrected result to server and retrain

19. Security
• Direct connections between Mobile devices and enterprise back-ends are
a security nightmare (for developers and operators)

20. Gluon CloudLink
Gluon CloudLink
Dashboard
(https://gluon.io)
Backend
Application
Mobile Application
Public cloud
infrastructure
Cloud
Application Container
Cloud
Configuration
Storage DB

21. Monitor calls via Gluon CloudLink

22. Did we have permission to do this?
• Sending personal, raw data to cloud systems might be
sensitive.
• Pictures, locations, audio, video,… are all very relevant
input for AI and Deep Learning
• If users have to give permission for this, chances are they
will refuse.

23. Distributed Deep Learning
• Since we are using Java on the Client, we can run the
exact same code to use the model and to predictions
• Model is still retrieved from the server
• Predictions are made on client, using raw input data from
client

24. Quality concerns
• Supervised learning not possible anymore?
• We can do local training
• And synchronize model to server

25. Distributed Deep Learning

26. Client-side learning
• Retrain model on the (mobile) client.
– Synchronous
– At night
– Use raw user data to improve the local model
– Send new gradient (or delta with previous gradient) to server

27. Server-side update
• Accept changes from clients
– Merge weights in a smart way
– Send updated model to all clients

28. Conclusions
• Deep Learning algorithms provide many opportunities
• Java developers can use Deep Learning API’s
• Deep Learning applications run very well on Cloud systems
• Java Mobile applications can be used as clients for Deep
Learning cloud applications
• Java on Mobile allows to do more processing on the mobile
client, respecting user privacy and improving cloud model.

29. Thanks For Attending
Any Questions?
@johanvos
Johan.vos@gluonhq.com
@gluonhq
https://gluonhq.com
support@gluonhq.com