Deep Learning with H2O and R In my previous TNO4U talk I gave an introduction about how I addressed the classification problem for autonomous driving using fuzzy logic based insights. I also gave a very concise introduction on deep learning. In this talk I want to go more into the details of deep learning - what is it - and why people think it is so important. Due to the duration of the talk I will not go through the complete history of Artificial Intelligence from the perceptron, via the Hopfield net, towards modern  Restricted Bolzmann Machines and Convoluted Neural Networks. Nor get philosophical and do a Gödel, Escher, Bach exposé.   I will just give some basic theoretical considerations and  demonstrate how one easy it is to get results with deep learning using – open source- tools like R and H2O. You can install these for free on any computer, Windows, Linux or Mac. R is of course the computer language of choice for data science, H2O is an easy to use interface between R and Big Data (like Spark).     During the talk we will do some small workshop style examples. Handwriting recognition with a Restricted Bolzmann Machine, analyze heartbeats with machine learning and do a little predictive modelling on an industrial process.     Are this the heartbeats of a healthy person? Let’s ask our algorithm (The computer has seen more heartbeats than any living doctor)

1. DEEP LEARNING WITH R AND H2O
Performance Analysis of Complex Processes | Jan Eite Bullema

2. CONTENT:
DEEP LEARNING WITH R AND H2O
What is Deep Learning
What is R
What is H2O
Deep Learning with R and H2O
Example(s)
2 | Deep Learning with R and H2O 17 May 2016

3. WHAT IS DEEP LEARNING
WHAT CAN DEEP NETS DO
3 | Deep Learning with R and H2O 17 May 2016
Coursera: Neural Networks for Machine Learning

4. WHAT IS DEEP LEARNING
DEEP NETS OUTPERFORM OTHER NETS
4 | Deep Learning with R and H2O 17 May 2016
https://gigaom.com/2014/08/22/with-enlitic-a-veteran-data-scientist-plans-to-fight-disease-using-deep-learning/

5. WHAT IS DEEP LEARNING
NEURAL NETWORKS: THE PERCEPTRON
5 | Deep Learning with R and H2O 17 May 2016
1957 Frank Rosenblatt

6. WHAT IS DEEP LEARNING
ADJUSTING WEIGHTS FIND A FUNCTION
6 | Deep Learning with R and H2O 17 May 2016
EXAMPLE TRAIN A NEURAL NET FOR FINDING A SQUARE ROOT
net.sqrt <- neuralnet(Output~Input,trainingdata, hidden=c(4), threshold=0.1)

7. WHAT IS DEEP LEARNING
ADJUSTING WEIGHTS FIND A FUNCTION
7 | Deep Learning with R and H2O 17 May 2016

8. WHAT IS DEEP LEARNING
PRINCIPLES ORIGINATE FROM
JOHN VON NEUMANN
8 | Deep Learning with R and H2O 17 May 2016
John von Neumann, “The Computer and the Brain”, 1958 (unfinished manuscript)
Computation not
by Formal Logic but
with Thermodynamics

9. WHAT IS DEEP LEARNING
RESTRICTED BOLTZMANN MACHINES
9 | Deep Learning with R and H2O 17 May 2016
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines
Visible
Hidden
wiwj

10. WHAT IS DEEP LEARNING
RESTRICTED BOLTZMANN MACHINES
10 | Deep Learning with R and H2O 17 May 2016
A joint configuration, (v, h) of the visible and hidden units has an energy
Assigns probability to every pair of a visible and a hidden vector via this energy
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines
Partition Function

11. WHAT IS DEEP LEARNING
RESTRICTED BOLTZMANN MACHINES
11 | Deep Learning with R and H2O 17 May 2016
The probability assigned to a visible vector is given by summing over all possible
hidden vectors:
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines

12. WHAT IS DEEP LEARNING
RESTRICTED BOLTZMANN MACHINES
12 | Deep Learning with R and H2O 17 May 2016
The derivative of the log probability of a training vector with respect to a
weight is surprisingly simple
This leads to a very simple learning rule for performing stochastic steepest
ascent in the log probability of the training data
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines
Simple Learning Rule

13. WHAT IS DEEP LEARNING
CONTRASTIVE DIVERGENCE (GIBBS SAMPLING)
13 | Deep Learning with R and H2O 17 May 2016
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines

14. WHAT IS DEEP LEARNING
RESTRICTED BOLTZMANN MACHINES
14 | Deep Learning with R and H2O 17 May 2016
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines
Visible
Hidden
wiwj

15. WHAT IS DEEP LEARNING
AUTOENCODER
15 | Deep Learning with R and H2O 17 May 2016
Geoffrey Hinton: A Practical Guide to Training Restricted Boltzmann Machines

16. WHAT IS DEEP LEARNING
AUTOENCODER
16 | Deep Learning with R and H2O 17 May 2016
Coursera: Neural Networks for Machine Learning
The first two hidden layers are
learned without using labels.

17. WHAT IS DEEP LEARNING
AUTOENCODER
17 | Deep Learning with R and H2O 17 May 2016
Coursera: Neural Networks for Machine Learning
The top layer is learned as an
RBM for modeling the labels
concatenated with the features
in the second hidden layer

18. WHAT IS DEEP LEARNING
AUTOENCODER
18 | Deep Learning with R and H2O 17 May 2016
Coursera: Neural Networks for Machine Learning
Weights are then fine-tuned
using contrastive wake-sleep

19. WHAT IS DEEP LEARNING
AUTOENCODER
19 | Deep Learning with R and H2O 17 May 2016
Coursera: Neural Networks for Machine Learning

20. WHAT IS DEEP LEARNING
AUTOENCODER
20 | Deep Learning with R and H2O 17 May 2016
Coursera: Neural Networks for Machine Learning

21. WHAT IS DEEP LEARNING
GENERATE IMAGES
21 | Deep Learning with R and H2O 17 May 2016
Lunchcolloquium december 2015 'Deep Sense' Applying the Art of Deep Learning to Sensing in Defence and Security Domain

22. WHAT IS DEEP LEARNING
GENERATE IMAGES
22 | Deep Learning with R and H2O 17 May 2016
http://www.nu.nl/gadgets/4241850/nieuwe-rembrandt-gemaakt-met-behulp-van-data.html#the-next-rembrandt-2

23. WHAT IS DEEP LEARNING
WHAT IS IN A LAYER
23 | Deep Learning with R and H2O 17 May 2016
https://www.youtube.com/watch?v=0qVOUD76JOg The art of neural networks | Mike Tyka | TEDxTUM

24. WHAT IS DEEP LEARNING
INJECT NOISE AND ZOOM IN A LAYER
24 | Deep Learning with R and H2O 17 May 2016
https://www.youtube.com/watch?v=0qVOUD76JOg The art of neural networks | Mike Tyka | TEDxTUM

25. WHAT IS DEEP LEARNING
EXAMPLES OF DEEP NETS
25 | Deep Learning with R and H2O 17 May 2016
Recurrent Neural Network
Restricted Boltzmann Machines
Convolutional Neural Network

26. CONTENT:
DEEP LEARNING WITH R AND H2O
What is Deep Learning
What is R
What is H2O
Deep Learning with R and H2O
Example(s)
26 | Deep Learning with R and H2O 17 May 2016

27. WHAT IS R
27 | Deep Learning with R and H2O 17 May 2016
https://www.r-project.org/logo/

28. WHAT IS R
POPULAR WITH MACHINE LEARNING
28 | Deep Learning with R and H2O 17 May 2016
http://machinelearningmastery.com/best-programming-language-for-machine-learning/

29. WHAT IS R
R PROGRAMMERS EARN MORE MONEY
29 | Deep Learning with R and H2O 17 May 2016
https://www.datacamp.com/community/tutorials/r-or-python-for-data-analysis/

30. WHAT IS R
BIG THING WITH R ARE PACKAGES
30 | Deep Learning with R and H2O 17 May 2016
http://www.r-bloggers.com/on-the-growth-of-cran-packages/
More than 8,000 Packages
in April, 2016
install.packages("ggplot2")
library(ggplot2)
dim(available.packages())
8281 17

31. WHAT IS R
R STUDIO IS THE USER INTERFACE FOR R
31 | Deep Learning with R and H2O 17 May 2016https://www.rstudio.com/

32. WHAT IS R
EXAMPLE: DYNAMIC TIME WARPING
32 | Deep Learning with R and H2O 17 May 2016

33. WHAT IS R
EXAMPLE: DYNAMIC TIME WARPING
33 | Deep Learning with R and H2O 17 May 2016
library(dtw)
data(aami3a)
ref <- window(aami3a, start = stref, end = refend )
test <- window(aami3a, start = st, end = ed )
alignment <- dtw(test, ref, ask= FALSE)

34. WHAT IS R
DEEP LEARNING WITH R
34 | Deep Learning with R and H2O 17 May 2016
http://stats.stackexchange.com/questions/41771/r-libraries-for-deep-learning
Darch (based upon Hinton’s Deep Belief Networks)
Deepnet (incorporated in caret package)
RBM (works with GPU enhancement)

35. WHAT IS R
THE R INFERNO
35 | Deep Learning with R and H2O 17 May 2016
/
From ‘The R Inferno’ by Patrick Burns:The giants by Sandro Botticelli.

36. CONTENT:
DEEP LEARNING WITH R AND H2O
What is Deep Learning
What is R
What is H2O
Deep Learning with R and H2O
Example(s)
36 | Deep Learning with R and H2O 17 May 2016

37. WHAT IS H2O
DEEP LEARNING PLATFORMS
Theano
Dato Graph Lab
H2O
37 | Deep Learning with R and H2O 17 May 2016

38. WHAT IS H2O
H2O.AI IS A START-UP COMPANY
38 | Deep Learning with R and H2O 17 May 2016
http://techcrunch.com/2015/11/09/h2o-ai-raises-20m-for-its-open-source-machine-learning-platform/--

39. WHAT IS H2O
THE BEST "OPEN-SOURCE JAVA
MACHINE LEARNING PROJECT"
39 | Deep Learning with R and H2O 17 May 2016

40. WHAT IS H2O
H2O: PREDICTIVE ANALYTICS PLATFORM
40 | Deep Learning with R and H2O 17 May 2016

41. CONTENT:
DEEP LEARNING WITH R AND H2O
What is Deep Learning
What is R
What is H2O
Deep Learning with R and H2O
Example(s)
41 | Deep Learning with R and H2O 17 May 2016

42. DEEP LEARNING WITH R AND H2O
HANDWRITING RECOGNITION IN H2O
42 | Deep Learning with R and H2O 17 May 2016

43. DEEP LEARNING WITH R AND H2O
HANDWRITING RECOGNITION IN H2O
43 | Deep Learning with R and H2O 17 May 2016
h2o.init()
Connection successful!
R is connected to the H2O cluster:
H2O cluster uptime: 2 hours 5 minutes
H2O cluster version: 3.9.1.3405
H2O cluster name: H2O_started_from_R_bullemaje_mwo670
H2O cluster total nodes: 1
H2O cluster total memory: 1.56 GB
H2O cluster total cores: 8
H2O cluster allowed cores: 2
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
R Version: R version 3.2.3 (2015-12-10)

44. DEEP LEARNING WITH R AND H2O
HANDWRITING RECOGNITION IN H2O
44 | Deep Learning with R and H2O 17 May 2016

45. DEEP LEARNING WITH R AND H2O
HANDWRITING RECOGNITION IN H2O
deep_mnist <- h2o.deeplearning(x = 2:784, y=1,
training_frame = train,
validation_frame = valid,
activation = "RectifierWithDropout",
max_confusion_matrix_size = 10,
autoencoder =FALSE,
input_dropout_ratio = 0.2,
hidden_dropout_ratios = c(0.5,0.5,0.5),
balance_classes = FALSE,
hidden = c(500,500,2000),
epochs = 1000)
45 | Deep Learning with R and H2O 17 May 2016

46. DEEP LEARNING WITH R AND H2O
HANDWRITING RECOGNITION IN H2O
46 | Deep Learning with R and H2O 17 May 2016

47. CONTENT:
DEEP LEARNING WITH R AND H2O
What is Deep Learning
What is R
What is H2O
Deep Learning with R and H2O
Example(s)
47 | Deep Learning with R and H2O 17 May 2016

48. EXAMPLE(S)
ANOMALY IN HEARTBEAT DETECTION
48 | Deep Learning with R and H2O 17 May 2016

49. EXAMPLE(S)
ANOMALY IN HEARTBEAT DETECTION
49 | Deep Learning with R and H2O 17 May 2016

50. EXAMPLE(S)
PROCESS CONTROL FOR 3D PRINTING
50 | Deep Learning with R and H2O 17 May 2016
SERPENTINE MIXER

51. EXAMPLE(S)
MIXING MODEL FOR SERPENTINE MIXER
51 | Deep Learning with R and H2O 17 May 2016

52. EXAMPLE(S)
ACTUAL PRINTED CHANNELS
52 | Deep Learning with R and H2O 17 May 2016

53. EXAMPLE(S)
MICRO CT OF PRINTED CHANNELS
53 | Deep Learning with R and H2O 17 May 2016

54. EXAMPLE(S)
ADVANCED PROCESS CONTROL
FOR 3D PRINTING
54 | Deep Learning with R and H2O 17 May 2016
PROCESS DATA
DEEP NET
ADVANCED
PROCESS
CONTROL
MICRO CT
SCAN DATA

55. EXAMPLE(S)
OTHER POTENTIAL APPLICATIONS
Text mining
Structure and Taste of Food
Predictive Maintenance / Condition Monitoring
Credit Card Fraud
Tax Evasion
Predict new material properties from data
55 | Deep Learning with R and H2O 17 May 2016

56. DEEP LEARNING WITH R AND H2O
CONCLUSION: THERE ARE NEW TOOLS
56 | Deep Learning with R and H2O 17 May 2016
30 years ago I had to learn to use these type of
drawing tools to make nice graphs

57. THANK YOU FOR YOUR ATTENTION