The document discusses the growth of connected devices and the Internet of Things. It then summarizes machine learning approaches for IoT data, including online learning which allows real-time model updates and historic learning which enables algorithmic improvements but requires high storage. Deep learning techniques for IoT like neural networks, convolutional neural networks, and LSTMs are also overviewed. Finally, the document presents challenges of deep learning and potential solutions like distributed hardware.

1. Unless stated otherwise all images are taken from wikipedia.org or openclipart.org

3. Why IoT (now) ?
• 15 Billion connected devices in 2015
• 40 Billion connected devices in 2020
• World population 7.4 Billion in 2016

15. Machine Learning on
historic data
Source: deeplearning4j.org

16. Online Learning
Source: deeplearning4j.org

17. online vs. historic
• Pros
• low storage costs
• real-time model update
• Cons
• algorithm support
• software support
• no algorithmic improvement
• compute power to be inline
with data rate
• Pros
• all algorithms
• abundance of software
• model re-scoring / re-
parameterisation (algorithmic
improvement)
• batch processing
• Cons
• high storage costs
• batch model update

19. DeepLearning
DeepLearning
Apache Spark
Hadoop

20. Neural Networks

21. Neural Networks

22. Deeper (more) Layers

23. Convolutional

24. Convolutional
+ =

25. Convolutional

27. Learning of a function
A neural network can basically learn any
mathematical function

28. Recurrent

29. LSTM
“vanishing error problem” == influence of past inputs decay
quickly over time

30. LSTM

31. http://karpathy.github.io/2015/05/21/rnn-effectiveness/

35. • Outperformed traditional methods, such as
• cumulative sum (CUSUM)
• exponentially weighted moving average (EWMA)
• Hidden Markov Models (HMM)
• Learned what “Normal” is
• Raised error if time series pattern haven't been seen
before

38. Learning of an algorithm
A LSTM network is touring complete

39. Problems
• Neural Networks are computationally very complex
•especially during training
•but also during scoring
CPU (2009) GPU (2016) IBM TrueNorth (2017)

40. IBM TrueNorth
• Scalable
• Parallel
• Distributed
• Fault Tolerant
• No Clock ! :)
• IBM Cluster
• 4.096 chips
• 4 billion neurons
• 1 trillion synapses
• Human Brain
• 100 billion neurons
• 100 trillion synapses
• 1.000.000 neurons
• 250.000.000 synapses

41. DeepLearning
the future in cloud based analytics
Storage Layer (OpenStack SWIFT / Hadoop HDFS / IBM GPFS)
Execution Layer (Spark Executor, YARN, Platform Symphony)
Hardware Layer (Bare Metal High Performance Cluster)
GraphXStreaming SQL MLLib BlinkDB
DeepLearning4J
ND4J
R MLBase H2O
Y
O
U
GPUAVX
Intel Xeon E7-4850 v2 48 core, 3 TB RAM, 72 GB HDD, 10Gbps, NVIDIA TESLA M60 GPU
(cu)BLAS
jcuBLAS
S
T
R
E
A
M
S

46. data
https://github.com/romeokienzler/pmqsimulator
https://ibm.biz/joinIBMCloud