SlideShare a Scribd company logo
1 of 91
Melanie Swan
Philosophy Department, Purdue University
melanie@BlockchainStudies.org
Deep Learning Explained
The future of Smart Networks
Boulder Futurists: Solid State Depot Hackspace
Boulder CO, August 12, 2017
Slides: http://slideshare.net/LaBlogga
Image credit: Nvidia
12 Aug 2017
Deep Learning 1
Melanie Swan, Technology Theorist
 Philosophy and Economic Theory, Purdue
University, Indiana, USA
 Founder, Institute for Blockchain Studies
 Singularity University Instructor; Institute for Ethics and
Emerging Technology Affiliate Scholar; EDGE
Essayist; FQXi Advisor
Traditional Markets Background
Economics and Financial
Theory Leadership
New Economies research group
Source: http://www.melanieswan.com, http://blockchainstudies.org/NSNE.pdf, http://blockchainstudies.org/Metaphilosophy_CFP.pdf
https://www.facebook.com/groups/NewEconomies
12 Aug 2017
Deep Learning
Agenda
 Deep Learning
 Definition
 Technical details
 Applications
 Deep Qualia: Deep Learning and the Brain
 Smart Network Convergence Theory
 Conclusion
2
Image Source: http://www.opennn.net
12 Aug 2017
Deep Learning
Deep Learning vocabulary
What do these terms mean?
 Deep Learning, Machine Learning, Artificial Intelligence
 Perceptron, Artificial Neuron, Logit
 Deep Belief Net, Artificial Neural Net, Boltzmann Machine
 Google DeepDream, Google Brain, Google DeepMind
 Supervised and Unsupervised Learning
 Convolutional Neural Nets
 Recurrent NN & LSTM (Long Short Term Memory)
 Activation Function ReLU (Rectified Linear Unit)
 Deep Learning libraries and frameworks
 TensorFlow, Caffe, Theano, Torch, DL4J
 Backpropagation, gradient descent, loss function
3
12 Aug 2017
Deep Learning 4
Conceptual Definition:
Deep learning is a computer program that can
identify what something is
Technical Definition:
Deep learning is a class of machine learning
algorithms in the form of a neural network that
uses a cascade of layers (tiers) of processing
units to extract features from data and make
predictive guesses about new data
Source: Extending Jann LeCun, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun-
on-deep-learning
12 Aug 2017
Deep Learning
Deep Learning Theory
 System is “dumb” (i.e. mechanical)
 “Learns” with big data (lots of input examples) and trial-and-error
guesses to adjust weights and bias to establish key features
 Creates a predictive system to identity new examples
 Same AI argument: big enough data is what makes a
difference (“simple” algorithms run over large data sets)
5
Input: Big Data (e.g.;
many examples)
Method: Trial-and-error
guesses to adjust node weights
Output: system identifies
new examples
12 Aug 2017
Deep Learning
Sample task: is that a Car?
 Create an image recognition system that determines
which features are relevant (at increasingly higher levels
of abstraction) and correctly identifies new examples
6
Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
12 Aug 2017
Deep Learning
Broader Computer Science Context
7
Source: Machine Learning Guide, 9. Deep Learning
 Within the Computer Science discipline, in the field of
Artificial Intelligence, Deep Learning is a class of
Machine Learning algorithms, that are in the form of a
Neural Network
12 Aug 2017
Deep Learning
Statistical Mechanics
Deep Learning is inspired by Physics
8
 Sigmoid function suggested as a model for neurons,
per statistical mechanical behavior (Jack Cowan)
 Stationary solutions for dynamic models (asymmetric
weights create an oscillator to model neuron signaling)
 Hopfield Neural Network: content-addressable
memory system with binary threshold nodes,
converges to a local minimum (John Hopfield)
 Can use an Ising model (of ferromagnetism) for neurons
 Restricted Boltzmann Machine (Geoffrey Hinton)
 Studied in theoretical physics, condensed matter field
theory; Statistical Mechanics concepts: Renormalization,
Boltzmann Distribution, Free Energies, Gibbs Sampling
Source: https://www.quora.com/Is-deep-learning-related-to-statistical-physics-particularly-network-science
12 Aug 2017
Deep Learning
What is a Neural Net?
9
 Motivation: create an Artificial Neural Network to solve
problems the same way a human brain would
12 Aug 2017
Deep Learning
What is a Neural Net?
10
 Structure: input-processing-output
 Mimic neuronal signal firing structure of brain with
computational processing units
Source: https://www.slideshare.net/ThomasDaSilvaPaula/an-introduction-to-machine-learning-and-a-little-bit-of-deep-learning,
http://cs231n.github.io/convolutional-networks/
12 Aug 2017
Deep Learning
What is an Artificial Neural Network?
 Collection of connected units called artificial
neurons (analogous to axons in a biological brain)
 Organized in layers of signaling cascades
 Each neuron transmits a signal to another neuron
 Neurons may have state
 Represented by a number between 0 and 1
 Variable parameters
 Neurons may have a weight that varies as learning
proceeds, which can increase or decrease the strength of
the signal that it sends downstream
 Neurons may have a threshold (bias) such that only if the
aggregate signal is below (or above) that level is the
downstream signal sent
11
12 Aug 2017
Deep Learning
Why is it called Deep Learning?
 Deep: Hidden layers (cascading tiers) of processing
 “Deep” networks (3+ layers) versus “shallow” (1-2 layers)
 Learning: Algorithms “learn” from data by modeling
features and updating probability weights assigned to
feature nodes in testing how relevant specific features
are in determining the general type of item
12
Deep: Hidden processing layers Learning: Updating probability
weights re: feature importance
12 Aug 2017
Deep Learning
Supervised and Unsupervised Learning
 Supervised (classify
labeled data)
 Unsupervised (find
patterns in unlabeled
data)
13
Source: https://www.slideshare.net/ThomasDaSilvaPaula/an-introduction-to-machine-learning-and-a-little-bit-of-deep-learning
12 Aug 2017
Deep Learning
Early success in Supervised Learning (2011)
 YouTube: user-classified data
perfect for Supervised Learning
14
Source: Google Brain: Le, QV, Dean, Jeff, Ng, Andrew, et al. 2012. Building high-level features using large scale unsupervised
learning. https://arxiv.org/abs/1112.6209
12 Aug 2017
Deep Learning
2 main kinds of Deep Learning neural nets
15
Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ
 Convolutional Neural Nets
 Image recognition
 Convolve: roll up to higher
levels of abstraction in feature
sets
 Recurrent Neural Nets
 Speech, text, audio recognition
 Recur: iterate over sequential
inputs with a memory function
 LSTM (Long Short-Term
Memory) remembers
sequences and avoids
gradient vanishing
12 Aug 2017
Deep Learning
Image Recognition and Computer Vision
16
Source: Quoc Le, https://arxiv.org/abs/1112.6209; Yann LeCun, NIPS 2016,
https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view
Marv Minsky, 1966
“summer project”
Jeff Hawkins, 2004, Hierarchical
Temporal Memory (HTM)
Quoc Le, 2011, Google
Brain cat recognition
Convolutional net for autonomous driving, http://cs231n.github.io/convolutional-networks/
History
Current state of
the art - 2017
12 Aug 2017
Deep Learning
Progression in AI Deep Learning machines
17
Single-purpose AI:
Hard-coded rules
Multi-purpose AI:
Algorithm detects rules,
reusable template
Question-answering AI:
Natural-language processing
Deep Learning prototypeHard-coded AI machine Deep Learning machine
Deep Blue, 1997 Watson, 2011 AlphaGo, 2016
12 Aug 2017
Deep Learning
Why do we need Deep Learning?
18
 A contemporary data science method to keep up
with the growth in data, older learning algorithms no
longer performing
Source: http://blog.algorithmia.com/introduction-to-deep-learning-2016
12 Aug 2017
Deep Learning
Agenda
 Deep Learning
 Definition
 Technical details
 Applications
 Deep Qualia: Deep Learning and the Brain
 Smart Network Convergence Theory
 Conclusion
19
Image Source: http://www.opennn.net
12 Aug 2017
Deep Learning
3 Key Technical Principles of Deep Learning
20
Reduce combinatoric
dimensionality
Core processing unit
(input-processing-output)
Levers: weights and bias
Squash values into
Sigmoidal S-curve
-Binary values (Y/N, 0/1)
-Probability values (0 to 1)
-Tanh values 9(-1) to 1)
Loss FunctionPerceptron StructureSigmoid Function
“Dumb” system learns by
adjusting parameters and
checking against outcome
Loss function
optimizes efficiency
of solution
Non-linear formulation
as a logistic regression
problem means
greater mathematical
manipulation
What
Why
12 Aug 2017
Deep Learning
Linear Regression
21
House price vs. Size (square feet)
y=mx+b
House price
Size (square feet)
Source: https://www.statcrunch.com/5.0/viewreport.php?reportid=5647
12 Aug 2017
Deep Learning
Logistic Regression
22
Source: http://www.simafore.com/blog/bid/99443/Understand-3-critical-steps-in-developing-logistic-regression-models
12 Aug 2017
Deep Learning
Logistic Regression
23
 Higher-order mathematical
formulation
 Sigmoid function
 S-shaped and bounded
 Maps the whole real axis into a finite
interval (0-1)
 Non-linear
 Can fit probability
 Can apply optimization techniques
 Deep Learning classification
predictions are in the form of a
probability value
Source: https://www.quora.com/Logistic-Regression-Why-sigmoid-function
Sigmoid Function
Unit Step Function
12 Aug 2017
Deep Learning
Sigmoid function: Taleb
24
Source: http://www.fooledbyrandomness.com/medicine.pdf
 Thesis: if can map a phenomenon onto
a sigmoid curve (“convexify” it), then
can control its risk
 Antifragility = convexity = risk-manageable
 Fragility = concavity
 Non-linearity of dose-response in
medicine, and therefore suggested
treatment optimality
12 Aug 2017
Deep Learning
Regression
 Logistic regression
 Predict binary outcomes:
 Perceptron (0 or 1)
 Predict probabilities:
 Sigmoid Neuron (values 0-1)
 Tanh Hyperbolic Tangent
Neuron (values (-1)-1)
25
Logistic Regression (Sigmoid function)
(0-1) or Tanh ((-1)-1)
Linear Regression
 Linear regression
 Predict continuous set
of values (house prices)
12 Aug 2017
Deep Learning
Deep Learning Architecture
26
Source: Michael A. Nielsen, Neural Networks and Deep Learning
12 Aug 2017
Deep Learning
Processing Unit, Perceptron, Neuron
27
Source: http://deeplearning.stanford.edu/tutorial
1. Input 2. Hidden layers 3. Output
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
 Unit (processing unit, logistic regression
unit), perceptron (“multilayer perceptron”),
artificial neuron
12 Aug 2017
Deep Learning
Example: Image recognition
1. Obtain training data set
2. Digitize pixels (convert images to numbers)
 Divide image into 28x28 grid, assign a value (0-255) to each
square based on brightness
3. Read into vector (array; list of numbers)
 28x28 = 784 elements per image)
28
Source: Quoc V. Le, A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm, 2015, Google
Brain, https://cs.stanford.edu/~quocle/tutorial1.pdf
12 Aug 2017
Deep Learning
Deep Learning Architecture
4. Load spreadsheet of vectors into deep learning system
 Each row of spreadsheet (784-element array) is an input
29
Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist
1. Input 2. Hidden layers 3. Output
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Vector data
784-element array
12 Aug 2017
Deep Learning
What happens in the Hidden Layers?
30
Source: Michael A. Nielsen, Neural Networks and Deep Learning
 First layer learns primitive features (line, edge, tiniest
unit of sound) by finding combinations of the input vector
data that occur more frequently than by chance
 Logistic regression performed and encoded at each processing
node (Y/N (0,1)), does this example have this feature?
 Feeds these basic features to next layer, which trains
itself to recognize slightly more complicated features
(corner, combination of speech sounds)
 Feeds features to new layers until recognizes full objects
12 Aug 2017
Deep Learning
Image Recognition
Higher Abstractions of Feature Recognition
31
Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
Edges Object Parts
(combinations of edges)
Object Models
12 Aug 2017
Deep Learning
Speech, Text, Audio Recognition
Sequence-to-sequence Recognition + LSTM
32
Source: Andrew Ng
12 Aug 2017
Deep Learning
Example: NVIDIA Facial Recognition
33
Source: Nvidia
 First hidden layer extracts all possible low-level features
from data (lines, edges, contours); next layers abstract
into more complex features of possible relevance
12 Aug 2017
Deep Learning
Deep Learning
34
Source: Quoc V. Le et al, Building high-level features using large scale unsupervised learning, 2011, https://arxiv.org/abs/1112.6209
12 Aug 2017
Deep Learning
Deep Learning Architecture
35
Source: Michael A. Nielsen, Neural Networks and Deep Learning
1. Input 2. Hidden layers 3. Output
(0,1)
12 Aug 2017
Deep Learning
Mathematical methods update weights
36
1. Input 2. Hidden layers 3. Output
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist
 Linear algebra: matrix multiplications of input vectors
 Statistics: logistic regression units (Y/N (0,1)), probability
weighting and updating, inference for outcome prediction
 Calculus: optimization (minimization), gradient descent in
back-propagation to avoid local minima with saddle points
Feed-forward pass (0,1)
0.5
Backward pass to update probabilities
.5.5
.5.5.5
0
01
.75
.25
Inference
Guess
Actual
12 Aug 2017
Deep Learning
More complicated in actual use
 Convolutional neural net scale-up for
number recognition
 Example data: MNIST dataset
 http://yann.lecun.com/exdb/mnist
37
Source: http://www.kdnuggets.com/2016/04/deep-learning-vs-svm-random-forest.html
12 Aug 2017
Deep Learning
Structure of a Node: Computation Graph
38
Edge
(input value)
Architecture
Node
(operation)
Edge
(input value)
Edge
(output value)
Example 1
3
4
Add
??
Example 2
3
4
Mult
??
12 Aug 2017
Deep Learning
Neural net unit: perceptron, neuron, node
39
Source: http://neuralnetworksanddeeplearning.com/chap1.html
(0,1)
(0,1)
(0,1)
(0,1)
Oper-
ation
 Sigmoid function means all inputs and outputs in the
system are (0,1)
12 Aug 2017
Deep Learning
Other parameters: weights and bias
40
Source: http://neuralnetworksanddeeplearning.com/chap1.html
Values have
Weights
Operation node
has Bias
W1 = (-2)
B=3
W2 = (-2)
 Weight and bias are variable parameters that
get adjusted as the system iterates and learns
Values have
Weights
Operation node
has Bias
W1 = (-2)
B=3
W2 = (-2)
= 0
(-2)*0 + (-2)*0 + 3 = 3
= 0
Output
= 0
0,0
0,1
1,0
1,1 (-2)*1 + (-2)*1 + 3 = (-1)
(-2)*0 + (-2)*1 + 3 = 1
(-2)*1 + (-2)*0 + 3 = 1
W1*X1 + W2*X2 + Bias = n Output (0,1)Input (0,1) X1, X2
Weight and Bias are
“randomly” assigned at the
beginning: (here (-2) and 3)
Mimics NAND gate
1
1
1
0
12 Aug 2017
Deep Learning
Actual: same structure, more complicated
41
12 Aug 2017
Deep Learning
Neural net: massive scale-up of nodes
42
Source: http://neuralnetworksanddeeplearning.com/chap1.html
12 Aug 2017
Deep Learning
Same Structure
43
12 Aug 2017
Deep Learning
How does the neural net actually learn?
 Vary the weights
and biases to see if
a better outcome is
obtained
 Repeat until the net
correctly classifies
the data
44
Source: http://neuralnetworksanddeeplearning.com/chap2.html
 Structural system based on cascading layers of
neurons with variable parameters: weight and bias
12 Aug 2017
Deep Learning
Backpropagation
 Problem: Inefficient to test the combinatorial
explosion of all possible parameter variations
 Solution: Backpropagation (1986 Nature paper)
 Backpropagation is an optimization method used to
calculate the error contribution of each neuron after
a batch of data is processed
45
Source: http://neuralnetworksanddeeplearning.com/chap2.html
12 Aug 2017
Deep Learning
Backpropagation of error
 Calculate the total error
 Calculate the contribution to the error at each step
going backwards
 Variety of Error Calculation methods: Mean Square Error
(MSE), sum of squared errors of prediction (SSE), Cross-
Entropy (Softmax), Softplus
46
12 Aug 2017
Deep Learning
Backpropagation
 Heart of Deep Learning
 Backpropagation: algorithm dynamically calculates
the gradient (derivative) of the loss function with
respect to the weights in a network to find the
minimum and optimize the function from there
 Algorithms optimize the performance of the network by
adjusting the weights, e.g.; in the gradient descent algorithm
 Error and gradient are computed for each node
 Intermediate errors transmitted backwards through the
network (backpropagation)
 Objective: optimize the weights so that the neural
network can learn how to correctly map arbitrary
inputs to outputs
47
Source: http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4,
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
12 Aug 2017
Deep Learning
Gradient Descent
 Gradient: derivative to find the minimum of a function
 Gradient descent: optimization algorithm to find the
biggest errors (minima) most quickly
 Error = MSE, log loss, cross-entropy; e.g.; least correct
predictions to correctly identify data
48
Source: http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4
12 Aug 2017
Deep Learning
 Optimization Technique
 Mathematical tool used in statistics, finance, decision theory,
biological modeling, computational neuroscience
 State as non-linear equation to optimize
 Minimize loss or cost
 Maximize reward, utility, profit, or fitness
 Loss function links instance of an event to its cost
 Accident (event) means $1,000 damage on average (cost)
 5 cm height (event) confers 5% fitness advantage (reward)
 Deep learning: system feedback loop
 Use penalty cost for incorrect classifications to train system
 CNN (classification): cross-entropy; RNN (regression): MSE
Loss Function
49
Laplace
12 Aug 2017
Deep Learning
Overfitting
 Regularization
 Introduce additional information
such as a lambda parameter in the
cost function (to update the theta
parameters in the gradient descent
algorithm)
 Dropout: prevent complex
adaptations on training data by
dropping out units (both hidden and
visible)
 Test new datasets
50
12 Aug 2017
Deep Learning
Research Topics
 Layer depth vs. height (1x9, 3x3, etc.); L1/2 slow-downs
 Backpropagation, gradient descent, loss function
 Saddle-free optimization, vanishing gradients
 Composition of non-linearities
 Non-parametric manifold learning, auto-encoders
 Activation maximization
 Synthesizing preferred inputs for neurons
51
Source: http://cs231n.github.io/convolutional-networks, https://arxiv.org/abs/1605.09304,
https://www.iro.umontreal.ca/~bengioy/talks/LondonParisMeetup_15April2015.pdf
12 Aug 2017
Deep Learning
Advanced
Deep Learning Architectures
52
Source: http://prog3.com/sbdm/blog/zouxy09/article/details/8781396
 Deep Belief Network
 Connections between layers not units
 Establish weighting guesses for
processing units before run deep
learning system
 Used to pre-train systems to assign
initial probability weights (more efficient)
 Deep Boltzmann Machine
 Stochastic recurrent neural network
 Runs learning on internal
representations
 Represent and solve combinatoric
problems
Deep
Boltzmann
Machine
Deep
Belief
Network
12 Aug 2017
Deep Learning
Convolutional net: Image Enhancement
 Google DeepDream: Convolutional neural network
enhances (potential) patterns in images; deliberately
over-processing images
53
Source: Georges Seurat, Un dimanche après-midi à l'Île de la Grande Jatte, 1884-1886;
http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil722; Google DeepDream uses algorithmic pareidolia (seeing an image
when none is present) to create a dream-like hallucinogenic appearance
12 Aug 2017
Deep Learning
Hardware and Software Tools
54
12 Aug 2017
Deep Learning
Deep Learning frameworks and libraries
55
Source: http://www.infoworld.com/article/3163525/analytics/review-the-best-frameworks-for-machine-learning-and-deep-
learning.html#tk.ifw-ifwsb
12 Aug 2017
Deep Learning
What is TensorFlow?
56
Source: https://www.youtube.com/watch?v=uHaKOFPpphU
Python code invoking TensorFlowTensorBoard (TensorFlow) visualization
Computation graph Design in TensorFlow
 Google’s open-source machine learning library
 “Tensor” = multidimensional arrays used in NN operations
12 Aug 2017
Deep Learning
Hardware
 Advances in chip design
 GPU chips (graphics processing unit):
3D graphics cards designed to do fast
matrix multiplication
 Google TPU chip (tensor processing
unit): custom ASICs for machine
learning, used in AlphaGo
 TPUs process matrix
multiplications without storing
intermediate values in memory
 NVIDIA DGX-1 integrated deep
learning system
 Eight Tesla P100 GPU accelerators
57
Google TPU chip (Tensor
Processing Unit), 2016
Source: http://www.techradar.com/news/computing-components/processors/google-s-tensor-processing-unit-explained-this-is-what-
the-future-of-computing-looks-like-1326915
NVIDIA DGX-1
Deep Learning System
12 Aug 2017
Deep Learning
USB and Browser-based Machine Learning
 Intel: Movidius Visual Processing
Unit (VPU): USB ML for IOT
 Security cameras, industrial
equipment, robots, drones
 Apple: ML acquisition Turi (Dato)
 Browser-based Deep Learning
 ConvNetJS; TensorFire
 Javascript library to run Deep
Learning (Neural Networks) in a
browser
 Smart Network in a browser
 JavaScript Deep Learning
 Blockchain EtherWallets
58
Source: http://cs.stanford.edu/people/karpathy/convnetjs/, http://www.infoworld.com/article/3212884/machine-learning/machine-learning-
comes-to-your-browser-via-javascript.html
12 Aug 2017
Deep Learning
How big are Deep Learning neural nets?
 Google Deep Brain cat recognition, 2011
 1 billion connections, 10 million images (200x200
pixel), 1,000 machines (16,000 cores), 3 days, each
instantiation of the network spanned 170 servers, and
20,000 object categories
 State of the art, 2016-2017
 NVIDIA facial recognition, 100 million images, 10
layers, 1 bn parameters, 30 exaflops, 30 GPU days
 Google, 11.2-billion parameter system
 Lawrence Livermore Lab, 15-billion parameter system
 Digital Reasoning, cognitive computing (Nashville TN),
160 billion parameters, trained on three multi-core
computers overnight
59
Source: https://futurism.com/biggest-neural-network-ever-pushes-ai-deep-learning, Digital Reasoning paper:
https://arxiv.org/pdf/1506.02338v3.pdf
12 Aug 2017
Deep Learning
Agenda
 Deep Learning
 Definition
 Technical details
 Applications
 Deep Qualia: Deep Learning and the Brain
 Smart Network Convergence Theory
 Conclusion
60
Image Source: http://www.opennn.net
12 Aug 2017
Deep Learning
Applications: Cats to Cancer to Cognition
61
Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ
Computational imaging: Machine learning for 3D microscopy
https://www.nature.com/nature/journal/v523/n7561/full/523416a.html
12 Aug 2017
Deep Learning
Tumor Image Recognition
62
Source: https://www.nature.com/articles/srep24454
 Computer-Aided
Diagnosis with
Deep Learning
Architecture
 Breast tissue
lesions in images
and pulmonary
nodules in CT
Scans
12 Aug 2017
Deep Learning
Melanoma Image Recognition
63
Source: http://www.nature.com/nature/journal/v542/n7639/full/nature21056.html
12 Aug 2017
Deep Learning
DIY Image Recognition: use Contrast
64
Source: https://developer.clarifai.com/modelshttps://developer.clarifai.com/models
How many orange pixels?
Apple or Orange? Melanoma risk or healthy skin?
Degree of contrast in photo colors?
12 Aug 2017
Deep Learning
Deep Learning and Genomics
 Large classes of hypothesized but unknown correlations
 Genotype-phenotype disease linkage unknown
 Computer-identifiable patterns in genomic data
 CNN: genome symmetries; RNN: textual analysis
65
Source: http://ieeexplore.ieee.org/document/7347331
12 Aug 2017
Deep Learning
Deep Learning and the Brain
66
12 Aug 2017
Deep Learning
 Deep learning neural networks are inspired by the
structure of the cerebral cortex
 The processing unit, perceptron, artificial neuron is the
mathematical representation of a biological neuron
 In the cerebral cortex, there can be several layers of
interconnected perceptrons
67
Deep Qualia machine? General purpose AI
Mutual inspiration of neurological and computing research
12 Aug 2017
Deep Learning
Deep Qualia machine?
 Visual cortex is hierarchical with intermediate layers
 The ventral (recognition) pathway in the visual cortex has multiple
stages: Retina - LGN - V1 - V2 - V4 - PIT – AIT
 Human brain simulation projects
 Swiss Blue Brain project, European Human Brain Project
68
Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
12 Aug 2017
Deep Learning
Social Impact of Deep Learning
 WHO estimates 400 million people without
access to essential health services
 6% in extreme poverty due to healthcare costs
 Next leapfrog technology: Deep Learning
 Last-mile build out of brick-and-mortar clinics
does not make sense in era of digital medicine
 Medical diagnosis via image recognition, natural
language processing symptoms description
 Convergence Solution: Digital Health Wallet
 Deep Learning medical diagnosis + Blockchain-
based EMRs (electronic medical records)
 Empowerment Effect: Deep learning = “tool I
use,” not hierarchically “doctor-administered”
69
Source: http://www.who.int/mediacentre/news/releases/2015/uhc-report/en/
Digital Health Wallet:
Deep Learning diagnosis
Blockchain-based EMRs
12 Aug 2017
Deep Learning
Agenda
 Deep Learning
 Definition
 Technical details
 Applications
 Deep Qualia: Deep Learning and the Brain
 Smart Network Convergence Theory
 Conclusion
70
Image Source: http://www.opennn.net
12 Aug 2017
Deep Learning 71
Better horse AND new car
New Technology
12 Aug 2017
Deep Learning 72
Smart networks are computing networks with
intelligence built in such that identification
and transfer is performed by the network
itself through protocols that automatically
identify (deep learning), and validate,
confirm, and route transactions (blockchain)
within the network
Smart Network Convergence Theory
12 Aug 2017
Deep Learning
Smart Network Convergence Theory
 Network intelligence “baked in” to smart networks
 Deep Learning algorithms for predictive identification
 Blockchains to transfer value, confirm authenticity
73
Source: Expanded from Mark Sigal, http://radar.oreilly.com/2011/10/post-pc-revolution.html
Two Fundamental Eras of Network Computing
12 Aug 2017
Deep Learning 74
Blockchain is the tamper-resistant
distributed ledger software underlying
cryptocurrencies such as Bitcoin, for the
secure transfer of money, assets, and
information via the Internet without a third-
party intermediary
Source: http://www.amazon.com/Bitcoin-Blueprint-New-World-Currency/dp/1491920491
12 Aug 2017
Deep Learning
Blockchain Deep Learning nets
 Provide increasingly sophisticated automated network
computational infrastructure
 Make predictive guesses of reality states of the world
 Predictive inference (deep learning) and cryptographic nonce-
guesses (blockchain)
 Instantiate decentralization
 Hierarchical models do not scale
75
12 Aug 2017
Deep Learning
Next Phase
 Put Deep Learning systems on the Internet
 Deep Learning Blockchain Networks
 Combine Deep Learning and Blockchain Technology
 Blockchain offers secure audit ledger of activity
 Advanced computational infrastructure to tackle
larger-scale problems
 Genomic disease, protein modeling, energy storage,
global financial risk assessment, voting, astronomical data
76
12 Aug 2017
Deep Learning
Example: Autonomous Driving
 Requires the smart network functionality
of deep learning and blockchain
 Deep Learning: identify what things are
 Convolutional neural nets core element of
machine vision system
 Blockchain: secure automation
technology
 Track arbitrarily-many fleet units
 Legal accountability
 Software upgrades
 Remuneration
77
12 Aug 2017
Deep Learning
The Very Small
Blockchain Deep Learning nets in Cells
 Medical nanorobotics for cell repair
 Deep Learning: identify what things are
(diagnosis)
 Blockchain: secure automation technology
 Bio-cryptoeconomics: secure automation
of medical nanorobotics for cell repair
 Medical nanorobotics as coming-onboard
repair platform for the human body
 High number of agents and “transactions”
 Identification and automation is obvious
78
Sources: Swan, M. Blockchain Thinking: The Brain as a DAC (Decentralized Autonomous Corporation). Technology and Society Magazine,
IEEE 2015; 34(4): 41-52 , https://www.slideshare.net/lablogga/biocryptoeconomy-smart-contract-blockchainbased-bionano-repair-dacs
12 Aug 2017
Deep Learning
The Very Large
Blockchain Deep Learning nets in Space
 Automated space
construction bots/agents
 Deep Learning: identify
what things are
(classification)
 Blockchain: secure
automation technology
 Applications: asteroid
mining, terraforming,
radiation-monitoring,
space-based solar power,
debris tracking net
79
12 Aug 2017
Deep Learning
Agenda
 Deep Learning
 Definition
 Technical details
 Applications
 Deep Qualia: Deep Learning and the Brain
 Smart Network Convergence Theory
 Conclusion
80
Image Source: http://www.opennn.net
12 Aug 2017
Deep Learning
Our human future
81
 Are we doomed?
12 Aug 2017
Deep Learning
Human-machine collaboration
82
 Team-members excel at different things
 Differently-abled agents in society
Source: Swan, M. (2017). Is Technological Unemployment Real? In: Surviving the Machine Age.
http://www.springer.com/us/book/9783319511641
12 Aug 2017
Deep Learning 83
Conceptual Definition:
Deep learning is a computer program that can
identify what something is
Technical Definition:
Deep learning is a class of machine learning
algorithms in the form of a neural network that
uses a cascade of layers (tiers) of processing
units to extract features from data and make
predictive guesses about new data
Source: Extending Jann LeCun, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun-
on-deep-learning
12 Aug 2017
Deep Learning
Deep Learning Theory
 System is “dumb” (i.e. mechanical)
 “Learns” with big data (lots of input examples) and trial-and-error
guesses to adjust weights and bias to establish key features
 Creates a predictive system to identity new examples
 Same AI argument: big enough data is what makes a
difference (“simple” algorithms run over large data sets)
84
Input: Big Data (e.g.;
many examples)
Method: Trial-and-error
guesses to adjust node weights
Output: system identifies
new examples
12 Aug 2017
Deep Learning
3 Key Technical Principles of Deep Learning
85
Reduce combinatoric
dimensionality
Core processing unit
(input-processing-output)
Levers: weights and bias
Squash values into
probability function
(Sigmoid (0-1);
Tanh ((-1)-1))
Loss FunctionPerceptron StructureSigmoid Function
“Dumb” system learns by
adjusting parameters and
checking against outcome
Loss function
optimizes efficiency
of solution
Formulate as a logistic
regression problem for
greater mathematical
manipulation
What
Why
12 Aug 2017
Deep Learning
Conclusion
 Next-generation global infrastructure:
Deep Learning Blockchain Networks
merging deep learning systems and
blockchain technology
 Smart Network Convergence Theory:
pushing more complexity and
automation through Internet pipes
 Blockchain Deep Learning nets: Ability to
identify what something is (machine
learning) and securely verify and transact it
(blockchain)
86
12 Aug 2017
Deep Learning
 Neural Networks and Deep Learning, Michael Nielsen,
http://neuralnetworksanddeeplearning.com/
 Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron
Courville, http://www.deeplearningbook.org/Machine learning and deep neural nets
 Machine Learning Guide podcast, Tyler Renelle,
http://ocdevel.com/podcasts/machine-learning
 notMNIST dataset http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html
 Metacademy; Fast.ai; Keras.io
Resources
87
Distill (visual ML journal)
http://distill.pubSource: http://cs231n.stanford.edu
https://www.deeplearning.ai/
Melanie Swan
Philosophy Department, Purdue University
melanie@BlockchainStudies.org
Deep Learning Explained
The future of Smart Networks
Boulder Futurists: Solid State Depot Hackspace
Boulder CO, August 12, 2017
Slides: http://slideshare.net/LaBlogga
Image credit: Nvidia
Thank You! Questions?
12 Aug 2017
Deep Learning
Deep Learning Taxonomy
89
Source: Machine Learning Guide, 9. Deep Learning;
AI (artificial intelligence)
Machine learning Other methods
Supervised learning
(labeled data:
classification)
Unsupervised learning
(unlabeled data: pattern
recognition)
Reinforcement learning
Shallow learning
(1-2 layers)
Deep learning
(5-20 layers)
Recurrent nets (text, speech)
Convolutional nets (images)
Neural Nets (NN) Other methods
Bayesian inference
Support Vector Machines
Decision trees
K-means clustering
K-nearest neighbor
12 Aug 2017
Deep Learning
Kinds of Deep Learning Systems
What Deep Learning net to choose?
90
Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ
 Supervised algorithms (classify labeled data)
 Image (object) recognition
 Convolutional net (image processing), deep belief
network, recursive neural tensor network
 Text analysis (name recognition, sentiment
analysis)
 Recurrent net (iteration; character level text),
recursive neural tensor network
 Speech recognition
 Recurrent net
 Unsupervised algorithms (find patterns in
unlabeled data)
 Boltzmann machine or autoencoder

More Related Content

What's hot

Deep neural networks
Deep neural networksDeep neural networks
Deep neural networksSi Haem
 
Neural networks and deep learning
Neural networks and deep learningNeural networks and deep learning
Neural networks and deep learningJörgen Sandig
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learningleopauly
 
Intro to deep learning
Intro to deep learning Intro to deep learning
Intro to deep learning David Voyles
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine LearningLior Rokach
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksChristian Perone
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep LearningSebastian Ruder
 
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckAI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckSlideTeam
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep LearningJulien SIMON
 
An introduction to Machine Learning
An introduction to Machine LearningAn introduction to Machine Learning
An introduction to Machine Learningbutest
 
Deep Learning With Neural Networks
Deep Learning With Neural NetworksDeep Learning With Neural Networks
Deep Learning With Neural NetworksAniket Maurya
 
Artificial neural network
Artificial neural networkArtificial neural network
Artificial neural networkDEEPASHRI HK
 
Explainable AI (XAI) - A Perspective
Explainable AI (XAI) - A Perspective Explainable AI (XAI) - A Perspective
Explainable AI (XAI) - A Perspective Saurabh Kaushik
 

What's hot (20)

Deep learning
Deep learningDeep learning
Deep learning
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
Neural networks and deep learning
Neural networks and deep learningNeural networks and deep learning
Neural networks and deep learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 
Deep learning
Deep learning Deep learning
Deep learning
 
Intro to deep learning
Intro to deep learning Intro to deep learning
Intro to deep learning
 
Support Vector Machines ( SVM )
Support Vector Machines ( SVM ) Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
 
Transfer Learning
Transfer LearningTransfer Learning
Transfer Learning
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Cnn
CnnCnn
Cnn
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep Learning
 
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckAI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
 
Machine learning
Machine learning Machine learning
Machine learning
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep Learning
 
An introduction to Machine Learning
An introduction to Machine LearningAn introduction to Machine Learning
An introduction to Machine Learning
 
Deep Learning With Neural Networks
Deep Learning With Neural NetworksDeep Learning With Neural Networks
Deep Learning With Neural Networks
 
Artificial neural network
Artificial neural networkArtificial neural network
Artificial neural network
 
Explainable AI (XAI) - A Perspective
Explainable AI (XAI) - A Perspective Explainable AI (XAI) - A Perspective
Explainable AI (XAI) - A Perspective
 

Viewers also liked

Blockchain Smartnetworks: Bitcoin and Blockchain Explained
Blockchain Smartnetworks: Bitcoin and Blockchain ExplainedBlockchain Smartnetworks: Bitcoin and Blockchain Explained
Blockchain Smartnetworks: Bitcoin and Blockchain ExplainedMelanie Swan
 
Construisons ensemble le chatbot bancaire dedemain !
Construisons ensemble le chatbot bancaire dedemain !Construisons ensemble le chatbot bancaire dedemain !
Construisons ensemble le chatbot bancaire dedemain !LINAGORA
 
Cs231n 2017 lecture13 Generative Model
Cs231n 2017 lecture13 Generative ModelCs231n 2017 lecture13 Generative Model
Cs231n 2017 lecture13 Generative ModelYanbin Kong
 
Exploring Session Context using Distributed Representations of Queries and Re...
Exploring Session Context using Distributed Representations of Queries and Re...Exploring Session Context using Distributed Representations of Queries and Re...
Exploring Session Context using Distributed Representations of Queries and Re...Bhaskar Mitra
 
Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...
Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...
Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...Association for Computational Linguistics
 
Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...
Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...
Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...Association for Computational Linguistics
 
Deep Learning for Chatbot (1/4)
Deep Learning for Chatbot (1/4)Deep Learning for Chatbot (1/4)
Deep Learning for Chatbot (1/4)Jaemin Cho
 
Cs231n 2017 lecture11 Detection and Segmentation
Cs231n 2017 lecture11 Detection and SegmentationCs231n 2017 lecture11 Detection and Segmentation
Cs231n 2017 lecture11 Detection and SegmentationYanbin Kong
 
Cs231n 2017 lecture10 Recurrent Neural Networks
Cs231n 2017 lecture10 Recurrent Neural NetworksCs231n 2017 lecture10 Recurrent Neural Networks
Cs231n 2017 lecture10 Recurrent Neural NetworksYanbin Kong
 
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...Association for Computational Linguistics
 
Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop
Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 WorkshopSatoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop
Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 WorkshopAssociation for Computational Linguistics
 
Deep Learning for Chatbot (3/4)
Deep Learning for Chatbot (3/4)Deep Learning for Chatbot (3/4)
Deep Learning for Chatbot (3/4)Jaemin Cho
 
Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...
Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...
Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...Association for Computational Linguistics
 
Using Text Embeddings for Information Retrieval
Using Text Embeddings for Information RetrievalUsing Text Embeddings for Information Retrieval
Using Text Embeddings for Information RetrievalBhaskar Mitra
 
State of Blockchain 2017: Smartnetworks and the Blockchain Economy
State of Blockchain 2017:  Smartnetworks and the Blockchain EconomyState of Blockchain 2017:  Smartnetworks and the Blockchain Economy
State of Blockchain 2017: Smartnetworks and the Blockchain EconomyMelanie Swan
 
Cs231n 2017 lecture12 Visualizing and Understanding
Cs231n 2017 lecture12 Visualizing and UnderstandingCs231n 2017 lecture12 Visualizing and Understanding
Cs231n 2017 lecture12 Visualizing and UnderstandingYanbin Kong
 
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...Association for Computational Linguistics
 

Viewers also liked (20)

Blockchain Smartnetworks: Bitcoin and Blockchain Explained
Blockchain Smartnetworks: Bitcoin and Blockchain ExplainedBlockchain Smartnetworks: Bitcoin and Blockchain Explained
Blockchain Smartnetworks: Bitcoin and Blockchain Explained
 
Construisons ensemble le chatbot bancaire dedemain !
Construisons ensemble le chatbot bancaire dedemain !Construisons ensemble le chatbot bancaire dedemain !
Construisons ensemble le chatbot bancaire dedemain !
 
Cs231n 2017 lecture13 Generative Model
Cs231n 2017 lecture13 Generative ModelCs231n 2017 lecture13 Generative Model
Cs231n 2017 lecture13 Generative Model
 
Exploring Session Context using Distributed Representations of Queries and Re...
Exploring Session Context using Distributed Representations of Queries and Re...Exploring Session Context using Distributed Representations of Queries and Re...
Exploring Session Context using Distributed Representations of Queries and Re...
 
Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...
Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...
Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...
 
Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...
Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...
Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...
 
Deep Learning for Chatbot (1/4)
Deep Learning for Chatbot (1/4)Deep Learning for Chatbot (1/4)
Deep Learning for Chatbot (1/4)
 
Roee Aharoni - 2017 - Towards String-to-Tree Neural Machine Translation
Roee Aharoni - 2017 - Towards String-to-Tree Neural Machine TranslationRoee Aharoni - 2017 - Towards String-to-Tree Neural Machine Translation
Roee Aharoni - 2017 - Towards String-to-Tree Neural Machine Translation
 
Cs231n 2017 lecture11 Detection and Segmentation
Cs231n 2017 lecture11 Detection and SegmentationCs231n 2017 lecture11 Detection and Segmentation
Cs231n 2017 lecture11 Detection and Segmentation
 
Cs231n 2017 lecture10 Recurrent Neural Networks
Cs231n 2017 lecture10 Recurrent Neural NetworksCs231n 2017 lecture10 Recurrent Neural Networks
Cs231n 2017 lecture10 Recurrent Neural Networks
 
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
 
Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop
Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 WorkshopSatoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop
Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop
 
Care your Child
Care your ChildCare your Child
Care your Child
 
Deep Learning for Chatbot (3/4)
Deep Learning for Chatbot (3/4)Deep Learning for Chatbot (3/4)
Deep Learning for Chatbot (3/4)
 
Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...
Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...
Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...
 
Chenchen Ding - 2015 - NICT at WAT 2015
Chenchen Ding - 2015 - NICT at WAT 2015Chenchen Ding - 2015 - NICT at WAT 2015
Chenchen Ding - 2015 - NICT at WAT 2015
 
Using Text Embeddings for Information Retrieval
Using Text Embeddings for Information RetrievalUsing Text Embeddings for Information Retrieval
Using Text Embeddings for Information Retrieval
 
State of Blockchain 2017: Smartnetworks and the Blockchain Economy
State of Blockchain 2017:  Smartnetworks and the Blockchain EconomyState of Blockchain 2017:  Smartnetworks and the Blockchain Economy
State of Blockchain 2017: Smartnetworks and the Blockchain Economy
 
Cs231n 2017 lecture12 Visualizing and Understanding
Cs231n 2017 lecture12 Visualizing and UnderstandingCs231n 2017 lecture12 Visualizing and Understanding
Cs231n 2017 lecture12 Visualizing and Understanding
 
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...
 

Similar to Deep Learning Explained

Philosophy of Deep Learning
Philosophy of Deep LearningPhilosophy of Deep Learning
Philosophy of Deep LearningMelanie Swan
 
Deep learning: challenges and applications
Deep learning: challenges and  applicationsDeep learning: challenges and  applications
Deep learning: challenges and applicationsAboul Ella Hassanien
 
Future of AI: Blockchain & Deep Learning
Future of AI: Blockchain & Deep LearningFuture of AI: Blockchain & Deep Learning
Future of AI: Blockchain & Deep LearningMelanie Swan
 
Philosophy of Deep Learning
Philosophy of Deep LearningPhilosophy of Deep Learning
Philosophy of Deep LearningMelanie Swan
 
Introduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolutionIntroduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolutionDarian Frajberg
 
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...Melanie Swan
 
Smart Networks: Blockchain, Deep Learning, and Quantum Computing
Smart Networks: Blockchain, Deep Learning, and Quantum ComputingSmart Networks: Blockchain, Deep Learning, and Quantum Computing
Smart Networks: Blockchain, Deep Learning, and Quantum ComputingMelanie Swan
 
Deep learning 1.0 and Beyond, Part 1
Deep learning 1.0 and Beyond, Part 1Deep learning 1.0 and Beyond, Part 1
Deep learning 1.0 and Beyond, Part 1Deakin University
 
20140327 - Hashing Object Embedding
20140327 - Hashing Object Embedding20140327 - Hashing Object Embedding
20140327 - Hashing Object EmbeddingJacob Xu
 
Automatic Attendance System using Deep Learning Framework
Automatic Attendance System using Deep Learning FrameworkAutomatic Attendance System using Deep Learning Framework
Automatic Attendance System using Deep Learning FrameworkPinaki Ranjan Sarkar
 
Deep Learning and Watson Studio
Deep Learning and Watson StudioDeep Learning and Watson Studio
Deep Learning and Watson StudioSasha Lazarevic
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Model-Agnostic Meta-Learning for Fast Adaptation of Deep NetworksModel-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Model-Agnostic Meta-Learning for Fast Adaptation of Deep NetworksYoonho Lee
 
Big data analytics 1
Big data analytics 1Big data analytics 1
Big data analytics 1gauravsc36
 
Big Data in Learning Analytics - Analytics for Everyday Learning
Big Data in Learning Analytics - Analytics for Everyday LearningBig Data in Learning Analytics - Analytics for Everyday Learning
Big Data in Learning Analytics - Analytics for Everyday LearningStefan Dietze
 
Data-centric AI and the convergence of data and model engineering: opportunit...
Data-centric AI and the convergence of data and model engineering:opportunit...Data-centric AI and the convergence of data and model engineering:opportunit...
Data-centric AI and the convergence of data and model engineering: opportunit...Paolo Missier
 

Similar to Deep Learning Explained (20)

Philosophy of Deep Learning
Philosophy of Deep LearningPhilosophy of Deep Learning
Philosophy of Deep Learning
 
Deep learning: challenges and applications
Deep learning: challenges and  applicationsDeep learning: challenges and  applications
Deep learning: challenges and applications
 
Future of AI: Blockchain & Deep Learning
Future of AI: Blockchain & Deep LearningFuture of AI: Blockchain & Deep Learning
Future of AI: Blockchain & Deep Learning
 
Philosophy of Deep Learning
Philosophy of Deep LearningPhilosophy of Deep Learning
Philosophy of Deep Learning
 
Introduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolutionIntroduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolution
 
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...
 
Smart Networks: Blockchain, Deep Learning, and Quantum Computing
Smart Networks: Blockchain, Deep Learning, and Quantum ComputingSmart Networks: Blockchain, Deep Learning, and Quantum Computing
Smart Networks: Blockchain, Deep Learning, and Quantum Computing
 
Deep learning 1.0 and Beyond, Part 1
Deep learning 1.0 and Beyond, Part 1Deep learning 1.0 and Beyond, Part 1
Deep learning 1.0 and Beyond, Part 1
 
AI Science
AI Science AI Science
AI Science
 
20140327 - Hashing Object Embedding
20140327 - Hashing Object Embedding20140327 - Hashing Object Embedding
20140327 - Hashing Object Embedding
 
Automatic Attendance System using Deep Learning Framework
Automatic Attendance System using Deep Learning FrameworkAutomatic Attendance System using Deep Learning Framework
Automatic Attendance System using Deep Learning Framework
 
Deep Learning and Watson Studio
Deep Learning and Watson StudioDeep Learning and Watson Studio
Deep Learning and Watson Studio
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
Mini Project PPT
Mini Project PPTMini Project PPT
Mini Project PPT
 
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Model-Agnostic Meta-Learning for Fast Adaptation of Deep NetworksModel-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
 
Big data analytics 1
Big data analytics 1Big data analytics 1
Big data analytics 1
 
Big Data in Learning Analytics - Analytics for Everyday Learning
Big Data in Learning Analytics - Analytics for Everyday LearningBig Data in Learning Analytics - Analytics for Everyday Learning
Big Data in Learning Analytics - Analytics for Everyday Learning
 
BrightTALK - Semantic AI
BrightTALK - Semantic AI BrightTALK - Semantic AI
BrightTALK - Semantic AI
 
Deep learning
Deep learningDeep learning
Deep learning
 
Data-centric AI and the convergence of data and model engineering: opportunit...
Data-centric AI and the convergence of data and model engineering:opportunit...Data-centric AI and the convergence of data and model engineering:opportunit...
Data-centric AI and the convergence of data and model engineering: opportunit...
 

More from Melanie Swan

AI Health Agents: Longevity as a Service in the Web3 GenAI Quantum Revolution
AI Health Agents: Longevity as a Service in the Web3 GenAI Quantum RevolutionAI Health Agents: Longevity as a Service in the Web3 GenAI Quantum Revolution
AI Health Agents: Longevity as a Service in the Web3 GenAI Quantum RevolutionMelanie Swan
 
Quantum Intelligence: Responsible Human-AI Entities
Quantum Intelligence: Responsible Human-AI EntitiesQuantum Intelligence: Responsible Human-AI Entities
Quantum Intelligence: Responsible Human-AI EntitiesMelanie Swan
 
The Human-AI Odyssey: Homerian Aspirations towards Non-labor Identity
The Human-AI Odyssey: Homerian Aspirations towards Non-labor IdentityThe Human-AI Odyssey: Homerian Aspirations towards Non-labor Identity
The Human-AI Odyssey: Homerian Aspirations towards Non-labor IdentityMelanie Swan
 
AdS Biology and Quantum Information Science
AdS Biology and Quantum Information ScienceAdS Biology and Quantum Information Science
AdS Biology and Quantum Information ScienceMelanie Swan
 
Quantum Information Science and Quantum Neuroscience.ppt
Quantum Information Science and Quantum Neuroscience.pptQuantum Information Science and Quantum Neuroscience.ppt
Quantum Information Science and Quantum Neuroscience.pptMelanie Swan
 
Quantum Information
Quantum InformationQuantum Information
Quantum InformationMelanie Swan
 
Critical Theory of Silence
Critical Theory of SilenceCritical Theory of Silence
Critical Theory of SilenceMelanie Swan
 
Quantum-Classical Reality
Quantum-Classical RealityQuantum-Classical Reality
Quantum-Classical RealityMelanie Swan
 
Derrida-Hegel: Différance-Difference
Derrida-Hegel: Différance-DifferenceDerrida-Hegel: Différance-Difference
Derrida-Hegel: Différance-DifferenceMelanie Swan
 
The Quantum Mindset
The Quantum MindsetThe Quantum Mindset
The Quantum MindsetMelanie Swan
 
Blockchains in Space
Blockchains in SpaceBlockchains in Space
Blockchains in SpaceMelanie Swan
 
Complexity and Quantum Information Science
Complexity and Quantum Information ScienceComplexity and Quantum Information Science
Complexity and Quantum Information ScienceMelanie Swan
 
Quantum Blockchains
Quantum BlockchainsQuantum Blockchains
Quantum BlockchainsMelanie Swan
 
Quantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIs
Quantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIsQuantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIs
Quantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIsMelanie Swan
 
Art Theory: Two Cultures Synthesis of Art and Science
Art Theory: Two Cultures Synthesis of Art and ScienceArt Theory: Two Cultures Synthesis of Art and Science
Art Theory: Two Cultures Synthesis of Art and ScienceMelanie Swan
 
Quantum Computing Lecture 1: Basic Concepts
Quantum Computing Lecture 1: Basic ConceptsQuantum Computing Lecture 1: Basic Concepts
Quantum Computing Lecture 1: Basic ConceptsMelanie Swan
 

More from Melanie Swan (20)

AI Health Agents: Longevity as a Service in the Web3 GenAI Quantum Revolution
AI Health Agents: Longevity as a Service in the Web3 GenAI Quantum RevolutionAI Health Agents: Longevity as a Service in the Web3 GenAI Quantum Revolution
AI Health Agents: Longevity as a Service in the Web3 GenAI Quantum Revolution
 
AI Math Agents
AI Math AgentsAI Math Agents
AI Math Agents
 
Quantum Intelligence: Responsible Human-AI Entities
Quantum Intelligence: Responsible Human-AI EntitiesQuantum Intelligence: Responsible Human-AI Entities
Quantum Intelligence: Responsible Human-AI Entities
 
The Human-AI Odyssey: Homerian Aspirations towards Non-labor Identity
The Human-AI Odyssey: Homerian Aspirations towards Non-labor IdentityThe Human-AI Odyssey: Homerian Aspirations towards Non-labor Identity
The Human-AI Odyssey: Homerian Aspirations towards Non-labor Identity
 
AdS Biology and Quantum Information Science
AdS Biology and Quantum Information ScienceAdS Biology and Quantum Information Science
AdS Biology and Quantum Information Science
 
Space Humanism
Space HumanismSpace Humanism
Space Humanism
 
Quantum Information Science and Quantum Neuroscience.ppt
Quantum Information Science and Quantum Neuroscience.pptQuantum Information Science and Quantum Neuroscience.ppt
Quantum Information Science and Quantum Neuroscience.ppt
 
Quantum Information
Quantum InformationQuantum Information
Quantum Information
 
Critical Theory of Silence
Critical Theory of SilenceCritical Theory of Silence
Critical Theory of Silence
 
Quantum-Classical Reality
Quantum-Classical RealityQuantum-Classical Reality
Quantum-Classical Reality
 
Derrida-Hegel: Différance-Difference
Derrida-Hegel: Différance-DifferenceDerrida-Hegel: Différance-Difference
Derrida-Hegel: Différance-Difference
 
Quantum Moreness
Quantum MorenessQuantum Moreness
Quantum Moreness
 
Crypto Jamming
Crypto JammingCrypto Jamming
Crypto Jamming
 
The Quantum Mindset
The Quantum MindsetThe Quantum Mindset
The Quantum Mindset
 
Blockchains in Space
Blockchains in SpaceBlockchains in Space
Blockchains in Space
 
Complexity and Quantum Information Science
Complexity and Quantum Information ScienceComplexity and Quantum Information Science
Complexity and Quantum Information Science
 
Quantum Blockchains
Quantum BlockchainsQuantum Blockchains
Quantum Blockchains
 
Quantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIs
Quantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIsQuantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIs
Quantum Neuroscience: CRISPR for Alzheimer’s, Connectomes & Quantum BCIs
 
Art Theory: Two Cultures Synthesis of Art and Science
Art Theory: Two Cultures Synthesis of Art and ScienceArt Theory: Two Cultures Synthesis of Art and Science
Art Theory: Two Cultures Synthesis of Art and Science
 
Quantum Computing Lecture 1: Basic Concepts
Quantum Computing Lecture 1: Basic ConceptsQuantum Computing Lecture 1: Basic Concepts
Quantum Computing Lecture 1: Basic Concepts
 

Recently uploaded

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxAbida Shariff
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?Mark Billinghurst
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfFIDO Alliance
 
Buy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptxBuy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptxEasyPrinterHelp
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyUXDXConf
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeCzechDreamin
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKUXDXConf
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityScyllaDB
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024Stephanie Beckett
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutesconfluent
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...FIDO Alliance
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfFIDO Alliance
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Julian Hyde
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIES VE
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlPeter Udo Diehl
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastUXDXConf
 
Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024TopCSSGallery
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfFIDO Alliance
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoTAnalytics
 

Recently uploaded (20)

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Buy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptxBuy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptx
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 

Deep Learning Explained

  • 1. Melanie Swan Philosophy Department, Purdue University melanie@BlockchainStudies.org Deep Learning Explained The future of Smart Networks Boulder Futurists: Solid State Depot Hackspace Boulder CO, August 12, 2017 Slides: http://slideshare.net/LaBlogga Image credit: Nvidia
  • 2. 12 Aug 2017 Deep Learning 1 Melanie Swan, Technology Theorist  Philosophy and Economic Theory, Purdue University, Indiana, USA  Founder, Institute for Blockchain Studies  Singularity University Instructor; Institute for Ethics and Emerging Technology Affiliate Scholar; EDGE Essayist; FQXi Advisor Traditional Markets Background Economics and Financial Theory Leadership New Economies research group Source: http://www.melanieswan.com, http://blockchainstudies.org/NSNE.pdf, http://blockchainstudies.org/Metaphilosophy_CFP.pdf https://www.facebook.com/groups/NewEconomies
  • 3. 12 Aug 2017 Deep Learning Agenda  Deep Learning  Definition  Technical details  Applications  Deep Qualia: Deep Learning and the Brain  Smart Network Convergence Theory  Conclusion 2 Image Source: http://www.opennn.net
  • 4. 12 Aug 2017 Deep Learning Deep Learning vocabulary What do these terms mean?  Deep Learning, Machine Learning, Artificial Intelligence  Perceptron, Artificial Neuron, Logit  Deep Belief Net, Artificial Neural Net, Boltzmann Machine  Google DeepDream, Google Brain, Google DeepMind  Supervised and Unsupervised Learning  Convolutional Neural Nets  Recurrent NN & LSTM (Long Short Term Memory)  Activation Function ReLU (Rectified Linear Unit)  Deep Learning libraries and frameworks  TensorFlow, Caffe, Theano, Torch, DL4J  Backpropagation, gradient descent, loss function 3
  • 5. 12 Aug 2017 Deep Learning 4 Conceptual Definition: Deep learning is a computer program that can identify what something is Technical Definition: Deep learning is a class of machine learning algorithms in the form of a neural network that uses a cascade of layers (tiers) of processing units to extract features from data and make predictive guesses about new data Source: Extending Jann LeCun, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun- on-deep-learning
  • 6. 12 Aug 2017 Deep Learning Deep Learning Theory  System is “dumb” (i.e. mechanical)  “Learns” with big data (lots of input examples) and trial-and-error guesses to adjust weights and bias to establish key features  Creates a predictive system to identity new examples  Same AI argument: big enough data is what makes a difference (“simple” algorithms run over large data sets) 5 Input: Big Data (e.g.; many examples) Method: Trial-and-error guesses to adjust node weights Output: system identifies new examples
  • 7. 12 Aug 2017 Deep Learning Sample task: is that a Car?  Create an image recognition system that determines which features are relevant (at increasingly higher levels of abstraction) and correctly identifies new examples 6 Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
  • 8. 12 Aug 2017 Deep Learning Broader Computer Science Context 7 Source: Machine Learning Guide, 9. Deep Learning  Within the Computer Science discipline, in the field of Artificial Intelligence, Deep Learning is a class of Machine Learning algorithms, that are in the form of a Neural Network
  • 9. 12 Aug 2017 Deep Learning Statistical Mechanics Deep Learning is inspired by Physics 8  Sigmoid function suggested as a model for neurons, per statistical mechanical behavior (Jack Cowan)  Stationary solutions for dynamic models (asymmetric weights create an oscillator to model neuron signaling)  Hopfield Neural Network: content-addressable memory system with binary threshold nodes, converges to a local minimum (John Hopfield)  Can use an Ising model (of ferromagnetism) for neurons  Restricted Boltzmann Machine (Geoffrey Hinton)  Studied in theoretical physics, condensed matter field theory; Statistical Mechanics concepts: Renormalization, Boltzmann Distribution, Free Energies, Gibbs Sampling Source: https://www.quora.com/Is-deep-learning-related-to-statistical-physics-particularly-network-science
  • 10. 12 Aug 2017 Deep Learning What is a Neural Net? 9  Motivation: create an Artificial Neural Network to solve problems the same way a human brain would
  • 11. 12 Aug 2017 Deep Learning What is a Neural Net? 10  Structure: input-processing-output  Mimic neuronal signal firing structure of brain with computational processing units Source: https://www.slideshare.net/ThomasDaSilvaPaula/an-introduction-to-machine-learning-and-a-little-bit-of-deep-learning, http://cs231n.github.io/convolutional-networks/
  • 12. 12 Aug 2017 Deep Learning What is an Artificial Neural Network?  Collection of connected units called artificial neurons (analogous to axons in a biological brain)  Organized in layers of signaling cascades  Each neuron transmits a signal to another neuron  Neurons may have state  Represented by a number between 0 and 1  Variable parameters  Neurons may have a weight that varies as learning proceeds, which can increase or decrease the strength of the signal that it sends downstream  Neurons may have a threshold (bias) such that only if the aggregate signal is below (or above) that level is the downstream signal sent 11
  • 13. 12 Aug 2017 Deep Learning Why is it called Deep Learning?  Deep: Hidden layers (cascading tiers) of processing  “Deep” networks (3+ layers) versus “shallow” (1-2 layers)  Learning: Algorithms “learn” from data by modeling features and updating probability weights assigned to feature nodes in testing how relevant specific features are in determining the general type of item 12 Deep: Hidden processing layers Learning: Updating probability weights re: feature importance
  • 14. 12 Aug 2017 Deep Learning Supervised and Unsupervised Learning  Supervised (classify labeled data)  Unsupervised (find patterns in unlabeled data) 13 Source: https://www.slideshare.net/ThomasDaSilvaPaula/an-introduction-to-machine-learning-and-a-little-bit-of-deep-learning
  • 15. 12 Aug 2017 Deep Learning Early success in Supervised Learning (2011)  YouTube: user-classified data perfect for Supervised Learning 14 Source: Google Brain: Le, QV, Dean, Jeff, Ng, Andrew, et al. 2012. Building high-level features using large scale unsupervised learning. https://arxiv.org/abs/1112.6209
  • 16. 12 Aug 2017 Deep Learning 2 main kinds of Deep Learning neural nets 15 Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ  Convolutional Neural Nets  Image recognition  Convolve: roll up to higher levels of abstraction in feature sets  Recurrent Neural Nets  Speech, text, audio recognition  Recur: iterate over sequential inputs with a memory function  LSTM (Long Short-Term Memory) remembers sequences and avoids gradient vanishing
  • 17. 12 Aug 2017 Deep Learning Image Recognition and Computer Vision 16 Source: Quoc Le, https://arxiv.org/abs/1112.6209; Yann LeCun, NIPS 2016, https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view Marv Minsky, 1966 “summer project” Jeff Hawkins, 2004, Hierarchical Temporal Memory (HTM) Quoc Le, 2011, Google Brain cat recognition Convolutional net for autonomous driving, http://cs231n.github.io/convolutional-networks/ History Current state of the art - 2017
  • 18. 12 Aug 2017 Deep Learning Progression in AI Deep Learning machines 17 Single-purpose AI: Hard-coded rules Multi-purpose AI: Algorithm detects rules, reusable template Question-answering AI: Natural-language processing Deep Learning prototypeHard-coded AI machine Deep Learning machine Deep Blue, 1997 Watson, 2011 AlphaGo, 2016
  • 19. 12 Aug 2017 Deep Learning Why do we need Deep Learning? 18  A contemporary data science method to keep up with the growth in data, older learning algorithms no longer performing Source: http://blog.algorithmia.com/introduction-to-deep-learning-2016
  • 20. 12 Aug 2017 Deep Learning Agenda  Deep Learning  Definition  Technical details  Applications  Deep Qualia: Deep Learning and the Brain  Smart Network Convergence Theory  Conclusion 19 Image Source: http://www.opennn.net
  • 21. 12 Aug 2017 Deep Learning 3 Key Technical Principles of Deep Learning 20 Reduce combinatoric dimensionality Core processing unit (input-processing-output) Levers: weights and bias Squash values into Sigmoidal S-curve -Binary values (Y/N, 0/1) -Probability values (0 to 1) -Tanh values 9(-1) to 1) Loss FunctionPerceptron StructureSigmoid Function “Dumb” system learns by adjusting parameters and checking against outcome Loss function optimizes efficiency of solution Non-linear formulation as a logistic regression problem means greater mathematical manipulation What Why
  • 22. 12 Aug 2017 Deep Learning Linear Regression 21 House price vs. Size (square feet) y=mx+b House price Size (square feet) Source: https://www.statcrunch.com/5.0/viewreport.php?reportid=5647
  • 23. 12 Aug 2017 Deep Learning Logistic Regression 22 Source: http://www.simafore.com/blog/bid/99443/Understand-3-critical-steps-in-developing-logistic-regression-models
  • 24. 12 Aug 2017 Deep Learning Logistic Regression 23  Higher-order mathematical formulation  Sigmoid function  S-shaped and bounded  Maps the whole real axis into a finite interval (0-1)  Non-linear  Can fit probability  Can apply optimization techniques  Deep Learning classification predictions are in the form of a probability value Source: https://www.quora.com/Logistic-Regression-Why-sigmoid-function Sigmoid Function Unit Step Function
  • 25. 12 Aug 2017 Deep Learning Sigmoid function: Taleb 24 Source: http://www.fooledbyrandomness.com/medicine.pdf  Thesis: if can map a phenomenon onto a sigmoid curve (“convexify” it), then can control its risk  Antifragility = convexity = risk-manageable  Fragility = concavity  Non-linearity of dose-response in medicine, and therefore suggested treatment optimality
  • 26. 12 Aug 2017 Deep Learning Regression  Logistic regression  Predict binary outcomes:  Perceptron (0 or 1)  Predict probabilities:  Sigmoid Neuron (values 0-1)  Tanh Hyperbolic Tangent Neuron (values (-1)-1) 25 Logistic Regression (Sigmoid function) (0-1) or Tanh ((-1)-1) Linear Regression  Linear regression  Predict continuous set of values (house prices)
  • 27. 12 Aug 2017 Deep Learning Deep Learning Architecture 26 Source: Michael A. Nielsen, Neural Networks and Deep Learning
  • 28. 12 Aug 2017 Deep Learning Processing Unit, Perceptron, Neuron 27 Source: http://deeplearning.stanford.edu/tutorial 1. Input 2. Hidden layers 3. Output X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X  Unit (processing unit, logistic regression unit), perceptron (“multilayer perceptron”), artificial neuron
  • 29. 12 Aug 2017 Deep Learning Example: Image recognition 1. Obtain training data set 2. Digitize pixels (convert images to numbers)  Divide image into 28x28 grid, assign a value (0-255) to each square based on brightness 3. Read into vector (array; list of numbers)  28x28 = 784 elements per image) 28 Source: Quoc V. Le, A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm, 2015, Google Brain, https://cs.stanford.edu/~quocle/tutorial1.pdf
  • 30. 12 Aug 2017 Deep Learning Deep Learning Architecture 4. Load spreadsheet of vectors into deep learning system  Each row of spreadsheet (784-element array) is an input 29 Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist 1. Input 2. Hidden layers 3. Output X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X Vector data 784-element array
  • 31. 12 Aug 2017 Deep Learning What happens in the Hidden Layers? 30 Source: Michael A. Nielsen, Neural Networks and Deep Learning  First layer learns primitive features (line, edge, tiniest unit of sound) by finding combinations of the input vector data that occur more frequently than by chance  Logistic regression performed and encoded at each processing node (Y/N (0,1)), does this example have this feature?  Feeds these basic features to next layer, which trains itself to recognize slightly more complicated features (corner, combination of speech sounds)  Feeds features to new layers until recognizes full objects
  • 32. 12 Aug 2017 Deep Learning Image Recognition Higher Abstractions of Feature Recognition 31 Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf Edges Object Parts (combinations of edges) Object Models
  • 33. 12 Aug 2017 Deep Learning Speech, Text, Audio Recognition Sequence-to-sequence Recognition + LSTM 32 Source: Andrew Ng
  • 34. 12 Aug 2017 Deep Learning Example: NVIDIA Facial Recognition 33 Source: Nvidia  First hidden layer extracts all possible low-level features from data (lines, edges, contours); next layers abstract into more complex features of possible relevance
  • 35. 12 Aug 2017 Deep Learning Deep Learning 34 Source: Quoc V. Le et al, Building high-level features using large scale unsupervised learning, 2011, https://arxiv.org/abs/1112.6209
  • 36. 12 Aug 2017 Deep Learning Deep Learning Architecture 35 Source: Michael A. Nielsen, Neural Networks and Deep Learning 1. Input 2. Hidden layers 3. Output (0,1)
  • 37. 12 Aug 2017 Deep Learning Mathematical methods update weights 36 1. Input 2. Hidden layers 3. Output X X X X X X X X X X X X X X X Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist  Linear algebra: matrix multiplications of input vectors  Statistics: logistic regression units (Y/N (0,1)), probability weighting and updating, inference for outcome prediction  Calculus: optimization (minimization), gradient descent in back-propagation to avoid local minima with saddle points Feed-forward pass (0,1) 0.5 Backward pass to update probabilities .5.5 .5.5.5 0 01 .75 .25 Inference Guess Actual
  • 38. 12 Aug 2017 Deep Learning More complicated in actual use  Convolutional neural net scale-up for number recognition  Example data: MNIST dataset  http://yann.lecun.com/exdb/mnist 37 Source: http://www.kdnuggets.com/2016/04/deep-learning-vs-svm-random-forest.html
  • 39. 12 Aug 2017 Deep Learning Structure of a Node: Computation Graph 38 Edge (input value) Architecture Node (operation) Edge (input value) Edge (output value) Example 1 3 4 Add ?? Example 2 3 4 Mult ??
  • 40. 12 Aug 2017 Deep Learning Neural net unit: perceptron, neuron, node 39 Source: http://neuralnetworksanddeeplearning.com/chap1.html (0,1) (0,1) (0,1) (0,1) Oper- ation  Sigmoid function means all inputs and outputs in the system are (0,1)
  • 41. 12 Aug 2017 Deep Learning Other parameters: weights and bias 40 Source: http://neuralnetworksanddeeplearning.com/chap1.html Values have Weights Operation node has Bias W1 = (-2) B=3 W2 = (-2)  Weight and bias are variable parameters that get adjusted as the system iterates and learns Values have Weights Operation node has Bias W1 = (-2) B=3 W2 = (-2) = 0 (-2)*0 + (-2)*0 + 3 = 3 = 0 Output = 0 0,0 0,1 1,0 1,1 (-2)*1 + (-2)*1 + 3 = (-1) (-2)*0 + (-2)*1 + 3 = 1 (-2)*1 + (-2)*0 + 3 = 1 W1*X1 + W2*X2 + Bias = n Output (0,1)Input (0,1) X1, X2 Weight and Bias are “randomly” assigned at the beginning: (here (-2) and 3) Mimics NAND gate 1 1 1 0
  • 42. 12 Aug 2017 Deep Learning Actual: same structure, more complicated 41
  • 43. 12 Aug 2017 Deep Learning Neural net: massive scale-up of nodes 42 Source: http://neuralnetworksanddeeplearning.com/chap1.html
  • 44. 12 Aug 2017 Deep Learning Same Structure 43
  • 45. 12 Aug 2017 Deep Learning How does the neural net actually learn?  Vary the weights and biases to see if a better outcome is obtained  Repeat until the net correctly classifies the data 44 Source: http://neuralnetworksanddeeplearning.com/chap2.html  Structural system based on cascading layers of neurons with variable parameters: weight and bias
  • 46. 12 Aug 2017 Deep Learning Backpropagation  Problem: Inefficient to test the combinatorial explosion of all possible parameter variations  Solution: Backpropagation (1986 Nature paper)  Backpropagation is an optimization method used to calculate the error contribution of each neuron after a batch of data is processed 45 Source: http://neuralnetworksanddeeplearning.com/chap2.html
  • 47. 12 Aug 2017 Deep Learning Backpropagation of error  Calculate the total error  Calculate the contribution to the error at each step going backwards  Variety of Error Calculation methods: Mean Square Error (MSE), sum of squared errors of prediction (SSE), Cross- Entropy (Softmax), Softplus 46
  • 48. 12 Aug 2017 Deep Learning Backpropagation  Heart of Deep Learning  Backpropagation: algorithm dynamically calculates the gradient (derivative) of the loss function with respect to the weights in a network to find the minimum and optimize the function from there  Algorithms optimize the performance of the network by adjusting the weights, e.g.; in the gradient descent algorithm  Error and gradient are computed for each node  Intermediate errors transmitted backwards through the network (backpropagation)  Objective: optimize the weights so that the neural network can learn how to correctly map arbitrary inputs to outputs 47 Source: http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4, https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
  • 49. 12 Aug 2017 Deep Learning Gradient Descent  Gradient: derivative to find the minimum of a function  Gradient descent: optimization algorithm to find the biggest errors (minima) most quickly  Error = MSE, log loss, cross-entropy; e.g.; least correct predictions to correctly identify data 48 Source: http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4
  • 50. 12 Aug 2017 Deep Learning  Optimization Technique  Mathematical tool used in statistics, finance, decision theory, biological modeling, computational neuroscience  State as non-linear equation to optimize  Minimize loss or cost  Maximize reward, utility, profit, or fitness  Loss function links instance of an event to its cost  Accident (event) means $1,000 damage on average (cost)  5 cm height (event) confers 5% fitness advantage (reward)  Deep learning: system feedback loop  Use penalty cost for incorrect classifications to train system  CNN (classification): cross-entropy; RNN (regression): MSE Loss Function 49 Laplace
  • 51. 12 Aug 2017 Deep Learning Overfitting  Regularization  Introduce additional information such as a lambda parameter in the cost function (to update the theta parameters in the gradient descent algorithm)  Dropout: prevent complex adaptations on training data by dropping out units (both hidden and visible)  Test new datasets 50
  • 52. 12 Aug 2017 Deep Learning Research Topics  Layer depth vs. height (1x9, 3x3, etc.); L1/2 slow-downs  Backpropagation, gradient descent, loss function  Saddle-free optimization, vanishing gradients  Composition of non-linearities  Non-parametric manifold learning, auto-encoders  Activation maximization  Synthesizing preferred inputs for neurons 51 Source: http://cs231n.github.io/convolutional-networks, https://arxiv.org/abs/1605.09304, https://www.iro.umontreal.ca/~bengioy/talks/LondonParisMeetup_15April2015.pdf
  • 53. 12 Aug 2017 Deep Learning Advanced Deep Learning Architectures 52 Source: http://prog3.com/sbdm/blog/zouxy09/article/details/8781396  Deep Belief Network  Connections between layers not units  Establish weighting guesses for processing units before run deep learning system  Used to pre-train systems to assign initial probability weights (more efficient)  Deep Boltzmann Machine  Stochastic recurrent neural network  Runs learning on internal representations  Represent and solve combinatoric problems Deep Boltzmann Machine Deep Belief Network
  • 54. 12 Aug 2017 Deep Learning Convolutional net: Image Enhancement  Google DeepDream: Convolutional neural network enhances (potential) patterns in images; deliberately over-processing images 53 Source: Georges Seurat, Un dimanche après-midi à l'Île de la Grande Jatte, 1884-1886; http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil722; Google DeepDream uses algorithmic pareidolia (seeing an image when none is present) to create a dream-like hallucinogenic appearance
  • 55. 12 Aug 2017 Deep Learning Hardware and Software Tools 54
  • 56. 12 Aug 2017 Deep Learning Deep Learning frameworks and libraries 55 Source: http://www.infoworld.com/article/3163525/analytics/review-the-best-frameworks-for-machine-learning-and-deep- learning.html#tk.ifw-ifwsb
  • 57. 12 Aug 2017 Deep Learning What is TensorFlow? 56 Source: https://www.youtube.com/watch?v=uHaKOFPpphU Python code invoking TensorFlowTensorBoard (TensorFlow) visualization Computation graph Design in TensorFlow  Google’s open-source machine learning library  “Tensor” = multidimensional arrays used in NN operations
  • 58. 12 Aug 2017 Deep Learning Hardware  Advances in chip design  GPU chips (graphics processing unit): 3D graphics cards designed to do fast matrix multiplication  Google TPU chip (tensor processing unit): custom ASICs for machine learning, used in AlphaGo  TPUs process matrix multiplications without storing intermediate values in memory  NVIDIA DGX-1 integrated deep learning system  Eight Tesla P100 GPU accelerators 57 Google TPU chip (Tensor Processing Unit), 2016 Source: http://www.techradar.com/news/computing-components/processors/google-s-tensor-processing-unit-explained-this-is-what- the-future-of-computing-looks-like-1326915 NVIDIA DGX-1 Deep Learning System
  • 59. 12 Aug 2017 Deep Learning USB and Browser-based Machine Learning  Intel: Movidius Visual Processing Unit (VPU): USB ML for IOT  Security cameras, industrial equipment, robots, drones  Apple: ML acquisition Turi (Dato)  Browser-based Deep Learning  ConvNetJS; TensorFire  Javascript library to run Deep Learning (Neural Networks) in a browser  Smart Network in a browser  JavaScript Deep Learning  Blockchain EtherWallets 58 Source: http://cs.stanford.edu/people/karpathy/convnetjs/, http://www.infoworld.com/article/3212884/machine-learning/machine-learning- comes-to-your-browser-via-javascript.html
  • 60. 12 Aug 2017 Deep Learning How big are Deep Learning neural nets?  Google Deep Brain cat recognition, 2011  1 billion connections, 10 million images (200x200 pixel), 1,000 machines (16,000 cores), 3 days, each instantiation of the network spanned 170 servers, and 20,000 object categories  State of the art, 2016-2017  NVIDIA facial recognition, 100 million images, 10 layers, 1 bn parameters, 30 exaflops, 30 GPU days  Google, 11.2-billion parameter system  Lawrence Livermore Lab, 15-billion parameter system  Digital Reasoning, cognitive computing (Nashville TN), 160 billion parameters, trained on three multi-core computers overnight 59 Source: https://futurism.com/biggest-neural-network-ever-pushes-ai-deep-learning, Digital Reasoning paper: https://arxiv.org/pdf/1506.02338v3.pdf
  • 61. 12 Aug 2017 Deep Learning Agenda  Deep Learning  Definition  Technical details  Applications  Deep Qualia: Deep Learning and the Brain  Smart Network Convergence Theory  Conclusion 60 Image Source: http://www.opennn.net
  • 62. 12 Aug 2017 Deep Learning Applications: Cats to Cancer to Cognition 61 Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ Computational imaging: Machine learning for 3D microscopy https://www.nature.com/nature/journal/v523/n7561/full/523416a.html
  • 63. 12 Aug 2017 Deep Learning Tumor Image Recognition 62 Source: https://www.nature.com/articles/srep24454  Computer-Aided Diagnosis with Deep Learning Architecture  Breast tissue lesions in images and pulmonary nodules in CT Scans
  • 64. 12 Aug 2017 Deep Learning Melanoma Image Recognition 63 Source: http://www.nature.com/nature/journal/v542/n7639/full/nature21056.html
  • 65. 12 Aug 2017 Deep Learning DIY Image Recognition: use Contrast 64 Source: https://developer.clarifai.com/modelshttps://developer.clarifai.com/models How many orange pixels? Apple or Orange? Melanoma risk or healthy skin? Degree of contrast in photo colors?
  • 66. 12 Aug 2017 Deep Learning Deep Learning and Genomics  Large classes of hypothesized but unknown correlations  Genotype-phenotype disease linkage unknown  Computer-identifiable patterns in genomic data  CNN: genome symmetries; RNN: textual analysis 65 Source: http://ieeexplore.ieee.org/document/7347331
  • 67. 12 Aug 2017 Deep Learning Deep Learning and the Brain 66
  • 68. 12 Aug 2017 Deep Learning  Deep learning neural networks are inspired by the structure of the cerebral cortex  The processing unit, perceptron, artificial neuron is the mathematical representation of a biological neuron  In the cerebral cortex, there can be several layers of interconnected perceptrons 67 Deep Qualia machine? General purpose AI Mutual inspiration of neurological and computing research
  • 69. 12 Aug 2017 Deep Learning Deep Qualia machine?  Visual cortex is hierarchical with intermediate layers  The ventral (recognition) pathway in the visual cortex has multiple stages: Retina - LGN - V1 - V2 - V4 - PIT – AIT  Human brain simulation projects  Swiss Blue Brain project, European Human Brain Project 68 Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
  • 70. 12 Aug 2017 Deep Learning Social Impact of Deep Learning  WHO estimates 400 million people without access to essential health services  6% in extreme poverty due to healthcare costs  Next leapfrog technology: Deep Learning  Last-mile build out of brick-and-mortar clinics does not make sense in era of digital medicine  Medical diagnosis via image recognition, natural language processing symptoms description  Convergence Solution: Digital Health Wallet  Deep Learning medical diagnosis + Blockchain- based EMRs (electronic medical records)  Empowerment Effect: Deep learning = “tool I use,” not hierarchically “doctor-administered” 69 Source: http://www.who.int/mediacentre/news/releases/2015/uhc-report/en/ Digital Health Wallet: Deep Learning diagnosis Blockchain-based EMRs
  • 71. 12 Aug 2017 Deep Learning Agenda  Deep Learning  Definition  Technical details  Applications  Deep Qualia: Deep Learning and the Brain  Smart Network Convergence Theory  Conclusion 70 Image Source: http://www.opennn.net
  • 72. 12 Aug 2017 Deep Learning 71 Better horse AND new car New Technology
  • 73. 12 Aug 2017 Deep Learning 72 Smart networks are computing networks with intelligence built in such that identification and transfer is performed by the network itself through protocols that automatically identify (deep learning), and validate, confirm, and route transactions (blockchain) within the network Smart Network Convergence Theory
  • 74. 12 Aug 2017 Deep Learning Smart Network Convergence Theory  Network intelligence “baked in” to smart networks  Deep Learning algorithms for predictive identification  Blockchains to transfer value, confirm authenticity 73 Source: Expanded from Mark Sigal, http://radar.oreilly.com/2011/10/post-pc-revolution.html Two Fundamental Eras of Network Computing
  • 75. 12 Aug 2017 Deep Learning 74 Blockchain is the tamper-resistant distributed ledger software underlying cryptocurrencies such as Bitcoin, for the secure transfer of money, assets, and information via the Internet without a third- party intermediary Source: http://www.amazon.com/Bitcoin-Blueprint-New-World-Currency/dp/1491920491
  • 76. 12 Aug 2017 Deep Learning Blockchain Deep Learning nets  Provide increasingly sophisticated automated network computational infrastructure  Make predictive guesses of reality states of the world  Predictive inference (deep learning) and cryptographic nonce- guesses (blockchain)  Instantiate decentralization  Hierarchical models do not scale 75
  • 77. 12 Aug 2017 Deep Learning Next Phase  Put Deep Learning systems on the Internet  Deep Learning Blockchain Networks  Combine Deep Learning and Blockchain Technology  Blockchain offers secure audit ledger of activity  Advanced computational infrastructure to tackle larger-scale problems  Genomic disease, protein modeling, energy storage, global financial risk assessment, voting, astronomical data 76
  • 78. 12 Aug 2017 Deep Learning Example: Autonomous Driving  Requires the smart network functionality of deep learning and blockchain  Deep Learning: identify what things are  Convolutional neural nets core element of machine vision system  Blockchain: secure automation technology  Track arbitrarily-many fleet units  Legal accountability  Software upgrades  Remuneration 77
  • 79. 12 Aug 2017 Deep Learning The Very Small Blockchain Deep Learning nets in Cells  Medical nanorobotics for cell repair  Deep Learning: identify what things are (diagnosis)  Blockchain: secure automation technology  Bio-cryptoeconomics: secure automation of medical nanorobotics for cell repair  Medical nanorobotics as coming-onboard repair platform for the human body  High number of agents and “transactions”  Identification and automation is obvious 78 Sources: Swan, M. Blockchain Thinking: The Brain as a DAC (Decentralized Autonomous Corporation). Technology and Society Magazine, IEEE 2015; 34(4): 41-52 , https://www.slideshare.net/lablogga/biocryptoeconomy-smart-contract-blockchainbased-bionano-repair-dacs
  • 80. 12 Aug 2017 Deep Learning The Very Large Blockchain Deep Learning nets in Space  Automated space construction bots/agents  Deep Learning: identify what things are (classification)  Blockchain: secure automation technology  Applications: asteroid mining, terraforming, radiation-monitoring, space-based solar power, debris tracking net 79
  • 81. 12 Aug 2017 Deep Learning Agenda  Deep Learning  Definition  Technical details  Applications  Deep Qualia: Deep Learning and the Brain  Smart Network Convergence Theory  Conclusion 80 Image Source: http://www.opennn.net
  • 82. 12 Aug 2017 Deep Learning Our human future 81  Are we doomed?
  • 83. 12 Aug 2017 Deep Learning Human-machine collaboration 82  Team-members excel at different things  Differently-abled agents in society Source: Swan, M. (2017). Is Technological Unemployment Real? In: Surviving the Machine Age. http://www.springer.com/us/book/9783319511641
  • 84. 12 Aug 2017 Deep Learning 83 Conceptual Definition: Deep learning is a computer program that can identify what something is Technical Definition: Deep learning is a class of machine learning algorithms in the form of a neural network that uses a cascade of layers (tiers) of processing units to extract features from data and make predictive guesses about new data Source: Extending Jann LeCun, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun- on-deep-learning
  • 85. 12 Aug 2017 Deep Learning Deep Learning Theory  System is “dumb” (i.e. mechanical)  “Learns” with big data (lots of input examples) and trial-and-error guesses to adjust weights and bias to establish key features  Creates a predictive system to identity new examples  Same AI argument: big enough data is what makes a difference (“simple” algorithms run over large data sets) 84 Input: Big Data (e.g.; many examples) Method: Trial-and-error guesses to adjust node weights Output: system identifies new examples
  • 86. 12 Aug 2017 Deep Learning 3 Key Technical Principles of Deep Learning 85 Reduce combinatoric dimensionality Core processing unit (input-processing-output) Levers: weights and bias Squash values into probability function (Sigmoid (0-1); Tanh ((-1)-1)) Loss FunctionPerceptron StructureSigmoid Function “Dumb” system learns by adjusting parameters and checking against outcome Loss function optimizes efficiency of solution Formulate as a logistic regression problem for greater mathematical manipulation What Why
  • 87. 12 Aug 2017 Deep Learning Conclusion  Next-generation global infrastructure: Deep Learning Blockchain Networks merging deep learning systems and blockchain technology  Smart Network Convergence Theory: pushing more complexity and automation through Internet pipes  Blockchain Deep Learning nets: Ability to identify what something is (machine learning) and securely verify and transact it (blockchain) 86
  • 88. 12 Aug 2017 Deep Learning  Neural Networks and Deep Learning, Michael Nielsen, http://neuralnetworksanddeeplearning.com/  Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, http://www.deeplearningbook.org/Machine learning and deep neural nets  Machine Learning Guide podcast, Tyler Renelle, http://ocdevel.com/podcasts/machine-learning  notMNIST dataset http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html  Metacademy; Fast.ai; Keras.io Resources 87 Distill (visual ML journal) http://distill.pubSource: http://cs231n.stanford.edu https://www.deeplearning.ai/
  • 89. Melanie Swan Philosophy Department, Purdue University melanie@BlockchainStudies.org Deep Learning Explained The future of Smart Networks Boulder Futurists: Solid State Depot Hackspace Boulder CO, August 12, 2017 Slides: http://slideshare.net/LaBlogga Image credit: Nvidia Thank You! Questions?
  • 90. 12 Aug 2017 Deep Learning Deep Learning Taxonomy 89 Source: Machine Learning Guide, 9. Deep Learning; AI (artificial intelligence) Machine learning Other methods Supervised learning (labeled data: classification) Unsupervised learning (unlabeled data: pattern recognition) Reinforcement learning Shallow learning (1-2 layers) Deep learning (5-20 layers) Recurrent nets (text, speech) Convolutional nets (images) Neural Nets (NN) Other methods Bayesian inference Support Vector Machines Decision trees K-means clustering K-nearest neighbor
  • 91. 12 Aug 2017 Deep Learning Kinds of Deep Learning Systems What Deep Learning net to choose? 90 Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ  Supervised algorithms (classify labeled data)  Image (object) recognition  Convolutional net (image processing), deep belief network, recursive neural tensor network  Text analysis (name recognition, sentiment analysis)  Recurrent net (iteration; character level text), recursive neural tensor network  Speech recognition  Recurrent net  Unsupervised algorithms (find patterns in unlabeled data)  Boltzmann machine or autoencoder