SlideShare a Scribd company logo

Structure learning with Deep Neural Networks

•
•
Patrick Michl
Patrick Michl

Conference talk about highly nonlinear association analysis in gene expression data by using deep neural networks. Given at 6th Network Modeling Workshop, Jena, Germany, 2013

1 of 75
Download to read offline
Structure learning with deep neuronal networks 6th Network Modeling Workshop, 6/6/2013 Patrick Michl
Page 26/6/2013 Patrick Michl Network Modeling Agenda Autoencoders Biological Model Validation & Implementation
Page 36/6/2013 Patrick Michl Network Modeling Real world data usually is high dimensional … x1 x2 Dataset Model Autoencoders
Page 46/6/2013 Patrick Michl Network Modeling … which makes structural analysis and modeling complicated! x1 x2 x1 x2 Dataset Model 𝐹(𝑥1, 𝑥2) ? Autoencoders
Page 56/6/2013 Patrick Michl Network Modeling Dimensionality reduction techinques like PCA … x1 x2 PCA Dataset Model Autoencoders
Page 66/6/2013 Patrick Michl Network Modeling … can not preserve complex structures! x1 x2 PCA Dataset Model x1 x2 𝑥2 = α𝑥1 + β Autoencoders
Page 76/6/2013 Patrick Michl Network Modeling Therefore the analysis of unknown structures … x1 x2 Dataset Model Autoencoders
Page 86/6/2013 Patrick Michl Network Modeling … needs more considerate nonlinear techniques! x1 x2 Dataset Model x1 x2 𝑥2 = 𝑓(𝑥1) Autoencoders
Page 96/6/2013 Patrick Michl Network Modeling Autoencoders are artificial neuronal networks … Autoencoder • Artificial Neuronal Network Autoencoders input data X output data X‘ Perceptrons Gaussian Units
Page 106/6/2013 Patrick Michl Network Modeling Autoencoders are artificial neuronal networks … Autoencoder • Artificial Neuronal Network Autoencoders input data X output data X‘ Perceptrons Gaussian Units Perceptron 1 0 Gauss Units R
Page 116/6/2013 Patrick Michl Network Modeling Autoencoders are artificial neuronal networks … Autoencoder • Artificial Neuronal Network Autoencoders input data X output data X‘ Perceptrons Gaussian Units
Page 126/6/2013 Patrick Michl Network Modeling Autoencoder • Artificial Neuronal Network • Multiple hidden layers Autoencoders … with multiple hidden layers. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers)
Page 136/6/2013 Patrick Michl Network Modeling Autoencoder • Artificial Neuronal Network • Multiple hidden layers Autoencoders Such networks are called deep networks. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers)
Page 146/6/2013 Patrick Michl Network Modeling Autoencoder • Artificial Neuronal Network • Multiple hidden layers Autoencoders Such networks are called deep networks. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers)Definition (deep network) Deep networks are artificial neuronal networks with multiple hidden layers
Page 156/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers) Such networks are called deep networks. • Deep network
Page 166/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders Autoencoders have a symmetric topology … Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers) • Deep network • Symmetric topology
Page 176/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … with an odd number of hidden layers. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers) • Deep network • Symmetric topology
Page 186/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders The small layer in the center works lika an information bottleneck input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck Bottleneck
Page 196/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders ... that creates a low dimensional code for each sample in the input data. input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck Bottleneck
Page 206/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders The upper stack does the encoding … input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck • Encoder Encoder
Page 216/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … and the lower stack does the decoding. input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck • Encoder • Decoder Encoder Decoder
Page 226/6/2013 Patrick Michl Network Modeling • Deep network • Symmetric topology • Information bottleneck • Encoder • Decoder Autoencoder Autoencoders … and the lower stack does the decoding. input data X output data X‘ Encoder Decoder Definition (deep network) Deep networks are artificial neuronal networks with multiple hidden layers Definition (autoencoder) Autoencoders are deep networks with a symmetric topology and an odd number of hiddern layers, containing a encoder, a low dimensional representation and a decoder.
Page 236/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders Autoencoders can be used to reduce the dimension of data … input data X output data X‘ Problem: dimensionality of data Idea: 1. Train autoencoder to minimize the distance between input X and output X‘ 2. Encode X to low dimensional code Y 3. Decode low dimensional code Y to output X‘ 4. Output X‘ is low dimensional
Page 246/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … if we can train them! input data X output data X‘ Problem: dimensionality of data Idea: 1. Train autoencoder to minimize the distance between input X and output X‘ 2. Encode X to low dimensional code Y 3. Decode low dimensional code Y to output X‘ 4. Output X‘ is low dimensional
Page 256/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders In feedforward ANNs backpropagation is a good approach. input data X output data X‘ Training Backpropagation
Page 266/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function 𝑋′ = 𝐹 𝑋 error = 𝑋′2 − 𝑌 In feedforward ANNs backpropagation is a good approach.
Page 276/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders In feedforward ANNs backpropagation is the choice input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function Example (linear neuronal unit with two inputs)
Page 286/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function (2) By calculating −𝛻𝑒𝑟𝑟𝑜𝑟 we get a vector that shows in a direction which decreases the error (3) We update the parameters to decrease the error In feedforward ANNs backpropagation is a good approach.
Page 296/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders In feedforward ANNs backpropagation is the choice input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function (2) By calculating −𝛻𝑒𝑟𝑟𝑜𝑟 we get a vector that shows in a direction which decreases the error (3) We update the parameters to decrease the error (4) We repeat that
Page 306/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … the problem are the multiple hidden layers! input data X output data X‘ Training Backpropagation Problem: Deep Network
Page 316/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation is known to be slow far away from the output layer … Backpropagation Problem: Deep Network • Very slow training
Page 326/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training … and can converge to poor local minima. Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution
Page 336/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution The task is to initialize the parameters close to a good solution!
Page 346/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining Therefore the training of autoencoders has a pretraining phase …
Page 356/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs)
Page 366/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine • RBMs are Markov Random Fields
Page 376/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine • RBMs are Markov Random Fields Markov Random Field Every unit influences every neighbor The coupling is undirected Motivation (Ising Model) A set of magnetic dipoles (spins) is arranged in a graph (lattice) where neighbors are coupled with a given strengt
Page 386/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine • RBMs are Markov Random Fields • Bipartite topology: visible (v), hidden (h) • Use local energy to calculate the probabilities of values Training: contrastive divergency (Gibbs Sampling) h1 v1 v2 v3 v4 h2 h3
Page 396/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine Gibbs Sampling
Page 406/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder The top layer RBM transforms real value data into binary codes. 𝑉 ≔ set of visible units 𝑥 𝑣 ≔ value of unit 𝑣, ∀𝑣 ∈ 𝑉 𝑥 𝑣 ∈ 𝑹, ∀𝑣 ∈ 𝑉 𝐻 ≔ set of hidden units 𝑥ℎ ≔ value of unit ℎ, ∀ℎ ∈ 𝐻 𝑥ℎ ∈ {𝟎, 𝟏}, ∀ℎ ∈ 𝐻 Top Training
Page 416/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Top Therefore visible units are modeled with gaussians to encode data … 𝑥 𝑣~𝑁 𝑏 𝑣 + 𝑤 𝑣ℎ ℎ 𝑥ℎ, 𝜎𝑣 𝜎𝑣 ≔ std. dev. of unit 𝑣 𝑏 𝑣 ≔ bias of unit 𝑣 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h2 v1 v2 v3 v4 h3 h4 h5h1 Training
Page 426/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Top … and many hidden units with simoids to encode dependencies 𝑥ℎ~sigm 𝑏ℎ + 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝜎𝑣 𝜎𝑣 ≔ std. dev. of unit 𝑣 𝑏ℎ ≔ bias of unit ℎ 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h2 v1 v2 v3 v4 h3 h4 h5h1 Training
Page 436/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Top The objective function is the sum of the local energies. Local Energy 𝐸ℎ ≔ − 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝜎𝑣 𝑥ℎ + 𝑥ℎ 𝑏ℎ 𝐸 𝑣 ≔ − 𝑤 𝑣ℎ ℎ 𝑥 𝑣 𝜎𝑣 𝑥ℎ + 𝑥 𝑣 − 𝑏 𝑣 2 2𝜎𝑣 2 h2 v1 v2 v3 v4 h3 h4 h5h1 Training
Page 446/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction 𝑉 ≔ set of visible units 𝑥 𝑣 ≔ value of unit 𝑣, ∀𝑣 ∈ 𝑉 𝑥 𝑣 ∈ {𝟎, 𝟏}, ∀𝑣 ∈ 𝑉 𝐻 ≔ set of hidden units 𝑥ℎ ≔ value of unit ℎ, ∀ℎ ∈ 𝐻 𝑥ℎ ∈ {𝟎, 𝟏}, ∀ℎ ∈ 𝐻 The next RBM layer maps the dependency encoding… Training
Page 456/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction … from the upper layer … 𝑥 𝑣~sigm 𝑏 𝑣 + 𝑤 𝑣ℎ ℎ 𝑥ℎ 𝑏 𝑣 ≔ bias of unit v 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h1 v1 v2 v3 v4 h2 h3 Training
Page 466/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction … to a smaller number of simoids … 𝑥ℎ~sigm 𝑏ℎ + 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝑏ℎ ≔ bias of unit h 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h1 v1 v2 v3 v4 h2 h3 Training
Page 476/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction … which can be trained faster than the top layer Local Energy 𝐸 𝑣 ≔ − 𝑤 𝑣ℎ ℎ 𝑥 𝑣 𝑥ℎ + 𝑥ℎ 𝑏ℎ 𝐸ℎ ≔ − 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝑥ℎ + 𝑥 𝑣 𝑏 𝑣 h1 v1 v2 v3 v4 h2 h3 Training
Page 486/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Unrolling The symmetric topology allows us to skip further training. Training
Page 496/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Unrolling The symmetric topology allows us to skip further training. Training
Page 506/6/2013 Patrick Michl Network Modeling After pretraining backpropagation usually finds good solutions Autoencoders Autoencoder Training • Pretraining Top RBM (GRBM) Reduction RBMs Unrolling • Finetuning Backpropagation
Page 516/6/2013 Patrick Michl Network Modeling The algorithmic complexity of RBM training depends on the network size Autoencoders Autoencoder Training • Complexity: O(inw) i: number of iterations n: number of nodes w: number of weights • Memory Complexity: O(w)
Page 526/6/2013 Patrick Michl Network Modeling Agenda Autoencoders Biological Model Validation & Implementation
Page 536/6/2013 Patrick Michl Network Modeling Network Modeling Restricted Boltzmann Machines (RBM) How to model the topological structure? S E TF
Page 546/6/2013 Patrick Michl Network Modeling We define S and E as visible data Layer … S E TF Network Modeling Restricted Boltzmann Machines (RBM)
Page 556/6/2013 Patrick Michl Network Modeling S E TF Network Modeling Restricted Boltzmann Machines (RBM) We identify S and E with the visible layer …
Page 566/6/2013 Patrick Michl Network Modeling S E … and the TFs with the hidden layer in a RBM TF Network Modeling Restricted Boltzmann Machines (RBM)
Page 576/6/2013 Patrick Michl Network Modeling S E The training of the RBM gives us a model TF Network Modeling Restricted Boltzmann Machines (RBM)
Page 586/6/2013 Patrick Michl Network Modeling Agenda Autoencoder Biological Model Implementation & Results
Page 596/6/2013 Patrick Michl Network Modeling Results Validation of the results • Needs information about the true regulation • Needs information about the descriptive power of the data
Page 606/6/2013 Patrick Michl Network Modeling Results Validation of the results • Needs information about the true regulation • Needs information about the descriptive power of the data Without this infomation validation can only be done, using artificial datasets!
Page 616/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps:
Page 626/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps Step 1 Choose number of Genes (E+S) and create random bimodal distributed data
Page 636/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps Step 1 Choose number of Genes (E+S) and create random bimodal distributed data Step 2 Manipulate data in a fixed order
Page 646/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps Step 1 Choose number of Genes (E+S) and create random bimodal distributed data Step 2 Manipulate data in a fixed order Step 3 Add noise to manipulated data and normalize data
Page 656/6/2013 Patrick Michl Network Modeling Simulation Results Step 1 Number of visible nodes 8 (4E, 4S) Create random data: Random {-1, +1} + N(0, 𝜎 = 0.5)
Page 666/6/2013 Patrick Michl Network Modeling Simulation Results 𝑒1 = 0.25𝑠1 + 0.25𝑠2 + 0.25𝑠3 + 0.25𝑠4 𝑒2 = 0.5𝑠1 + 0.5 Noise 𝑒3 = 0.5𝑠1 + 0.5 𝑁𝑜𝑖𝑠𝑒4 𝑒4 = 0.5𝑠1 + 0.5 𝑁𝑜𝑖𝑠𝑒 Step 2 Manipulate data
Page 676/6/2013 Patrick Michl Network Modeling Simulation Results Step 3 Add noise: N(0, 𝜎 = 0.5)
Page 686/6/2013 Patrick Michl Network Modeling Results We analyse the data X with an RBM
Page 696/6/2013 Patrick Michl Network Modeling Results We train an autoencoder with 9 hidden layers and 165 nodes: Layer 1 & 9: 32 hidden units Layer 2 & 8: 24 hidden units Layer 3 & 7: 16 hidden units Layer 4 & 6: 8 hidden units Layer 5: 5 hidden units input data X output data X‘
Page 706/6/2013 Patrick Michl Network Modeling Results We transform the data from X to X‘ And reduce the dimensionality
Page 716/6/2013 Patrick Michl Network Modeling Results We analyse the transformed data X‘ with an RBM
Page 726/6/2013 Patrick Michl Network Modeling Results Lets compare the models
Page 736/6/2013 Patrick Michl Network Modeling Results Another Example with more nodes and larger autoencoder
Page 746/6/2013 Patrick Michl Network Modeling Conclusion Conclusion • Autoencoders can improve modeling significantly by reducing the dimensionality of data • Autoencoders preserve complex structures in their multilayer perceptron network. Analysing those networks (for example with knockout tests) could give more structural information • The drawback are high computational costs Since the field of deep learning is getting more popular (Face recognition / Voice recognition, Image transformation). Many new improvements in facing the computational costs have been made.
Page 756/6/2013 Patrick Michl Network Modeling Acknowledgement eilsLABS Prof. Dr. Rainer König Prof. Dr. Roland Eils Network Modeling Group

Recommended

Structure learning with Deep Neural Networks
Structure learning with Deep Neural NetworksStructure learning with Deep Neural Networks
Structure learning with Deep Neural Networks
Patrick Michl
 
Structure learning with Deep Autoencoders
Structure learning with Deep AutoencodersStructure learning with Deep Autoencoders
Structure learning with Deep Autoencoders
Patrick Michl
 
AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...
AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...
AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...
AILABS Academy
 
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Codemotion
 
Data Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQLData Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQL
EDB
 
ENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITS
ENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITSENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITS
ENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITS
IRJET Journal
 
Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
LogeekNightUkraine
 
Accelerating algorithmic and hardware advancements for power efficient on-dev...
Accelerating algorithmic and hardware advancements for power efficient on-dev...Accelerating algorithmic and hardware advancements for power efficient on-dev...
Accelerating algorithmic and hardware advancements for power efficient on-dev...
Qualcomm Research
 

Recommended

Structure learning with Deep Neural Networks
Structure learning with Deep Neural NetworksStructure learning with Deep Neural Networks
Structure learning with Deep Neural Networks
Patrick Michl
 
Structure learning with Deep Autoencoders
Structure learning with Deep AutoencodersStructure learning with Deep Autoencoders
Structure learning with Deep Autoencoders
Patrick Michl
 
AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...
AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...
AILABS Lecture Series - Is AI The New Electricity. Topic - Deep Learning - Ev...
AILABS Academy
 
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Codemotion
 
Data Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQLData Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQL
EDB
 
ENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITS
ENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITSENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITS
ENCRYPTION KEY GENERATION FOR DIGITAL CIRCUITS USING ANALOG CIRCUITS
IRJET Journal
 
Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
LogeekNightUkraine
 
Accelerating algorithmic and hardware advancements for power efficient on-dev...
Accelerating algorithmic and hardware advancements for power efficient on-dev...Accelerating algorithmic and hardware advancements for power efficient on-dev...
Accelerating algorithmic and hardware advancements for power efficient on-dev...
Qualcomm Research
 
Android Malware 2020 (CCCS-CIC-AndMal-2020)
Android Malware 2020 (CCCS-CIC-AndMal-2020)Android Malware 2020 (CCCS-CIC-AndMal-2020)
Android Malware 2020 (CCCS-CIC-AndMal-2020)
Indraneel Dabhade
 
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing WorldCloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Omer Kilic
 
Distributed machine learning 101 using apache spark from a browser devoxx.b...
Distributed machine learning 101 using apache spark from a browser devoxx.b...Distributed machine learning 101 using apache spark from a browser devoxx.b...
Distributed machine learning 101 using apache spark from a browser devoxx.b...
Andy Petrella
 
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Generating 3D Models Using 3D Generative Adversarial NetworkIRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET Journal
 
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st CenturyThe von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
Perry Lea
 
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
TigerGraph
 
Eye deep
Eye deepEye deep
Eye deep
sveitser
 
Deploying Pretrained Model In Edge IoT Devices.pdf
Deploying Pretrained Model In Edge IoT Devices.pdfDeploying Pretrained Model In Edge IoT Devices.pdf
Deploying Pretrained Model In Edge IoT Devices.pdf
Object Automation
 
Pres Tesi LM-2016+transcript_eng
Pres Tesi LM-2016+transcript_engPres Tesi LM-2016+transcript_eng
Pres Tesi LM-2016+transcript_eng
Daniele Ciriello
 
Apache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise CompanyApache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise Company
Databricks
 
H2O for IoT - Jo-Fai (Joe) Chow, H2O
H2O for IoT - Jo-Fai (Joe) Chow, H2OH2O for IoT - Jo-Fai (Joe) Chow, H2O
H2O for IoT - Jo-Fai (Joe) Chow, H2O
Data Science Milan
 
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario SpacagnaAnomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Data Science Milan
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
Raffael Marty
 
Internet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! nightInternet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! night
Andy Gelme
 
Coco co-desing and co-verification of masked software implementations on cp us
Coco co-desing and co-verification of masked software implementations on cp usCoco co-desing and co-verification of masked software implementations on cp us
Coco co-desing and co-verification of masked software implementations on cp us
RISC-V International
 
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
digitaldesign-s20-lecture3b-fpga-afterlecture.pdfdigitaldesign-s20-lecture3b-fpga-afterlecture.pdf
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
Duy-Hieu Bui
 
Dream3D and its Extension to Abaqus Input Files
Dream3D and its Extension to Abaqus Input FilesDream3D and its Extension to Abaqus Input Files
Dream3D and its Extension to Abaqus Input Files
Matthew Priddy
 
Parcel Lot Division with cGAN
Parcel Lot Division with cGANParcel Lot Division with cGAN
Parcel Lot Division with cGAN
Matthew To
 
Basic signal processing system design on fpga using lms based adaptive filter
Basic signal processing system design on fpga using lms based adaptive filterBasic signal processing system design on fpga using lms based adaptive filter
Basic signal processing system design on fpga using lms based adaptive filter
eSAT Journals
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
CHENHuiMei
 
Attention Please! 2022
Attention Please! 2022Attention Please! 2022
Attention Please! 2022
Patrick Michl
 
Spec2Vec - Energy based non-linear Calibration in NIRS
Spec2Vec - Energy based non-linear Calibration in NIRSSpec2Vec - Energy based non-linear Calibration in NIRS
Spec2Vec - Energy based non-linear Calibration in NIRS
Patrick Michl
 

More Related Content

Similar to Structure learning with Deep Neural Networks

Android Malware 2020 (CCCS-CIC-AndMal-2020)
Android Malware 2020 (CCCS-CIC-AndMal-2020)Android Malware 2020 (CCCS-CIC-AndMal-2020)
Android Malware 2020 (CCCS-CIC-AndMal-2020)
Indraneel Dabhade
 
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing WorldCloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Omer Kilic
 
Distributed machine learning 101 using apache spark from a browser devoxx.b...
Distributed machine learning 101 using apache spark from a browser devoxx.b...Distributed machine learning 101 using apache spark from a browser devoxx.b...
Distributed machine learning 101 using apache spark from a browser devoxx.b...
Andy Petrella
 
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Generating 3D Models Using 3D Generative Adversarial NetworkIRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET Journal
 
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st CenturyThe von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
Perry Lea
 
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
TigerGraph
 
Eye deep
Eye deepEye deep
Eye deep
sveitser
 
Deploying Pretrained Model In Edge IoT Devices.pdf
Deploying Pretrained Model In Edge IoT Devices.pdfDeploying Pretrained Model In Edge IoT Devices.pdf
Deploying Pretrained Model In Edge IoT Devices.pdf
Object Automation
 
Pres Tesi LM-2016+transcript_eng
Pres Tesi LM-2016+transcript_engPres Tesi LM-2016+transcript_eng
Pres Tesi LM-2016+transcript_eng
Daniele Ciriello
 
Apache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise CompanyApache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise Company
Databricks
 
H2O for IoT - Jo-Fai (Joe) Chow, H2O
H2O for IoT - Jo-Fai (Joe) Chow, H2OH2O for IoT - Jo-Fai (Joe) Chow, H2O
H2O for IoT - Jo-Fai (Joe) Chow, H2O
Data Science Milan
 
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario SpacagnaAnomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Data Science Milan
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
Raffael Marty
 
Internet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! nightInternet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! night
Andy Gelme
 
Coco co-desing and co-verification of masked software implementations on cp us
Coco co-desing and co-verification of masked software implementations on cp usCoco co-desing and co-verification of masked software implementations on cp us
Coco co-desing and co-verification of masked software implementations on cp us
RISC-V International
 
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
digitaldesign-s20-lecture3b-fpga-afterlecture.pdfdigitaldesign-s20-lecture3b-fpga-afterlecture.pdf
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
Duy-Hieu Bui
 
Dream3D and its Extension to Abaqus Input Files
Dream3D and its Extension to Abaqus Input FilesDream3D and its Extension to Abaqus Input Files
Dream3D and its Extension to Abaqus Input Files
Matthew Priddy
 
Parcel Lot Division with cGAN
Parcel Lot Division with cGANParcel Lot Division with cGAN
Parcel Lot Division with cGAN
Matthew To
 
Basic signal processing system design on fpga using lms based adaptive filter
Basic signal processing system design on fpga using lms based adaptive filterBasic signal processing system design on fpga using lms based adaptive filter
Basic signal processing system design on fpga using lms based adaptive filter
eSAT Journals
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
CHENHuiMei
 

Similar to Structure learning with Deep Neural Networks (20)

Android Malware 2020 (CCCS-CIC-AndMal-2020)
Android Malware 2020 (CCCS-CIC-AndMal-2020)Android Malware 2020 (CCCS-CIC-AndMal-2020)
Android Malware 2020 (CCCS-CIC-AndMal-2020)
 
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing WorldCloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
Cloud, Distributed, Embedded: Erlang in the Heterogeneous Computing World
 
Distributed machine learning 101 using apache spark from a browser devoxx.b...
Distributed machine learning 101 using apache spark from a browser devoxx.b...Distributed machine learning 101 using apache spark from a browser devoxx.b...
Distributed machine learning 101 using apache spark from a browser devoxx.b...
 
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Generating 3D Models Using 3D Generative Adversarial NetworkIRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
 
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st CenturyThe von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
 
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
Graph Gurus Episode 19: Deep Learning Implemented by GSQL on a Native Paralle...
 
Eye deep
Eye deepEye deep
Eye deep
 
Deploying Pretrained Model In Edge IoT Devices.pdf
Deploying Pretrained Model In Edge IoT Devices.pdfDeploying Pretrained Model In Edge IoT Devices.pdf
Deploying Pretrained Model In Edge IoT Devices.pdf
 
Pres Tesi LM-2016+transcript_eng
Pres Tesi LM-2016+transcript_engPres Tesi LM-2016+transcript_eng
Pres Tesi LM-2016+transcript_eng
 
Apache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise CompanyApache Spark for Cyber Security in an Enterprise Company
Apache Spark for Cyber Security in an Enterprise Company
 
H2O for IoT - Jo-Fai (Joe) Chow, H2O
H2O for IoT - Jo-Fai (Joe) Chow, H2OH2O for IoT - Jo-Fai (Joe) Chow, H2O
H2O for IoT - Jo-Fai (Joe) Chow, H2O
 
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario SpacagnaAnomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
 
Internet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! nightInternet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! night
 
Coco co-desing and co-verification of masked software implementations on cp us
Coco co-desing and co-verification of masked software implementations on cp usCoco co-desing and co-verification of masked software implementations on cp us
Coco co-desing and co-verification of masked software implementations on cp us
 
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
digitaldesign-s20-lecture3b-fpga-afterlecture.pdfdigitaldesign-s20-lecture3b-fpga-afterlecture.pdf
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
 
Dream3D and its Extension to Abaqus Input Files
Dream3D and its Extension to Abaqus Input FilesDream3D and its Extension to Abaqus Input Files
Dream3D and its Extension to Abaqus Input Files
 
Parcel Lot Division with cGAN
Parcel Lot Division with cGANParcel Lot Division with cGAN
Parcel Lot Division with cGAN
 
Basic signal processing system design on fpga using lms based adaptive filter
Basic signal processing system design on fpga using lms based adaptive filterBasic signal processing system design on fpga using lms based adaptive filter
Basic signal processing system design on fpga using lms based adaptive filter
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
 

More from Patrick Michl

Attention Please! 2022
Attention Please! 2022Attention Please! 2022
Attention Please! 2022
Patrick Michl
 
Spec2Vec - Energy based non-linear Calibration in NIRS
Spec2Vec - Energy based non-linear Calibration in NIRSSpec2Vec - Energy based non-linear Calibration in NIRS
Spec2Vec - Energy based non-linear Calibration in NIRS
Patrick Michl
 
Attention please! Attention Mechanism in Neural Networks
Attention please! Attention Mechanism in Neural NetworksAttention please! Attention Mechanism in Neural Networks
Attention please! Attention Mechanism in Neural Networks
Patrick Michl
 
Regulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann MachinesRegulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann Machines
Patrick Michl
 
Concept of Regulation Analysis using Restricted Boltzmann Machines
Concept of Regulation Analysis using Restricted Boltzmann MachinesConcept of Regulation Analysis using Restricted Boltzmann Machines
Concept of Regulation Analysis using Restricted Boltzmann Machines
Patrick Michl
 
Synchronization and Collective Dynamics
Synchronization and Collective DynamicsSynchronization and Collective Dynamics
Synchronization and Collective Dynamics
Patrick Michl
 
Epidemic Spreading
Epidemic SpreadingEpidemic Spreading
Epidemic Spreading
Patrick Michl
 

More from Patrick Michl (7)

Attention Please! 2022
Attention Please! 2022Attention Please! 2022
Attention Please! 2022
 
Spec2Vec - Energy based non-linear Calibration in NIRS
Spec2Vec - Energy based non-linear Calibration in NIRSSpec2Vec - Energy based non-linear Calibration in NIRS
Spec2Vec - Energy based non-linear Calibration in NIRS
 
Attention please! Attention Mechanism in Neural Networks
Attention please! Attention Mechanism in Neural NetworksAttention please! Attention Mechanism in Neural Networks
Attention please! Attention Mechanism in Neural Networks
 
Regulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann MachinesRegulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann Machines
 
Concept of Regulation Analysis using Restricted Boltzmann Machines
Concept of Regulation Analysis using Restricted Boltzmann MachinesConcept of Regulation Analysis using Restricted Boltzmann Machines
Concept of Regulation Analysis using Restricted Boltzmann Machines
 
Synchronization and Collective Dynamics
Synchronization and Collective DynamicsSynchronization and Collective Dynamics
Synchronization and Collective Dynamics
 
Epidemic Spreading
Epidemic SpreadingEpidemic Spreading
Epidemic Spreading
 

Recently uploaded

SAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdfSAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdf
KrushnaDarade1
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
Leonel Morgado
 
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
Scintica Instrumentation
 
The cost of acquiring information by natural selection
The cost of acquiring information by natural selectionThe cost of acquiring information by natural selection
The cost of acquiring information by natural selection
Carl Bergstrom
 
Basics of crystallography, crystal systems, classes and different forms
Basics of crystallography, crystal systems, classes and different formsBasics of crystallography, crystal systems, classes and different forms
Basics of crystallography, crystal systems, classes and different forms
MaheshaNanjegowda
 
Direct Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart AgricultureDirect Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart Agriculture
International Food Policy Research Institute- South Asia Office
 
20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx
Sharon Liu
 
Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.
Aditi Bajpai
 
Sharlene Leurig - Enabling Onsite Water Use with Net Zero Water
Sharlene Leurig - Enabling Onsite Water Use with Net Zero WaterSharlene Leurig - Enabling Onsite Water Use with Net Zero Water
Sharlene Leurig - Enabling Onsite Water Use with Net Zero Water
Texas Alliance of Groundwater Districts
 
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
vluwdy49
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
University of Hertfordshire
 
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
Sérgio Sacani
 
Equivariant neural networks and representation theory
Equivariant neural networks and representation theoryEquivariant neural networks and representation theory
Equivariant neural networks and representation theory
Daniel Tubbenhauer
 
Katherine Romanak - Geologic CO2 Storage.pdf
Katherine Romanak - Geologic CO2 Storage.pdfKatherine Romanak - Geologic CO2 Storage.pdf
Katherine Romanak - Geologic CO2 Storage.pdf
Texas Alliance of Groundwater Districts
 
Pests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdfPests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdf
PirithiRaju
 
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
Advanced-Concepts-Team
 
ESR spectroscopy in liquid food and beverages.pptx
ESR spectroscopy in liquid food and beverages.pptxESR spectroscopy in liquid food and beverages.pptx
ESR spectroscopy in liquid food and beverages.pptx
PRIYANKA PATEL
 
The binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defectsThe binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defects
Sérgio Sacani
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
RDhivya6
 
aziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobelaziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobel
İsa Badur
 

Recently uploaded (20)

SAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdfSAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdf
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
 
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
 
The cost of acquiring information by natural selection
The cost of acquiring information by natural selectionThe cost of acquiring information by natural selection
The cost of acquiring information by natural selection
 
Basics of crystallography, crystal systems, classes and different forms
Basics of crystallography, crystal systems, classes and different formsBasics of crystallography, crystal systems, classes and different forms
Basics of crystallography, crystal systems, classes and different forms
 
Direct Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart AgricultureDirect Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart Agriculture
 
20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx
 
Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.
 
Sharlene Leurig - Enabling Onsite Water Use with Net Zero Water
Sharlene Leurig - Enabling Onsite Water Use with Net Zero WaterSharlene Leurig - Enabling Onsite Water Use with Net Zero Water
Sharlene Leurig - Enabling Onsite Water Use with Net Zero Water
 
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
 
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
 
Equivariant neural networks and representation theory
Equivariant neural networks and representation theoryEquivariant neural networks and representation theory
Equivariant neural networks and representation theory
 
Katherine Romanak - Geologic CO2 Storage.pdf
Katherine Romanak - Geologic CO2 Storage.pdfKatherine Romanak - Geologic CO2 Storage.pdf
Katherine Romanak - Geologic CO2 Storage.pdf
 
Pests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdfPests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdf
 
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
 
ESR spectroscopy in liquid food and beverages.pptx
ESR spectroscopy in liquid food and beverages.pptxESR spectroscopy in liquid food and beverages.pptx
ESR spectroscopy in liquid food and beverages.pptx
 
The binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defectsThe binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defects
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
 
aziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobelaziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobel
 

Structure learning with Deep Neural Networks

  • 1. Structure learning with deep neuronal networks 6th Network Modeling Workshop, 6/6/2013 Patrick Michl
  • 2. Page 26/6/2013 Patrick Michl Network Modeling Agenda Autoencoders Biological Model Validation & Implementation
  • 3. Page 36/6/2013 Patrick Michl Network Modeling Real world data usually is high dimensional … x1 x2 Dataset Model Autoencoders
  • 4. Page 46/6/2013 Patrick Michl Network Modeling … which makes structural analysis and modeling complicated! x1 x2 x1 x2 Dataset Model 𝐹(𝑥1, 𝑥2) ? Autoencoders
  • 5. Page 56/6/2013 Patrick Michl Network Modeling Dimensionality reduction techinques like PCA … x1 x2 PCA Dataset Model Autoencoders
  • 6. Page 66/6/2013 Patrick Michl Network Modeling … can not preserve complex structures! x1 x2 PCA Dataset Model x1 x2 𝑥2 = α𝑥1 + β Autoencoders
  • 7. Page 76/6/2013 Patrick Michl Network Modeling Therefore the analysis of unknown structures … x1 x2 Dataset Model Autoencoders
  • 8. Page 86/6/2013 Patrick Michl Network Modeling … needs more considerate nonlinear techniques! x1 x2 Dataset Model x1 x2 𝑥2 = 𝑓(𝑥1) Autoencoders
  • 9. Page 96/6/2013 Patrick Michl Network Modeling Autoencoders are artificial neuronal networks … Autoencoder • Artificial Neuronal Network Autoencoders input data X output data X‘ Perceptrons Gaussian Units
  • 10. Page 106/6/2013 Patrick Michl Network Modeling Autoencoders are artificial neuronal networks … Autoencoder • Artificial Neuronal Network Autoencoders input data X output data X‘ Perceptrons Gaussian Units Perceptron 1 0 Gauss Units R
  • 11. Page 116/6/2013 Patrick Michl Network Modeling Autoencoders are artificial neuronal networks … Autoencoder • Artificial Neuronal Network Autoencoders input data X output data X‘ Perceptrons Gaussian Units
  • 12. Page 126/6/2013 Patrick Michl Network Modeling Autoencoder • Artificial Neuronal Network • Multiple hidden layers Autoencoders … with multiple hidden layers. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers)
  • 13. Page 136/6/2013 Patrick Michl Network Modeling Autoencoder • Artificial Neuronal Network • Multiple hidden layers Autoencoders Such networks are called deep networks. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers)
  • 14. Page 146/6/2013 Patrick Michl Network Modeling Autoencoder • Artificial Neuronal Network • Multiple hidden layers Autoencoders Such networks are called deep networks. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers)Definition (deep network) Deep networks are artificial neuronal networks with multiple hidden layers
  • 15. Page 156/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers) Such networks are called deep networks. • Deep network
  • 16. Page 166/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders Autoencoders have a symmetric topology … Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers) • Deep network • Symmetric topology
  • 17. Page 176/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … with an odd number of hidden layers. Gaussian Units input data X output data X‘ Perceptrons (Visible layers) (Hidden layers) • Deep network • Symmetric topology
  • 18. Page 186/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders The small layer in the center works lika an information bottleneck input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck Bottleneck
  • 19. Page 196/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders ... that creates a low dimensional code for each sample in the input data. input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck Bottleneck
  • 20. Page 206/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders The upper stack does the encoding … input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck • Encoder Encoder
  • 21. Page 216/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … and the lower stack does the decoding. input data X output data X‘ • Deep network • Symmetric topology • Information bottleneck • Encoder • Decoder Encoder Decoder
  • 22. Page 226/6/2013 Patrick Michl Network Modeling • Deep network • Symmetric topology • Information bottleneck • Encoder • Decoder Autoencoder Autoencoders … and the lower stack does the decoding. input data X output data X‘ Encoder Decoder Definition (deep network) Deep networks are artificial neuronal networks with multiple hidden layers Definition (autoencoder) Autoencoders are deep networks with a symmetric topology and an odd number of hiddern layers, containing a encoder, a low dimensional representation and a decoder.
  • 23. Page 236/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders Autoencoders can be used to reduce the dimension of data … input data X output data X‘ Problem: dimensionality of data Idea: 1. Train autoencoder to minimize the distance between input X and output X‘ 2. Encode X to low dimensional code Y 3. Decode low dimensional code Y to output X‘ 4. Output X‘ is low dimensional
  • 24. Page 246/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … if we can train them! input data X output data X‘ Problem: dimensionality of data Idea: 1. Train autoencoder to minimize the distance between input X and output X‘ 2. Encode X to low dimensional code Y 3. Decode low dimensional code Y to output X‘ 4. Output X‘ is low dimensional
  • 25. Page 256/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders In feedforward ANNs backpropagation is a good approach. input data X output data X‘ Training Backpropagation
  • 26. Page 266/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function 𝑋′ = 𝐹 𝑋 error = 𝑋′2 − 𝑌 In feedforward ANNs backpropagation is a good approach.
  • 27. Page 276/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders In feedforward ANNs backpropagation is the choice input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function Example (linear neuronal unit with two inputs)
  • 28. Page 286/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function (2) By calculating −𝛻𝑒𝑟𝑟𝑜𝑟 we get a vector that shows in a direction which decreases the error (3) We update the parameters to decrease the error In feedforward ANNs backpropagation is a good approach.
  • 29. Page 296/6/2013 Patrick Michl Network Modeling Backpropagation Autoencoder Autoencoders In feedforward ANNs backpropagation is the choice input data X output data X‘ Training Definition (autoencoder) Backpropagation (1) The distance (error) between current output X‘ and wanted output Y is computed. This gives a error function (2) By calculating −𝛻𝑒𝑟𝑟𝑜𝑟 we get a vector that shows in a direction which decreases the error (3) We update the parameters to decrease the error (4) We repeat that
  • 30. Page 306/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders … the problem are the multiple hidden layers! input data X output data X‘ Training Backpropagation Problem: Deep Network
  • 31. Page 316/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation is known to be slow far away from the output layer … Backpropagation Problem: Deep Network • Very slow training
  • 32. Page 326/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training … and can converge to poor local minima. Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution
  • 33. Page 336/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution The task is to initialize the parameters close to a good solution!
  • 34. Page 346/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining Therefore the training of autoencoders has a pretraining phase …
  • 35. Page 356/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs)
  • 36. Page 366/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine • RBMs are Markov Random Fields
  • 37. Page 376/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine • RBMs are Markov Random Fields Markov Random Field Every unit influences every neighbor The coupling is undirected Motivation (Ising Model) A set of magnetic dipoles (spins) is arranged in a graph (lattice) where neighbors are coupled with a given strengt
  • 38. Page 386/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine • RBMs are Markov Random Fields • Bipartite topology: visible (v), hidden (h) • Use local energy to calculate the probabilities of values Training: contrastive divergency (Gibbs Sampling) h1 v1 v2 v3 v4 h2 h3
  • 39. Page 396/6/2013 Patrick Michl Network Modeling Autoencoder Autoencoders input data X output data X‘ Training Backpropagation Problem: Deep Network • Very slow training • Maybe bad solution Idea: Initialize close to a good solution • Pretraining • Restricted Boltzmann Machines … which uses Restricted Boltzmann Machines (RBMs) Restricted Boltzmann Machine Gibbs Sampling
  • 40. Page 406/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder The top layer RBM transforms real value data into binary codes. 𝑉 ≔ set of visible units 𝑥 𝑣 ≔ value of unit 𝑣, ∀𝑣 ∈ 𝑉 𝑥 𝑣 ∈ 𝑹, ∀𝑣 ∈ 𝑉 𝐻 ≔ set of hidden units 𝑥ℎ ≔ value of unit ℎ, ∀ℎ ∈ 𝐻 𝑥ℎ ∈ {𝟎, 𝟏}, ∀ℎ ∈ 𝐻 Top Training
  • 41. Page 416/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Top Therefore visible units are modeled with gaussians to encode data … 𝑥 𝑣~𝑁 𝑏 𝑣 + 𝑤 𝑣ℎ ℎ 𝑥ℎ, 𝜎𝑣 𝜎𝑣 ≔ std. dev. of unit 𝑣 𝑏 𝑣 ≔ bias of unit 𝑣 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h2 v1 v2 v3 v4 h3 h4 h5h1 Training
  • 42. Page 426/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Top … and many hidden units with simoids to encode dependencies 𝑥ℎ~sigm 𝑏ℎ + 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝜎𝑣 𝜎𝑣 ≔ std. dev. of unit 𝑣 𝑏ℎ ≔ bias of unit ℎ 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h2 v1 v2 v3 v4 h3 h4 h5h1 Training
  • 43. Page 436/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Top The objective function is the sum of the local energies. Local Energy 𝐸ℎ ≔ − 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝜎𝑣 𝑥ℎ + 𝑥ℎ 𝑏ℎ 𝐸 𝑣 ≔ − 𝑤 𝑣ℎ ℎ 𝑥 𝑣 𝜎𝑣 𝑥ℎ + 𝑥 𝑣 − 𝑏 𝑣 2 2𝜎𝑣 2 h2 v1 v2 v3 v4 h3 h4 h5h1 Training
  • 44. Page 446/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction 𝑉 ≔ set of visible units 𝑥 𝑣 ≔ value of unit 𝑣, ∀𝑣 ∈ 𝑉 𝑥 𝑣 ∈ {𝟎, 𝟏}, ∀𝑣 ∈ 𝑉 𝐻 ≔ set of hidden units 𝑥ℎ ≔ value of unit ℎ, ∀ℎ ∈ 𝐻 𝑥ℎ ∈ {𝟎, 𝟏}, ∀ℎ ∈ 𝐻 The next RBM layer maps the dependency encoding… Training
  • 45. Page 456/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction … from the upper layer … 𝑥 𝑣~sigm 𝑏 𝑣 + 𝑤 𝑣ℎ ℎ 𝑥ℎ 𝑏 𝑣 ≔ bias of unit v 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h1 v1 v2 v3 v4 h2 h3 Training
  • 46. Page 466/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction … to a smaller number of simoids … 𝑥ℎ~sigm 𝑏ℎ + 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝑏ℎ ≔ bias of unit h 𝑤 𝑣ℎ ≔ weight of edge (𝑣, ℎ) h1 v1 v2 v3 v4 h2 h3 Training
  • 47. Page 476/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Reduction … which can be trained faster than the top layer Local Energy 𝐸 𝑣 ≔ − 𝑤 𝑣ℎ ℎ 𝑥 𝑣 𝑥ℎ + 𝑥ℎ 𝑏ℎ 𝐸ℎ ≔ − 𝑤 𝑣ℎ 𝑣 𝑥 𝑣 𝑥ℎ + 𝑥 𝑣 𝑏 𝑣 h1 v1 v2 v3 v4 h2 h3 Training
  • 48. Page 486/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Unrolling The symmetric topology allows us to skip further training. Training
  • 49. Page 496/6/2013 Patrick Michl Network Modeling Autoencoders Autoencoder Unrolling The symmetric topology allows us to skip further training. Training
  • 50. Page 506/6/2013 Patrick Michl Network Modeling After pretraining backpropagation usually finds good solutions Autoencoders Autoencoder Training • Pretraining Top RBM (GRBM) Reduction RBMs Unrolling • Finetuning Backpropagation
  • 51. Page 516/6/2013 Patrick Michl Network Modeling The algorithmic complexity of RBM training depends on the network size Autoencoders Autoencoder Training • Complexity: O(inw) i: number of iterations n: number of nodes w: number of weights • Memory Complexity: O(w)
  • 52. Page 526/6/2013 Patrick Michl Network Modeling Agenda Autoencoders Biological Model Validation & Implementation
  • 53. Page 536/6/2013 Patrick Michl Network Modeling Network Modeling Restricted Boltzmann Machines (RBM) How to model the topological structure? S E TF
  • 54. Page 546/6/2013 Patrick Michl Network Modeling We define S and E as visible data Layer … S E TF Network Modeling Restricted Boltzmann Machines (RBM)
  • 55. Page 556/6/2013 Patrick Michl Network Modeling S E TF Network Modeling Restricted Boltzmann Machines (RBM) We identify S and E with the visible layer …
  • 56. Page 566/6/2013 Patrick Michl Network Modeling S E … and the TFs with the hidden layer in a RBM TF Network Modeling Restricted Boltzmann Machines (RBM)
  • 57. Page 576/6/2013 Patrick Michl Network Modeling S E The training of the RBM gives us a model TF Network Modeling Restricted Boltzmann Machines (RBM)
  • 58. Page 586/6/2013 Patrick Michl Network Modeling Agenda Autoencoder Biological Model Implementation & Results
  • 59. Page 596/6/2013 Patrick Michl Network Modeling Results Validation of the results • Needs information about the true regulation • Needs information about the descriptive power of the data
  • 60. Page 606/6/2013 Patrick Michl Network Modeling Results Validation of the results • Needs information about the true regulation • Needs information about the descriptive power of the data Without this infomation validation can only be done, using artificial datasets!
  • 61. Page 616/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps:
  • 62. Page 626/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps Step 1 Choose number of Genes (E+S) and create random bimodal distributed data
  • 63. Page 636/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps Step 1 Choose number of Genes (E+S) and create random bimodal distributed data Step 2 Manipulate data in a fixed order
  • 64. Page 646/6/2013 Patrick Michl Network Modeling Results Artificial datasets We simulate data in three steps Step 1 Choose number of Genes (E+S) and create random bimodal distributed data Step 2 Manipulate data in a fixed order Step 3 Add noise to manipulated data and normalize data
  • 65. Page 656/6/2013 Patrick Michl Network Modeling Simulation Results Step 1 Number of visible nodes 8 (4E, 4S) Create random data: Random {-1, +1} + N(0, 𝜎 = 0.5)
  • 66. Page 666/6/2013 Patrick Michl Network Modeling Simulation Results 𝑒1 = 0.25𝑠1 + 0.25𝑠2 + 0.25𝑠3 + 0.25𝑠4 𝑒2 = 0.5𝑠1 + 0.5 Noise 𝑒3 = 0.5𝑠1 + 0.5 𝑁𝑜𝑖𝑠𝑒4 𝑒4 = 0.5𝑠1 + 0.5 𝑁𝑜𝑖𝑠𝑒 Step 2 Manipulate data
  • 67. Page 676/6/2013 Patrick Michl Network Modeling Simulation Results Step 3 Add noise: N(0, 𝜎 = 0.5)
  • 68. Page 686/6/2013 Patrick Michl Network Modeling Results We analyse the data X with an RBM
  • 69. Page 696/6/2013 Patrick Michl Network Modeling Results We train an autoencoder with 9 hidden layers and 165 nodes: Layer 1 & 9: 32 hidden units Layer 2 & 8: 24 hidden units Layer 3 & 7: 16 hidden units Layer 4 & 6: 8 hidden units Layer 5: 5 hidden units input data X output data X‘
  • 70. Page 706/6/2013 Patrick Michl Network Modeling Results We transform the data from X to X‘ And reduce the dimensionality
  • 71. Page 716/6/2013 Patrick Michl Network Modeling Results We analyse the transformed data X‘ with an RBM
  • 72. Page 726/6/2013 Patrick Michl Network Modeling Results Lets compare the models
  • 73. Page 736/6/2013 Patrick Michl Network Modeling Results Another Example with more nodes and larger autoencoder
  • 74. Page 746/6/2013 Patrick Michl Network Modeling Conclusion Conclusion • Autoencoders can improve modeling significantly by reducing the dimensionality of data • Autoencoders preserve complex structures in their multilayer perceptron network. Analysing those networks (for example with knockout tests) could give more structural information • The drawback are high computational costs Since the field of deep learning is getting more popular (Face recognition / Voice recognition, Image transformation). Many new improvements in facing the computational costs have been made.
  • 75. Page 756/6/2013 Patrick Michl Network Modeling Acknowledgement eilsLABS Prof. Dr. Rainer König Prof. Dr. Roland Eils Network Modeling Group