Concept of Regulation Analysis using Restricted Boltzmann Machines
•
0 likes•12 views
The document discusses using Restricted Boltzmann Machines (RBMs) to analyze regulation of metabolism through biological signals, transcription factors, and enzymes. An RBM model is presented with signals and enzymes as visible units and transcription factors as hidden units. The document provides an example application with 8 visible and 2 hidden units to learn weights between units from sample data. The results show a strong dependency between one signal and enzyme, while others have little effect, demonstrating RBMs can learn regulatory patterns from data. Further applications of deep learning strategies using RBMs are mentioned.
Recommended
Recommended
More Related Content
More from Patrick Michl
More from Patrick Michl (7)
Recently uploaded
Recently uploaded (20)
Concept of Regulation Analysis using Restricted Boltzmann Machines
- 1. Regulation Analysis using Restricted Boltzmann Machines iBIOS, 2/2/2012 Patrick Michl Heidelberg University
- 2. Page 21/31/2012 | Author Department Agenda Biological Problem Analysing the regulation of metabolism Machine Learning Implementation
- 3. Page 31/31/2012 | Author Department Biological Problem Analysing the regulation of metabolism Signal Regulation Metabolism A linear metabolic pathway of enzymes (E) …
- 4. Page 41/31/2012 | Author Department Biological Problem Analysing the regulation of metabolism Signal Regulation Metabolism … is regulated by transcription factors (TF) …
- 5. Page 51/31/2012 | Author Department Biological Problem Analysing the regulation of metabolism Signal Regulation Metabolism … which respond to signals (S)
- 6. Page 61/31/2012 | Author Department P 4 P 3 P 2 P 1 Biological Problem Analysing the regulation of metabolism Upregulated linear pathways …
- 7. Page 71/31/2012 | Author Department P 4 P 3 P 2 P 1 Biological Problem Analysing the regulation of metabolism … can appear in different patterns
- 8. Page 81/31/2012 | Author Department S TF Biological Problem Analysing the regulation of metabolism Which transcription factors and signals cause this patterns? S TF S TF
- 9. Page 91/31/2012 | Author Department Agenda Biological Problem Analysing the regulation of metabolism Machine Learning Restricted Boltzmann Machines (RBM) Implementation
- 10. Page 101/31/2012 | Author Department Restricted Boltzmann Machines A Restricted Boltzmann Machine (RBM) is an ANN … Neuron like Units
- 11. Page 111/31/2012 | Author Department Restricted Boltzmann Machines … with two layers: visible units (v) and hidden units (h) h1 v1 v2 v3 v4 h2 h3
- 12. Page 121/31/2012 | Author Department Restricted Boltzmann Machines Visible units are strictly connected with hidden units h1 v1 v2 v3 v4 h2 h3
- 13. Page 131/31/2012 | Author Department Restricted Boltzmann Machines In the most common model all units have binary values … 𝑉 ≔ set of visible units 𝐻 ≔ set of hidden units 𝑠 𝑣: = value of 𝑣, ∀𝑣 ∈ 𝑉 𝑠ℎ ≔ value of ℎ, ∀ℎ ∈ 𝐻 𝑠 𝑣 ∈ 0, 1 , ∀𝑣 ∈ 𝑉 𝑠ℎ ∈ 0, 1 , ∀ℎ ∈ 𝐻
- 14. Page 141/31/2012 | Author Department Restricted Boltzmann Machines … and arbitrary tresholds which allow us … 𝑉 ≔ set of visible units 𝐻 ≔ set of hidden units 𝑠 𝑣: = value of 𝑣, ∀𝑣 ∈ 𝑉 𝑠ℎ ≔ value of ℎ, ∀ℎ ∈ 𝐻 𝑠 𝑣 ∈ 0, 1 , ∀𝑣 ∈ 𝑉 𝑠ℎ ∈ 0, 1 , ∀ℎ ∈ 𝐻 𝜃 𝑣 ≔ threshold of 𝑣, ∀𝑣 ∈ 𝑉 𝜃ℎ ≔ threshold of ℎ, ∀ℎ ∈ 𝐻
- 15. Page 151/31/2012 | Author Department Restricted Boltzmann Machines … to define energy functions: Local energies Ev and Eh … 𝐸 𝑣 ≔ − 𝑤 𝑣, ℎ 𝑠 𝑣 𝑠ℎ ℎ + 𝜃 𝑣 𝑠 𝑣 𝐸ℎ ≔ − 𝑤 𝑣, ℎ 𝑠 𝑣 𝑠ℎ 𝑣 + 𝜃ℎ 𝑠ℎ 𝑤 𝑣, ℎ ≔ weight of 𝑒𝑑𝑔𝑒(𝑣, ℎ) Local Energy
- 16. Page 161/31/2012 | Author Department Restricted Boltzmann Machines … and the global Energy E. We want to minimize E 𝐸 ≔ 𝐸 𝑣 𝑣 + 𝐸ℎ ℎ 𝑤 𝑣, ℎ ≔ weight of 𝑒𝑑𝑔𝑒(𝑣, ℎ) Local Energy Global Energy 𝐸 𝑣 ≔ − 𝑤 𝑣, ℎ 𝑠 𝑣 𝑠ℎ ℎ + 𝜃 𝑣 𝑠 𝑣 𝐸ℎ ≔ − 𝑤 𝑣, ℎ 𝑠 𝑣 𝑠ℎ 𝑣 + 𝜃ℎ 𝑠ℎ 1
- 17. Page 171/31/2012 | Author Department Restricted Boltzmann Machines If we aproximate a Boltzmann Distribution for Ev … Energy Delta for visible units ∆𝐸𝑣 = 𝐸 𝑣, 𝑜𝑓𝑓 − 𝐸 𝑣, 𝑜𝑛 = ...
- 18. Page 181/31/2012 | Author Department Restricted Boltzmann Machines … we can use the Boltzmann Factor … ∆𝐸𝑣 = 𝐸 𝑣, 𝑜𝑓𝑓 − 𝐸 𝑣, 𝑜𝑛 = −𝑘 𝐵 𝑇 𝑙𝑛 P 𝑣,off −(−𝑘 𝐵 𝑇 𝑙𝑛 P 𝑣,on ) Energy Delta for visible units
- 19. Page 191/31/2012 | Author Department Restricted Boltzmann Machines … to get a term for the probability [v, on] P 𝑣,off = 1−P 𝑣,on P 𝑣,on = 1 1 + 𝑒− ∆𝐸𝑣 𝑘 𝐵 𝑇 ∆𝐸𝑣 = 𝐸 𝑣, 𝑜𝑓𝑓 − 𝐸 𝑣, 𝑜𝑛 = −𝑘 𝐵 𝑇 𝑙𝑛 P 𝑣,off −(−𝑘 𝐵 𝑇 𝑙𝑛 P 𝑣,on ) Energy Delta for visible units
- 20. Page 201/31/2012 | Author Department Restricted Boltzmann Machines … and similary for hidden units Probalilities P 𝑣,on = 1 1 + 𝑒− ∆𝐸𝑣 𝑘 𝐵 𝑇 P ℎ,on = 1 1 + 𝑒− ∆𝐸ℎ 𝑘 𝐵 𝑇 2
- 21. Page 211/31/2012 | Author Department Restricted Boltzmann Machines With (1) an (2) we can perform Simulated Annealing set T= TMax while (T> TMin) forall v if (P[v,on] > rand(0,1)) set sv = 1 forall h if (P[h,on] > rand(0,1)) set sh = 1 set Tsmaller Simulated Annealing 𝐸 → min
- 22. Page 221/31/2012 | Author Department Agenda Biological Problem Analysing the regulation of metabolism Machine Learning Restricted Boltzmann Machines (RBM) Implementation Modeling the Problem / Example
- 23. Page 231/31/2012 | Author Department Modeling Regulation as RBM To model regulation as an RBM … S TF E
- 24. Page 241/31/2012 | Author Department Modeling Regulation as RBM … we define S and E as visible Layer … S E TF
- 25. Page 251/31/2012 | Author Department Modeling Regulation as RBM S E … we define S and E as visible Layer … TF
- 26. Page 261/31/2012 | Author Department Modeling Regulation as RBM S E … and TF as hidden Layer TF
- 27. Page 271/31/2012 | Author Department Example Let‘s try it as simple as possible S E TF
- 28. Page 281/31/2012 | Author Department Example … so we get 8 visible and 2 hidden units, fully connected S E TF
- 29. Page 291/31/2012 | Author Department Example Let‘s feed the machine with learning samples … S E 1,0,0,1 1,0,0,0 1,0,0,1 1,1,0,0 1,0,0,1 1,0,1,0 1,0,0,1 1,0,0,1 1,0,1,1 0,0,0,0 1,0,1,1 0,1,0,0 1,0,1,1 0,0,1,0 1,0,1,1 0,0,0,1 Learning samples
- 30. Page 301/31/2012 | Author Department Example .. to get the calculated weight matrix TF1 TF2 S1 0,3 0,8 S2 0,5 0,6 S3 0,9 0,1 S4 0,3 0,8 E1 1,0 0,0 E2 0,1 0,0 E3 0,1 0,0 E4 0,2 0,0 Weight matrix
- 31. Page 311/31/2012 | Author Department Example The weights are visualized by the intensity of the edges S E TF TF1 TF2 S1 0,3 0,8 S2 0,5 0,6 S3 0,9 0,1 S4 0,3 0,8 E1 1,0 0,0 E2 0,1 0,0 E3 0,1 0,0 E4 0,2 0,0 Weight matrix
- 32. Page 321/31/2012 | Author Department Example Now we can compare the results with the samples S E 1,0,0,1 1,0,0,0 1,0,0,1 1,1,0,0 1,0,0,1 1,0,1,0 1,0,0,1 1,0,0,1 1,0,1,1 0,0,0,0 1,0,1,1 0,1,0,0 1,0,1,1 0,0,1,0 1,0,1,1 0,0,0,1 Learning samples S E TF
- 33. Page 331/31/2012 | Author Department Example There‘s a strong dependency between S3 an E1 S E 1,0,0,1 1,0,0,0 1,0,0,1 1,1,0,0 1,0,0,1 1,0,1,0 1,0,0,1 1,0,0,1 1,0,1,1 0,0,0,0 1,0,1,1 0,1,0,0 1,0,1,1 0,0,1,0 1,0,1,1 0,0,0,1 Learning samples S E TF
- 34. Page 341/31/2012 | Author Department Example S1, S2 and S4 do almost not affect the metabolism S E TF S E 1,0,0,1 1,0,0,0 1,0,0,1 1,1,0,0 1,0,0,1 1,0,1,0 1,0,0,1 1,0,0,1 1,0,1,1 0,0,0,0 1,0,1,1 0,1,0,0 1,0,1,1 0,0,1,0 1,0,1,1 0,0,0,1 Learning samples
- 35. Page 351/31/2012 | Author Department Further Objectives Since 2006 RBMs have successfully be used to train (pre- train) Multi-Layer ANNs (Hinton, Osindero, 2006) This new branch in machine learning („deep learning“) already has a wide area of applications, incuding: • Face recognition / Voice recognition • Unsupervised detection of features • Imagetransformation It has to be asumed that deep learning strategies also provide further capabilities in regulatory analysis