Successfully reported this slideshow.
Your SlideShare is downloading. ×

idalab seminar #13 - Dr. Kristof T. Schütt

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 24 Ad

idalab seminar #13 - Dr. Kristof T. Schütt

What if we could build batteries for electric cars that would take us further than a full tank of gasoline? If we could grow affordable, tasty and nutritious meat in the laboratory instead of occupying one third of the land on our planet with animal farming? What if we could easily identify promising targets in the human body for new cancer drugs?
Underpinning all of these challenges is: chemistry. The challenge is not a scarcity of potential high-value compounds. Quite the opposite: The problem is that it just takes too long to test every possible compound for the desired chemical properties. Therefore, we must find ways to reliably predict these properties.

While quantum-chemical simulations allow us to accurately calculate chemical properties, their large computational cost as well as the huge number of molecules and materials make an exhaustive exploration infeasible. This talk introduces deep learning models for a variety of use cases in quantum chemistry. By analyzing the learned representations, we get a glimpse into the inner working of the neural network to find out whether the model has learned known chemical concepts – or has even uncovered hitherto unknown mechanisms!

What if we could build batteries for electric cars that would take us further than a full tank of gasoline? If we could grow affordable, tasty and nutritious meat in the laboratory instead of occupying one third of the land on our planet with animal farming? What if we could easily identify promising targets in the human body for new cancer drugs?
Underpinning all of these challenges is: chemistry. The challenge is not a scarcity of potential high-value compounds. Quite the opposite: The problem is that it just takes too long to test every possible compound for the desired chemical properties. Therefore, we must find ways to reliably predict these properties.

While quantum-chemical simulations allow us to accurately calculate chemical properties, their large computational cost as well as the huge number of molecules and materials make an exhaustive exploration infeasible. This talk introduces deep learning models for a variety of use cases in quantum chemistry. By analyzing the learned representations, we get a glimpse into the inner working of the neural network to find out whether the model has learned known chemical concepts – or has even uncovered hitherto unknown mechanisms!

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to idalab seminar #13 - Dr. Kristof T. Schütt (20)

Advertisement

More from idalab GmbH (20)

Recently uploaded (20)

Advertisement

idalab seminar #13 - Dr. Kristof T. Schütt

  1. 1. © 2018 | idalab GmbH | Potsdamer Straße 68 | 10785 Berlin | idalab.de page 1 | confidential Agency for Data Science Machine learning & AI Mathematical modelling Data strategy Dr. Kristof T. Schütt Exploring Chemical Space with Deep Learning idalab seminar #13 | September 28th 2018
  2. 2. Exploring Chemical Space with Deep Learning Kristof T. Sch¨utt
  3. 3. Exploring Chemical Space
  4. 4. ML for Quantum Chemistry Method Scaling Hartree Fock O(n3 ) − O(n4 ) Density Functional Theory O(n3 ) − O(n4 ) MP2 O(n5 ) CCSD O(n6 ) CCSD(T) O(n7 ) Full CI O(n!)
  5. 5. Representation Neural network architectures for atomistic systems
  6. 6. Atomistic Systems Molecules Materials S = {(Zi, ri)| i ∈ [1, natoms]} Challenges: rotational / translational invariance permutational invariance symmetries / local features
  7. 7. There is an abundance of features for atomistic systems... α β β α α β r dr α β
  8. 8. ... so what is missing? Drawbacks of fixed representations Property-specific similarity Do similar systems w.r.t energy also have similar dipole moments? Cross-element generalization How does silicon behave compared to carbon? Different tasks at different scales Chemical Compound Space vs. Potentials Energy Surfaces Solution End-to-end learning from atom types and positions Representation is adapted during training
  9. 9. The Deep Tensor Neural Network (DTNN) framework 1. Embed atom types x (0) i = AZi ∈ Rd 2. Add interactions x (t+1) i = x (t) i + j=i v(t) x (t) j , ri − rj 3. Predict via atom-wise contributions: ˆE = natoms i=1 E(x (T) i ) Sch¨utt, Arbabzadah, Chmiela, M¨uller, Tkatchenko, Nature Communications 8, 13890 (2017)
  10. 10. The Deep Tensor Neural Network (DTNN) framework 1. Embed atom types x (0) i = AZi ∈ Rd 2. Add interactions x (t+1) i = x (t) i + j=i v(t) x (t) j , ri − rj 3. Predict via atom-wise contributions: ˆE = natoms i=1 E(x (T) i ) Sch¨utt, Arbabzadah, Chmiela, M¨uller, Tkatchenko, Nature Communications 8, 13890 (2017)
  11. 11. The Deep Tensor Neural Network (DTNN) framework 1. Embed atom types x (0) i = AZi ∈ Rd 2. Add interactions x (t+1) i = x (t) i + j=i v(t) x (t) j , ri − rj 3. Predict via atom-wise contributions: ˆE = natoms i=1 E(x (T) i ) Sch¨utt, Arbabzadah, Chmiela, M¨uller, Tkatchenko, Nature Communications 8, 13890 (2017)
  12. 12. SchNet – quantum interactions from convolutions (x ∗ W )(ri) = Natom j=1 x (t) j ◦ W (t) [ri−rj] parameter tensor (x ∗ W )(ri) = Natom j=1 x (t) j ◦ W (t) (ri − rj) neural network
  13. 13. SchNet - a continuous-filter convolutional neural network K.T. Sch¨utt, P.-J. Kindermans, H.E. Sauceda, S. Chmiela, A. Tkatchenko, K.-R. M¨uller (2017). SchNet: A continuous-filter convolutional neural network for modeling quantum interactions. NIPS 30.
  14. 14. Property-specific output layers Internal energy U0 Dipole moment µ SchNet output layer E = i E(xi) µ = i q(xi)(ri − r0) QM9 – 110k ref. calculations – mean abs. errors SchNet (T=6, SGDR) 0.218 kcal mol−1 0.017 Debye HIP-NN[1] 0.256 kcal mol−1 – Message-passing NN[2] 0.450 kcal mol−1 0.030 Debye [1] N. Lubbers, J.S. Smith, K. Barros (2018). Hierarchical modeling of molecular energies using a deep neural network. The Journal of Chemical Physics, 148(24), 241715. [2] J. Gilmer, S.S. Schoenholz, P.F. Riley, O. Vinyals, G.E. Dahl. Neural-Message Passing for Quantum Chemistry (2017). ICML.
  15. 15. Analysis Insights about trained models and underlying data
  16. 16. Local chemical potentials virtual atom with charge Zp at position rp x (t+1) p = x (t) i + j v(t) x (t) j , rp − rj probe energy from output network ΩZp (r) = fout(x(T) p )
  17. 17. Moving through alchemical space
  18. 18. Learning the periodic table of elements 1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 1st principal component 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02nd principalcomponent H Li CsRb K Na Ba Sr Mg Ca Be Ga In Al B Tl Si C PbSn Ge As Bi P N Sb Se Te S O I F Cl Br Ne Xe Ar He Kr I II III IV V VI VII VIII Group II Group I G roup V 1.5 Trained on 60k bulk crystals from the Materials Project.
  19. 19. Application Accelerated molecular dynamics simulations
  20. 20. Molecular dynamics – Training with forces Atomic forces can efficiently be obtained by differentiating the network: Fi(r1, . . . , rn) = − ∂E ∂ri (r1, . . . , rn), yielding energy-conserving, rotationally equivariant force field predictions. Combined loss of total energy E and atomic forces Fi for training: ( ˆE, (E, F1, . . . , Fn)) = ρ E− ˆE 2 + 1 Natoms Natoms i=0 − ∂ ˆE ∂ri −Fi 2 . 0 100 200 time step 0 10 20 30 40 totalenergy[kcalmol1 ]
  21. 21. Molecular dynamics – PIMD of the fullerene C20
  22. 22. Molecular dynamics – PIMD of the fullerene C20 Accurate prediction of vibrational frequencies 0 200 400 600 800 1000 1200 1400 frequency [cm 1] spectrum DFT (PBE-TS) SchNet (PBE-TS) 0 200 400 600 800 1000 1200 1400 frequency [cm 1] 5 0 5 DFTSchNet[cm1] PIMD@SchNet shows delocalization of bonds 1.5 2.0 2.5 3.0 3.5 4.0 4.5 r [Å] 0.0 0.5 1.0 1.5 2.0 2.5 h(r)[a.u.] P= 1 P= 8 1.3 1.4 1.5 1.6 1.7 nearest C-C [Å] 0 2 4 6 8 10 distribution 3.8 4.0 4.2 4.4 4.6 diameter [Å] 0 1 2 3 distribution SchNet enables generation of 1.25ns PIMD trajectory by 3-4 orders of magnitude: 7 years ⇒ 7 hours
  23. 23. SchNetPack P spk.Atomwise SchNet spk.atomistic spk.DipoleMoment wACSFZ spk.representation R Z R Angular Concatenate Radial Embedding Interaction Interaction RZ www.quantum-machine.org/schnetpack
  24. 24. SchNetPack P spk.Atomwise SchNet spk.atomistic spk.DipoleMoment wACSFZ spk.representation R Z R Angular Concatenate Radial Embedding Interaction Interaction RZ www.quantum-machine.org/schnetpack Thank you!

×