Your SlideShare is downloading.
×

- 1. © 2018 | idalab GmbH | Potsdamer Straße 68 | 10785 Berlin | idalab.de page 1 | confidential Agency for Data Science Machine learning & AI Mathematical modelling Data strategy Dr. Kristof T. Schütt Exploring Chemical Space with Deep Learning idalab seminar #13 | September 28th 2018
- 2. Exploring Chemical Space with Deep Learning Kristof T. Sch¨utt
- 3. Exploring Chemical Space
- 4. ML for Quantum Chemistry Method Scaling Hartree Fock O(n3 ) − O(n4 ) Density Functional Theory O(n3 ) − O(n4 ) MP2 O(n5 ) CCSD O(n6 ) CCSD(T) O(n7 ) Full CI O(n!)
- 5. Representation Neural network architectures for atomistic systems
- 6. Atomistic Systems Molecules Materials S = {(Zi, ri)| i ∈ [1, natoms]} Challenges: rotational / translational invariance permutational invariance symmetries / local features
- 7. There is an abundance of features for atomistic systems... α β β α α β r dr α β
- 8. ... so what is missing? Drawbacks of ﬁxed representations Property-speciﬁc similarity Do similar systems w.r.t energy also have similar dipole moments? Cross-element generalization How does silicon behave compared to carbon? Diﬀerent tasks at diﬀerent scales Chemical Compound Space vs. Potentials Energy Surfaces Solution End-to-end learning from atom types and positions Representation is adapted during training
- 9. The Deep Tensor Neural Network (DTNN) framework 1. Embed atom types x (0) i = AZi ∈ Rd 2. Add interactions x (t+1) i = x (t) i + j=i v(t) x (t) j , ri − rj 3. Predict via atom-wise contributions: ˆE = natoms i=1 E(x (T) i ) Sch¨utt, Arbabzadah, Chmiela, M¨uller, Tkatchenko, Nature Communications 8, 13890 (2017)
- 10. The Deep Tensor Neural Network (DTNN) framework 1. Embed atom types x (0) i = AZi ∈ Rd 2. Add interactions x (t+1) i = x (t) i + j=i v(t) x (t) j , ri − rj 3. Predict via atom-wise contributions: ˆE = natoms i=1 E(x (T) i ) Sch¨utt, Arbabzadah, Chmiela, M¨uller, Tkatchenko, Nature Communications 8, 13890 (2017)
- 11. The Deep Tensor Neural Network (DTNN) framework 1. Embed atom types x (0) i = AZi ∈ Rd 2. Add interactions x (t+1) i = x (t) i + j=i v(t) x (t) j , ri − rj 3. Predict via atom-wise contributions: ˆE = natoms i=1 E(x (T) i ) Sch¨utt, Arbabzadah, Chmiela, M¨uller, Tkatchenko, Nature Communications 8, 13890 (2017)
- 12. SchNet – quantum interactions from convolutions (x ∗ W )(ri) = Natom j=1 x (t) j ◦ W (t) [ri−rj] parameter tensor (x ∗ W )(ri) = Natom j=1 x (t) j ◦ W (t) (ri − rj) neural network
- 13. SchNet - a continuous-ﬁlter convolutional neural network K.T. Sch¨utt, P.-J. Kindermans, H.E. Sauceda, S. Chmiela, A. Tkatchenko, K.-R. M¨uller (2017). SchNet: A continuous-ﬁlter convolutional neural network for modeling quantum interactions. NIPS 30.
- 14. Property-speciﬁc output layers Internal energy U0 Dipole moment µ SchNet output layer E = i E(xi) µ = i q(xi)(ri − r0) QM9 – 110k ref. calculations – mean abs. errors SchNet (T=6, SGDR) 0.218 kcal mol−1 0.017 Debye HIP-NN[1] 0.256 kcal mol−1 – Message-passing NN[2] 0.450 kcal mol−1 0.030 Debye [1] N. Lubbers, J.S. Smith, K. Barros (2018). Hierarchical modeling of molecular energies using a deep neural network. The Journal of Chemical Physics, 148(24), 241715. [2] J. Gilmer, S.S. Schoenholz, P.F. Riley, O. Vinyals, G.E. Dahl. Neural-Message Passing for Quantum Chemistry (2017). ICML.
- 15. Analysis Insights about trained models and underlying data
- 16. Local chemical potentials virtual atom with charge Zp at position rp x (t+1) p = x (t) i + j v(t) x (t) j , rp − rj probe energy from output network ΩZp (r) = fout(x(T) p )
- 17. Moving through alchemical space
- 18. Learning the periodic table of elements 1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 1st principal component 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02nd principalcomponent H Li CsRb K Na Ba Sr Mg Ca Be Ga In Al B Tl Si C PbSn Ge As Bi P N Sb Se Te S O I F Cl Br Ne Xe Ar He Kr I II III IV V VI VII VIII Group II Group I G roup V 1.5 Trained on 60k bulk crystals from the Materials Project.
- 19. Application Accelerated molecular dynamics simulations
- 20. Molecular dynamics – Training with forces Atomic forces can eﬃciently be obtained by diﬀerentiating the network: Fi(r1, . . . , rn) = − ∂E ∂ri (r1, . . . , rn), yielding energy-conserving, rotationally equivariant force ﬁeld predictions. Combined loss of total energy E and atomic forces Fi for training: ( ˆE, (E, F1, . . . , Fn)) = ρ E− ˆE 2 + 1 Natoms Natoms i=0 − ∂ ˆE ∂ri −Fi 2 . 0 100 200 time step 0 10 20 30 40 totalenergy[kcalmol1 ]
- 21. Molecular dynamics – PIMD of the fullerene C20
- 22. Molecular dynamics – PIMD of the fullerene C20 Accurate prediction of vibrational frequencies 0 200 400 600 800 1000 1200 1400 frequency [cm 1] spectrum DFT (PBE-TS) SchNet (PBE-TS) 0 200 400 600 800 1000 1200 1400 frequency [cm 1] 5 0 5 DFTSchNet[cm1] PIMD@SchNet shows delocalization of bonds 1.5 2.0 2.5 3.0 3.5 4.0 4.5 r [Å] 0.0 0.5 1.0 1.5 2.0 2.5 h(r)[a.u.] P= 1 P= 8 1.3 1.4 1.5 1.6 1.7 nearest C-C [Å] 0 2 4 6 8 10 distribution 3.8 4.0 4.2 4.4 4.6 diameter [Å] 0 1 2 3 distribution SchNet enables generation of 1.25ns PIMD trajectory by 3-4 orders of magnitude: 7 years ⇒ 7 hours
- 23. SchNetPack P spk.Atomwise SchNet spk.atomistic spk.DipoleMoment wACSFZ spk.representation R Z R Angular Concatenate Radial Embedding Interaction Interaction RZ www.quantum-machine.org/schnetpack
- 24. SchNetPack P spk.Atomwise SchNet spk.atomistic spk.DipoleMoment wACSFZ spk.representation R Z R Angular Concatenate Radial Embedding Interaction Interaction RZ www.quantum-machine.org/schnetpack Thank you!