Frank Noé's slides on using deep learning for solving physics many-body problems. From talks at Perimeter Institute and Vector AI Institute, Canada, Nov 2019.
4. Adaptive Markov State Model — seconds to hours kinetics
Reinforcement learning
Plattner, Doerr, De Fabritiis, Noé
Nature Chemistry 9, 1005 (2017)
total 2 ms data
Samping problem of many-body physics systems
10. Flows and Normalizing Flows
pX (x) = pZ(f(x))
@f(x)
@x>
<latexit sha1_base64="3rYWslgzJsBx6CLZP+nUj15gDRw=">AAACVHicbZFNS8NAEIY3qdVatVY9elksQr2URAS9CEUvHitYLTY1bLabdukmWXYnYknzI/Ug+Eu8eHDTFvwcWHh5ZoaZeTeQgmtwnDfLLq2UV9cq69WNza3adn1n91YnqaKsSxORqF5ANBM8Zl3gIFhPKkaiQLC7YHJZ5O8emdI8iW9gKtkgIqOYh5wSMMivT6Tfa3oRgXEQZk/5ET7H0r9vht/ZEfYEC2GGvVARmnmSKOBE4B9F+Rf/og8eJDLHnuKjMcz8esNpOfPAf4W7FA20jI5ff/GGCU0jFgMVROu+60gYZMUYKlhe9VLNJKETMmJ9I2MSMT3I5qbk+NCQIQ4TZV4MeE6/d2Qk0noaBaay2Ff/zhXwv1w/hfBskPFYpsBiuhgUpgJDgguH8ZArRkFMjSBUcbMrpmNinAPzD1Vjgvv75L/i9rjlOi33+qTRvljaUUH76AA1kYtOURtdoQ7qIoqe0buFLMt6tT7skl1elNrWsmcP/Qi79gn6x7RT</latexit><latexit sha1_base64="3rYWslgzJsBx6CLZP+nUj15gDRw=">AAACVHicbZFNS8NAEIY3qdVatVY9elksQr2URAS9CEUvHitYLTY1bLabdukmWXYnYknzI/Ug+Eu8eHDTFvwcWHh5ZoaZeTeQgmtwnDfLLq2UV9cq69WNza3adn1n91YnqaKsSxORqF5ANBM8Zl3gIFhPKkaiQLC7YHJZ5O8emdI8iW9gKtkgIqOYh5wSMMivT6Tfa3oRgXEQZk/5ET7H0r9vht/ZEfYEC2GGvVARmnmSKOBE4B9F+Rf/og8eJDLHnuKjMcz8esNpOfPAf4W7FA20jI5ff/GGCU0jFgMVROu+60gYZMUYKlhe9VLNJKETMmJ9I2MSMT3I5qbk+NCQIQ4TZV4MeE6/d2Qk0noaBaay2Ff/zhXwv1w/hfBskPFYpsBiuhgUpgJDgguH8ZArRkFMjSBUcbMrpmNinAPzD1Vjgvv75L/i9rjlOi33+qTRvljaUUH76AA1kYtOURtdoQ7qIoqe0buFLMt6tT7skl1elNrWsmcP/Qi79gn6x7RT</latexit><latexit sha1_base64="3rYWslgzJsBx6CLZP+nUj15gDRw=">AAACVHicbZFNS8NAEIY3qdVatVY9elksQr2URAS9CEUvHitYLTY1bLabdukmWXYnYknzI/Ug+Eu8eHDTFvwcWHh5ZoaZeTeQgmtwnDfLLq2UV9cq69WNza3adn1n91YnqaKsSxORqF5ANBM8Zl3gIFhPKkaiQLC7YHJZ5O8emdI8iW9gKtkgIqOYh5wSMMivT6Tfa3oRgXEQZk/5ET7H0r9vht/ZEfYEC2GGvVARmnmSKOBE4B9F+Rf/og8eJDLHnuKjMcz8esNpOfPAf4W7FA20jI5ff/GGCU0jFgMVROu+60gYZMUYKlhe9VLNJKETMmJ9I2MSMT3I5qbk+NCQIQ4TZV4MeE6/d2Qk0noaBaay2Ff/zhXwv1w/hfBskPFYpsBiuhgUpgJDgguH8ZArRkFMjSBUcbMrpmNinAPzD1Vjgvv75L/i9rjlOi33+qTRvljaUUH76AA1kYtOURtdoQ7qIoqe0buFLMt6tT7skl1elNrWsmcP/Qi79gn6x7RT</latexit><latexit sha1_base64="3rYWslgzJsBx6CLZP+nUj15gDRw=">AAACVHicbZFNS8NAEIY3qdVatVY9elksQr2URAS9CEUvHitYLTY1bLabdukmWXYnYknzI/Ug+Eu8eHDTFvwcWHh5ZoaZeTeQgmtwnDfLLq2UV9cq69WNza3adn1n91YnqaKsSxORqF5ANBM8Zl3gIFhPKkaiQLC7YHJZ5O8emdI8iW9gKtkgIqOYh5wSMMivT6Tfa3oRgXEQZk/5ET7H0r9vht/ZEfYEC2GGvVARmnmSKOBE4B9F+Rf/og8eJDLHnuKjMcz8esNpOfPAf4W7FA20jI5ff/GGCU0jFgMVROu+60gYZMUYKlhe9VLNJKETMmJ9I2MSMT3I5qbk+NCQIQ4TZV4MeE6/d2Qk0noaBaay2Ff/zhXwv1w/hfBskPFYpsBiuhgUpgJDgguH8ZArRkFMjSBUcbMrpmNinAPzD1Vjgvv75L/i9rjlOi33+qTRvljaUUH76AA1kYtOURtdoQ7qIoqe0buFLMt6tT7skl1elNrWsmcP/Qi79gn6x7RT</latexit>
Main idea: Transformation of random variables for bijective transformations:
RealNVP: Dinh, Sohl-Dickstein, S. Bengio, ICLR 2017
PixelRNN: van den Oord et al, ICML 2016
*Neural ODEs: Chen et Al, NeurIPS 2018
*
PDE Flows: Tabak, Vanden-Eijnden, Commun. Math. Sci. 2010
NICE: Dinh, Krueger, Y. Bengio, ICLR 2015
Normalizing Flows: Rezende, Mohamed, ICML 2015
11. Boltzmann Generators: Exact probability generator + reweighting
Noé, Olsson, Köhler, Wu, Science 365: eaww1147 (2019). arXiv:1812.01729
Boltzmann Generators learn to generate
unbiased samples from target distribution e-u(x)
12. Invertible Networks
T neural network
. . . x1
x2
y1
y2
NICE Layer
T
+
x1
x2
y1
y2
NICE-1 Layer
T
-
. . .
...
...
...
...
=σ(Σiwixi+b)
RealNVP: Dinh, Sohl-Dickstein, S. Bengio, ICLR 2017
PixelRNN: van den Oord et al, ICML 2016
*Neural ODEs: Chen et Al, NeurIPS 2018
PDE Flows: Tabak, Vanden-Eijnden, Commun. Math. Sci. 2010
NICE: Dinh, Krueger, Y. Bengio, ICLR 2015
Normalizing Flows: Rezende, Mohamed, ICML 2015
13. S/T neural network
. . .
S
x1
x2
y1
y2*
RealNVP Layer
T
+
S
x1
x2
y1
y2/
RealNVP-1 Layer
T
-
. . .
...
...
...
...
=σ(Σiwixi+b)
z
x
pX(x)
Fzx Fxz
...
f1
...
...
f1
...
-1
fn fn
-1
pZ(z)
PDE Flows: Tabak, Vanden-Eijnden, Commun. Math. Sci. 2010
NICE: Dinh, Krueger, Y. Bengio, ICLR 2015
Normalizing Flows: Rezende, Mohamed, ICML 2015
RealNVP: Dinh, Sohl-Dickstein, S. Bengio, ICLR 2017
PixelRNN: van den Oord et al, ICML 2016
*Neural ODEs: Chen et Al, NeurIPS 2018
Invertible Networks
14. Boltzmann Generators: training with variational free energy
Noé, Olsson, Köhler, Wu, Science 365: eaww1147 (2019). arXiv:1812.01729
1. Train with known energy u(x)
Minimize KL divergence between generated and Boltzmann distribution
Sample
Gaussian
Energy of
generated samples
Jacobian (Entropy) of
generated samples
2. Maximum generator likelihood of training data x
Minimize KL divergence between latent and Gaussian distribution
Sample
Data
Energy in
Harmonic oscillator
Jacobian (Entropy) of
data samples
15. Boltzmann Generators: Double well
Noé, Olsson, Köhler, Wu, Science 365: eaww1147 (2019). arXiv:1812.01729
Data X
Boltzmann
Generator
train
latent-space
MCMC
Adaptive Exploration from one configuration:
Train with data from multiple states:
24. 1/10
Schrödinger Equation
Electronic Schrödinger Equation:
ˆHy(r1,...,rN) = Ey(r1,...,rN)
r = (r1,...,rN)> 2 R3N spatial electron coordinates
E electronic energy, y electronic wave function.
Hamiltonian operator:
ˆH := Â
i
✓
1
2 —2
ri Â
I
ZI
|ri RI|
◆
+ Â
i<j
1
|ri rj|
ZI nuclear charges
RI fixed nuclear coordinates (Born--Oppenheimer approximation).
Electronic spins si 2 {",#}.
y must be antisymmetric wrt exchange of equal-spin electrons
y(...,ri ,...,rj,...) = y(...,rj,...,ri ,...)
Electronic Schrödinger Equation
25. 1/10
Schrödinger Equation
Electronic Schrödinger Equation:
ˆHy(r1,...,rN) = Ey(r1,...,rN)
r = (r1,...,rN)> 2 R3N spatial electron coordinates
E electronic energy, y electronic wave function.
Hamiltonian operator:
ˆH := Â
i
✓
1
2 —2
ri Â
I
ZI
|ri RI|
◆
+ Â
i<j
1
|ri rj|
ZI nuclear charges
RI fixed nuclear coordinates (Born--Oppenheimer approximation).
Electronic spins si 2 {",#}.
y must be antisymmetric wrt exchange of equal-spin electrons
y(...,ri ,...,rj,...) = y(...,rj,...,ri ,...)
Electronic Schrödinger Equation
26. 1/10
Schrödinger Equation
Electronic Schrödinger Equation:
ˆHy(r1,...,rN) = Ey(r1,...,rN)
r = (r1,...,rN)> 2 R3N spatial electron coordinates
E electronic energy, y electronic wave function.
Hamiltonian operator:
ˆH := Â
i
✓
1
2 —2
ri Â
I
ZI
|ri RI|
◆
+ Â
i<j
1
|ri rj|
ZI nuclear charges
RI fixed nuclear coordinates (Born--Oppenheimer approximation).
Electronic spins si 2 {",#}.
y must be antisymmetric wrt exchange of equal-spin electrons
y(...,ri ,...,rj,...) = y(...,rj,...,ri ,...)
Electronic Schrödinger Equation - Antisymmetric wavefunctions
1/10
r = (r1,...,rN)> 2 R3N spatial electron coordinates
E electronic energy, y electronic wave function.
Hamiltonian operator:
ˆH := Â
i
✓
1
2 —2
ri Â
I
ZI
|ri RI|
◆
+ Â
i<j
1
|ri rj|
ZI nuclear charges
RI fixed nuclear coordinates (Born--Oppenheimer approximation).
Electronic spins si 2 {",#}.
y must be antisymmetric wrt exchange of equal-spin electrons
y(...,ri ,...,rj,...) = y(...,rj,...,ri ,...)
Pauli exclusion
27. Variational Approach — Keystone for machine learning
2/10
Variational Monte Carlo
Variational Approach (used in Hartree-Fock, DMRG, QMC)
E0 = min
y
E[y] min
q
E[y(r;q)] E[y] =
Z
dry(r)ˆHy(r)
VMC: Variational Quantum Monte Carlo:
E[y;q] = Er⇠|y|2
⇥
Eloc[y](r;q)
⇤
Eloc[y](r;q) = ˆHy(r;q)/y(r;q)
Deep VMC idea: use deep neural network to represent y(r;q),
minimize variational energy E[y;q] over q.
G. Carleo, M. Troyer, Science 355:602-606 (2017): Discrete lattice
systems, Restricted Boltzmann Machines.
DeepWF: J. Han et al, arXiv:1807.07014 (2018):
Atoms+Molecules, Poor accuracy, moderate computational cost.
FermiNet: D. Pfau et al, arXiv:1909.02487 (Sept 2019):
Atoms+Molecules, highest accuracy, insane computational cost.
PauliNet: This work, arXiv:1909.08423 (Sept 2019):
Atoms+Molecules, high accuracy, moderate computational cost.
Machine Learning loss function Represent with neural networkariational Monte Carlo
Variational Approach (used in Hartree-Fock, DMRG, QMC)
E0 = min
y
E[y] min
q
E[y(r;q)] E[y] =
Z
dry(r)ˆHy(r)
VMC: Variational Quantum Monte Carlo:
E[y;q] = Er⇠|y|2
⇥
Eloc[y](r;q)
⇤
Eloc[y](r;q) = ˆHy(r;q)/y(r;q)
Deep VMC idea: use deep neural network to represent y(r;q),
minimize variational energy E[y;q] over q.
G. Carleo, M. Troyer, Science 355:602-606 (2017): Discrete lattice
systems, Restricted Boltzmann Machines.
riational Monte Carlo
Variational Approach (used in Hartree-Fock, DMRG, QMC)
E0 = min
y
E[y] min
q
E[y(r;q)] E[y] =
Z
dry(r)ˆHy(r)
VMC: Variational Quantum Monte Carlo:
E[y;q] = Er⇠|y|2
⇥
Eloc[y](r;q)
⇤
Eloc[y](r;q) = ˆHy(r;q)/y(r;q)
Deep VMC idea: use deep neural network to represent y(r;q),
minimize variational energy E[y;q] over q.
Questions:
• Computation: How do we evaluate ?
• Representation: How do we build physics into ?
(antisymmetry, asymptotics, …)
28. Quantum Monte Carlo is the natural choice!
2/10
Variational Monte Carlo
Variational Approach (used in Hartree-Fock, DMRG, QMC)
E0 = min
y
E[y] min
q
E[y(r;q)] E[y] =
Z
dry(r)ˆHy(r)
VMC: Variational Quantum Monte Carlo:
E[y;q] = Er⇠|y|2
⇥
Eloc[y](r;q)
⇤
Eloc[y](r;q) = ˆHy(r;q)/y(r;q)
Deep VMC idea: use deep neural network to represent y(r;q),
minimize variational energy E[y;q] over q.
G. Carleo, M. Troyer, Science 355:602-606 (2017): Discrete lattice
systems, Restricted Boltzmann Machines.
DeepWF: J. Han et al, arXiv:1807.07014 (2018):
Atoms+Molecules, Poor accuracy, moderate computational cost.
FermiNet: D. Pfau et al, arXiv:1909.02487 (Sept 2019):
Atoms+Molecules, highest accuracy, insane computational cost.
PauliNet: This work, arXiv:1909.08423 (Sept 2019):
Atoms+Molecules, high accuracy, moderate computational cost.
Sample Minibatches
29. Why is Quantum Monte Carlo an accurate method?
Var{E[ ; ✓]} =
Varr{Eloc[ ](r; ✓)}
Nsample<latexit sha1_base64="taghiYfPmtqV60nXzLAmcZItaQc=">AAAFcnicjVRdb9MwFM22Fkb52kC8gASGqWiVRpV0SCChSRNo0l6YhsQ+pDqLHMdprTkfsp2xyvM7v483fgUv/ADsLG2zdRuzFOXm3HPPPb5JHOaMCum6v+fmFxrNO3cX77XuP3j46PHS8pN9kRUckz2csYwfhkgQRlOyJ6lk5DDnBCUhIwfh8RebPzghXNAs/S5HOfETNEhpTDGSBgqWF37CBMkhT9Q+4hqqrT7MBf0E5ZBI5EMNNgCMOcKqTgvKhzBWZUUwTrEM67LcX50SKqkO1FrtTKgCJTkjWrfa+YR7qDu2G01loL5O0FPd2ZgyNMjrGTDWi/QUrWueGq8SFaWwhTBiavccu6AUuJ1WGw6RVNva7sDwt4Jp2kLtOt1b63a7azVgpwOMwOQZQEGTiTuij9S7rf/UG98wIrIPw4xFYpSYm4IniOdDagde1GYa0CMFixxxnv3QHd+YNbWMxPLMBiEZ0FSZJBpphc3SFj1XCry6jFeXAW/BmNS7kYSjTIoae/16NoTXdO7dpvMMadz56ta9K1pPrFbBOLzW2PptjM2QbhzJ+qwvkkbVC7KvhtPBUJ4FSytu1y0XmA28KlhxqrUbLP2CUYaLhKQSMyRE33Nz6RtdSbH5t1qwECRH+BgNSN+EKUqI8FV5ZGjQNkgE4oybK5WgROsVCiXCfoSGaXciLucseFWuX8j4o69omheSpPi8UVwwIDNgzx8QUU6wZCMTIMyp8QrwEJkTRppTqmWG4F3e8myw3+t6btf79n5l83M1jkXnhfPGWXU854Oz6Ww7u86egxf+NJ41XjZeNf42nzdfN6vZzc9VNU+dC6u59g8fndj6</latexit><latexit sha1_base64="taghiYfPmtqV60nXzLAmcZItaQc=">AAAFcnicjVRdb9MwFM22Fkb52kC8gASGqWiVRpV0SCChSRNo0l6YhsQ+pDqLHMdprTkfsp2xyvM7v483fgUv/ADsLG2zdRuzFOXm3HPPPb5JHOaMCum6v+fmFxrNO3cX77XuP3j46PHS8pN9kRUckz2csYwfhkgQRlOyJ6lk5DDnBCUhIwfh8RebPzghXNAs/S5HOfETNEhpTDGSBgqWF37CBMkhT9Q+4hqqrT7MBf0E5ZBI5EMNNgCMOcKqTgvKhzBWZUUwTrEM67LcX50SKqkO1FrtTKgCJTkjWrfa+YR7qDu2G01loL5O0FPd2ZgyNMjrGTDWi/QUrWueGq8SFaWwhTBiavccu6AUuJ1WGw6RVNva7sDwt4Jp2kLtOt1b63a7azVgpwOMwOQZQEGTiTuij9S7rf/UG98wIrIPw4xFYpSYm4IniOdDagde1GYa0CMFixxxnv3QHd+YNbWMxPLMBiEZ0FSZJBpphc3SFj1XCry6jFeXAW/BmNS7kYSjTIoae/16NoTXdO7dpvMMadz56ta9K1pPrFbBOLzW2PptjM2QbhzJ+qwvkkbVC7KvhtPBUJ4FSytu1y0XmA28KlhxqrUbLP2CUYaLhKQSMyRE33Nz6RtdSbH5t1qwECRH+BgNSN+EKUqI8FV5ZGjQNkgE4oybK5WgROsVCiXCfoSGaXciLucseFWuX8j4o69omheSpPi8UVwwIDNgzx8QUU6wZCMTIMyp8QrwEJkTRppTqmWG4F3e8myw3+t6btf79n5l83M1jkXnhfPGWXU854Oz6Ww7u86egxf+NJ41XjZeNf42nzdfN6vZzc9VNU+dC6u59g8fndj6</latexit><latexit sha1_base64="taghiYfPmtqV60nXzLAmcZItaQc=">AAAFcnicjVRdb9MwFM22Fkb52kC8gASGqWiVRpV0SCChSRNo0l6YhsQ+pDqLHMdprTkfsp2xyvM7v483fgUv/ADsLG2zdRuzFOXm3HPPPb5JHOaMCum6v+fmFxrNO3cX77XuP3j46PHS8pN9kRUckz2csYwfhkgQRlOyJ6lk5DDnBCUhIwfh8RebPzghXNAs/S5HOfETNEhpTDGSBgqWF37CBMkhT9Q+4hqqrT7MBf0E5ZBI5EMNNgCMOcKqTgvKhzBWZUUwTrEM67LcX50SKqkO1FrtTKgCJTkjWrfa+YR7qDu2G01loL5O0FPd2ZgyNMjrGTDWi/QUrWueGq8SFaWwhTBiavccu6AUuJ1WGw6RVNva7sDwt4Jp2kLtOt1b63a7azVgpwOMwOQZQEGTiTuij9S7rf/UG98wIrIPw4xFYpSYm4IniOdDagde1GYa0CMFixxxnv3QHd+YNbWMxPLMBiEZ0FSZJBpphc3SFj1XCry6jFeXAW/BmNS7kYSjTIoae/16NoTXdO7dpvMMadz56ta9K1pPrFbBOLzW2PptjM2QbhzJ+qwvkkbVC7KvhtPBUJ4FSytu1y0XmA28KlhxqrUbLP2CUYaLhKQSMyRE33Nz6RtdSbH5t1qwECRH+BgNSN+EKUqI8FV5ZGjQNkgE4oybK5WgROsVCiXCfoSGaXciLucseFWuX8j4o69omheSpPi8UVwwIDNgzx8QUU6wZCMTIMyp8QrwEJkTRppTqmWG4F3e8myw3+t6btf79n5l83M1jkXnhfPGWXU854Oz6Ww7u86egxf+NJ41XjZeNf42nzdfN6vZzc9VNU+dC6u59g8fndj6</latexit><latexit sha1_base64="taghiYfPmtqV60nXzLAmcZItaQc=">AAAFcnicjVRdb9MwFM22Fkb52kC8gASGqWiVRpV0SCChSRNo0l6YhsQ+pDqLHMdprTkfsp2xyvM7v483fgUv/ADsLG2zdRuzFOXm3HPPPb5JHOaMCum6v+fmFxrNO3cX77XuP3j46PHS8pN9kRUckz2csYwfhkgQRlOyJ6lk5DDnBCUhIwfh8RebPzghXNAs/S5HOfETNEhpTDGSBgqWF37CBMkhT9Q+4hqqrT7MBf0E5ZBI5EMNNgCMOcKqTgvKhzBWZUUwTrEM67LcX50SKqkO1FrtTKgCJTkjWrfa+YR7qDu2G01loL5O0FPd2ZgyNMjrGTDWi/QUrWueGq8SFaWwhTBiavccu6AUuJ1WGw6RVNva7sDwt4Jp2kLtOt1b63a7azVgpwOMwOQZQEGTiTuij9S7rf/UG98wIrIPw4xFYpSYm4IniOdDagde1GYa0CMFixxxnv3QHd+YNbWMxPLMBiEZ0FSZJBpphc3SFj1XCry6jFeXAW/BmNS7kYSjTIoae/16NoTXdO7dpvMMadz56ta9K1pPrFbBOLzW2PptjM2QbhzJ+qwvkkbVC7KvhtPBUJ4FSytu1y0XmA28KlhxqrUbLP2CUYaLhKQSMyRE33Nz6RtdSbH5t1qwECRH+BgNSN+EKUqI8FV5ZGjQNkgE4oybK5WgROsVCiXCfoSGaXciLucseFWuX8j4o69omheSpPi8UVwwIDNgzx8QUU6wZCMTIMyp8QrwEJkTRppTqmWG4F3e8myw3+t6btf79n5l83M1jkXnhfPGWXU854Oz6Ww7u86egxf+NJ41XjZeNf42nzdfN6vZzc9VNU+dC6u59g8fndj6</latexit>
Monte Carlo converges slowly:
But: Variance of local energy decreases during training
2/10
Monte Carlo
al Approach (used in Hartree-Fock, DMRG, QMC)
min
y
E[y] min
q
E[y(r;q)] E[y] =
Z
dry(r)ˆHy(r)
riational Quantum Monte Carlo:
Er⇠|y|2
⇥
Eloc[y](r;q)
⇤
Eloc[y](r;q) = ˆHy(r;q)/y(r;q)
MC idea: use deep neural network to represent y(r;q),
variational energy E[y;q] over q.
rleo, M. Troyer, Science 355:602-606 (2017): Discrete lattice
ms, Restricted Boltzmann Machines.
WF: J. Han et al, arXiv:1807.07014 (2018):
s+Molecules, Poor accuracy, moderate computational cost.
iNet: D. Pfau et al, arXiv:1909.02487 (Sept 2019):
s+Molecules, highest accuracy, insane computational cost.
Net: This work, arXiv:1909.08423 (Sept 2019):
s+Molecules, high accuracy, moderate computational cost.
At the eigenstate: = constant in r !
ˆH = E<latexit sha1_base64="k812nidaC5QC9teNzsV6KtlheyM=">AAAFqHicjVRta9swEHbbZOuyl7bbx30RCxnN6IKdDjYYhbIRKIyVDJq2I3KNLMuJqPyCJHcNqv7afsS+7d9MdpzEbdquB8an55577k6y5aeMCmnbf1dW12r1R4/XnzSePnv+YmNz6+WxSDKOyQAnLOGnPhKE0ZgMJJWMnKacoMhn5MQ//5rHTy4IFzSJj+QkJW6ERjENKUbSQN7W2u8WjJAc80gdI66h6g1hKuhnKMdEIhdqsAdgyBFWVZpXLPxQFRneLMQSrIt0d3tBKKXaUGt1OKcKFKWMaN1opXPuqW7n1WgsPfV9jl7q9t6CoUFajYCZXqAXaFXz0vQqUVYI5xBGTPWn2DUlz2434BhJdaDzAQy9l79bVYqz0+l0dirAYRs0WjeyvEX8oQLzNYCCRvORiD5T73v/yTfDwoDIIfQTFohJZF4KXiCejml+SlnlIDx6pmCWIs6TX7rtmmZNLiOhvModn4xorEwQTbTCxnSOTpU8pyrjVGXAWzAjde8l4SCRosLevZsN4R2Vuw+pvESaVb69dPeW0vNWS2fm3tnY7kMaWyLduyW7y32ROCgPKD8aTkdjeeVtNu2OXRhYdpzSaVql9b3NPzBIcBaRWGKGhBg6dipdoyspNj9kA2aCpAifoxEZGjdGERGuKi4aDVoGCUCYcPPEEhRoNUOhSOQfoWHmk4ibsRy8LTbMZPjJVTROM0liPC0UZgzIBOS3FggoJ1iyiXEQ5tT0CvAYmWtJmrutYTbBuTnysnPc7Th2x/nxobn/pdyOdeu19cbathzro7VvHVh9a2DhWqv2rXZUG9Tf1fv1k/rPKXV1pcx5ZV2zuv8PBNHrYw==</latexit><latexit sha1_base64="k812nidaC5QC9teNzsV6KtlheyM=">AAAFqHicjVRta9swEHbbZOuyl7bbx30RCxnN6IKdDjYYhbIRKIyVDJq2I3KNLMuJqPyCJHcNqv7afsS+7d9MdpzEbdquB8an55577k6y5aeMCmnbf1dW12r1R4/XnzSePnv+YmNz6+WxSDKOyQAnLOGnPhKE0ZgMJJWMnKacoMhn5MQ//5rHTy4IFzSJj+QkJW6ERjENKUbSQN7W2u8WjJAc80gdI66h6g1hKuhnKMdEIhdqsAdgyBFWVZpXLPxQFRneLMQSrIt0d3tBKKXaUGt1OKcKFKWMaN1opXPuqW7n1WgsPfV9jl7q9t6CoUFajYCZXqAXaFXz0vQqUVYI5xBGTPWn2DUlz2434BhJdaDzAQy9l79bVYqz0+l0dirAYRs0WjeyvEX8oQLzNYCCRvORiD5T73v/yTfDwoDIIfQTFohJZF4KXiCejml+SlnlIDx6pmCWIs6TX7rtmmZNLiOhvModn4xorEwQTbTCxnSOTpU8pyrjVGXAWzAjde8l4SCRosLevZsN4R2Vuw+pvESaVb69dPeW0vNWS2fm3tnY7kMaWyLduyW7y32ROCgPKD8aTkdjeeVtNu2OXRhYdpzSaVql9b3NPzBIcBaRWGKGhBg6dipdoyspNj9kA2aCpAifoxEZGjdGERGuKi4aDVoGCUCYcPPEEhRoNUOhSOQfoWHmk4ibsRy8LTbMZPjJVTROM0liPC0UZgzIBOS3FggoJ1iyiXEQ5tT0CvAYmWtJmrutYTbBuTnysnPc7Th2x/nxobn/pdyOdeu19cbathzro7VvHVh9a2DhWqv2rXZUG9Tf1fv1k/rPKXV1pcx5ZV2zuv8PBNHrYw==</latexit><latexit sha1_base64="k812nidaC5QC9teNzsV6KtlheyM=">AAAFqHicjVRta9swEHbbZOuyl7bbx30RCxnN6IKdDjYYhbIRKIyVDJq2I3KNLMuJqPyCJHcNqv7afsS+7d9MdpzEbdquB8an55577k6y5aeMCmnbf1dW12r1R4/XnzSePnv+YmNz6+WxSDKOyQAnLOGnPhKE0ZgMJJWMnKacoMhn5MQ//5rHTy4IFzSJj+QkJW6ERjENKUbSQN7W2u8WjJAc80gdI66h6g1hKuhnKMdEIhdqsAdgyBFWVZpXLPxQFRneLMQSrIt0d3tBKKXaUGt1OKcKFKWMaN1opXPuqW7n1WgsPfV9jl7q9t6CoUFajYCZXqAXaFXz0vQqUVYI5xBGTPWn2DUlz2434BhJdaDzAQy9l79bVYqz0+l0dirAYRs0WjeyvEX8oQLzNYCCRvORiD5T73v/yTfDwoDIIfQTFohJZF4KXiCejml+SlnlIDx6pmCWIs6TX7rtmmZNLiOhvModn4xorEwQTbTCxnSOTpU8pyrjVGXAWzAjde8l4SCRosLevZsN4R2Vuw+pvESaVb69dPeW0vNWS2fm3tnY7kMaWyLduyW7y32ROCgPKD8aTkdjeeVtNu2OXRhYdpzSaVql9b3NPzBIcBaRWGKGhBg6dipdoyspNj9kA2aCpAifoxEZGjdGERGuKi4aDVoGCUCYcPPEEhRoNUOhSOQfoWHmk4ibsRy8LTbMZPjJVTROM0liPC0UZgzIBOS3FggoJ1iyiXEQ5tT0CvAYmWtJmrutYTbBuTnysnPc7Th2x/nxobn/pdyOdeu19cbathzro7VvHVh9a2DhWqv2rXZUG9Tf1fv1k/rPKXV1pcx5ZV2zuv8PBNHrYw==</latexit><latexit sha1_base64="k812nidaC5QC9teNzsV6KtlheyM=">AAAFqHicjVRta9swEHbbZOuyl7bbx30RCxnN6IKdDjYYhbIRKIyVDJq2I3KNLMuJqPyCJHcNqv7afsS+7d9MdpzEbdquB8an55577k6y5aeMCmnbf1dW12r1R4/XnzSePnv+YmNz6+WxSDKOyQAnLOGnPhKE0ZgMJJWMnKacoMhn5MQ//5rHTy4IFzSJj+QkJW6ERjENKUbSQN7W2u8WjJAc80gdI66h6g1hKuhnKMdEIhdqsAdgyBFWVZpXLPxQFRneLMQSrIt0d3tBKKXaUGt1OKcKFKWMaN1opXPuqW7n1WgsPfV9jl7q9t6CoUFajYCZXqAXaFXz0vQqUVYI5xBGTPWn2DUlz2434BhJdaDzAQy9l79bVYqz0+l0dirAYRs0WjeyvEX8oQLzNYCCRvORiD5T73v/yTfDwoDIIfQTFohJZF4KXiCejml+SlnlIDx6pmCWIs6TX7rtmmZNLiOhvModn4xorEwQTbTCxnSOTpU8pyrjVGXAWzAjde8l4SCRosLevZsN4R2Vuw+pvESaVb69dPeW0vNWS2fm3tnY7kMaWyLduyW7y32ROCgPKD8aTkdjeeVtNu2OXRhYdpzSaVql9b3NPzBIcBaRWGKGhBg6dipdoyspNj9kA2aCpAifoxEZGjdGERGuKi4aDVoGCUCYcPPEEhRoNUOhSOQfoWHmk4ibsRy8LTbMZPjJVTROM0liPC0UZgzIBOS3FggoJ1iyiXEQ5tT0CvAYmWtJmrutYTbBuTnysnPc7Th2x/nxobn/pdyOdeu19cbathzro7VvHVh9a2DhWqv2rXZUG9Tf1fv1k/rPKXV1pcx5ZV2zuv8PBNHrYw==</latexit>
Var{E[ ; ✓]} = 0<latexit sha1_base64="+SnZsH5g5Ndit21HyxBQCnNzgAE=">AAAFzHicjVRba9swFHbbZOuyW7s97kWsZDTQBTsZbDAKZaPQPax0sF4gco0sy4mofEGSuwZVr/uBe9vrfskkx0ncprcDxkff+c53zpFshTmjQrru36XllUbz0ePVJ62nz56/eLm2/upIZAXH5BBnLOMnIRKE0ZQcSioZOck5QUnIyHF49tXGj88JFzRLf8pxTvwEDVMaU4ykgYL1lX8wQXLEE3WEuIZqdwBzQT9DOSIS+VCDbeC22vdyYMwRVnVaUC7CWJUZwTTEMqzLdH9zTqikOlBrtT+jCpTkjGjdaucz7onu2Go0lYH6PkMvdGd7ztAgr0fAVC/Sc7SueWF6lagohS2EEVMHE+yKUuB2zE6MkFR72k5g+Lv23a5zvK1ut7tVA/Y7YCErmMcfKjBbAyhoMpuJ6FP1fveefDMtjIgcwDBjkRgn5qXgOeL5iNpjKmonEdBTBYsccZ790h3fNGtyGYnlpXVCMqSpMkE01gob0xadKAVeXcary4B3YErq3UnCUSZFjd2/nQ3hLZV7D6m8QJpWvrl074bSs1YrZ+re2lj/IY0tkO7ckv5iXySNqgOyR8PpcCQvg7UNt+uWBhYdr3I2nMoOgrU/MMpwkZBUYoaEGHhuLn2jKyk2f2QLFoLkCJ+hIRkYN0UJEb4qLyMN2gaJQJxx86QSlGg9Q6FE2I/QMO0k4nrMgjfFBoWMP/mKpnkhSYonheKCAZkBe7OBiHKCJRsbB2FOTa8Aj5C5l6S5/1pmE7zrIy86R72u53a9Hx82dr5U27HqvHHeOpuO53x0dpw958A5dHDjWyNrXDTGzf2mbKqmnlCXl6qc184Va/7+D+SP+a8=</latexit><latexit sha1_base64="+SnZsH5g5Ndit21HyxBQCnNzgAE=">AAAFzHicjVRba9swFHbbZOuyW7s97kWsZDTQBTsZbDAKZaPQPax0sF4gco0sy4mofEGSuwZVr/uBe9vrfskkx0ncprcDxkff+c53zpFshTmjQrru36XllUbz0ePVJ62nz56/eLm2/upIZAXH5BBnLOMnIRKE0ZQcSioZOck5QUnIyHF49tXGj88JFzRLf8pxTvwEDVMaU4ykgYL1lX8wQXLEE3WEuIZqdwBzQT9DOSIS+VCDbeC22vdyYMwRVnVaUC7CWJUZwTTEMqzLdH9zTqikOlBrtT+jCpTkjGjdaucz7onu2Go0lYH6PkMvdGd7ztAgr0fAVC/Sc7SueWF6lagohS2EEVMHE+yKUuB2zE6MkFR72k5g+Lv23a5zvK1ut7tVA/Y7YCErmMcfKjBbAyhoMpuJ6FP1fveefDMtjIgcwDBjkRgn5qXgOeL5iNpjKmonEdBTBYsccZ790h3fNGtyGYnlpXVCMqSpMkE01gob0xadKAVeXcary4B3YErq3UnCUSZFjd2/nQ3hLZV7D6m8QJpWvrl074bSs1YrZ+re2lj/IY0tkO7ckv5iXySNqgOyR8PpcCQvg7UNt+uWBhYdr3I2nMoOgrU/MMpwkZBUYoaEGHhuLn2jKyk2f2QLFoLkCJ+hIRkYN0UJEb4qLyMN2gaJQJxx86QSlGg9Q6FE2I/QMO0k4nrMgjfFBoWMP/mKpnkhSYonheKCAZkBe7OBiHKCJRsbB2FOTa8Aj5C5l6S5/1pmE7zrIy86R72u53a9Hx82dr5U27HqvHHeOpuO53x0dpw958A5dHDjWyNrXDTGzf2mbKqmnlCXl6qc184Va/7+D+SP+a8=</latexit><latexit sha1_base64="+SnZsH5g5Ndit21HyxBQCnNzgAE=">AAAFzHicjVRba9swFHbbZOuyW7s97kWsZDTQBTsZbDAKZaPQPax0sF4gco0sy4mofEGSuwZVr/uBe9vrfskkx0ncprcDxkff+c53zpFshTmjQrru36XllUbz0ePVJ62nz56/eLm2/upIZAXH5BBnLOMnIRKE0ZQcSioZOck5QUnIyHF49tXGj88JFzRLf8pxTvwEDVMaU4ykgYL1lX8wQXLEE3WEuIZqdwBzQT9DOSIS+VCDbeC22vdyYMwRVnVaUC7CWJUZwTTEMqzLdH9zTqikOlBrtT+jCpTkjGjdaucz7onu2Go0lYH6PkMvdGd7ztAgr0fAVC/Sc7SueWF6lagohS2EEVMHE+yKUuB2zE6MkFR72k5g+Lv23a5zvK1ut7tVA/Y7YCErmMcfKjBbAyhoMpuJ6FP1fveefDMtjIgcwDBjkRgn5qXgOeL5iNpjKmonEdBTBYsccZ790h3fNGtyGYnlpXVCMqSpMkE01gob0xadKAVeXcary4B3YErq3UnCUSZFjd2/nQ3hLZV7D6m8QJpWvrl074bSs1YrZ+re2lj/IY0tkO7ckv5iXySNqgOyR8PpcCQvg7UNt+uWBhYdr3I2nMoOgrU/MMpwkZBUYoaEGHhuLn2jKyk2f2QLFoLkCJ+hIRkYN0UJEb4qLyMN2gaJQJxx86QSlGg9Q6FE2I/QMO0k4nrMgjfFBoWMP/mKpnkhSYonheKCAZkBe7OBiHKCJRsbB2FOTa8Aj5C5l6S5/1pmE7zrIy86R72u53a9Hx82dr5U27HqvHHeOpuO53x0dpw958A5dHDjWyNrXDTGzf2mbKqmnlCXl6qc184Va/7+D+SP+a8=</latexit><latexit sha1_base64="+SnZsH5g5Ndit21HyxBQCnNzgAE=">AAAFzHicjVRba9swFHbbZOuyW7s97kWsZDTQBTsZbDAKZaPQPax0sF4gco0sy4mofEGSuwZVr/uBe9vrfskkx0ncprcDxkff+c53zpFshTmjQrru36XllUbz0ePVJ62nz56/eLm2/upIZAXH5BBnLOMnIRKE0ZQcSioZOck5QUnIyHF49tXGj88JFzRLf8pxTvwEDVMaU4ykgYL1lX8wQXLEE3WEuIZqdwBzQT9DOSIS+VCDbeC22vdyYMwRVnVaUC7CWJUZwTTEMqzLdH9zTqikOlBrtT+jCpTkjGjdaucz7onu2Go0lYH6PkMvdGd7ztAgr0fAVC/Sc7SueWF6lagohS2EEVMHE+yKUuB2zE6MkFR72k5g+Lv23a5zvK1ut7tVA/Y7YCErmMcfKjBbAyhoMpuJ6FP1fveefDMtjIgcwDBjkRgn5qXgOeL5iNpjKmonEdBTBYsccZ790h3fNGtyGYnlpXVCMqSpMkE01gob0xadKAVeXcary4B3YErq3UnCUSZFjd2/nQ3hLZV7D6m8QJpWvrl074bSs1YrZ+re2lj/IY0tkO7ckv5iXySNqgOyR8PpcCQvg7UNt+uWBhYdr3I2nMoOgrU/MMpwkZBUYoaEGHhuLn2jKyk2f2QLFoLkCJ+hIRkYN0UJEb4qLyMN2gaJQJxx86QSlGg9Q6FE2I/QMO0k4nrMgjfFBoWMP/mKpnkhSYonheKCAZkBe7OBiHKCJRsbB2FOTa8Aj5C5l6S5/1pmE7zrIy86R72u53a9Hx82dr5U27HqvHHeOpuO53x0dpw958A5dHDjWyNrXDTGzf2mbKqmnlCXl6qc184Va/7+D+SP+a8=</latexit>
30. Quantum Monte Carlo + Neural Networks
2/10
Variational Monte Carlo
Variational Approach (used in Hartree-Fock, DMRG, QMC)
E0 = min
y
E[y] min
q
E[y(r;q)] E[y] =
Z
dry(r)ˆHy(r)
VMC: Variational Quantum Monte Carlo:
E[y;q] = Er⇠|y|2
⇥
Eloc[y](r;q)
⇤
Eloc[y](r;q) = ˆHy(r;q)/y(r;q)
Deep VMC idea: use deep neural network to represent y(r;q),
minimize variational energy E[y;q] over q.
G. Carleo, M. Troyer, Science 355:602-606 (2017): Discrete lattice
systems, Restricted Boltzmann Machines.
DeepWF: J. Han et al, arXiv:1807.07014 (2018):
Atoms+Molecules, Poor accuracy, moderate computational cost.
FermiNet: D. Pfau et al, arXiv:1909.02487 (Sept 2019):
Atoms+Molecules, highest accuracy, insane computational cost.
PauliNet: This work, arXiv:1909.08423 (Sept 2019):
Atoms+Molecules, high accuracy, moderate computational cost.
QMC with Restricted Boltzmann Machines:
Carleo, Troyer: Solving the Quantum Many-Body Problem with Artificial Neural Networks,
Science 355: 602-606 (2011)
32. PauliNet Wavefunction
4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
1) Enforce Antisymmetry
Backflow
2) Baseline Performance
Hartree Fock Input
3) Correlation Energy
Slater-Jastrow Backflow
4) Asymptotics
Cusp conditions
Incorporating Physics
33. PauliNet Wavefunction
4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
Assignment to spin-groups avoids spin coordinates
spin-up electrons spin-down electrons
34. PauliNet Wavefunction
4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018) 4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
1) Enforcing antisymmetry: Standard QM approach
antisymmetric part
Limited representative power
Many Slater determinants needed for high accuracy (CI)
det['µ(r"
i )] =
'1(r"
1) '2(r"
1) · · · 'N (r"
1)
'1(r"
2) '2(r"
2) · · · 'N (r"
2)
...
...
...
'1(r"
N ) '2(r"
N ) · · · 'N (r"
N )<latexit sha1_base64="nw4oXDF+2ROBn0tFrLJmP47cakc=">AAAFxHicjVRbb9MwFM62Fka4bfDIi8VUtEqjSiokkNCkARraC9WQ2EWqs8hxnNaac8F2xirP/Eje4NfgpGmbLbtZinzyne985+LEQcaokI7zd2l5pdV+8HD1kf34ydNnz9fWXxyKNOeYHOCUpfw4QIIwmpADSSUjxxknKA4YOQpOvxT+ozPCBU2TH3KSES9Go4RGFCNpIH995V8HxkiOeawOEddQ7Q5hJuhHKMdEIg9qsA0c+24OjDjCqk7zy5cgUmWEP3OxFOsy3NtcECqpLtRaDeZUgeKMEa3tTjbnHutukY0m0lff5ui57m4vGBpkdQ+Y6YV6gdY1z02tEuWlcAFhxNT+FLuk5DtdM4kxkmpPFx0Y/m6xd+ocd6vX623VgEEXNKL8hf++AvN3AAWN5z0RfaLe7t4Rr20YEjmEQcpCMYnNpuAZ4tmYFqeU1w7CpycK5hniPP2lu56p1YaMRPLC7AEZ0UQZF5pohc3SBpzK+G5dw61rgDdgRurfSsJhKkWNPbiZDeH1ifv3SdwgzRJfn7nfzDwvtDJm5k1lDe5TVoN06zwGjapIElZnY06F09FYXvhrG07PKRdoGm5lbFjV2vfX/sAwxXlMEokZEmLoOpn0jKyk2PyINswFyRA+RSMyNGaCYiI8VV5CGnQMEoIo5eZJJCjReoRCsSg+PsMs+hBXfQV4nW+Yy+iDp2iS5ZIkeJooyhmQKShuNBBSTrBkE2MgzKmpFeAxMteRNPeebYbgXm25aRz2e67Tc7+/29j5XI1j1XplvbY2Ldd6b+1Ye9a+dWDh1qfWqJW1fra/tllbtPMpdXmpinlpXVrt3/8B+lP42Q==</latexit><latexit sha1_base64="nw4oXDF+2ROBn0tFrLJmP47cakc=">AAAFxHicjVRbb9MwFM62Fka4bfDIi8VUtEqjSiokkNCkARraC9WQ2EWqs8hxnNaac8F2xirP/Eje4NfgpGmbLbtZinzyne985+LEQcaokI7zd2l5pdV+8HD1kf34ydNnz9fWXxyKNOeYHOCUpfw4QIIwmpADSSUjxxknKA4YOQpOvxT+ozPCBU2TH3KSES9Go4RGFCNpIH995V8HxkiOeawOEddQ7Q5hJuhHKMdEIg9qsA0c+24OjDjCqk7zy5cgUmWEP3OxFOsy3NtcECqpLtRaDeZUgeKMEa3tTjbnHutukY0m0lff5ui57m4vGBpkdQ+Y6YV6gdY1z02tEuWlcAFhxNT+FLuk5DtdM4kxkmpPFx0Y/m6xd+ocd6vX623VgEEXNKL8hf++AvN3AAWN5z0RfaLe7t4Rr20YEjmEQcpCMYnNpuAZ4tmYFqeU1w7CpycK5hniPP2lu56p1YaMRPLC7AEZ0UQZF5pohc3SBpzK+G5dw61rgDdgRurfSsJhKkWNPbiZDeH1ifv3SdwgzRJfn7nfzDwvtDJm5k1lDe5TVoN06zwGjapIElZnY06F09FYXvhrG07PKRdoGm5lbFjV2vfX/sAwxXlMEokZEmLoOpn0jKyk2PyINswFyRA+RSMyNGaCYiI8VV5CGnQMEoIo5eZJJCjReoRCsSg+PsMs+hBXfQV4nW+Yy+iDp2iS5ZIkeJooyhmQKShuNBBSTrBkE2MgzKmpFeAxMteRNPeebYbgXm25aRz2e67Tc7+/29j5XI1j1XplvbY2Ldd6b+1Ye9a+dWDh1qfWqJW1fra/tllbtPMpdXmpinlpXVrt3/8B+lP42Q==</latexit><latexit sha1_base64="nw4oXDF+2ROBn0tFrLJmP47cakc=">AAAFxHicjVRbb9MwFM62Fka4bfDIi8VUtEqjSiokkNCkARraC9WQ2EWqs8hxnNaac8F2xirP/Eje4NfgpGmbLbtZinzyne985+LEQcaokI7zd2l5pdV+8HD1kf34ydNnz9fWXxyKNOeYHOCUpfw4QIIwmpADSSUjxxknKA4YOQpOvxT+ozPCBU2TH3KSES9Go4RGFCNpIH995V8HxkiOeawOEddQ7Q5hJuhHKMdEIg9qsA0c+24OjDjCqk7zy5cgUmWEP3OxFOsy3NtcECqpLtRaDeZUgeKMEa3tTjbnHutukY0m0lff5ui57m4vGBpkdQ+Y6YV6gdY1z02tEuWlcAFhxNT+FLuk5DtdM4kxkmpPFx0Y/m6xd+ocd6vX623VgEEXNKL8hf++AvN3AAWN5z0RfaLe7t4Rr20YEjmEQcpCMYnNpuAZ4tmYFqeU1w7CpycK5hniPP2lu56p1YaMRPLC7AEZ0UQZF5pohc3SBpzK+G5dw61rgDdgRurfSsJhKkWNPbiZDeH1ifv3SdwgzRJfn7nfzDwvtDJm5k1lDe5TVoN06zwGjapIElZnY06F09FYXvhrG07PKRdoGm5lbFjV2vfX/sAwxXlMEokZEmLoOpn0jKyk2PyINswFyRA+RSMyNGaCYiI8VV5CGnQMEoIo5eZJJCjReoRCsSg+PsMs+hBXfQV4nW+Yy+iDp2iS5ZIkeJooyhmQKShuNBBSTrBkE2MgzKmpFeAxMteRNPeebYbgXm25aRz2e67Tc7+/29j5XI1j1XplvbY2Ldd6b+1Ye9a+dWDh1qfWqJW1fra/tllbtPMpdXmpinlpXVrt3/8B+lP42Q==</latexit><latexit sha1_base64="nw4oXDF+2ROBn0tFrLJmP47cakc=">AAAFxHicjVRbb9MwFM62Fka4bfDIi8VUtEqjSiokkNCkARraC9WQ2EWqs8hxnNaac8F2xirP/Eje4NfgpGmbLbtZinzyne985+LEQcaokI7zd2l5pdV+8HD1kf34ydNnz9fWXxyKNOeYHOCUpfw4QIIwmpADSSUjxxknKA4YOQpOvxT+ozPCBU2TH3KSES9Go4RGFCNpIH995V8HxkiOeawOEddQ7Q5hJuhHKMdEIg9qsA0c+24OjDjCqk7zy5cgUmWEP3OxFOsy3NtcECqpLtRaDeZUgeKMEa3tTjbnHutukY0m0lff5ui57m4vGBpkdQ+Y6YV6gdY1z02tEuWlcAFhxNT+FLuk5DtdM4kxkmpPFx0Y/m6xd+ocd6vX623VgEEXNKL8hf++AvN3AAWN5z0RfaLe7t4Rr20YEjmEQcpCMYnNpuAZ4tmYFqeU1w7CpycK5hniPP2lu56p1YaMRPLC7AEZ0UQZF5pohc3SBpzK+G5dw61rgDdgRurfSsJhKkWNPbiZDeH1ifv3SdwgzRJfn7nfzDwvtDJm5k1lDe5TVoN06zwGjapIElZnY06F09FYXvhrG07PKRdoGm5lbFjV2vfX/sAwxXlMEokZEmLoOpn0jKyk2PyINswFyRA+RSMyNGaCYiI8VV5CGnQMEoIo5eZJJCjReoRCsSg+PsMs+hBXfQV4nW+Yy+iDp2iS5ZIkeJooyhmQKShuNBBSTrBkE2MgzKmpFeAxMteRNPeebYbgXm25aRz2e67Tc7+/29j5XI1j1XplvbY2Ldd6b+1Ye9a+dWDh1qfWqJW1fra/tllbtPMpdXmpinlpXVrt3/8B+lP42Q==</latexit>
35. PauliNet Wavefunction
4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
1) Enforcing antisymmetry in PauliNet: Backflow
antisymmetric part
4/10
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
4/10
PauliNet – Physics 1: Antisymmetry
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
Greater representative power
Hope: few or one Slater determinant give high accuracy
36. PauliNet Wavefunction
4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
5/10
PauliNet – Physics 2: HF Baseline
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Use fixed Hartree-Fock (HF) orbitals jµ (r"
i )
jµ (r"
i ): linear combination of basis functions centered on atomic
nuclei, coe cients optimized variationally.
Pro: HF fast, captures physics of atoms and molecules qualitatively
Con: poor quantitative predictions, no electron-electron correlations.
2) Baseline Performance: Hartree-Fock
5/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Use fixed Hartree-Fock (HF) orbitals jµ (r"
i )
jµ (r"
i ): linear combination of basis functions centered on atomic
nuclei, coe cients optimized variationally.
Pro: HF fast, captures physics of atoms and molecules qualitatively
Con: poor quantitative predictions, no electron-electron correlations.
37. PauliNet Wavefunction
4/10
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Assign electrons to " and # sets.
y r"
1,...,r"
N"
,r#
N"+1,...,r#
N ⌘ y(r"
,r#
) ⌘ y(r)
Antisymmetry for single-electron functions: Slater determinants.
yHF(r) := det[jµ (r"
i )]det[jµ (r#
i )]
Antisymmetry for many-electron functions jµ (ri ) ! jµi (r) holds
if jµi (r) is equivariant wrt exchange P of same-spin electrons1,2
jµ (ri ) ! jµi (r),
Pjµi (r) = jµi (Pr)
PauliNet: jµi (r) = jµ (r"
i )fµi (r;q), fµi equivariant – SchNet3.
1P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
2D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
3K. T. Schütt et al., J. Chem. Phys. 148:241722 (2018)
3) Correlation Energy (beyond Hartree-Fock): deep learning
6/10
PauliNet – Physics 3: Beyond HF
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Jastrow factor4 J(r;q): Antisymmetric part is multiplied by
nonnegative symmetric function eJ(r;q)
. Can capture complex
electron correlations but not change the nodal surface.
Backflow5,6 f(r;q): Generalize one-electron orbitals jµ (ri ) to
many-electron orbitals jµi (r). Can change the nodal surface.
Exploit the flexibility of deep learning to achieve higher accuracy!
4N. D. Drummond et al. Phys. Rev. B 70:235119 (2004)
5P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
6D. Luo, B. K. Clark. Phys. Rev. Lett. 122:226401 (2019)
PauliNet – Physics 3: Beyond HF
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Jastrow factor4 J(r;q): Antisymmetric part is multiplied by
nonnegative symmetric function eJ(r;q)
. Can capture complex
electron correlations but not change the nodal surface.
Backflow5,6 f(r;q): Generalize one-electron orbitals jµ (ri ) to
many-electron orbitals jµi (r). Can change the nodal surface.
Exploit the flexibility of deep learning to achieve higher accuracy!
4
PauliNet – Physics 3: Beyond HF
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Jastrow factor4 J(r;q): Antisymmetric part is multiplied by
nonnegative symmetric function eJ(r;q)
. Can capture complex
electron correlations but not change the nodal surface.
Backflow5,6 f(r;q): Generalize one-electron orbitals jµ (ri ) to
many-electron orbitals jµi (r). Can change the nodal surface.
Exploit the flexibility of deep learning to achieve higher accuracy!
4N. D. Drummond et al. Phys. Rev. B 70:235119 (2004)
5P. López Ríos et al, Phys. Rev. E, 74:066701(2006)
PauliNet – Physics 3: Beyond HF
PauliNet Wavefunction:
y(r;q) = det[jµ (r"
i )fµi (r;q)]det[jµ (r#
i )fµi (r;q)]eg(r)+J(r;q)
Jastrow factor4 J(r;q): Antisymmetric part is multiplied by
nonnegative symmetric function eJ(r;q)
. Can capture complex
electron correlations but not change the nodal surface.
Backflow5,6 f(r;q): Generalize one-electron orbitals jµ (ri ) to
many-electron orbitals jµi (r). Can change the nodal surface.
Exploit the flexibility of deep learning to achieve higher accuracy!
4N. D. Drummond et al. Phys. Rev. B 70:235119 (2004)
5
