1.
Kohei Shinohara
@ Preferred Networks Summer Internship 2021
Charge Transfer Modeling
in Neural Network Potential

2.
Neural network potential for materials simulation 2
Density functional theory (DFT)
● Materials simulation routinely rely on DFT
calculation
● Accurate in most cases, but slow
Neural Network Potential (NNP)
● Train NN with DFT dataset and predict
energy, forces, …
● Similar accuracy with DFT and still fast!
adapted from MPNN

3.
Challenging systems with charge transfer 3
Long-range interaction
● Usual NNP only predicts short-range energy
● Long-range interaction is crucial in some systems
○ ionic crystals, catalysts, and nanoclusters
Charge transfer
● Long-range interaction comes from the Coulomb
interaction between charges of atoms
● We need to model charges (charge transfer) and
correct the long-range interaction for accurate
prediction
local-environment change may cause
non-local charge transfer
(adapted from 4G-BPNN)

4.
Objective of this work 4
● Verify various techniques for modeling charge transfer
● Study effective ways to incorporate charge transfer in NNP
● This work should contribute to extending the application systems of NNP
4G-BPNN SpookyNet
GNN Charge+Eele Qeq
2G-BPNN ✖
3G-BPNN ✖ ✅
4G-BPNN ✖ ✅ ✅
SpookyNet ✅ ✅ ✖
Ours ✅ ✖/✅ ✖/
✅

5.
Baseline architecture: NequIP [1] 5
GNN
● update features on atoms (node) by neighbor atoms
Short range
● predict short-term atomic energy and sum all
Forces via automatic differentiation
position
atomic species
trained model
adapted from [1]
[1] S. Batzner et al., arxiv:2101.03164

6.
Electrostatic correction 6
Additional inputs and outputs
● Original NequIP does not output charges
● Qtot: (input) total charge of a system
● Qi: (output) atomic charge
Coulomb term
● Model charge density with gaussian distributions
● Electrostatic energy is a quadratic form of {Qi}
{Zi}
{Qi}
{Ei}
{ri}
Qtot
Embedding
Conv. Layers
Output Block
Sum Pooling
Linear
Coulomb
Linear
Eshort
Etot
Eele
Compare with
Hirshfeld charges

7.
Charge Equilibration (Qeq) [1] 7
● Add self-interaction term to Eele
● Predict the chemically motivated 𝜒i and Ji by NN [2]
● Determine charges by minimizing EQeq
○ Quadratic programming with equality
constraint
○ Equivalent to solve linear equations, O(N3)
○ torch.linalg.solve, no need to implement
gradient
● Reuse predicted charges as node features [3]
[1] A. K. Rappe and W. A. Goddard III, J. Phys. Chem. 95, 8 (1991).
[2] S. A. Ghasemi et al., Phys. Rev. B 92, 045131 (2015).
[3] Tsz Wai Ko et al., Nat. Commun. 12, 398 (2021).
{Zi}
{𝜒i}, {Ji}
{Ei}
{ri}
Qtot
Embedding
Conv. Layers
Output Block
Sum Pooling
Linear
Coulomb
Linear
Eshort
Etot
EQeq
Compare with
Hirshfeld charges
Qeq
{Qi}

8.
Electrostatic energy for periodic system 8
Ewald summation [1,2]
● The sum in Eele for periodic systems is conditional convergent
● Standard technique to calculate Eele for periodic systems
● Eele is a quadratic form → Qeq works as well as nonperiodic
Periodic boundary condition
(adapted from Wikimedia Commons)
[1] P. P. Ewald, Ann. Phys. 64, 253 (1921). [2] P. T. Kiss et al., J. Chem. Theory Comput. 10, 12 (2014).
Lecture note on Ewald summation: link
Neighbor search
Naive: O(N2), Neighbor list: O(N)
Fourier transform
Naive: O(N2), FFT: O(N log N)

9.
Datasets [1] 9
[1] Tsz Wai Ko et al., Nat. Commun. 12, 398 (2021).
Carbon chain
(C10H2/C10H3
+)
NaCl
(Na8Cl8
+/Na9Cl8
+)
Ag cluster
(Ag3
+/Ag3
-)
Au2-MgO
(undoped/doped)
nonperiodic nonperiodic nonperiodic periodic
Trajectory from MD and
relaxation
10019 structures
Blue atom (Na) is randomly
displaced
5000 structures
Trajectory from MD and
relaxation
11013 structures
Au2 cluster on MgO (001)
Half of dataset are Al-doped
(blue atom)
5000 structures
adapted from [1]

10.
Effects of Eele and Qeq on forces RMSE 10
● GNNs give better accuracy than MLPs (dotted lines)
● Qeq improve accuracies compared to naive charge prediction
● But, baseline GNN is often superior to these models (except Ag cluster)
○ GNN can learn Eele effect in these datasets?

11.
Effects of Qeq on charges RMSE 11
● Qeq improves charges prediction except Ag cluster dataset
● Worse accuracies than MLP (green dotted)
○ Need more params. for predicting 𝜒i and Ji ?

12.
Evaluation time 12
● Au2-MgO, 110 atoms in unit cell, V100 x1
● Current implementation calculates Ewald sum and Qeq serially
○ Batched lu_solve will speed up Qeq
Order #params. Eval time ↓
(ms/structure)
Performance ↑
(Katoms-step/sec)
Baseline O(N) 48024 27.8 4.61
w/ Eele (Ewald) O(N2) 48104 49.8 3.20
w/ Qeq O(N3) 48104 66.7 2.34

13.
Conclusion 13
Inspect techniques for charge transfer on GNN
● Qeq improves charges prediction in most cases
● Qeq improve accuracies compared to naive charge prediction
● but, baseline GNN is often superior to these models…
○ Effective cutoff radii of GNN can see charge transfer in the present datasets
Implementation aspects
● pytorch implementation of Qeq
○ no need to care about derivatives of linear equations!
● pytorch implementation of Ewald summation
○ electrostatic interaction for periodic system