Download as PDF, PPTX
![Outline
3
• Motivation [4]
• Introduction to JARVIS infrastructure [5-9]
• ALIGNN model for solids and molecules [10-28]
• AtomVision model for STM and STEM images [29-36]
• AtomQC: Quantum computation for solids [37-46]
• Summary & Future Work [47]
Contents Slide #](https://image.slidesharecdn.com/jarvis-miracle-220226231414/75/Materials-Design-in-the-Age-of-Deep-Learning-and-Quantum-Computation-3-2048.jpg)
![13
From ANNs to Graph Convolution Networks
𝑧[𝑙]
= 𝑊
[𝑙]
𝑎
[𝑙−1]
+ 𝑏
[𝑙]
𝑎[1]
= σ( 𝑧[𝑙]
); 𝑎[0]
= 𝑋
1) Forward propagation
2) 𝐶𝑜𝑠𝑡, 𝐽(𝑊, 𝑏) = 𝑓(𝑦 −
𝑦)
X1
X2
Hidden
Layer
Input
Layer
Output
layer
𝑦
3) Gradient descent (∇J):
minimize cost with W,b
4) Backpropagation:
chain rule to get,
𝜕𝐽
𝜕𝑊
1) Convolution:
element-wise multiplication & sum
2) Pool: Max, Average, Sum
3) Fully Connected: Standard NN
Shared weights (Learnable filters),
regularized version of NNs
መ
𝐴 = ෩
𝐷−
1
2 𝐴෩
𝐷−
1
2 𝐻𝑙+1 = σ 𝑊 መ
𝐴𝐻𝑙
Adjacency matrix, A
1 0 1
0 1 0
1 0 1
1) Adjacency matrix, N x N (N: #nodes),
2) D: degree of node
3) Update node representation using
message passing, GPU efficient
4) Update equation is local, neighborhood
of a node only, independent of graph size
Standard NN ConvolutionNN GraphConvNN
Types: un/weighted, un/directed, line,
Hetero/Homogenous, Multigraph](https://image.slidesharecdn.com/jarvis-miracle-220226231414/75/Materials-Design-in-the-Age-of-Deep-Learning-and-Quantum-Computation-13-2048.jpg)
![Dynamical Mean Field Theory
45
Imaginary part of Al’s DMFT hybridization function for a few components considering zero self-energy. a)Δ00, b)Δ01,
c)Δ10, d)Δ11
• Dynamical mean-field theory (DMFT): commonly used
techniques for solving predicting electronic structure of
correlated systems using impurity solver models.
• DMFT maps a many-body lattice problem to a many-
body local problem with impurity models.
• In DMFT one of the central quantities of interest is the
Green’s function such as
𝐺(𝑘, ꞷ𝑛) = [ꞷ𝑛 + 𝜇 − 𝐻(𝑘) − 𝛴(ꞷ𝑛)]−1
• Spectral function (𝐴) & DMFT hybridization function (𝛥)
𝐴(ꞷ) = −
1
𝜋
𝐼𝑚(𝐺(ꞷ + 𝑖𝛿))
𝑘
𝛥(ꞷ + 𝑖𝛿) = ꞷ − (𝐺)−1
• Next, integrate with quantum impurity solvers
𝛴 = 0](https://image.slidesharecdn.com/jarvis-miracle-220226231414/75/Materials-Design-in-the-Age-of-Deep-Learning-and-Quantum-Computation-45-2048.jpg)
More Related Content
![[論文紹介] BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Ne...](https://cdn.slidesharecdn.com/ss_thumbnails/blendedmvsslideshare-200630044345-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Materials Design in the Age of Deep Learning and Quantum Computation
Materials Design in the Age of Deep Learning and Quantum Computation
- 1. Materials Design inthe Age of Deep Learning and Quantum Computation ALIGNN, AtomVision, AtomQC Kamal Choudhary Researcher: NIST, Gaithersburg, MD, USA Developer & Founder: https://jarvis.nist.gov MIrACLE seminars, AFRL 11/05/2021 1 Joint Automated Repository for Various Integrated Simulations
- 2. Acknowledgement and Collaboration 2 F.Tavazza (NIST) C. Campbell (NIST) A. Reid (NIST) A. Biacchi (NIST) B. DeCost (NIST) Igor Mazin (George Mason University) A. Agarwal (Northwestern University) S. Kalidindi (GAtech) D. Siderius (NIST) Ruth Pachter (AFRL) Karen Sauer (George Mason University) K. Garrity (NIST) David Vanderbilt (Rutgers University) Sergei Kalinin (ORNL) F. Tavazza (NIST) B. DeCost (NIST) K. Garrity (NIST)
- 3. Outline 3 • Motivation [4] •Introduction to JARVIS infrastructure [5-9] • ALIGNN model for solids and molecules [10-28] • AtomVision model for STM and STEM images [29-36] • AtomQC: Quantum computation for solids [37-46] • Summary & Future Work [47] Contents Slide #
- 4. Motivation 4 • Accelerate traditionalcomputational and experimental methods • Automate experimental data analysis, • Discover new materials, • Develop new methods, • Find new physical equations/phenomenon, • Enhance collaboration, • Uncertainty quantification, reproducibility,… Data-challenge: discrete, continuous and images Materials Genome Initiative 2011 Crystals Molecules Proteins TEM images Band-structure XRD spectra
- 5. NIST-JARVIS: Success Stories 5 https://jarvis.nist.gov “Youguys are doing something really beneficial…” “I find JARVIS-DFT very useful for my research…” User-comments: Established: January 2017 (MGI funded) Published: >25 articles Users: >6000 users worldwide Downloads: >300K Workshops: 2 AIMS, 1 QMMS (~200 attendees for each) jarvis.nist.gov: Requires login credentials, free registration Choudhary et al., npj Computational Materials 6, 173 (2020).
- 6. 2017 2018 20192020 2021 JARVIS-FF (Evaluate FF) JARVIS-DFT 2D (OptB88vdW, Exf. En.) JARVIS-DFT Optoelectronics (TBmBJ) JARVIS-DFT Elastic Tensor 3D & 2D JARVIS-ML CFID descriptors JARVIS-FF (Evaluate FF, defects) JARVIS-DFT Topological SOC spillage 3D JARVIS-DFT /ML K-point convergence JARVIS-DFT Solar SLME JARVIS-DFT Topological SOC spillage 2D (Mag/Non-Mag.) JARVIS-DFT/ML 2D Heterostructures JARVIS-DFT/ML DFPT Dielec., Piezo., IR JARVIS-DFT/ML Thermoelectrics 3D & 2D Seebeck, PF JARVIS-DFT EFG NQR, NMR JARVIS-DFT STM 2D JARVIS-DFT AQCE 2D JARVIS-DFT WTBH JARVIS-DFT/ML Topological SOC spillage 3D Mag., non-mag, Exp. JARVIS-AtomQC VQE/VQD JARVIS-DAC MOFs JARVIS- AtomVison (STEM/STM) JARVIS-TB TB3PY JARVIS- ALIGNN (Solids Molecules) JARVIS-ML UQ, Phonons JARVIS- OPTIMADE 6
- 7. JARVIS-DFT, FF, ML,STM, Tools, … 7 ~60000 bulk, 1000 monolayer materials, >110 FFs, ~1 million properties
- 8. JARVIS-DFT MP OQMD #Materials(Struct., Ef, Eg ) 60870 144595 (41697 common) 1022663 DFT functional/methods vdW-DFT-OptB88, TBmBJ, DFT+SOC GGA-PBE, PBE+U, GLLBSC GGA-PBE, PBE+U K-point/cut-off Converged for each material Fixed (1000-3000) kp/atom, 520 eV Fixed kp/atom, cutoff SCF convergence criteria Energy & Forces Energy Energy Elastic tensors & point phonos 17402 14072 - Piezoelectric, IR spect. 4801 3402 - Dielectric tensors (w/o ion) 4801 (15860) 3402 - Electric field gradients 11865 - - XANES spectra - 22000 - 2D monolayers 1011 - - Raman spectra 400 50 - Seebeck, Power F 23210 48000 - Solar SLME 8614 - - Spin-orbit Coupling Spillage 11383 - - WannierTB 1771 - - STM images 1432 - -
- 9. 9 Automated Dataset &DL Training Dataset snapshots for DL training https://jarvis-tools.readthedocs.io/en/master/databases.html https://github.com/usnistgov/jarvis https://www.ctcms.nist.gov/~knc6/static/JARVIS-DFT/JVASP-1002.xml XML format webpages
- 10. 10 ALIGNN Atomistic Line GraphNeural Network Accepted in NPJ Computational Materials
- 11.
- 12.
- 13. 13 From ANNs toGraph Convolution Networks 𝑧[𝑙] = 𝑊 [𝑙] 𝑎 [𝑙−1] + 𝑏 [𝑙] 𝑎[1] = σ( 𝑧[𝑙] ); 𝑎[0] = 𝑋 1) Forward propagation 2) 𝐶𝑜𝑠𝑡, 𝐽(𝑊, 𝑏) = 𝑓(𝑦 − 𝑦) X1 X2 Hidden Layer Input Layer Output layer 𝑦 3) Gradient descent (∇J): minimize cost with W,b 4) Backpropagation: chain rule to get, 𝜕𝐽 𝜕𝑊 1) Convolution: element-wise multiplication & sum 2) Pool: Max, Average, Sum 3) Fully Connected: Standard NN Shared weights (Learnable filters), regularized version of NNs መ 𝐴 = ෩ 𝐷− 1 2 𝐴෩ 𝐷− 1 2 𝐻𝑙+1 = σ 𝑊 መ 𝐴𝐻𝑙 Adjacency matrix, A 1 0 1 0 1 0 1 0 1 1) Adjacency matrix, N x N (N: #nodes), 2) D: degree of node 3) Update node representation using message passing, GPU efficient 4) Update equation is local, neighborhood of a node only, independent of graph size Standard NN ConvolutionNN GraphConvNN Types: un/weighted, un/directed, line, Hetero/Homogenous, Multigraph
- 14.
- 15. 15 Graph Convolution Networksfor Materials Scientists Consider BaTiO3: start with -dimensional atom feature vectors The local environment of Ti can be modeled as: In GCNs, f is parameterized by one or more neural network layers GCNs implicitly represent multi-body interactions by composing multiple layers e.g. edge-gated graph convolution: is a neural network modeling pairwise atom interactions is a neural network modeling the importance of output channels as a function of bond character Elementwise multiplication denotes concatenation arxiv:1711.07553 Graph embedding transform and encode the data structure in high dimensional and non- Euclidean feature space to a low dimensional space
- 16. 16 Edge-gated Graph ConvolutionLayer Node i Nbrs {j} Edges {ij} For computational efficiency, the parameters are split into two separate matrices see https://docs.dgl.ai/guide/message-efficient.html adapted from:
- 17. 17 Line Graph Explicitly representpairwise and triplet (bond angle) interactions using line graph Possible to extend for n-body, e.g. line graph of line graph nisaba.nist.gov Tesla V100
- 18. 18 ALIGNN Model Initial atom, bond, angle features Feature-wise sum across atoms Property prediction Compose multipleALIGNN and EdgeGatedGCN layers: learnable local atom environment representation Global crystal representation: Feature-wise average across atoms in the crystal Final property regression model: linear model for regression, logistic regression for classification
- 19.
- 20. 20 Google Colab NotebookExample https://colab.research.google.com/github/knc6/jarvis-tools-notebooks/blob/master/jarvis-tools-notebooks/Training_ALIGNN_model_example.ipynb https://github.com/usnistgov/alignn
- 21. 21 Performance on theMaterials Project Dataset Trained on 69239 materials (DFT data) #Epochs: 300 Batch_size: 64
- 22. 22 Performance on theJARVIS-DFT Dataset Trained on ~55k materials • Total energy, Formation energy , Ehull • Bandgap (OPT), Bandgap (MBJ) • Kv, Gv • Mag. mom • єx (OPT/MBJ), єy (OPT), єz (OPT), є (DFPT:elec+ionic) • Max. piezo. stress coeff (eij) • Solar-SLME (%) • Topological-Spillage • 2D-Exfo. energy • Kpoint-length • Plane-wave cutoff • Max. Electric field gradient • avg. me, avg. mh • n-Seebeck, n-PF, p-Seebeck, p-PF
- 23. 23 How did weget here? Hyperparameters https://docs.ray.io
- 24.
- 25. 25 Training Time Study Numberof epochs to reach desired accuracy is different: 1000s vs 100s
- 26. 26 Applications: Pre-screening ofMaterials Above ones with CFID descriptors, can be done with ALIGNN More: ALIGNN+AtomVision
- 27. 27 Applications: AI forClimate Change DL model for predicting CO2 adsorption in MOFs (using hMOF and NIST-isotherm databases) Unpublished work MOF: Metal ion +Organic linkers hMOF DB: 137k MOFs GCMC calculations for CO2 adsorption available 5 Best 5 Worst
- 28. 28 AtomVision A deep learningframework for atomistic image data
- 29. 29 Scanning Tunneling MicroscopeImage Tersoff-Hamman approach, constant height/current images Tunneling current is proportional to the integrated local density of states (ILDOS)
- 30. 30 Scanning Transmission ElectronMicroscope Image PPdSe: JVASP-6316 C: JVASP-667 FeTe: JVASP-6667 Convolution approximation: accurate for thin films mainly Based on Rutherford scattering model
- 31. (a) (b) (c)(d) (e) (f) (g) (h) (i) (j) (k) (l) STM STEM C FeTe MoS2 31
- 32.
- 33.
- 34. Google Colab NotebookExample 34 https://colab.research.google.com/github/knc6/jarvis-tools-notebooks/blob/master/jarvis-tools-notebooks/AtomVisionExample.ipynb
- 35. 35 Feature Extraction forDefect Detection Image → Segmentation → ALIGNN model AtomVision+ALIGNN+Experiments (ongoing work) Mo vacancies in 2H-MoS2 Fe vacancies in FeTe
- 36. 36 AtomQC Atomistic Calculations onQuantum Computers
- 37.
- 38. Variational Quantum Eigensolver(VQE) & Variation Quantum Deflation (VQD) 38 http://openqemist.1qbit.com/docs/vqe_microsoft_qsharp.html Notes: • Quantum computers are good in preparing states, not good at sum, optimizers, multiplying etc. • QC to prepare a wavefunction ansatz of the system and estimate the expectation value VQD: Deflate other eigensatets once ground state is found using VQE VQE: a hybrid classical-quantum algorithm using Ritz variational principle
- 39. Typical Flowchart 39 https://github.com/usnistgov/atomqc K. Choudhary,J. Phys.: Condens. Matter 33 (2021) 385501 Wannier Functions: • Complete orthonormalized basis set, • Acts as a bridge between a delocalized plane wave representation and a localized atomic orbital basis • All major density functional theory (DFT) codes support generation WFs for a material 𝐻 = ℎ𝑃𝑃 𝑃∈ 𝐼,𝑋,𝑌,𝑍 ⨂𝑛 𝐻𝑗 = 𝐻 + 𝛽𝑖|𝜓(𝜽0 ∗) 𝜓(𝜽0 ∗)| 𝑗−1 𝑖=0 https://github.com/usnistgov/jarvis
- 40. Circuit Trials 40 RealAmplitudes PauliTwoDesign EfficientSU2 jarvis.core.circuits.QuantumCircuitLibrary RYand RZ: parametrized circuits with parameters ө, Wires and boxes with ‘X’ :Controlled-X gate. Wires with two solid squares: Controlled-Z gates
- 41. Circuit Trials 41 Finding rightcircuit and number of repeat units are important a) Al for Gamma point, b) Al for X point, c) PbS for X point for different repeat units of Circuit-6
- 42. Google Colab NotebookExample 42 https://colab.research.google.com/github/knc6/jarvis-tools-notebooks/blob/master/jarvis-tools- https://github.com/usnistgov/jarvis K. Choudhary, J. Phys.: Condens. Matter 33 (2021) 385501 Can be used for: • Qiskit • Tequila • Pennylane,… https://github.com/usnistgov/atomqc
- 43. FCC Aluminum Example 43 a)Monitoring VQE optimization progress with several local optimizers such COBYLA, L_BFGS_B, SLSQP, CG, and SPSA for Al electronic WTBH and at X-point. b) Electronic bandstructure calculated from classical diagonalization (Numpy-based exact solution) and VQD algorithm for Al. c) Phonon bandstructure for Al
- 44. Application to ~1000Systems 44 Comparison of minimum (Min.) and maximum (Max.) energy levels at Г-point for electronic and phonon WTBH using classical eigenvalue routine in Numpy (Np.) and VQE solver. (N_qubits <=5) a) and b) comparison of phonon (Phn.) minimum and maximum energy levels for 930 materials, c) and d) comparison of electronic (El.) minimum and maximum energy levels for 300 materials. The colorbar represents the number of Wannier orbitals.
- 45. Dynamical Mean FieldTheory 45 Imaginary part of Al’s DMFT hybridization function for a few components considering zero self-energy. a)Δ00, b)Δ01, c)Δ10, d)Δ11 • Dynamical mean-field theory (DMFT): commonly used techniques for solving predicting electronic structure of correlated systems using impurity solver models. • DMFT maps a many-body lattice problem to a many- body local problem with impurity models. • In DMFT one of the central quantities of interest is the Green’s function such as 𝐺(𝑘, ꞷ𝑛) = [ꞷ𝑛 + 𝜇 − 𝐻(𝑘) − 𝛴(ꞷ𝑛)]−1 • Spectral function (𝐴) & DMFT hybridization function (𝛥) 𝐴(ꞷ) = − 1 𝜋 𝐼𝑚(𝐺(ꞷ + 𝑖𝛿)) 𝑘 𝛥(ꞷ + 𝑖𝛿) = ꞷ − (𝐺)−1 • Next, integrate with quantum impurity solvers 𝛴 = 0
- 46. 46 Summary & FutureWork • NIST-JARVIS infrastructure with multiple components for materials discovery & design • Using GAN with ALIGNN & AtomVision • ALIGNN+TB, ALIGNN+FF • VQD with DMFT Impurity solvers https://jarvis.nist.gov https://github.com/usnistgov/jarvis https://github.com/usnistgov/alignn https://github.com/usnistgov/atomvision https://github.com/usnistgov/atomqc Email: kamal.choudhary@nist.gov