In this work, we present the ChemNLP library that can be used for 1) curating open access datasets for materials and chemistry literature, developing and comparing traditional machine learning, transformers and graph neural network models for 2) classifying and clustering texts, 3) named entity recognition for large-scale text-mining, 4) abstractive summarization for generating titles of articles from abstracts, 5) text generation for suggesting abstracts from titles, 6) integration with density functional theory dataset for identifying potential candidate materials such as superconductors, and 7) web-interface development for text and reference query. We primarily use the publicly available arXiv and Pubchem datasets but the tools can be used for other datasets as well. Moreover, as new models are developed, they can be easily integrated in the library.

1. ChemNLP
A Natural Language Processing based Library
for Materials Chemistry Text Data
Kamal Choudhary
https://jarvis.nist.gov/
NIST, Gaithersburg, MD, USA
Polymer group
7/13/2023
1
Joint Automated Repository for Various Integrated Simulations

2. Outline
2
• Introduction
• AI for Materials
• JARVIS
• NLP basics
• ChemNLP
• Datasets
• TextClassification
• TokenClassification
• WebApp
• TextSummarization
• TextGeneration
• Integrating DFT database
• JARVIS-Leaderboard/benchmarking
• Hands-on
• Summary
Electronic structure
DFT,DMFT,
TB,QMC
Quantum
Computation
AtomQC
Force-Field
JARVIS-FF
ALIGNN-FF
AI/ML
CFID
ALIGNN
AtomVision
ChemNLP

3. AI for Materials Science
3

4. Established: January 2017
Published: >40 articles
Users: >20000+ users worldwide
Materials: >80000, millions of properties
Events:
• Quantum Matters in Materials Science (QMMS)
• Artificial Intelligence for Materials Science (AIMS)
• JARVIS-School
User-comments:
• “There are many different theoretical levels on which you can
approach the field. JARVIS is unusual in that it spans more levels
than other databases.”
• “A pure gold-mine for the data-quality effort…”
• Thanks for your generous sharing. Your works inspire me a lot.
• “You guys are doing something really beneficial…”
• “I find JARVIS-DFT very useful for my research…”
JARVIS: Databases, Tools, Events, Outreach
4
https://jarvis.nist.gov
Requires login credentials, free registration

5. Updates
• 80,000 materials
• QMC, tight binding, ALIGNN, ALIGNN-FF,
• AtomVision, ChemNLP, JARVIS-Leaderboard
• Quantum Computation algorithms
• Superconductors (bulk and 2D), magnetic topological mats.
Recent Updates to JARVIS

6. Tools
Used for hands-on session!

7. ChemNLP

8. Text classification & Token classification

9. Text summarization & Text-generation

10. Conventional NLP: TFIDF
https://www.kaggle.com/code/ashoksrinivas/nlp-with-tfidf-neural-networks
https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html
https://i.stack.imgur.com/mtmP6.png

11. Transformers & “Attention Is All You Need”
Much better than RNN, LSTM etc.
Attention: extremely long-term memory
https://towardsdatascience.com/illustrated-guide-to-transformers-step-by-step-explanation-f74876522bc0

12. ChemNLP Datasets
Exploratory data analysis (EDA)

14. Accuracy
Text classification

15. Named Entity Recognition/Token classification
87 % F1 score

16. ChemNLP Webpage for Composition Search
https://jarvis.nist.gov/jarvischemnlp/

17. ChemNLP
Abstractive summarization (Abstract to Title)
Google’s T5-225million transformer model
ROGUE-1 score: 46.5 %
Text generation (Title to Abstract)
GPT2-medium LLM model
ROGUE-1 score: 32 %
Without fine tuning: 26 %
ROUGE:
Recall-Oriented Understudy for Gisting Evaluation
T5: Text-to-Text Transfer Transformer

18. Other test cases

19. 19
ChemNLP for supercondutors
Confusion matrix for text classification (137927 articles) • arXiv cond-mat.supr-con and JARVIS-SuperconDB
• Venn diagram for chemical formula
ChatGPT response

20. JARVIS-Leaderboard: Large Scale Benchmark
Challenges in materials science community:
• Reproducibility
• Transparency
• Validation
• Fidelity
• Data vs. metadata
• What is the ground truth/reference to compare our models to?
How does this change depending on the model?
• Synergy of computational and experimental databases
Community
effort to tackle
challenges:
https://pages.nist.gov/jarvis_leaderboard/

21. JARVIS-Leaderboard: Contributors
• Growing list of collaborators
• Multi-institutional effort
• Contributions are welcomed and
encouraged from community!

22. JARVIS-Leaderboard: Methods and Data
Types of Data:
• Atomic structure (Molecule, Crystal)
• Material Property (Bandgap, bulk modulus)
• Images (Microscopy: SEM, TEM, STM)
• Spectra (Diffraction: X-ray, Neutron, PL)
• Text (Research articles, notebooks, blogs)
• Eigensolver (Quantum Computation algorithms)
1) Electronic Structure
2) Artificial Intelligence
3) Force Field
4) Quantum Computation
5) Experiment

23. JARVIS-Leaderboard: Benchmarks
Contributions
1) Electronic Structure
2) Artificial Intelligence
3) Force Field
4) Quantum Computation
5) Experiment
Benchmarks (reference point)
1) Experiment/s
2) Test dataset
3) Electronic Structure
4) Analytical results
5) Other Experiments
Error metrics
*Benchmarks must be well-defined with an associated DOI

24. JARVIS-Leaderboard: Snapshot

25. JARVIS-Leaderboard: Snapshot

26. Hands-on session notebooks (later)
Natural Language Processing [44,45]
1. ChemNLP example (Part I)
2. ChemNLP example (Part II)
JARVIS-Leaderboard [5]
Analyzing benchmarks in the JARVIS-Leaderboard

27. 27
Summary
• NIST-JARVIS infrastructure with multiple components
• ChemNLP for solids currently, expand to polymers…
• Several events to engage (sign-up today & Demo!)
• Continuously growing, contribute, collaborate…
https://jarvis.nist.gov
https://github.com/usnistgov/jarvis
https://github.com/usnistgov/alignn
https://github.com/usnistgov/atomvision
https://github.com/usnistgov/chemnlp
https://github.com/usnistgov/atomqc
https://github.com/usnistgov/jarvis_leaderboard
Email: kamal.choudhary@nist.gov,
@dr_k_choudhary
@knc6
Slides:https://www.slideshare.net/KAMALCHOUDHARY4