This document discusses progress in using natural language processing (NLP) techniques to extract information from unstructured materials science text data. It notes that text data is not readily interpretable by machines and manual extraction is expensive. It describes using machine learning approaches to automatically extract synthesis parameters for materials from body text. It outlines a continuum of NLP approaches from rules to deep learning and challenges in applying NLP to materials domains due to issues like ambiguity and linked recipes. Examples of applying NLP to materials include generating plausible negative synthesis examples and exploring conditions to stabilize desired materials.

1. Progress in Natural Language Processing
of Materials Science Text
Elsa Olivetti, MIT Gerbrand Ceder, UC Berkeley
Departments of Materials Science & Engineering
Andrew McCallum, UMass Amherst
Department of Computer Science & Engineering
1

2. Motivation: data is a key ingredient in machine
learning for materials

3. Text is unstructured and semi‐structured data
described by free‐flowing natural language that is not
readily interpretable by machines
Manual data extraction is expensive, labor‐intensive,
and error‐prone
3

4. Scope Motivation: Modern data‐driven and first‐
principles materials design accelerates
pace of what to make…
4

5. 5

6. Automated body text extraction of synthesis parameters for
materials science using primarily machine learning
approaches
How does this work align with others
6

7. Scientific Domain Progression for NLP
7
Riedel and McCallum, Empirical methods in NLP 2011
ChemTagger: Hawizy et al 2011
ChemSpot: Rocktäschel et al 2012
Tchoua et al, IEEE, 2017 DOI 10.1109/eScience.2017.23
Biology/medical domain:
Publicly available annotated collections
GENIA corpus
Unified Medical Language System
Chemistry domain:
Domain‐specific terminology,
numeric phrases
CHEMDNER corpus
Inorganic materials:
Polymer domain:

8. Continuum of text mining approaches
8
Nature, 533, 2016

9. Continuum of text mining approaches
• Collection of rules/dictionaries ; rule‐matching engine
9
J Chem Inf Model, 56, 2016
LeadMine: Lowe and Sayle, J
Chem Inf Model, 7, 2015

10. Continuum of text mining approaches
10
Court and Cole., Scientific Data 2018
Korvigo et al., J Cheminform 2018CNN: convolutional neural net
HS‐biGRU: half‐stateful bidirectional gated recurrent unit
FCN: fully‐connected network
CEM: chemical entity mention

11. NLP activities
Entity extraction, Event extraction, Relation extraction, Entity linking
Krallinger et al. Chem. Rev. 2017

12. Chemical Entity Recognition
12
Krallinger et al. Chem. Rev. 2017

13. Parts of a text, types of texts
13
Experimental methods
NaNi1/3Co1/3Fe1/3O2 was synthesized by solid‐state reaction.
Excess amounts of Na2O, NiO, Co3O4 and Fe2O3 were mixed and
ball milled for 4 h at 500 rpm rate, and the resulting material
was collected in the glove box. About 0.5 g of powder was fired
at 800 °C under O2 for 14 h before it was quenched to room
temperature and moved to a glove box filled with argon.

14. Challenges
• With NLP for materials:
• Transferability across materials domains
• Off‐stoichiometry
• Lexical ambiguity and evolution
• With overall approach:
• Age, quality
• Linked recipes
• Negative examples
14
“BaxMn1‐xO3 for x = 0.9”
Solid state, thin film,
templated synthesis
“following the method described by….”
Optical character recognition, pdf vs. html
Bias of literature toward success

15. ACS central science, 2017
• Example from chemical synthesis
• Data augmentation reaction databased supplemented with chemically plausible negative
examples

16. Learning Method Ontologies
16

17. 17
Unspecified reaction conditions/amounts
Appropriate amounts
Small amount
Large amount
Several times
Ambient conditions
Constant conditions
Reduced pressure
Vigorous stirring
We can measure improvement in readability

18. Some examples of what we’ve done…
Variational autoencoder:
• Loss = reconstruction + f(Gaussian)
• Also a generative model
Edward Kim et al., npj Computational Materials 2017
Collaborator, Stefanie Jegelka, CSAIL, MIT

19. Exploratory: suggesting synthesis conditions
for stabilizing desired materials
Polymorphs for MnO2
overlaid with most probable
alkali‐ion use in synthesis
(intercalation‐based phase
stability)
Edward Kim et al., npj Computational Materials 2017
Photocatalysts
Lithium‐ion batteries Molecular sieves
Alkaline batteries
10,200 articles

20. Concluding thoughts on NLP progress
• Natural language processing is young in its application to
materials science
• It takes effort to build up an annotation approach and corpus
• There are domain‐specific needs regarding accuracy and
ambiguity
• Tradeoff between accuracy and degree of generalizability

21. Bibliography
Audus, Debra J., and Juan J. de Pablo. "Polymer informatics: opportunities and challenges." (2017):
1078‐1082.
Coley, Connor W., et al. "Prediction of organic reaction outcomes using machine learning." ACS
central science 3.5 (2017): 434‐443.
Court, Callum J., and Jacqueline M. Cole. "Auto‐generated materials database of Curie and Néel
temperatures via semi‐supervised relationship extraction." Scientific data 5 (2018): 180111.
Hawizy, L.; Jessop, D. M.; Adams, N.; Murray‐Rust, P. ChemicalTagger: A Tool for Semantic Text‐
Mining in Chemistry. J. Cheminform. 2011, 3, 1–13.
Kim, E. et al. “A Data‐driven Framework for Materials Synthesis Discovery.” Chemistry of Materials.
2017, 29.
Kim, E. et al. "Virtual screening of inorganic materials synthesis parameters with deep learning" npj
Computational Materials, 2017, 53.
Rocktäschel, T.; Weidlich, M.; Leser, U. ChemSpot: A Hybrid System for Chemical Named Entity
Recognition. Bioinformatics 2012, 28, 1633–1640.
Swain, M. C.; Cole, J. M. ChemDataExtractor: A Toolkit for Automated Extraction of Chemical
Information from the Scientific Literature. J. Chem. Inf. Model. 2016, 56, 1894–1904.

22. Thank you
Edward Kim, Zachary Jensen, Kevin Huang
Teams at Berkeley and UMA
olivetti.mit.edu
synthesisproject.org
elsao@mit.edu
22