1. ChemExtractor:
Enhanced Rule-Based Capture and
Identification of PDF Based Property Data
Stuart J. Chalk, Department of Chemistry
University of North Florida
schalk@unf.edu
253rd ACS Meeting April 2017

2. Outline
 Motivation
 Research Approach
 Analyzing Tabular Data
 Types of Data
 Regular Expressions (Regex)
 Rules and Rulesets
 Examples of Data Extraction
 Contextualizing Data
 Data Storage (MySQL)
 Data Representation (SciData)
 Conclusion
Funding for this project provided by

3. Motivation
 The Landholt-Börnstein Database is +450 volumes of
curated chemical property data (18?? to date)
 With the move to data-driven science it is imperative we
leverage the time invested in, and scientific quality of, the
curation of this data
 This high value data is locked in PDF files 
 This data can be made more useful if it is extracted with
its metadata (chemical system, original reference,
physical property with unit, etc.)

4.  Optical Character Recognition (OCR) is a standard
process to extract text from scanned images
 Accurate extraction of text using OCR in a page layout
format not only captures the text but also the inferred
relationships between tabular data
 Utilize regular expression (regex) analysis of tabular
text to capture data and its position relative to other
data
 Contextualize captured data with metadata encoded
based on layout of table and string format
Research Approach

5. Analyzing Tabular Data
Chemical
Metadata
Series of
Property Data
Condition
Series Condition
Property Data
Reference

6.  Properties and Units
 Conditions, data, supplemental data
 Equation coefficients and variables
 Chemical properties (MW, BP, MP)
 Annotations
 Table headers, column headers, data notes
 Chemical Metadata
 Formula, name, CASRN
 Context Metadata
 Table #, refcodes, property headers, component #
Types of Data

7.  Implemented in every programming language
 Relatively uniform implementation
 Uses syntax to create regex string that matches
and/or captures characters in string
 Groups of characters – ., [A-Z], [a-z], [0-9]
 Character classes w, d, s, h
 Repetition - ? (optional), + (1 or more), * ( or more)
 Capture group – ()
Regular Expressions (Regex)

8. http://regex101.com
Regex
Example 1

9. Regex
Example 2
http://regex101.com

10. http://regex101.com
Regex
Example 3

11.  We write rules for specific line formats constructed using
 Rule templates – define basic structure of a line, i.e.
how many blocks of text that need to be captured
 Rule snippets – define small regex strings that capture a particular
format of text
 Rulesets are list of rules with associated actions indicating
sequentaily process text line by line
Rules and Rulesets
^@B1@h+@B2@h*@B3@h*@B4@h*@B5@h*@B6@$
(?:[A-Z][a-z]{0,2}d{0,3}h?)+(?:.?d*.?d*H2O)

12. Rules and Rulesets

13. Rules and Rulesets

14. Rules and Rulesets

15. Rules and Rulesets

16. Examples of Data Extraction

17. Examples of Data Extraction

18. Examples of Data Extraction

19. Examples of Data Extraction

20.  This data…
 … is detected by rule ‘Temperature (K) & refcode’
Contextualizing the Data
T/K 298.15 () () () () () () () () 81V1 ()

21. Data Storage (MySQL)

22. Data Storage (MySQL)

23. Data Presentation

24. Data Presentation

25. Data Representation (SciData)

26.  Higher accuracy capture of property data and equation data
(350,000 property data points, 10,000 equations)
 Integrated chemical and reference metadata
 Can be applied to other PDF-based curated datasets
 Open website for community use this summer
 Implementation for research article capture of chemical
property data
Conclusion

27.  schalk@unf.edu
 Phone: 904-620-1938
 Skype: stuartchalk
 LinkedIn: https://www.linkedin.com/in/stuchalk
 ORCID: http://orcid.org/0000-0002-0703-7776
Questions?