Over the last few years data has grown to big data, but so has computing power and knowledge of techniques to extract value, particularly from the unstructured information. We at the Royal Society of Chemistry have applied these to our own big data and developed methods, tools and associations to improve its discoverability and usefulness and do our part in advancing the chemical sciences via the dissemination of knowledge.

1. Advancing the chemical
sciences through big data
Dr Aileen Day
Data Science, Royal Society of Chemistry

2. The Royal Society of Chemistry

3. • The Royal Society of Chemistry
• We help other teams make evidence-
based decisions
• The chemical community:
• RSC members, authors, readers
• We help them to easily find our articles
and compound data
Who are Data Science’s customers?

4. We have access to:
• ChemSpider
What big data do we have?

5. What big data do we have?
We have access to:
• ChemSpider
• RSC publishing

6. What big data do we have?
We have access to:
• ChemSpider
• RSC publishing
• logs
2016-06-24 00:05:07 192.168.0.1 pubs.rsc.org – GET /en/content/articlepdf/2007/sm/b704827k - - - XXX.XXX.XXX.XXX -
Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/50.0.2661.102+Safari/537.36
ShowEUCookieLawBanner=true;+X-Mapping-hhmaobcf=5EFF013F0F2EB5C7479A967277AFB2F4;+ASP.NET_SessionId=tzmjdkojxv2jcqh25omqerui;
+Branding=50000XXX;+AuthSystemSessionId=261e0a91-7d73-4fd7-9380-e73e298d6047;+__utmt=1;+__utma=1.2022872114.1464909160
.XXXXXXXXXX.XXXXXXXXXX.X;+__utmb=X.X.X.XXXXXXXXXXXXX;+__utmc=1;+__utmz=X.XXXXXXXXXX.X.X.utmcsr=google|utmccn=(organic)|utmcmd
=organic|utmctr=(not%20provided);+iislog-host=pubs.rsc.org;+iislog-s-ip=172.30.229.101
http://pubs.rsc.org/en/Content/ArticleLanding/2007/SM/b704827k - 200 - - - 409353 - -

7. • Not massive, but big enough!
• A combination of structured
(ChemSpider) and unstructured
(articles and logs)
• We can exploit this unstructured data…
What can this big data do for chemistry?

8. What can Data Science and this big data
do for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

9. What can Data Science and this big data
do for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

10. • The RSC editorial
teams boost RSC
publishing
• Identify hot emerging
topics
Customers for trend and category
analysis

11. Categorisation of RSC
journals and articles
Categories – partly
defined by RSC journal
categorization (top
level), partly generated,
reviewed, organised

12. • Aim: tool for seeing how highly accessed
various subsets of our papers are
• Based on:
• article information title, abstract, year, journal
but not full text
• access data for those papers
Category dashboard

13. …
User Inputs

14. Categories of results
Upward trend,
but note small
total number of
accesses

15. Subcategories

16. Hot terms
words that are found in the titles and abstracts of papers in that
category that are accessed more frequently than usual

17. What Data Science and this big data do
for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

18. • Aim: to identify and understand
emergent research trends in the
chemical sciences
Trend visualisation

19. Trend visualisation
Things people have
searched articles for
internally
Words that are over-
represented in the
articles which are
returned
Occurrences of each
in articles over time
uses entropy-based
measures (mutual
information)

20. Trend visualisation - example
• Shows the dramatic
emergence of perovskites for
solar applications
• Enables editorial staff to
commission reviews and
special issues in these more
specific areas

21. What can Data Science and this big data
do for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

22. Unstructured Data Structured Data
NER

23. • Participated in public, competitive evaluation extracting chemical names from
patents:
• BioCreative V.5 (Critical Assessment of Information Extraction in Biology)
community-wide effort with the aim of evaluating biomedical text mining and
information extraction tools, submitted and evaluated using Becalm platform
• CEMP (chemical entity mention in patents) task
• Using deep learning techniques – recurrent artificial neural networks
Chemlistem
Named Entity Recognition (NER)

24. Traditional CRF approach - sequence
labelling
“… the quisqualic acid-induced increase in the intracellular
calcium ion concentration …”
Tokenise: “… the quisqualic acid - induced increase
in the intracellular calcium ion concentration …”
Tag: “… the_O quisqualic_B acid_E -_O induced_O
increase_O in_O the_O intracellular_O calcium_S
ion_O concentration_O …”
O = outside B = begin I = inside E = end S = singleton

25. • Compared 3 methods:
• “Traditional” Conditional Random Fields CRF
translated to deep learning
• Minimalist approach
• Ensemble combination of previous two methods:
• Run Traditional and Minimalist systems with a low threshold =>
generate 2 lists of entities
• Combine scores of entities in lists and apply threshold of 0.475
Chemlistem Methods

26. Embeddings
Other Features
Convolutional
Layer
Merge
Bidirectional
LSTM
SOBIESOBIESOBIE
Token n Token n+1Token n-1
Final Layer
Outputs
Inputs
“Traditional” CRF neural network
• Word inputs and
output
• Word-level
embeddings
(GLOVE)
• Lots of chemical
knowledge
(features and
chemical
dictionaries)
• Single LSTM layer

27. Embeddings
Bidirectional
LSTM 3
SOBIESOBIESOBIE
Character n Character n+1Character n-1
Final Layer
Outputs
Inputs
Bidirectional
LSTM 2
Bidirectional
LSTM 1
“Minimalist” neural network
• Character inputs
and output
• Character-level
embeddings
• No chemical
knowledge
• Just relies on
training outputs
from inputs
• 3 LSTM layers

28. Results
System Offical
F-score
Official
Precision
Official
Recall
Internal
F-score
Internal
Precision
Internal
Recall
Trad .8919 .8867 .8971 .8703 .8648 .8758
Minimal .8901 .8865 .8936 .8664 .8479 .8858
Ensemble .9032 .9002 .9062 .8807 .8646 .8976
• Participating in public, competitive evaluation (BioCreative V.5 Becalm)
 0.9006 precision, 0.9062 recall, .9032 F
 3rd place out of 17 (0.1% off 1st, “differences in the top three
weren’t statistically significant”)
 Compare with typical inter-annotator agreement studies for
manual annotators 90-93% (human level)*
*Peter Corbett, Colin Batchelor, and Simone Teufel. Annotation of chemical named entities. Proceedings of
the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing. Association for
Computational Linguistics, 2007.

29. • Peter Corbett, John Boyle. “Chemlistem - chemical named entity recognition
using recurrent neural networks” (2017)
http://www.biocreative.org/media/store/files/2017/BioCreative_V5_paper8.pdf
• Open source:
 http://bitbucket.org/rscapplications/chemlistem
 pip install chemlistem
Chemlistem

30. e.g. word2vec and
GloVe
• high-dimensional
(often 300) vector
per token but can
be reduced and
visualised
Additional output - Word embeddings?
• Trainable inside neural network…
• e.g. king + man - woman = queen

31. • Could our trained GloVE word embeddings be
another useful output?
• query word to find and visualise related
words
• also e.g.:
• benzene – hazard + solvent = ?
• KOH – base + acid = ?
Additional output - Word embeddings?

32. What can Data Science and this big data
do for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

33. Article Recommendation

34. • Built in Elastic Search
• Four alternative methods considered
• User testing…
A New Article Recommender

35. content
similarity
content
similarity
cited
together
combined
results
readers
also readMethod 2 Method 3 Method 4Method 1

36. • All find interesting papers
• Early-stage reading/review writing
• Recent is good
• Different preferred methods before and after
they know which is which…
Key user observations

37. User Preferences
0
1
2
3
4
5
6
7
Before After
combined results
cited together
readers also read
content similarity

38. What can Data Science and this big data
do for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

39. What about a molecule recommender?
What other molecules are “related” to vancomycin?
Vancomycin

40. • ChemSpider web
logs (2015-2016),
molecules
grouped by user
IDs, anonymised,
aggregated
• RSC corpus
(2000-2012), text-
mined for
chemical
compounds,
molecules
grouped by article
• “Morgan
(radius=2)
fingerprinting
• Topology
fingerprinting

41. • Initial user testing indicates that
researchers prefer a range of methods and
molecules to eyeball
• Not just one “I feel lucky…” guess
• Primarily used as a tool to decide what to
research next but within that many different
questions
User testing

42. For example…

43. What can Data Science and this big data
do for Chemistry?
Tech Development
• Data processing pipeline
• Term extraction from
literature
Applications
• Citation velocity
• Recommending papers
Cheminformatics
• Molecular characterisation
• Chemical similarity
• Molecule recommender
Business analytics
• Lead generation
• Data dashboards
• Trend analysis
• Category dashboard

44. Colin Batchelor
Most of this work was done by…
Peter Corbett
John Boyle
Nicholas Bailey
Jeff White
Aileen Day
With help from the rest of RSC Data Science…

45. www.rsc.org/data-science