Hi, I’m here to talk about AMI; a data extraction framework and tool. First, I just want highlight some of key contributors to the projects; Andy for his work on the ChemistryVisitor and Peter for the overall architecture.
In this talk, I’m going to impress the importance of data in a specific format and its utility to automated machine processing. Then I’m going to demonstrate AMI’s architecture and the transformation of data as it flows through the process. I’m going to dwell a little on a core format used, Scalable Vector Graphics (SVG) before introducing the concept of visitors, which are pluggable context specific data extractors. Next, I’m going to introduce Andy’s ChemVisitor, for extracting semantic chemistry data, along with a few other visitors that can process non-chemistry specific data. Finally, I will demonstrate some uses of the ChemVisitor, within the realm of validation and metabolism.
ContentMining in Neuroscience
Open Mining of the Bioscience Literature
ContentMine.org and the University of Cambridge
UNAM, MX 2015-10-09
Millions of data points are hidden in the bioscience literature.
ContentMine has Open technology to liberate them automatically.
Using OpenNotebook approaches
The major problem is politico-legal
This is an exploratory talk, looking for ideas and projects
The future depends on young people
Jenny Molloy with mascot AMI
Some particularly relevant Fellows/Alumni and projects:
• Rufus Pollock: Open Knowledge Foundation
• Mark Surman: Mozilla
• Dan Whaley: Hypothes.is
• Daniel Lombrana-Gonzales: PyBossa/Crowdcrafting
Erin McKiernan, 2015 Flash Award
ContentMine and Peter Murray-Rust are funded by:
The Right to Read is the Right to Mine
ContentMine Workshops and
Open Science Brazil, 2014-08
Easily distributed software
Get started in 30 mins
in a morning
Start simple: bagOfWords, Stemming, Regex, templates
Why do we publish science?
• Communicate our results
• Get feedback from peers.
• Provide material that others can re-use.
• Priority and esteem.
[Liberian Ministry of Health] were stunned recently when we stumbled across
an article by European researchers in Annals of Virology : “The results
seem to indicate that Liberia has to be included in the Ebola virus endemic
zone.” In the future, the authors asserted, “medical personnel in Liberian health
centers should be aware of the possibility that they may come across active
cases and thus be prepared to avoid nosocomial epidemics,” referring to
Adage in public health: “The road to inaction is paved with research
Bernice Dahn (chief medical officer of Liberia’s Ministry of Health)
Vera Mussah (director of county health services)
Cameron Nutt (Ebola response adviser to Partners in Health)
A System Failure of Scholarly Publishing
You cannot assume how others will want to re-use your
PM-R’s “first real paper”, doing science by
re-using the results of others in a novel way
Each point is a separate paper!
Needing 1-4 hours
in library – discovery,hardcopy delivery,
Transcription, hand calculation.
PMR and WDSM developed software
And protocols to search and analyze
Cambridge Crystallographic DB
Output of scholarly publishing
586,364 Crossref DOIs 201,507  per month
1.5 million (papers + supplemental data) /year *
each 3 mm thick
4500 m high per year 
* Most is not Publicly readable
Scientific and Medical publication (STM)[+]
• World Citizens pay $450,000,000,000…
• … for research in 1,500,000 articles …
• … cost $300,000 each to create …
• … $7000 each to “publish” [*]…
• … $10,000,000,000 from academic libraries …
• … to “publishers” who forbid access to 99.9% of citizens of
the world …
• 85% of medical research is wasted (not published, badly
conceived, duplicated, …) [Lancet 2009]
[+] Figures probably +- 50 %
[*] arXiV preprint server costs $7 USD per paper
What is “Content”?
contentmine.org tackles these
0. Open software, Open content, Open notebooks
1. Daily liberation of facts which are easy and widely
– Species (Bacillus subtilis, Okapia johnstoni)
– Genes (BRCA1*, APOE)
– Chemicals (acetone, CH3OH)
– Identifiers (RRIDs, museum specimens, )
1. CMunities of practice with bespoke tools:
– Clinical Trials
– Phylogenetic trees
– Systematic reviews
Typical chemical synthesis
Open Content Mining of FACTs
Machines can interpret chemical reactions
We have done 500,000 patents. There are >
3,000,000 reactions/year. Added value > 1B Eur.
C) What’s the problem with this spectrum?
Org. Lett., 2011, 13 (15), pp 4084–4087
Original thanks to ChemBark
After AMI2 processing…..
… AMI2 has detected a square
CORE , HAL,
peerJ… Nature, IEEE,
30, 000 pages/day
ContentMine will work with OpenTrials
“adult nonpregnant patients, aged ≥18 years”,
“randomization sequence using a permuted block design with random
block sizes stratified by study center”.
“blinding of the patients and caregivers is not possible”.
“Investigators performing analysis are blinded for the intervention”.
“Continuous normally distributed variables … mean and standard deviation,
counts (n) and percentages (%). … Student’s t-test … or the Mann–Whitney U test
… Categorical … Chi-square test or Fisher's exact tests. Statistical significance is
considered to be at a P value <0.05 …”
Formulaic language in reporting clinical trials
• Bag of words
(Term-frequency, inverse document frequency)
• Templates and regexes (regular expressions).
“Bag of Words”
Three fulltext articles from trialsjournal.com
Regular Expressions for Systematic Reviews of Animal Tests
In 30 minutes 6 scientists (most were unfamiliar with regex)
wrote 200 regexes for ARRIVE (NC3R guidelines)
Stuffed Tree of Life
mine-images-of-phylogenetic-trees-and-more/ for story of extraction
Planned Memorial Meeting
July 14th 2014 Cambridge
OPEN NOTEBOOK SCIENCE
Traditional Research and Publication
“Lab” work paper/th
Open Notebook Science
Problems are solved communally;
Nothing is needlessly duplicated; “publication“ is
continuous ; data are SEMANTIC
Open Notebook Content Mining
• “No insider knowledge”
• Anyone can become involved
• All raw non-copyright material on Github
• Planning and discussion on Open Discourse
• All output (however imperfect) on Github CC0
• Immediate upload
• Inspired by Free/Libre/Open Source, Wikipedia,
Automatic Open Notebook of computations
Everything is posted to Github before being analyzed
Bacillus subtilis *
Bacteroides fragilis 
Escherichia coli 
Flectobacillus major 
Flexibacter flexilis 
Gelidibacter algens 
Lentibacillus salicampi 
Psychroflexus torquis 
Pseudomonas aeruginosa 
Sagittula stellata 
• [Identifier in Wikidata]
• Missing = not found with Wikidata API
20 commonest organisms (in > 30 papers) in trees from IJSEM*
Half do not appear to be in Wikidata
Can the Wikipedia Scientists comment?
*Int. J. Syst. Evol. Microbiol.
Prof. Ian Hargreaves (2011): "David Cameron's
exam question”: "Could it be true that laws
designed more than three centuries ago with the
express purpose of creating economic incentives
for innovation by protecting creators' rights are
today obstructing innovation and economic
“yes. We have found that the UK's intellectual
property framework, especially with regard to
copyright, is falling behind what is needed.” "Digital
Opportunity" by Prof Ian Hargreaves - http://www.ipo.gov.uk/ipreview.htm. Licensed under CC BY 3.0 via Wikipedia -
Elsevier wants to control Open Data
[asked by Michelle Brook]
UPDATE 20150902: Ian Hargreaves "the voices of the digital many should not be
drowned out by the digital self-interested few"