Web Crawling Chemistry

Weaving a New Web for Chemistry Antony Williams

Structure-Enabled Articles

Searching from the Structure Balloon

Imagine a time when ….

The internet is searchable by chemical structure and substructure

Chemistry articles are indexed and searchable by a free online service

Publicly funded research data can be shared and discussed in the Open

Cheminformatics has as much of a public face as bioinformatics

ChemSpider - A Search Engine for Chemists

Questions a chemist might ask…

What is the melting point of n-butanol?

What is the chemical structure of Xanax?

Chemically, what is phenolphthalein?

What are the stereocenters of cholesterol?

Where can I find publications about xylene?

What are the different trade names for Ketoconazole?

What is the NMR spectrum of Aspirin?

What are the safety handling issues for Thymol Blue?

ChemSpider can answer all of these questions

ChemSpider Data Content

Over 21.5 million unique chemical structures from ca. 150 data sources

Online Databases –PubChem, Drugbank, HMDB, Wikipedia

Chemical Vendors – over 40 different vendors and growing

Personal Depositions – individual contributions

Journal Publishers

Content database vendors

Analytical data collections

Patents (9 MILLION Structures being deposited now )

Web scraping

Content is generally linked back to the original data sources

Tell me about Aspirin

Link outs

Links out to KEGG Kyoto Encyclopedia of Genes and Genomes

Tell me About Aspirin

Text- Indexing and ChemSpider?

ChemSpider text-indexes almost 500,000 Open Access and Free Access articles

Collection is growing weekly and more publishers have already agreed

Open Access Literature Search

Search PubMed – ChemSpider

Other Searches

Complex Search

The Quality of Data Online…

Aggregating data opens up quality issues

Structure-identifier associations are “dirty”

Structures are COMMONLY incorrect – stereochem issues

Manual curation of small databases is enough work – what about millions of structures?

Structures are far from perfect. What is a “correct structure”?

Full stereochemistry?

Historical timeline of structure?

Who is the authority?

Who holds THE Quality Authority?

Chemical Abstracts Service is the structural authority today. 1400 (?) employees, world standard in chemistry information

101 years of knowledge, process and expertise. MANUAL curation is key. Robotic curation is enabling

How can an online, free access system peacefully co-exist with the authority?

Quality is a Major Issue- Search Butanol

Wikipedia – Crowdsourcing Chemistry

Wikipedia Chemistry Curation project

Only ca. 5000 organic structures, 7000 total structures

MONTHS of work so far for a team of 6 people

Many errors removed in the process. Curation process is a daily event for users/depositors

Slow and torturous process for stereo molecules.

Thymol Blue on ChemSpider

Data online includes:

UV-vis spectrum

Measured experimental properties

Link to Wikipedia article

Links to chromatography details

Multiple identifiers/trade names etc.

Links to vendors/suppliers/other databases

Safety information

Differences between ChemSpider/Wikipedia No Analytical Data Active editors – about 50 (?) Active depositors/curators – 30 No Prediction of properties ???? 5000 people/day; 1100 registered Detailed compound monographs Compound monographs linked Text Complex queries – Properties, Text, structure/substructure, OA publishers, Data Sources, … ~5000 organics, 2000 others >21 million unique structures Wikipedia ChemSpider

Differences between Wikipedia/ChemSpider Growing reputation as focused on quality Worldwide reputation as quality source Chemistry is the focus of ‘Spider Chemistry is a subset of the ‘Pedia Mixed “licensing” GFL licensing for everything Growing team of WP:Chem advocates, curators and admins Strong team of WP:Chem advocates, curators and admins “ Out of a basement” on three servers and 5 volunteers Established infrastructure and Wikipedia Foundation Team Primarily Microsoft .NET technologies with OS components Supported by tried and tested Media-Wiki platform. ChemSpider Wikipedia

Crowd-sourcing Curation

How to curate data for millions of structures?

Robot processes can clean up depositions

Search for Chloride and check molecular formula for Cl

Check for stereochemistry and remove names with stereo

Provide a simple-to-use platform to curate, annotate and tag data

Provide curator administration to prevent vandalism (Veropedia)

Multi-level Curation and Approval

Post Comments

Anyone can “Post Comments” associated with a structure. To curate data we require login to track

Crowd-sourcing Chemistry

Crowd-sourced curation: identify and tag errors, edit names, synonyms, identify records for deprecation

ALSO

Crowd-sourced deposition: anyone can deposit data (structures, text, images, analytical data)

But, when registered and logged in…

Ability to curate and add to the database

Add structures

“ Clean” structures

Add data (spectra, CIFs, images)

Add links to other pages (URLs)

Add publication details

Adding to the Database - Structure

Adding New Text Data Add Publication Add Identifier Add URL

Adding Supplementary Info to a Structure

ChemSpider TouchGraph

Structure-Centric

We want to search Open-Access articles by structure, substructure, similarity of structure

Standard approaches would be:

Identify chemical names “entity extraction”

Convert chemical names to structures and index

ChemSpider has a validated dictionary of structure-name pairs

Use name extraction, name-conversion and dictionary look-up. THEN curate.

“Entity Extraction”

Rule-based recognition of systematic names:

Use a lexeme of name fragments

Rules for identifying bounds of a name

Look-up dictionary:

Drug Names

Trivial Names

Numbers : Registry IDs, EINECS/ELINCS/Beilstein IDs

Name Recognition

Azo aldehyde 2 was synthesized according to a reported method [17]. To a stirred solution of azo aldehyde 2 (1.08 g, 3.76 mmol ) in dry CH2Cl2 (30.00 mL) at 0 oC were successively added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excces of anhydrous MgSO4 (2.00 g,16.67 mmol) . The resulting mixture was stirred for 6 hours at room temperature [18]. The mixture was filtered and washed with dichloromethane . Then the solvent was evaporated under reduced pressure to give azo Schiff base 3 as a red solid which was recrystalized from ethanol 95% (1.28 g, 91 %)

Name Recognition

Azo aldehyde 2 was synthesized according to a reported method [17]. To a stirred solution of azo aldehyde 2 (1.08 g, 3.76 mmol ) in dry CH2Cl2 (30.00 mL) at 0 oC were successively added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excess of anhydrous MgSO 4 (2.00 g,16.67 mmol) .

The resulting mixture was stirred for 6 hours at room temperature [18]. The mixture was filtered and washed with dichloromethane . Then the solvent was evaporated under reduced pressure to give azo Schiff base 3 as a red solid which was recrystalized from ethanol 95% (1.28 g, 91 %)

How Many Chemical Names?

“ She had the drive to derive success in any venture and was well versed in Karate. When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil. He went home and took an aspirin after the beating.”

How Many Chemical Names?

“ She had the drive to derive success in any venture and was well versed in Karate . When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil . He went home and took an aspirin after the beating.”

Making Open Access Articles Searchable Proof of Concept