Crowdsourcing, Collaborations and Text-Mining in a World of Open Chemistry Antony Williams

Imagine a time when …. The internet is searchable by chemical structure and substructure (e.g.Wikipedia, Google Scholar) Chemistry articles are indexed and searchable by a free online service Publicly funded research data can be shared and discussed in the Open, maybe as ONS? Cheminformatics has as much of a public face as bioinformatics and protectionism of data related to “structures” has waned

The internet is searchable by chemical structure and substructure (e.g.Wikipedia, Google Scholar)

Chemistry articles are indexed and searchable by a free online service

Publicly funded research data can be shared and discussed in the Open, maybe as ONS?

Cheminformatics has as much of a public face as bioinformatics and protectionism of data related to “structures” has waned

ChemSpider - A Search Engine for Chemists Questions a chemist might ask… What is the melting point of n-butanol? What is the chemical structure of Xanax? Chemically, what is phenolphthalein? What are the stereocenters of cholesterol? Where can I find publications about xylene? What are the different trade names for Ketoconazole? What is the NMR spectrum of Aspirin? What are the safety handling issues for Thymol Blue? ChemSpider can answer all of these questions

Questions a chemist might ask…

What is the melting point of n-butanol?

What is the chemical structure of Xanax?

Chemically, what is phenolphthalein?

What are the stereocenters of cholesterol?

Where can I find publications about xylene?

What are the different trade names for Ketoconazole?

What is the NMR spectrum of Aspirin?

What are the safety handling issues for Thymol Blue?

ChemSpider can answer all of these questions

ChemSpider Data Content Over 21.5 million unique chemical structures from ca. 150 data sources Online Databases –PubChem, Drugbank, KEGG, Wikipedia Literature – PubMed, J Het Chem, Nature, RSC, Open Access Chemical Vendors – over 40 different vendors and growing Personal Depositions – individual contributions Content database vendors Analytical data collections Patents Web scraping Content is linked back to the original data sources

Over 21.5 million unique chemical structures from ca. 150 data sources

Online Databases –PubChem, Drugbank, KEGG, Wikipedia

Literature – PubMed, J Het Chem, Nature, RSC, Open Access

Chemical Vendors – over 40 different vendors and growing

Personal Depositions – individual contributions

Content database vendors

Analytical data collections

Patents

Web scraping

Content is linked back to the original data sources

Tell me about Aspirin

Tell me about Aspirin

Link outs

Links out to KEGG Kyoto Encyclopedia of Genes and Genomes

Tell me about Aspirin

Tell me About Aspirin

Tell me about Aspirin

Tell me about Aspirin

Tell me about Aspirin

Text- Indexing and ChemSpider? ChemSpider text-indexes almost 500,000 Open Access and Free Access articles Collection is growing and more publishers have already agreed. Including theses in the future.

ChemSpider text-indexes almost 500,000 Open Access and Free Access articles

Collection is growing and more publishers have already agreed. Including theses in the future.

Open Access Literature Search

Search PubMed – ChemSpider

Other Searches What compounds have a mass of 300+/-0.001? or search a combination of intrinsic/predicted properties

What compounds have a mass of 300+/-0.001?

or search a combination of intrinsic/predicted properties

Other Searches

Complex Search

The Quality of Data Online… Aggregating data opens up quality issues Structure-identifier associations are “dirty” Structures are COMMONLY incorrect Manual curation of small databases is enough work – what about millions of structures? Structures are far from perfect. What is a “correct structure”? Full stereochemistry? Historical timeline of structure? Who is the authority?

Aggregating data opens up quality issues

Structure-identifier associations are “dirty”

Structures are COMMONLY incorrect

Manual curation of small databases is enough work – what about millions of structures?

Structures are far from perfect. What is a “correct structure”?

Full stereochemistry?

Historical timeline of structure?

Who is the authority?

Who holds THE Quality Authority? Chemical Abstracts Service is the structural authority today. 1400 (?) employees, world standard in chemistry information 101 years of knowledge, process and expertise. MANUAL curation is key. Robotic curation is enabling – Chloride with no Cl How can an online, free access system peacefully co-exist with the authority?

Chemical Abstracts Service is the structural authority today. 1400 (?) employees, world standard in chemistry information

101 years of knowledge, process and expertise.

MANUAL curation is key. Robotic curation is enabling – Chloride with no Cl

How can an online, free access system peacefully co-exist with the authority?

Quality is a Major Issue- Search Butanol OLD EXAMPLE..now fixed

Wikipedia Chemistry Curation project Only ca. 5000 organic structures, 7000 total structures Almost a year of work so far for a team of 6 people Many errors removed in the process. Curation process is a daily event for users/depositors Slow and torturous process http://en.wikipedia.org/wiki/Talk:Tacrolimus#IUPAC_Name_and_structure

Only ca. 5000 organic structures, 7000 total structures

Almost a year of work so far for a team of 6 people

Many errors removed in the process. Curation process is a daily event for users/depositors

Slow and torturous process

http://en.wikipedia.org/wiki/Talk:Tacrolimus#IUPAC_Name_and_structure

Wikipedia Curation Looking for self-consistency across a Wikipedia Page Primary key is the article TITLE The chemical shown needs to match the title – then registry numbers, names, identifiers, outlinks need to match the chemical shown. Cyclic self-consistency – and decisions must get made

Looking for self-consistency across a Wikipedia Page

Primary key is the article TITLE

The chemical shown needs to match the title – then registry numbers, names, identifiers, outlinks need to match the chemical shown.

Cyclic self-consistency – and decisions must get made

Viagra Viagra is sildenafil…

Viagra is sildenafil…

Viagra is Sildenafil Citrate Sildenafil may be shown in the Wikipedia record but their might be a redirect from a search on Viagra CAS registry Number: 139755-83-2, 171599-83-0 (as citrate). If structure box shows neutral compound then CAS Number must match neutral compound OR annotate

Sildenafil may be shown in the Wikipedia record but their might be a redirect from a search on Viagra

CAS registry Number: 139755-83-2, 171599-83-0 (as citrate). If structure box shows neutral compound then CAS Number must match neutral compound OR annotate

Other issues… If structure shows no stereo then don’t put stereo in name Outlinks to external databases – links should be structure to structure not name to name Tautomers

If structure shows no stereo then don’t put stereo in name

Outlinks to external databases – links should be structure to structure not name to name

Tautomers

Charges

Sugars – Machine Readable vs Aesthetics Fischer Stereo Haworth

Wikipedia – Crowdsourcing Chemistry

Thymol Blue on ChemSpider Data online includes: UV-vis spectrum Measured experimental properties Link to Wikipedia article Links to chromatography details Multiple identifiers/trade names etc. Links to vendors/suppliers/other databases Safety information http://www.chemspider.com/q/thymol%20blue

Data online includes:

UV-vis spectrum

Measured experimental properties

Link to Wikipedia article

Links to chromatography details

Multiple identifiers/trade names etc.

Links to vendors/suppliers/other databases

Safety information

http://www.chemspider.com/q/thymol%20blue

Differences between ChemSpider/Wikipedia No, but links. Analytical Data Active editors > 50 (?) Active depositors/curators – 30 No Prediction of properties ???? 6000 people/day; 1900 registered Detailed compound monographs Compound monographs linked Text Complex queries – Properties, Text, structure/substructure, OA publishers, Data Sources, … ~5000 organics, 2000 others >21 million unique structures Wikipedia ChemSpider

Differences between Wikipedia/ChemSpider Growing reputation as focused on quality Worldwide reputation as quality source – good and bad Chemistry is the focus of ‘Spider Chemistry is a subset of the ‘Pedia Mixed “licensing” GFL licensing for everything Growing team of advocates, curators and users Strong team of WP:Chem advocates, curators and admins “ Out of a basement” on three servers and 5 volunteers Established infrastructure and Wikipedia Foundation Team Primarily Microsoft .NET technologies with OS components Supported by tried and tested Media-Wiki platform. ChemSpider Wikipedia

Usage Growth – Representative Plot

Crowd-sourcing Curation How to curate data for millions of structures? Robot processes can clean up depositions Search for Chloride and check molecular formula for Cl Check for stereochemistry and remove names with stereo Provide a simple-to-use platform to curate, annotate and tag data Provide curator administration to prevent vandalism (Veropedia)

How to curate data for millions of structures?

Robot processes can clean up depositions

Search for Chloride and check molecular formula for Cl

Check for stereochemistry and remove names with stereo

Provide a simple-to-use platform to curate, annotate and tag data

Provide curator administration to prevent vandalism (Veropedia)

Multi-level Curation and Approval

Post Comments Anyone can “Post Comments” associated with a structure. To curate data we require login to track

Anyone can “Post Comments” associated with a structure. To curate data we require login to track

Crowd-sourcing Chemistry Crowd-sourced curation: identify and tag errors, edit names, synonyms, identify records for deprecation ALSO Crowd-sourced deposition: anyone can deposit data (structures, text, images, analytical data)

Crowd-sourced curation: identify and tag errors, edit names, synonyms, identify records for deprecation

ALSO

Crowd-sourced deposition: anyone can deposit data (structures, text, images, analytical data)

Structure-Centric We want to search Open-Access articles by structure, substructure, similarity of structure Standard approaches would be: Identify chemical names “entity extraction” Convert chemical names to structures and index ChemSpider has a validated dictionary of structure-name pairs Use name extraction, name-conversion and dictionary look-up. THEN curate.

We want to search Open-Access articles by structure, substructure, similarity of structure

Standard approaches would be:

Identify chemical names “entity extraction”

Convert chemical names to structures and index

ChemSpider has a validated dictionary of structure-name pairs

Use name extraction, name-conversion and dictionary look-up. THEN curate.

“Entity Extraction” Rule-based recognition of systematic names: Use a lexeme of name fragments Rules for identifying bounds of a name Look-up dictionary: Drug Names Trivial Names Numbers : Registry IDs, EINECS/ELINCS/Beilstein IDs Massive look-up dictionary of validated identifiers on ChemSpider

Rule-based recognition of systematic names:

Use a lexeme of name fragments

Rules for identifying bounds of a name

Look-up dictionary:

Drug Names

Trivial Names

Numbers : Registry IDs, EINECS/ELINCS/Beilstein IDs

Massive look-up dictionary of validated identifiers on ChemSpider

Name-to-Structure and Lexemes

Name Recognition Azo aldehyde 2 was synthesized according to a reported method [17]. To a stirred solution of azo aldehyde 2 (1.08 g, 3.76 mmol ) in dry CH2Cl2 (30.00 mL) at 0 oC were successively added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excces of anhydrous MgSO4 (2.00 g,16.67 mmol) . The resulting mixture was stirred for 6 hours at room temperature [18]. The mixture was filtered and washed with dichloromethane . Then the solvent was evaporated under reduced pressure to give azo Schiff base 3 as a red solid which was recrystalized from ethanol 95% (1.28 g, 91 %)

Azo aldehyde 2 was synthesized according to a reported method [17]. To a stirred solution of azo aldehyde 2 (1.08 g, 3.76 mmol ) in dry CH2Cl2 (30.00 mL) at 0 oC were successively added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excces of anhydrous MgSO4 (2.00 g,16.67 mmol) .

The resulting mixture was stirred for 6 hours at room temperature [18]. The mixture was filtered and washed with dichloromethane . Then the solvent was evaporated under reduced pressure to give azo Schiff base 3 as a red solid which was recrystalized from ethanol 95% (1.28 g, 91 %)

Name Recognition Azo aldehyde 2 was synthesized according to a reported method [17]. To a stirred solution of azo aldehyde 2 (1.08 g, 3.76 mmol ) in dry CH2Cl2 (30.00 mL) at 0 oC were successively added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excess of anhydrous MgSO 4 (2.00 g,16.67 mmol) . The resulting mixture was stirred for 6 hours at room temperature [18]. The mixture was filtered and washed with dichloromethane . Then the solvent was evaporated under reduced pressure to give azo Schiff base 3 as a red solid which was recrystalized from ethanol 95% (1.28 g, 91 %)

Azo aldehyde 2 was synthesized according to a reported method [17]. To a stirred solution of azo aldehyde 2 (1.08 g, 3.76 mmol ) in dry CH2Cl2 (30.00 mL) at 0 oC were successively added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excess of anhydrous MgSO 4 (2.00 g,16.67 mmol) .

The resulting mixture was stirred for 6 hours at room temperature [18]. The mixture was filtered and washed with dichloromethane . Then the solvent was evaporated under reduced pressure to give azo Schiff base 3 as a red solid which was recrystalized from ethanol 95% (1.28 g, 91 %)

How Many Chemical Names? “ She had the drive to derive success in any venture and was well versed in Karate. When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil. He went home and took an aspirin after the beating.”

“ She had the drive to derive success in any venture and was well versed in Karate. When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil. He went home and took an aspirin after the beating.”

How Many Chemical Names? “ She had the drive to derive success in any venture and was well versed in Karate . When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil . He went home and took an aspirin after the beating.”

“ She had the drive to derive success in any venture and was well versed in Karate . When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil . He went home and took an aspirin after the beating.”

ChemMantis Chem ical M arkup A nd N omenclature T ransformation I ntegrated S ystem

Chem ical M arkup A nd N omenclature T ransformation I ntegrated S ystem

Making Open Access Articles Searchable Proof of Concept Can we HOST Chemistry Open Access articles on ChemSpider and add-value Can we identify chemical names in Open Access articles in a user-friendly manner Can we convert names to structures in Open-Access articles and expand ChemSpider and provide structure searching of Open Access chemistry articles? Can we provide an environment for chemists to mark-up their own articles and crowd-source markup of an archive?

Can we HOST Chemistry Open Access articles on ChemSpider and add-value

Can we identify chemical names in Open Access articles in a user-friendly manner

Can we convert names to structures in Open-Access articles and expand ChemSpider and provide structure searching of Open Access chemistry articles?

Can we provide an environment for chemists to mark-up their own articles and crowd-source markup of an archive?

Document markup ChemSpider now hosting Open Access articles from MDPI, Molecular Diversity Preservation International Hosting the Molbank collection at present

ChemSpider now hosting Open Access articles from MDPI, Molecular Diversity Preservation International

Hosting the Molbank collection at present

A Standard for Document Markup? NLM-DTD: National Library of Medicine; Document Type Definition Approved markup definitions to apply to journal articles – extended as necessary for our purposes

NLM-DTD: National Library of Medicine; Document Type Definition

Approved markup definitions to apply to journal articles – extended as necessary for our purposes

NLM/DTD markup

Chemistry and Biology Menus can be extended as necessary

Menus can be extended as necessary

Document markup

Markup – 3 seconds!

On the fly conversion

Shorthand Formulae Supported

Curators Tools During Markup

One Click to more Info…

Coming Soon….

Structure Image Conversion

Two Seconds Later

Not Always Perfect….

A Platform for Markup Can we provide a platform for document markup for chemists? Workflow: Upload word docs, RTF files or point to HTML and load Apply entity extraction, convert names to structures, mark-up automatically and ask for user participation Publish final version with NLM-DTD markup Deposit all structures on ChemSpider under embargo and wait for article DOI to release

Can we provide a platform for document markup for chemists?

Workflow:

Upload word docs, RTF files or point to HTML and load

Apply entity extraction, convert names to structures, mark-up automatically and ask for user participation

Publish final version with NLM-DTD markup

Deposit all structures on ChemSpider under embargo and wait for article DOI to release

Challenges Computer software can generate chemical names better than the majority of chemists The majority of chemical names are generated by humans, and Incorrect – convert to the wrong structure or are ambiguous One name, Multiple Structures

Computer software can generate chemical names better than the majority of chemists

The majority of chemical names are generated by humans, and Incorrect – convert to the wrong structure or are ambiguous

One name, Multiple Structures

Names and Structures Dichloroacetone Trichloromethylsilane

Dichloroacetone

Trichloromethylsilane

Ambiguity

Ambiguity in Abbreviations - DPA

Ambiguity in Abbreviations - THF

Import is Easy Make articles Public/Private (embargo date soon) Auto-markup and check by user

Make articles Public/Private (embargo date soon)

Auto-markup and check by user

IUPAC PAC Articles

Supports Word .DOC, HTML, RTF

Drexel University Documents

Drexel University Documents

Drexel University Documents

Patents

Organism Markup

Extensible Markup Process Markup process is easily extendable Configurable from one XML file NLM/DTD is incorporated but is easy to extend

Markup process is easily extendable

Configurable from one XML file

NLM/DTD is incorporated but is easy to extend

Markup Movie

DailyMed

DailyMed on ChemSpider

Quality of Structures!!!

Quality of Structures

DailyMed

Remaining Issues in Markup Ongoing curation of look-up dictionaries – one name linked to multiple structures…which is right? Remember tautomers! Import and markup of large documents – tested up to 2Mbyte. Needs extending for larger documents Add comments/tags to documents where markup finds issues.

Ongoing curation of look-up dictionaries – one name linked to multiple structures…which is right? Remember tautomers!

Import and markup of large documents – tested up to 2Mbyte. Needs extending for larger documents

Add comments/tags to documents where markup finds issues.

Oops… Online document markup and indexing is a very disruptive offering and a natural extension for ChemSpider

Online document markup and indexing is a very disruptive offering and a natural extension for ChemSpider

What’s Coming? Agreement with Royal Society of Chemistry that we can add their structure-based RSS feeds to ChemSpider Agreement with Nature Publishing Group to add their Nature Chemical Biology structure collections to ChemSpider as they issue Presently indexing Acta Chemica Scandanavica, 1947-1999 PDF backfile – our first foray into OCR Presently indexing PLoS journals directly More publishers have agreed…

Agreement with Royal Society of Chemistry that we can add their structure-based RSS feeds to ChemSpider

Agreement with Nature Publishing Group to add their Nature Chemical Biology structure collections to ChemSpider as they issue

Presently indexing Acta Chemica Scandanavica, 1947-1999 PDF backfile – our first foray into OCR

Presently indexing PLoS journals directly

More publishers have agreed…

What’s Coming? Now working on “organism” extraction in document markup to identify organisms and link out to external resources – bacteria, fungi, viruses Open platform for users to deposit and markup their documents Export the markup in XML format and then map to the NLM-DTD Extract machine-readable structures directly

Now working on “organism” extraction in document markup to identify organisms and link out to external resources – bacteria, fungi, viruses

Open platform for users to deposit and markup their documents

Export the markup in XML format and then map to the NLM-DTD

Extract machine-readable structures directly

Conclusions The quality of structure-based data online should always be questioned – that includes ChemSpider Robots and software algorithms can help but eyeballs are necessary Data on ChemSpider are being added and curated on a daily basis but we need more eyeballs helping always ChemSpider now has a large validated structure-name dictionary Chemical name extraction and document markup is very enabling

The quality of structure-based data online should always be questioned – that includes ChemSpider

Robots and software algorithms can help but eyeballs are necessary

Data on ChemSpider are being added and curated on a daily basis but we need more eyeballs helping always

ChemSpider now has a large validated structure-name dictionary

Chemical name extraction and document markup is very enabling

Further reading www.chemspider.com/blog Internet-based tools for communication and collaboration in chemistry, Drug Discovery Today, Volume 13, Numbers 11/12, June 2008 502-506, doi:10.1016/j.drudis.2008.03.015 A perspective of publicly accessible/open-access chemistry databases, Drug Discovery Today, Volume 13, Numbers 11/12, June 2008, 495-501, doi:10.1016/j.drudis.2008.03.017

www.chemspider.com/blog

Internet-based tools for communication and collaboration in chemistry, Drug Discovery Today, Volume 13, Numbers 11/12, June 2008 502-506, doi:10.1016/j.drudis.2008.03.015

A perspective of publicly accessible/open-access chemistry databases, Drug Discovery Today, Volume 13, Numbers 11/12, June 2008, 495-501, doi:10.1016/j.drudis.2008.03.017

ChemSpider Forums/Blogs Forum.chemspider.com ChemSpider Blog Open Chemistry Web ChemConnector Blog www.chemspider.com/blog

Forum.chemspider.com

ChemSpider Blog

Open Chemistry Web

ChemConnector Blog

www.chemspider.com/blog