Crowdsourcing, Collaborations and Text-Mining in a  World of Open Chemistry Nature Publishing Group 11/2008   Antony Willi...
Imagine a time when …. <ul><li>The internet is searchable by chemical structure and substructure (e.g.Wikipedia, Google Sc...
ChemSpider - A Search Engine for Chemists <ul><li>Questions a chemist might ask… </li></ul><ul><ul><li>What is the melting...
What is a Structure? Ask a computer…ask a chemist
Tell Me About Glutathione
Tell Me About Glutathione
Tell Me About Glutathione
Tell Me About Glutathione
Tell Me About Glutathione
Tell Me About Glutathione
Link outs
Links out to KEGG Kyoto Encyclopedia of Genes and Genomes
How many names does a compound have?
ChemSpider Data Content <ul><li>Over 21.5 million unique chemical structures from ca. 150 data sources </li></ul><ul><ul><...
Other Searches <ul><li>What compounds have a mass of 300+/-0.001? </li></ul><ul><li>or search a combination of intrinsic/p...
Other Searches
Complex Search
The Quality of Data Online… <ul><li>Aggregating data opens up quality issues </li></ul><ul><li>Structure-identifier associ...
Who holds THE Quality Authority? <ul><li>Chemical Abstracts Service is the structural authority today. 1400 employees, wor...
Quality is a Major Issue- Search Butanol OLD EXAMPLE..now fixed
Wikipedia Chemistry Curation project <ul><li>Only ca. 5000 organic structures, 7000 total structures </li></ul><ul><li>Alm...
Wikipedia Curation <ul><li>Looking for self-consistency across a Wikipedia Page </li></ul><ul><li>Primary key is the artic...
Viagra or Sildenafil
Other issues…
Charges
Sugars – Machine Readable vs Aesthetics Haworth  Stereo  Fischer
Wikipedia – Crowdsourcing Chemistry
Thymol Blue on ChemSpider <ul><li>Data online includes: </li></ul><ul><ul><li>UV-vis spectrum </li></ul></ul><ul><ul><li>M...
Differences between ChemSpider/Wikipedia No, but links. Analytical Data Active editors > 50 (?) Active depositors/curators...
Differences between Wikipedia/ChemSpider Growing reputation as focused on quality Worldwide reputation as quality source –...
Crowd-sourcing Curation <ul><li>How to curate data for millions of structures?  </li></ul><ul><li>Robot processes can clea...
Post Comments <ul><li>Anyone can “Post Comments” associated with a structure. To curate data we require login to track </l...
Multi-level Curation and Approval
Crowd-sourcing Chemistry <ul><li>Crowd-sourced curation: identify and tag errors, edit names, synonyms, identify records f...
DailyMed
Quality of Structures
Quality of Structures!!!
Structure-Centric  <ul><li>We want to search “information” by structure, substructure, similarity of structure </li></ul><...
“Entity Extraction” <ul><li>Rule-based recognition of systematic names: </li></ul><ul><ul><li>Use a lexeme of name fragmen...
 
Name Recognition <ul><li>Azo aldehyde  2   was  synthesized according to a reported  method [17]. To  a stirred  solution ...
Name Recognition <ul><li>Azo aldehyde  2   was  synthesized according to a reported  method [17]. To  a stirred  solution ...
How Many Chemical Names? <ul><li>“ She had the drive to derive success in any venture and was well versed in Karate. When ...
How Many Chemical Names? <ul><li>“ She had the  drive  to derive  success  in any venture and was well  versed  in  Karate...
ChemMantis <ul><li>Chem ical  M arkup  A nd  N omenclature  T ransformation  I ntegrated  S ystem </li></ul>
Making Open Access Articles Searchable Proof of Concept <ul><li>Can we HOST Chemistry Open Access articles on ChemSpider a...
Document markup <ul><li>ChemSpider now hosting Open Access articles from MDPI, Molecular Diversity Preservation Internatio...
A Standard for Document Markup? <ul><li>NLM-DTD: National Library of Medicine; Document Type Definition </li></ul><ul><li>...
NLM/DTD markup
Chemistry and Biology <ul><li>Menus can be extended as necessary  </li></ul>
Document markup
Markup – 3 seconds!
On the fly conversion
Shorthand Formulae Supported
One Click to more Info…
Structure Image Conversion
Two Seconds Later
Not Always Perfect….
A Platform for Markup <ul><li>Can we provide a platform for document markup for chemists? </li></ul><ul><li>Workflow: </li...
Challenges <ul><li>Computer software can generate chemical names better than the majority of chemists </li></ul><ul><li>Th...
Names and Structures <ul><li>Dichloroacetone </li></ul><ul><li>Trichloromethylsilane </li></ul>
Ambiguity
Ambiguity in Abbreviations - DPA
Ambiguity in Abbreviations - THF
Import is Easy <ul><li>Make articles Public/Private (embargo date soon) </li></ul><ul><li>Auto-markup and check by user </...
IUPAC PAC Articles
Supports Word .DOC, HTML, RTF
Drexel University Documents
Drexel University Documents
Drexel University Documents
Patents
<ul><li>Single Configuration File defines entities for markup </li></ul><ul><li>Algorithms can be built for certain entiti...
Nature Publications
Entity Balloons <ul><li>Structures are the language of chemistry </li></ul><ul><li>Show structures to chemists and search/...
Other Dictionaries - Species <ul><li>We are considering  </li></ul><ul><ul><li>Bacteria </li></ul></ul><ul><ul><li>Fungi <...
Integrations Out to Other Sources
Integrations Out to Other Sources
Reactions
Manual Curation is Always Necessary
Text- Indexing  and ChemSpider? <ul><li>ChemSpider text-indexes almost 500,000 Open Access and Free Access articles </li><...
Open Access Literature Search
Conclusions <ul><li>The quality of structure-based data online should always be questioned – that includes ChemSpider </li...
Oops…
Upcoming SlideShare
Loading in …5
×

A Presentation At Nature Publishing Group Crowdsourcing, Collaborations And Text Mining In A World Of Open Chemistry

1,753 views

Published on

This is a presentation at gave at the Nature Publishing Group office in London, UK in November 2008. It focuses on some of the recent work on document markup and advances in ChemMantis

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,753
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
23
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

A Presentation At Nature Publishing Group Crowdsourcing, Collaborations And Text Mining In A World Of Open Chemistry

  1. 1. Crowdsourcing, Collaborations and Text-Mining in a World of Open Chemistry Nature Publishing Group 11/2008 Antony Williams
  2. 2. Imagine a time when …. <ul><li>The internet is searchable by chemical structure and substructure (e.g.Wikipedia, Google Scholar) </li></ul><ul><li>Chemistry articles are indexed and searchable by a free online service </li></ul><ul><li>The web is linked together through the “language of chemistry” </li></ul><ul><li>Publicly funded research data can be shared and discussed in the Open, maybe as ONS? </li></ul><ul><li>Cheminformatics has as much of a public face as bioinformatics </li></ul>
  3. 3. ChemSpider - A Search Engine for Chemists <ul><li>Questions a chemist might ask… </li></ul><ul><ul><li>What is the melting point of n-butanol? </li></ul></ul><ul><ul><li>What is the chemical structure of Xanax? </li></ul></ul><ul><ul><li>Chemically, what is phenolphthalein? </li></ul></ul><ul><ul><li>What are the stereocenters of cholesterol? </li></ul></ul><ul><ul><li>Where can I find publications about xylene? </li></ul></ul><ul><ul><li>What are the different trade names for Ketoconazole? </li></ul></ul><ul><ul><li>What is the NMR spectrum of Aspirin? </li></ul></ul><ul><ul><li>What are the safety handling issues for Thymol Blue? </li></ul></ul><ul><ul><li>ChemSpider can answer all of these questions </li></ul></ul>
  4. 4. What is a Structure? Ask a computer…ask a chemist
  5. 5. Tell Me About Glutathione
  6. 6. Tell Me About Glutathione
  7. 7. Tell Me About Glutathione
  8. 8. Tell Me About Glutathione
  9. 9. Tell Me About Glutathione
  10. 10. Tell Me About Glutathione
  11. 11. Link outs
  12. 12. Links out to KEGG Kyoto Encyclopedia of Genes and Genomes
  13. 13. How many names does a compound have?
  14. 14. ChemSpider Data Content <ul><li>Over 21.5 million unique chemical structures from ca. 150 data sources </li></ul><ul><ul><li>Online Databases –PubChem, Drugbank, KEGG, Wikipedia </li></ul></ul><ul><ul><li>Literature – PubMed, J Het Chem, Nature, RSC, Open Access </li></ul></ul><ul><ul><li>Chemical Vendors – over 40 different vendors and growing </li></ul></ul><ul><ul><li>Personal Depositions – individual contributions </li></ul></ul><ul><ul><li>Content database vendors </li></ul></ul><ul><ul><li>Analytical data collections </li></ul></ul><ul><ul><li>Patents </li></ul></ul><ul><ul><li>Web scraping </li></ul></ul><ul><ul><li>Content is linked back to the original data sources </li></ul></ul>
  15. 15. Other Searches <ul><li>What compounds have a mass of 300+/-0.001? </li></ul><ul><li>or search a combination of intrinsic/predicted properties </li></ul>
  16. 16. Other Searches
  17. 17. Complex Search
  18. 18. The Quality of Data Online… <ul><li>Aggregating data opens up quality issues </li></ul><ul><li>Structure-identifier associations are “dirty” </li></ul><ul><li>Structures are COMMONLY incorrect </li></ul><ul><li>Manual curation of small databases is enough work – what about millions of structures? </li></ul><ul><li>Structures are far from perfect. What is a “correct structure”? </li></ul><ul><ul><li>Full stereochemistry? </li></ul></ul><ul><ul><li>Historical timeline of structure? </li></ul></ul><ul><ul><li>Who is the authority? </li></ul></ul>
  19. 19. Who holds THE Quality Authority? <ul><li>Chemical Abstracts Service is the structural authority today. 1400 employees, world standard in chemistry information </li></ul><ul><li>101 years of knowledge, process and expertise. </li></ul><ul><li>How can an online, free access system peacefully co-exist with the authority? </li></ul>
  20. 20. Quality is a Major Issue- Search Butanol OLD EXAMPLE..now fixed
  21. 21. Wikipedia Chemistry Curation project <ul><li>Only ca. 5000 organic structures, 7000 total structures </li></ul><ul><li>Almost a year of work so far for a team of 6 people </li></ul><ul><li>Many errors removed in the process. Curation process is a daily event for users/depositors </li></ul><ul><li>Slow and torturous process </li></ul><ul><li>http://en.wikipedia.org/wiki/Talk:Tacrolimus#IUPAC_Name_and_structure </li></ul>
  22. 22. Wikipedia Curation <ul><li>Looking for self-consistency across a Wikipedia Page </li></ul><ul><li>Primary key is the article TITLE </li></ul><ul><li>The chemical shown needs to match the title </li></ul><ul><li>Cyclic self-consistency – and decisions must get made </li></ul>
  23. 23. Viagra or Sildenafil
  24. 24. Other issues…
  25. 25. Charges
  26. 26. Sugars – Machine Readable vs Aesthetics Haworth Stereo Fischer
  27. 27. Wikipedia – Crowdsourcing Chemistry
  28. 28. Thymol Blue on ChemSpider <ul><li>Data online includes: </li></ul><ul><ul><li>UV-vis spectrum </li></ul></ul><ul><ul><li>Measured experimental properties </li></ul></ul><ul><ul><li>Link to Wikipedia article </li></ul></ul><ul><ul><li>Links to chromatography details </li></ul></ul><ul><ul><li>Multiple identifiers/trade names etc. </li></ul></ul><ul><ul><li>Links to vendors/suppliers/other databases </li></ul></ul><ul><ul><li>Safety information </li></ul></ul><ul><ul><li>http://www.chemspider.com/q/thymol%20blue </li></ul></ul>
  29. 29. Differences between ChemSpider/Wikipedia No, but links. Analytical Data Active editors > 50 (?) Active depositors/curators – 30 No Prediction of properties ???? 6000 people/day; 1900 registered Detailed compound monographs Compound monographs linked Text Complex queries – Properties, Text, structure/substructure, OA publishers, Data Sources, … ~5000 organics, 2000 others >21 million unique structures Wikipedia ChemSpider
  30. 30. Differences between Wikipedia/ChemSpider Growing reputation as focused on quality Worldwide reputation as quality source – good and bad Chemistry is the focus of ‘Spider Chemistry is a subset of the ‘Pedia Mixed “licensing” GFL licensing for everything Growing team of advocates, curators and users Strong team of WP:Chem advocates, curators and admins “ Out of a basement” on three servers and 5 volunteers Established infrastructure and Wikipedia Foundation Team Primarily Microsoft .NET technologies with OS components Supported by tried and tested Media-Wiki platform. ChemSpider Wikipedia
  31. 31. Crowd-sourcing Curation <ul><li>How to curate data for millions of structures? </li></ul><ul><li>Robot processes can clean up depositions </li></ul><ul><ul><li>Search for Chloride and check molecular formula for Cl </li></ul></ul><ul><ul><li>Check for stereochemistry and remove names with stereo </li></ul></ul><ul><li>Provide a simple-to-use platform to curate, annotate and tag data </li></ul><ul><li>Provide curator administration to prevent vandalism (Veropedia) </li></ul>
  32. 32. Post Comments <ul><li>Anyone can “Post Comments” associated with a structure. To curate data we require login to track </li></ul>
  33. 33. Multi-level Curation and Approval
  34. 34. Crowd-sourcing Chemistry <ul><li>Crowd-sourced curation: identify and tag errors, edit names, synonyms, identify records for deprecation </li></ul><ul><li>ALSO </li></ul><ul><li>Crowd-sourced deposition: anyone can deposit data (structures, text, images, analytical data) </li></ul>
  35. 35. DailyMed
  36. 36. Quality of Structures
  37. 37. Quality of Structures!!!
  38. 38. Structure-Centric <ul><li>We want to search “information” by structure, substructure, similarity of structure </li></ul><ul><li>Specific focus on Open Chemistry at present </li></ul><ul><li>Standard approaches would be: </li></ul><ul><ul><li>Identify chemical names “entity extraction” </li></ul></ul><ul><ul><li>Convert chemical names to structures and index </li></ul></ul><ul><li>ChemSpider has a validated dictionary of structure-name pairs </li></ul><ul><li>Use name extraction, name-conversion and dictionary look-up. THEN curate. </li></ul>
  39. 39. “Entity Extraction” <ul><li>Rule-based recognition of systematic names: </li></ul><ul><ul><li>Use a lexeme of name fragments </li></ul></ul><ul><ul><li>Rules for identifying bounds of a name </li></ul></ul><ul><li>Look-up dictionary: </li></ul><ul><ul><li>Drug Names </li></ul></ul><ul><ul><li>Trivial Names </li></ul></ul><ul><ul><li>Numbers : Registry IDs, EINECS/ELINCS </li></ul></ul><ul><ul><li>Massive look-up dictionary of validated identifiers on ChemSpider </li></ul></ul>
  40. 41. Name Recognition <ul><li>Azo aldehyde 2   was  synthesized according to a reported  method [17]. To  a stirred  solution  of azo aldehyde 2   (1.08 g, 3.76 mmol )  in  dry CH2Cl2  (30.00 mL) at  0 oC  were  successively  added (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excces of anhydrous MgSO4 (2.00 g,16.67 mmol) . </li></ul><ul><li>The resulting  mixture  was  stirred  for  6 hours  at room temperature [18]. The mixture was  filtered and washed with dichloromethane . Then the solvent was  evaporated under reduced pressure to  give azo Schiff base 3   as a red solid which was recrystalized from ethanol 95%    (1.28 g, 91 %) </li></ul>
  41. 42. Name Recognition <ul><li>Azo aldehyde 2   was  synthesized according to a reported  method [17]. To  a stirred  solution  of azo aldehyde 2   (1.08 g, 3.76 mmol )  in  dry CH2Cl2   (30.00 mL) at  0 oC  were  successively  added  (3,4-diaminophenyl)phenyl methanone 1 (0.40 g, 1.88 mmol) and a excess of anhydrous MgSO 4 (2.00 g,16.67 mmol) . </li></ul><ul><li>The resulting  mixture  was  stirred  for  6 hours  at room temperature [18]. The mixture was  filtered and washed with dichloromethane . Then the solvent was  evaporated under reduced pressure to  give azo Schiff base 3   as a red solid which was recrystalized from ethanol 95%    (1.28 g, 91 %) </li></ul>
  42. 43. How Many Chemical Names? <ul><li>“ She had the drive to derive success in any venture and was well versed in Karate. When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil. He went home and took an aspirin after the beating.” </li></ul>
  43. 44. How Many Chemical Names? <ul><li>“ She had the drive to derive success in any venture and was well versed in Karate . When the man in the tartan shirt approached her with a dagger in his hand she spat in his face, took the stance of a commando and took advantage of his shock to release the dagger from his grip and causing him to recoil . He went home and took an aspirin after the beating.” </li></ul>
  44. 45. ChemMantis <ul><li>Chem ical M arkup A nd N omenclature T ransformation I ntegrated S ystem </li></ul>
  45. 46. Making Open Access Articles Searchable Proof of Concept <ul><li>Can we HOST Chemistry Open Access articles on ChemSpider and add-value </li></ul><ul><li>Can we identify chemical names in Open Access articles in a user-friendly manner </li></ul><ul><li>Can we convert names to structures in Open-Access articles and expand ChemSpider and provide structure searching of Open Access chemistry articles? </li></ul><ul><li>Can we provide an environment for chemists to mark-up their own articles and crowd-source markup of an archive? </li></ul>
  46. 47. Document markup <ul><li>ChemSpider now hosting Open Access articles from MDPI, Molecular Diversity Preservation International </li></ul><ul><li>Hosting the Molbank collection at present </li></ul>
  47. 48. A Standard for Document Markup? <ul><li>NLM-DTD: National Library of Medicine; Document Type Definition </li></ul><ul><li>Approved markup definitions to apply to journal articles – extended as necessary for our purposes </li></ul>
  48. 49. NLM/DTD markup
  49. 50. Chemistry and Biology <ul><li>Menus can be extended as necessary </li></ul>
  50. 51. Document markup
  51. 52. Markup – 3 seconds!
  52. 53. On the fly conversion
  53. 54. Shorthand Formulae Supported
  54. 55. One Click to more Info…
  55. 56. Structure Image Conversion
  56. 57. Two Seconds Later
  57. 58. Not Always Perfect….
  58. 59. A Platform for Markup <ul><li>Can we provide a platform for document markup for chemists? </li></ul><ul><li>Workflow: </li></ul><ul><ul><li>Upload word docs, RTF files or point to HTML and load </li></ul></ul><ul><ul><li>Apply entity extraction, convert names to structures, mark-up automatically and ask for user participation </li></ul></ul><ul><ul><li>Publish final version with NLM-DTD markup </li></ul></ul><ul><ul><li>Deposit all structures on ChemSpider under embargo and wait for article DOI to release </li></ul></ul>
  59. 60. Challenges <ul><li>Computer software can generate chemical names better than the majority of chemists </li></ul><ul><li>The majority of chemical names are generated by humans, and Incorrect – convert to the wrong structure or are ambiguous </li></ul><ul><li>One name, Multiple Structures </li></ul>
  60. 61. Names and Structures <ul><li>Dichloroacetone </li></ul><ul><li>Trichloromethylsilane </li></ul>
  61. 62. Ambiguity
  62. 63. Ambiguity in Abbreviations - DPA
  63. 64. Ambiguity in Abbreviations - THF
  64. 65. Import is Easy <ul><li>Make articles Public/Private (embargo date soon) </li></ul><ul><li>Auto-markup and check by user </li></ul>
  65. 66. IUPAC PAC Articles
  66. 67. Supports Word .DOC, HTML, RTF
  67. 68. Drexel University Documents
  68. 69. Drexel University Documents
  69. 70. Drexel University Documents
  70. 71. Patents
  71. 72. <ul><li>Single Configuration File defines entities for markup </li></ul><ul><li>Algorithms can be built for certain entities but the majority are dictionaries – vendors, Phys Properties, Analytical </li></ul><ul><li>We can extend our system to support your needs based on dictionaries – what does NPG need/not need? </li></ul>
  72. 73. Nature Publications
  73. 74. Entity Balloons <ul><li>Structures are the language of chemistry </li></ul><ul><li>Show structures to chemists and search/link from there </li></ul>
  74. 75. Other Dictionaries - Species <ul><li>We are considering </li></ul><ul><ul><li>Bacteria </li></ul></ul><ul><ul><li>Fungi </li></ul></ul><ul><ul><li>Enzymes </li></ul></ul><ul><ul><li>Viruses </li></ul></ul><ul><ul><li>PDB codes…. </li></ul></ul>
  75. 76. Integrations Out to Other Sources
  76. 77. Integrations Out to Other Sources
  77. 78. Reactions
  78. 79. Manual Curation is Always Necessary
  79. 80. Text- Indexing and ChemSpider? <ul><li>ChemSpider text-indexes almost 500,000 Open Access and Free Access articles </li></ul><ul><li>Collection is growing and more publishers have already agreed. Including theses in the future. </li></ul>
  80. 81. Open Access Literature Search
  81. 82. Conclusions <ul><li>The quality of structure-based data online should always be questioned – that includes ChemSpider </li></ul><ul><li>Data on ChemSpider are being added and curated on a daily basis but we need more eyeballs helping always </li></ul><ul><li>ChemSpider has a large validated structure-name dictionary </li></ul><ul><li>Chemical name extraction and document markup is very enabling </li></ul>
  82. 83. Oops…

×