Linking Database Submissions to Primary Citations with PubMed Central Heather Piwowar and Wendy Chapman Department of Biomedical Informatics University of Pittsburgh BioLINK 2008

These links are important for several reasons

These links are important for several reasons

Sometimes the links are easy to discover

Sometimes the links are easy to discover

But the meaning of hyperlinks is ambiguous:

And often no hyperlinks at all:

One way to identify links: NLP systems that identify statements of shared data from within full text.

One way to identify links:

NLP systems that identify statements of shared data from within full text.

BUT this requires developing and maintaining a full-text archive!

BUT this requires developing and maintaining a full-text archive!

What about using PubMed Central?

What about using PubMed Central?

Usage? scientists looking for datasets for reuse curators looking for primary citations researchers studying data sharing behaviour

Usage?

scientists looking for datasets for reuse

curators looking for primary citations

researchers studying data sharing behaviour

Goal: Use the simple, full-text query interface of PubMed Central to identify articles with shared datasets

Goal:

Use the simple, full-text query interface of PubMed Central to identify articles with shared datasets

Method: Gene expression microarray data GEO database

Method:

Gene expression microarray data

GEO database

Method: Open Access articles to train Non-Open access articles to test Gene-expression articles selected by MeSH term query

Method:

Open Access articles to train

Non-Open access articles to test

Gene-expression articles selected by MeSH term query

Gold Standard: True positives (N=550) Articles with primary citation links from GEO + screening of full-text True negatives (N=165) The rest

Gold Standard:

True positives (N=550) Articles with primary citation links from GEO + screening of full-text

True negatives (N=165) The rest

Building the query: Used full-text of open-access cohort Removed words <40 occurrences Unigram bag-of-words vectors Tree and Rule algorithms, a variety of parameters

Building the query:

Used full-text of open-access cohort

Removed words <40 occurrences

Unigram bag-of-words vectors

Tree and Rule algorithms, a variety of parameters

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

(geo OR omnibus) AND microarray AND &quot;gene expression&quot; AND accession NOT (databases OR user OR users OR (public AND accessed) OR (downloaded AND published))

Evaluation Results 40% recall 94% precision, 65% for those not yet linked worse than full-NLP results (~ 89%,83%) slightly better than trivial query (34%,90%)

Evaluation Results

40% recall

94% precision, 65% for those not yet linked

worse than full-NLP results (~ 89%,83%)

slightly better than trivial query (34%,90%)

Limitations only one datatype database-centric performance so far is rather mediocre…

Limitations

only one datatype

database-centric

performance so far is rather mediocre…

Impact? Today’s performance: would increase GEO links by 2.6% by 5.5% annually when all NIH in PMC Double the recall, to 80%: double the numbers above GEO curators added the 40 links identified by this study

Impact?

Today’s performance:

would increase GEO links by 2.6%

by 5.5% annually when all NIH in PMC

Double the recall, to 80%:

double the numbers above

GEO curators added the 40 links identified by this study

We hope this work inspires future enhancements, and highlights the opportunities for simple full-text queries in PubMed Central given the mandated influx of NIH-funded research reports.

We hope this work

inspires future enhancements, and

highlights the opportunities for simple full-text queries in PubMed Central given the mandated influx of NIH-funded research reports.

Thank you Advisor: Dr. Wendy Chapman Funders: NLM and Pitt DBMI Enablers: Everyone who deposits their publications in PubMed Central! My shared data: www.dbmi.pitt.edu/piwowar Share your research data too!

Advisor: Dr. Wendy Chapman

Funders: NLM and Pitt DBMI

Enablers: Everyone who deposits their publications in PubMed Central!

My shared data: www.dbmi.pitt.edu/piwowar

Share your research data too!