This document summarizes an analysis that links different large datasets to explore the impact of National Science Foundation (NSF) funding on research outputs and downstream commercial licensing. The datasets included NSF awards, US patents, scientific publications, and technology licensing agreements. Machine learning and text analytics were used to identify relationships between these datasets by linking acknowledgements, citations, and other text references. Key relationships identified included NSF awards to publications, patents, and eventual technology licenses, with time analyses of the lag periods between stages. Examples of full linkages traced from an NSF award through a publication to a patent and finally to a technology licensing agreement are provided.

1. Knowledge Flows: linking big data sets
Christina Freyman (christina.freyman@sri.com), John Byrnes, Tulay Muezzinoglu, SRI International
References and Notes
1. Huang, Z., Chen, H., Li, X., & Roco, M. C. (2006). Connecting NSF funding to patent innovation in nanotechnology (2001–
2004). JOURNAL OF NANOPARTICLE RESEARCH, 8(6), 859-879. doi:10.1007/s11051-006-9147-9
2. Personal communication with Kevin Boyak on May 23, 2016.
3. Cutler, R. S. (1987). Patents resulting from NSF's engineering program. World Patent Information, 9(1), 38-42.
doi:http://dx.doi.org/10.1016/0172-2190(87)90193-1
4. Sampat, B. N., & Pincus, H. A. (2015). Citations in Life Science Patents to Publicly Funded Research at Academic Medical
Centers. Clinical and Translational Science, 8(6), 759-763. doi:10.1111/cts.12361
5. For details of the algorithm, see: Collins, M. (2002). Discriminative training methods for hidden Markov models. In
Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP ’02 (Vol. 10, pp. 1–
8). Morristown, NJ, USA: Association for Computational Linguistics. http://doi.org/10.3115/1118693.1118694
Disclaimer: This work was supported by the National Science Foundation [contract number NSFDACS14T1252]. Any
opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the
views of the National Science Foundation or its staff.
Overview
A few studies have looked at measuring knowledge flows from the U.S. science agency awards; all have focused on hand-curated, in some cases small, data sets. Examples include:
• A hand-curated, NSF “nano” awards set manually linked to USPTO “nano” patents.1
• Kevin Boyak recently reported on a project where he linked NSF awards to publications from three program element codes. He manually obtained the publication references from the
NSF awards database website, and then linked those references to Scopus data using an ElasticSearch index that he created using Scopus metadata.2
• Patrick Thomas and Anthony Breitzman published in a final report for a DOE project that they linked patents to awards via non-patent literature citations and publication funding
acknowledgements. They were able to connect 14,840 patents to NSF awards.3
• B. Sampat and H. Pincus linked NIH grants to patents via patent citations. They then linked the patents to marketed drugs.4
Objectives
• Build a large dataset of knowledge flows in innovation using large databases and text analytics.
• Build a unified database from 1) a technology licensing data set provided by ktMINE, a commercial provider of IP licensing data; 2) United States Patents; 3) peer-reviewed scientific
articles from the Web of Science bibliographic database; and 4) NSF awards.
• Exploit a text analytics and machine learning approach to identify the patents and papers that are derived from NSF-funded research, the commercial licenses resulting from these
research outputs, and the lags associated with a) the time from funding to publication of research results, and b) the time between research codification and commercial application.
Results: Links between data sets
Data
NSF Awards: 526,002
Years: 1992–2013
USPTO records: 9,836,702
Years: 2001 to 2016
Licenses in pharma and
semiconductors: 3,387
Licenses with patent references: 391
Year: 2009–2014
NSF Funding
(NSF Awards
Database)
76,322 awards with WOS
acknowledgements
391 with patent references
3387 total
9,368 links (unique)
17,791 total links (licenses
resolved to USPTO)
5,236,545 links
(unique resolved to WoS)
19,897,730 unique citations
402,621 links (resolved
to awards DB)
456,858 NSF
acknowledgements with
grant number
71,148 links: licence to patent to publication
20 links: licence to patent to publication to award
Scientific
Publications
(Web of Science)
U.S. Patents
(Lexis Nexus Patent
Data)
Technology
Licensing
Agreements
(ktMINE)
Linked by Funding
Acknowledgement
Linked by NPL Citation Linked by ktMINE license to
patent data
2,610 links: patent to publication to award
Timeanalysis
7,558 links: patent to award
Web of Science records:
48,679,673
Years: 1900–2014
Examples of full links
Mean
(years)
Median
(years)
Number
of links
Award to
publication
3.0 2.0 3
Award to patent
(direct)
11.8 11.0 17
Award to patent
(linked by
publication)
8.3 7.0 3
Award to license 15.6 19.5 20
NSF award to license full links
Mean
(years)
Median
(years)
Number
of links
Award to
publication
3.6 3 2610
Award to patent
(linked by
publication)
8.1 7 2610
Award to patent
(direct)
8.3 8 7558
NSF award to patent links
NSF Award
8618780
Biopolymer
Sequence Analysis:
New Methods and
Instrumentation
Scientific
Publication:
Oligopeptide
Sequence Analysis
by Collision-activated
Dissociation of
Multiply Charged Ions
U.S. Patent:
Protein
sequencing by
mass
spectrometry
Technology Licensing
Agreement
From: MDS INC., Canada
To: DH Technologies
Development Pte Ltd.
NSF Award:
9415861
Studies on FokI (a
Type IIS)
Restriction
Enzyme
U.S. Patent:
Methods for inactivating
target DNA and for
detecting conformational
change in a nucleic acid
Technology Licensing
Agreement
From: Johns Hopkins University et al.
To: SIGMA-ALDRICH CO.
Patents to Publications
• Unstructured NPL citation strings are first extracted from the patent record and parsed
into structured citation records including article title, journal details, author names, and
publication date. We use overlapping token sequences to map the citation records into
a large set of structured publication records. We use this map to train a perceptron
model to parse the remaining records.5
• After the NPL citations are parsed, the database is queried for matching scientific
publication records.
Patents to NSF Awards
• We extracted the NSF award acknowledgement from the description of the USPTO
record as a function of patent.
Licenses to Patents
• Direct links from the ktMINE data.
This work explores the impact of NSF funding on research output and on downstream
commercial licensing by attempting to link NSF grants to specific research papers and
patents and patents to commercial licenses.
Publications to NSF Awards
• First we extract a list of all NSF award IDs from the NSF awards database.
• Next, we search for these strings in unstructured funding acknowledgement records
and then train a model based on adjacent term occurrences that will recognize the
contexts in which these strings occur. Applying this model will generalize to detect
award references we had missed initially and will help remove string matches which
were not award references.
• After the funding acknowledgements are parsed, the database is queried for matching
award records.
Methods
Direct link Chained link
Legend
Below are two of the links we found in this analysis. These links illustrate knowledge flows from NSF
funding to a licensing agreement.