Innovation means many different things to many people. Ask five people and you will likely get ten answers. But all agree that it is a key driver behind the success of organizations, the growth of economies and provides major contributions in addressing global problems. This presentation will examine various analytical methods and possible metrics for measuring innovation and determining relative performance of organizations. The challenges involved in assessing innovation and how these can be addressed will be explored. The pros and cons associated with the metrics identified will also be discussed with a view to identifying a practical method for assessing innovation.

1. What is Innovation?
How can we measure it?
April 2017
Bob Stembridge
Senior IP Analyst
Clarivate Analytics

2. 2
Agenda
• What is Innovation?
• Some potential metrics
• Measuring Innovation
• Nations
• Government and academia
• Corporations
• Towards a practical benchmarking approach

3. 3
For Clarivate Analytics, Innovation is a Dominant Theme
Common questions we are asked:
• How does our research
output compare to other
groups and governments?
• How do we ensure we are
funding the best people,
institutions and projects?
• Where do we invest and
focus resources?
• How do we improve research
decision making?
• How can we secure more research funding for our institution?
• How do we drive the most value from our IP assets and deals?
• What should our licensing strategy be?
• With whom should we partner?
• How do we protect and
improve our innovation
rankings amongst global peer
groups?
• How do we brand our new
invention and ensure we are
free to use the brand
globally?
• How do we better align our
approach to IP protection with
our business strategy?

4. 4
We know it when we see it….but what is it?
Innovation:
“An innovation is the implementation of a new or
significantly improved product (good or service), or
process, a new marketing method, or a new
organizational method in business practices,
workplace organization or external relations.”
SOURCE: Oslo Manual: Guidelines for Collecting and Interpreting
Innovation Data, 3rd edition, OECD/Eurostat, 2005

5. 5
Innovation is more than discovery…
• Innovation is often based on discoveries, inventions and translation
• Academic research and education may support innovation but are insufficient by
themselves to drive the process
• The process of converting new ideas into commercially successful novel services
and products is driven by entrepreneurship
“The entrepreneur is driving innovation by combining assets, including
technologies, in new ways, creating new opportunities, new markets, new
economic values and […] the eagerness of millions of people as customers
seeking to improve their lives….”
Joseph Schumpeter

6. 6
The Lifecycle of Innovation
The Lifecycle of Innovation
comprises three elements:
• Discover
• Protect
• Commercialise
Each of these are valid areas to
examine in seeking to understand
the nature of innovation and may
provide elements that can be
measured to evaluate and
benchmark innovation
performance

7. Potential metrics

8. 8
Discover
• R&D expenditure
• Research articles published
• Scientific publications
• Conference proceedings
• Number of ideas generated
(invention disclosures)

9. 9
Protect
• Patents filed
• Patents published
• Patents granted
• Copyright
• Trademarks
• Trade secrets

10. 10
Commercialize
• Patents granted
• Trademarks
• Brand
• Royalty and fee trading
• Venture capital financing

11. 11
Measuring nations

12. 12
European Innovation Scoreboards (2001 – 2009)
Innovation Union Scoreboards (2010 – present)
• Enablers: the basic building
blocks which allow innovation
to take place.
• Firm Activities: which show
how innovative Europe’s firms
are.
• Outputs: which show the
benefits for the economy at
• large.

13. 13
An Array of Indicators
SOURCE: Dominique Foray and Hugo Hollanders, “An assessment of the Innovation Union Scoreboard as a tool to analyze national innovation
capacities: The case of Switzerland,” Research Evaluation, 24 (2): 213-228, April 2015.

14. 14
Measuring Academic &
Government Institutes

15. 15
Clarivate Analytics Indicators for World’s Most Innovative Universities
Indicator Description
Patent Volume The total number of patents by the organization
Patent Success The percentage of the patent portfolio that has been granted
Global Patents The percentage of the patents that have been submitted to global
patent authorities (US, Europe, Japan)
Patent to patent Citations The total number of times the patent portfolio was cited by other
patents
Patent to patent Citation Impact The average number of times a patent has been cited
(half weighting)
Percent Patents Cited The percentage of the patent portfolio that has been cited one or
more times (half weighting)
Patent to Article Citation Impact The average number of times an article has been cited by patents
Industry Article to Article Citation
Impact
The average number of times an article has been cited by articles
with an industry author affiliation
Percent Industry Collaborative Articles The percentage of articles that contain a co-author from industry
Web of Science Documents Journal articles indexed in Web of Science (articles and reviews)

16. 16
Citation Analysis to Identify Research Fronts…
Co-citation analysis of the specialty structure of contemporary research
Biological Sciences (March 2016)
Rank Research Fronts Core
Papers
Citations Ave
Year
Hot
Papers
in Core
1 Isolation and characterization of
Middle East Respiratory Syndrome
coronavirus
47 3556 2013.6 1
2 Physiology functions of melatonin in
humans and plants
37 2417 2013.5 2
3 Femtosecond x-ray
nanocrystallography of biological
macromolecules
23 2129 2013.5 1
4 Context-specific identities and
functions of macrophages
21 3641 2012.9 0
5 Genome-wide analysis of genetic loci
associated with Alzheimer disease
21 4815 2012.8 0
6 RNA secondary structure modeling 33 3683 2012.6 0
7 HIV vaccine design targeting trimeric
envelope (Env) spike
32 6031 2012.5 2
8 PINK1-generated phospho-ubiquitin
and Parkin induction of mitophagy
25 5749 2012.5 1
9 Metabolic regulation of T cells 39 5391 2012.5 0
10 C9orf72 hexanucleotide repeat
expansion and frontotemporal
dementia and amyotrophic lateral
sclerosis
34 5354 2012.5 1
SOURCE: Essential Science Indicators

17. 17
….And to Detect Emerging Research Fronts
Emerging Research Fronts (March
2016)
Rank Research Fronts Core
Papers
Citations Ave
Year
Hot
Papers
in Core
1 Long carrier diffusion lengths in
organolead trihalide perovskite single
crystals for solar cells
2 225 2015 2
2 Degradation dynamics of perovskite
solar cells
7 160 2015 2
3 Genome-wide identification of CRISPR
RNA-guided nucleases (RGNs) off-
target effects
4 152 2015 2
4 Programmed cell death 1 (PD-1)
inhibition for advanced non-small-cell
lung cancer
4 146 2015 4
5 Botanical extracts and nanoparticles
for mosquito control
12 132 2015 8
6 Gravitational wave detectors and
simulations of observations
4 129 2015 2
7 Multicolor light-emitting diodes based
on nanocrystalline perovskite
7 121 2015 5
8 Genome-wide association studies of
obesity
3 119 2015 1
9 Environmental contamination and
exposure of non-target organisms by
neonicotinoids and fipronil
7 108 2015 2
10 Activation of mechanistic target of
rapamycin complex 1 (mTORC1)
3 106 2015 0
SOURCE: Essential Science Indicators

18. 18
Measuring
Corporations

19. 19
• Studies show that the majority of technological improvements that companies believe to
be patentable and important are documented in patent applications
• Therefore patents are a useful source of innovation indicators for Corporations
• Various aspects can be measured
 Volume (Portfolio size)
• of applications
• of grants
 Ratio of grants to applications (Success)
 Breadth of filing (Globalization or Market Coverage)
 Citation impact (Influence or Technology Coverage)
 Legal status
 Remaining life
• Innovation indicators differ by industry:
Patents as Innovation indicators
Company Industry Invention volume
2011-2015
Company A Telecommunications 25617
Company B Chemicals 6013
Company C Pharmaceuticals 2418

20. 20
• Inventions vs patent document count
Volume
Company Volume of inventions
2012-2016
Volume of patents
2012-2016
Ratio
Company A 3665 36389 9.93
Company B 5169 33346 6.45
Company C 5679 56797 10.00
SOURCE: Inventions – Derwent World Patents Index (DWPI);
Patents – Thomson Innovation Full Text collections
Counting patents leads to variable degrees of double counting.
It is better to count inventions.

21. 21
• Ratio of inventions with a granted patent divided by total number of inventions
Success
Company Volume of inventions
2012-2016
Volume of grants
2012-2016
Ratio
Company A 3665 1605 0.44
Company B 5169 2923 0.57
Company C 5679 1938 0.34
SOURCE: Derwent World Patents Index (DWPI)

22. 22
• Ratio of inventions with patents published in multiple countries* divided by
total number of inventions
Globalization
Company Volume of inventions
2012-2016
Quads 2012-2016 Ratio
Company A 3665 1406 0.38
Company B 5169 605 0.12
Company C 5679 1283 0.23
* CN, EP, JP and US = Quadrilateral patents
SOURCE: Derwent World Patents Index (DWPI)

23. 23
• Ratio of the number of citations (excluding self-citations) divided by the total
number of citable inventions
Influence
Company Citable inventions
2012-2016
Citations (excl. self-cites)
2012-2016
Ratio
Company A 3051 1759 0.58
Company B 4001 2332 0.58
Company C 4937 3423 0.69
SOURCE: Derwent Patents Citation Index (DPCI)

24. 24
Towards a practical
benchmarking approach

25. 25
Putting it together
Company Inventions
2012-2016
A
Grants 2012-2016
B
Quads
2012-2016
C
Citable inventions
2012-2016
D
Citations (excl. self-
cites) 2012-2016
E
Company A 3665 1605 1406 3051 1759
Company B 5169 2923 605 4001 2332
Company C 5679 1938 1283 4937 3423
Company Volume score
VS= VM/Average VM
Success score
SS=SM/Average SM
Globalization score
GS=GM/Average GM
Influence score
IS=IM/Average IM
Overall score
OS=VS+SS+GS+IS
Company A 0.76 0.98 1.58 0.93 4.25
Company B 1.07 1.26 0.48 0.94 3.76
Company C 1.17 0.76 0.93 1.12 3.99
Company Volume metric
VM=A
Success metric
SM=B/A
Globalization metric
GM=C/A
Influence metric
IM=E/D
Company A 3665 0.44 0.38 0.88
Company B 5169 0.57 0.12 0.21
Company C 5679 0.34 0.23 0.66
• Numbers
• Ratios
• Scores

26. 26
Relative performance in each metric
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
4,00
4,50
Volume score Success score Globalization
score
Influence score Overall score
Company A
Company B
Company C

27. Final Remarks

28. 28
• The Kelvin Dictum: “If you can measure that of which you
speak, and can express it by a number, you know something
of your subject; but if you cannot measure it, your
knowledge is meager and unsatisfactory.” — William
Thomson (Lord Kelvin)
• The Viner Dictum: “When you can measure it, when you can
express it in numbers, your knowledge is still of a meager
and unsatisfactory kind.” — Jacob Viner
• The Knight Dictum: “If you cannot measure a thing, go
ahead and measure it anyway.” — Frank Knight
Is it Possible to Measure Innovative Capacity and Achievement?
SOURCE: Robert K. Merton, David L. Sills, and Stephen M. Stigler, “The Kelvin Dictum and Social Science: An
Excursion into the History of an Idea,” Journal of the History of the Behavioral Sciences, 20 (4): 319-331, 1984

29. Thank you

30. Bob Stembridge, Senior IP Analyst| +44-207-433-4690| bob.stembridge@clarivate.com | clarivate.com