This document presents a new proposed indicator called "Impact Vitality" to measure individual excellence in scientific publication activity. Impact Vitality calculates a weighted sum of the number of publications citing a researcher's work over time, with lower weights given to older citations. It is designed to reflect characteristics like prominence in a field, continued top-level performance, and increasing impact over a career. Initial test results found Impact Vitality scores correlated well with peer review selections for research fellowship applications. Further validation studies are suggested to explore the robustness and ability of Impact Vitality to assess excellence.

1. 17/05/2006 1Herhaling titel van presentatie
Impact Vitality –
A Measure for
Excellent Scientists
Nadine Rons (1,*), Lucy Amez (1,2)
(1) Vrije Universiteit Brussel (VUB); (2) Policy Research Centre for R&D Indicators VUB (SOOI-VUB)
(*) Corresponding Author
STI 10, Vienna, Austria, 17-20 September 2008
Version 12.09.2008

2. September 2008 - Slide2
STI 10
A MEASURE FOR
EXCELLENT SCIENTISTS
o Context & Research Question
o Approach
o Proposed Indicator
o Evaluation
o Conclusion

3. September 2008 - Slide3
STI 10
CONTEXT & RESEARCH QUESTION
o Context: Funding programmes aimed at researchers
performing at excellence level
o Need: Measures in support of peer review
o Focus: Individual excellence in terms of publication
activity
o Approach: Construction of an indicator reflecting a
description of excellence

4. September 2008 - Slide4
STI 10
Approach
=>
Test Case
Test Results
Proposed Indicator
=>
A definition of excellence:
Funding Programmes
=>Excellence in terms of publications
Indicator requirements:
Discussions on Bibliometric Indicators
=>Principles & Requirements verification of
requirements

5. September 2008 - Slide5
STI 10
Towards a DEFINITION of EXCELLENCE
Descriptions of excellence from programme outlines that can be
related to publications
o "at the forefront of the field",
o "prominence in the field",
o "substantial contribution to the development of the field",
o "continued performance at top level",
o "increasing productivity",
o "lead author of papers with significant impact in the field".
e.g. Spinoza prize (NL), Leibniz prize (DE), Research professor recruitment award (IR),
Methusalem programme (BE), Odysseus programme (BE)

6. September 2008 - Slide6
STI 10
EXCELLENCE
Individual performance at excellence level, as manifested in publication activity.
An excellent researcher is prominently present in the field,
continuously publishing new knowledge and ideas over a longer
period of time.
As an established reference in the field, his/her contributions are
eagerly followed by colleagues and his/her ideas are picked up
fast in their further research.
As such, he or she is a central figure in a strong research
dynamic, at the level of the researcher's own research team as
well as for the research area as a whole, increasing both volume
and impact of research in the field.

7. September 2008 - Slide7
STI 10
INDICATOR REQUIREMENTS /
CHALLENGES
principles
a reflection of relevant capacities
sufficiently well correlated with peer review
acknowledging characteristics of the individual career
independent of career length
a balanced appreciation of collaborative output
fit for common use
up to date
easily calculated
acknowledging the nature of science / the discipline
outlyer proof
avoiding bias
resisting human error / interventions
error proof
manipulation proof

8. September 2008 - Slide8
STI 10
PROPOSED INDICATOR: Impact Vitality
Inspiration: Klavans & Boyack, STI9, concerning a vitality measure for national & institutional comparison
V = (∑j=1→n 1 / ( age(j) + 1 )) / n, where n is the number of references j in a set of papers
Impact Vitality
Weighted sum of the numbers of citing publications P(y) over the years y,
Weights: Lower weight for higher age i in years,
Time window: n years,
Normalization: 1 = P(y) constant over time; 0 = no longer cited.
IV(y1, n) = [n (∑i=1→n P(yi) / i) / ∑i=1→n P(yi) - 1] / [∑i=1→n 1/i - 1]
with n > 1, yi+1 = yi - 1 and ∑i=1→n P(yi) > 0
or, as a sum over all m citing publications j in the time window:
IV = [ (∑j=1→m 1 / age(j)) / (m/n) - 1] / [∑i=2→n 1/i]
Profile over time
Options: moving with fixed length; growing with career from fixed start

9. September 2008 - Slide9
STI 10
WHAT IT REPRESENTS
A combination of publication and citation data:
counting publications that cite a body of work.
A value > 1 :
the number of publications citing the work increases over time,
increasing "impact area" of research (impact vitality).
An Impact Vitality profile over time above 1 :
a continuously growing uptake of the work in recent developments in
the field.

10. September 2008 - Slide10
STI 10
CHARACTERISTICS
Five fictive examples of numbers of citing publications per year
Year A B C D E F
2006 5 5 1 10 1 3
2005 5 4 2 8 2 2
2004 5 3 3 6 3 1
2003 5 2 4 4 2 2
2002 5 1 5 2 1 3
IV(2006, 5) 1,0 1,5 0,5 1,5 0,8 1,1
Constant amount of citing documents over time (case A): value = 1
Increasing amount of citing documents over time (case B): value > 1
Decreasing amount of citing documents over time (case C): value < 1
Same proportional increase of citing documents over time (cases B and D): same value
Fluctuating amount of citing documents over time: value according to the final trend,
< 1 for a final negative trend (case E) and > 1 for a final positive trend (case F).

11. September 2008 - Slide11
STI 10
SIMILAR STRUCTURE,
DIFFERENT CONTENT
AR-index
AR = (∑j=1→h cit (j) / age(j))^(1/2)
where age(j) is the age of article j,
cit(j) is the number of citations to it
and h is the number of elements in
the h-core (Jin et al., 2007).
Impact Vitality
IV = [(∑j=1→m 1 / age(j))/(m/n) - 1] / [∑i=2→n 1/i]
where age(j) is the age of citing documents j and
m is the number of citing documents in the time
window of n years.
BASIS numbers of citations numbers of citing publications
WEIGHTS lower weight for older cited
publications
lower weight for older citing publications
LIMITS citations to the h-core citing publications in a chosen time frame
UNLIMITED age of the citations age of the publications cited
CONTENT accumulated impact of the most
highly cited publications, with an
emphasis on the impact of the
most recent highly cited
publications
impact of the work in a specified period, with
an emphasis on the impact in the most
recent years

12. September 2008 - Slide12
STI 10
INDICATOR EVALUATION
principles
a reflection of relevant capacities: cfr. definition of excellence
sufficiently well correlated with peer review
acknowledging characteristics of the individual career
independent of career length: emphasis on recent citing publications
a balanced appreciation of collaborative output: trend calculation (not absolute value)
fit for common use
up to date: emphasis on recent citing publications
easily calculated: select set of documents citing the right author
acknowledging the nature of science / the discipline
outlyer proof: modulation of the impact vitality profile
avoiding bias: trend calculation (not absolute value); target expanded analysis; continued
impact of older work while impact in a new topic is built up
resisting human error / interventions
error proof: count of citing documents with at least one correctly cited name
manipulation proof: artificially raising publications, self citations, impact factor does not work
GENERAL ROBUSTNESS: errors, construction, data source

13. September 2008 - Slide13
STI 10
OUTCOMES: FIRST TEST RESULTS
Peer review based selection vs. Impact Vitality: first test results
IVPhD ≥ 1 for all years IVPhD < 1 for one or more years
Selected 5 —
Not selected 4 4
Source: VUB Research Fellowship calls 2000-2006, applications in predefined research themes, excluding themes in Social
Sciences and Humanities. Impact Vitality calculated using the Web of Science.
Senior Career Impact Vitality:
IVPhD(y1) = IV(y1, nPhD), where nPhD = y1 - yPhD + 1, the age in years of the PhD
Profile: growing window starting at PhD-year
Applicants selected by peers can be characterized by:
Senior Career Impact Vitality continuously above 1: prominence of recent impact during the whole career
Applicants not selected (for quality of budgetary reasons) also correspond to:
Start of career or Senior Career Impact Vitality with periods below 1

14. September 2008 - Slide14
STI 10
PROFILE OF SELECTED APPLICANTS

15. September 2008 - Slide15
STI 10
PROFILES OF NOT-SELECTED
APPLICANTS

16. September 2008 - Slide16
STI 10
CONCLUSIONS
A novel indicator is proposed that reflects what is expected from excellent
scientists.
Advantages of the indicator:
o relatively easy to calculate,
o hard to manipulate,
o limited sensitivity to outlyers in citation counts and to errors in references,
o a scope broader than the scientist's indexed publications,
o independence regarding size and citation culture of the research community.
First test results for a limited sample look promising.

17. September 2008 - Slide17
STI 10
SOME FUTURE WORK
Studies on robustness and validation.
Studies on larger samples of applicants and the selective power of different
indicator variants
-> recommendation of a particular variant for the assessment of individual
scientists? (within a set of indicators)
Further exploration of application possibilities.

18. September 2008 - Slide18
STI 10
THANK YOU!
Nadine Rons
Vrije Universiteit Brussel (VUB)
VUB R&D dept., Research Coordination Unit
Pleinlaan 2, B-1050 Brussels, Belgium
E-mail: nrons@vub.ac.be
Lucy Amez
Vrije Universiteit Brussel (VUB) and Policy Research Centre for R&D Indicators (SOOI)
VUB R&D dept., Research Coordination Unit
Pleinlaan 2, B-1050 Brussels, Belgium
E-mail: Lucy.Amez@vub.ac.be