Metrics for Measuring Scientific Performance

1. Metrics for Measuring Scientific Performance
Prepared by
Ahmed Ramadan Farag Ahmed
Supervisor
Prof. Dr. Nawal Ahmed El-Fishawy
Technical Writing

2. Outline :
 Impact Factor
 H-Index
 Quality Factor

3. Impact Factor

4. History
The creators of the factor, Eugene Garfield and Irving Sher, aimed to
invent a metric that will serve to evaluate the journals. Following the
Genetics Citation Index's first experimental publication in 1955,
Science Citation Index (SCI) was published in 1961. Garfield and Sher
introduced the metric for selecting the new journals to include in the
SCI. Since then, the assumption is there is a direct dependence
between the journal's impact factor and quality.

5. What is an Impact Factor?
Journal Impact Factor (JIF) is a metric of authority and importance for
academic journals, calculated every year. Since its establishment in
1975, the JIF has been calculated by different organizations.
Currently, Thomson Scientific ( Scopus) is calculating and publishing
Journal Citation Reports (JCR) annually. You may access the report at
the Scopus Preview website.

6. Impact Factor Calculation:
Impact Factor is calculated by dividing the number of citations of the
journal by the sum of its citable publications during the previous two
consecutive years. Consequently, you should understand the metric
as an indicator of how often a certain journal is cited in different
publications. Logically, the more journal is cited, the more people
trust and rely on its information validity. So, you can consider high-
impact journals as a good source to conduct your research. On the
other side, the Impact Factor will help choose where to publish your
Ph.D. or other research works.

7. The Impact Factor Criticisms :
However, there are critics of the metric, pointing to the
disadvantages or flaws of its calculation. There is a suggestion that it
would be more appropriate to present the median rather than the
mean of the data Impact Factor uses for calculations. Also, critics say
JIF can't be an accurate predictive measure of where to publish
works. A certain article can achieve few citations even if published in
a frequently cited journal. Addressing the wave of skepticism toward
the metric, the European Association of Science Editors restated its
purpose in 2017. JIF should be an indicator to assess the entire
journal's influence, not single publications or researchers separately.

8. H-Index

9. The H-Index
In 2005, Jorge E. Hirsch of UCSD
published a paper in PNAS in which he
put forward the h-index as a metric
for measuring and comparing the
overall scientific productivity of
individual scientists.
The h-index has been quickly adopted
as the metric of choice for many
committees and bodies.

10. The H-Index
Conceptually, the h-index is pretty
simple. we just plot the number of
papers versus the number of citations
you (or someone else) have received,
and the h-index is the number of papers
at which the 45-degree line
(citations=papers) intercepts the curve,
as shown in the diagram. That is, h
equals the number of papers that have
received at least h citations. For
example, do we have one publication
that has been cited at least once? If the
answer is yes, then we can go on to our
next publication. Have our two
publications each been cited twice? If
yes, then our h-index is at least 2. We
can keep going until we get to a “no.”

11. The H-Index
So, if we have an h-index of 20, then that means we have 20 papers
with at least 20 citations. It also means that we are doing pretty well
with our science!
The advantage of the h-index is that it combines productivity (i.e.
number of papers produced) and impact (number of citations) in a
single number. So, both productivity and impact are required for a
high h-index; neither a few highly cited papers nor a long list of
papers with only a handful of (or no!) citations will yield a high h-
index.
What is a Good h-Index?
Hirsch reckons that after 20 years of research, an h-index of 20 is
good, 40 is outstanding, and 60 is truly exceptional.

12. Limitations of the H-Index
Although having a single number that measures scientific performance
is attractive, the h-index is only a rough indicator of scientific
performance and should only be considered as such.
Limitations of the h-index include the following:
• It does not take into account the number of authors on a paper. A
scientist who is the sole author of a paper with 100 citations should
be given more credit than one who is on a similarly cited paper with
10 co-authors.
• It penalizes early-career scientists. Outstanding scientists with only
a small number of publications cannot have a high h-index, even if
all of those publications are ground-breaking and highly cited. For
example, if “Albert Einstein died in early 1906, then his h-index
would be stuck at 4 or 5, despite his being widely acknowledged as
one of the most important physicists, even considering only his
publications to that date.”

13. Limitations of the H-Index
• Review articles have a greater impact on the h-index than original
papers since they are generally cited more often.
• Use of the h-index has now broadened beyond science. However,
it’s difficult to compare fields and disciplines directly, so, really, a
‘good’ h-index is impossible to define.

14. Calculating the H-Index
There are several online resources and h-index calculators for
obtaining a scientist’s h-index. The most established are ISI Web of
Knowledge, and Scopus, both of which require a subscription
(probably via your institution), but there are free options too, one of
which is Publish or Perish.
If you check your own (or someone else’s) h-index with each of these
databases, you might get a different value. This is because each uses a
different database to count the total publications and citations. ISI and
Scopus use their own databases, and Publish or Perish uses Google
Scholar. Each database has different coverage, so will come up with
different h-index values. For
example, ISI has good coverage of journal publications, but poor
coverage of conferences, while Scopus covers conferences better, but
has poor journal coverage pre 1992.
h-index = the number of publications with a citation number greater
than or equal to h.

15. The H-Index Summed Up
The h-index provides a useful metric for scientific performance, but
only when viewed in the context of other factors. So, when making
decisions that are important to you (funding, job, finding a PI) be sure
to read through publication lists, talk to other scientists (and students)
and peers, and take account of career stage. So, keep in mind that an
h-index is only one consideration among many—and we should
definitely know our h-index—but it doesn’t define you (or anyone
else) as a scientist.

16. Quality Factor

17. Quality Factor Approach
Quality Factor is a system that seeks to improve the publication
services with an emphasis on future results. It involves based on the
Deming’s Method of PDCA (Plan-Do-Check-Act) Cycle to measure the
standard of the journals/Publications. If necessary, the plan may be
revised on the basis of the results, so that the improvement is
ongoing.

18. Quality Factor Process Map

19. Quality Factor Calculation
Journal Selection Criteria: Academic journal necessity to have ISSN
number is the eligible criteria to get Quality Factor Measurement.
Quality Factor is calculated using the below formula.
Quality Factor =
Q1+Q2+Q3+Q4+Q5+Q6+Q7+Q8+Q9+Q10+Q11+Q12+Q13+Q14+Q15
The Audit parameters and grid for Quality Factor calculation purpose
are listed .
Based on the criteria all journals
will be measured yearly and
share the Quality Factor ranking
to improve the journal standards.
The Journal Quality factor
Reports (JQR) will be published
and also includes a previous year
quality factor. The JQR also shows
rankings of journals by Quality
Factor.

20. Quality Factor Criticisms
Frequent criticisms have been made of the use of a quality factor. For
one thing, the quality factor might not be consistently reproduced in an
independent audit bases on the journal data we evaluate the Quality
Factor. There is a more universalargument on the rationality of the
quality factor as a measure of journal standards and the effect of
strategies that editors/publishers may adopt to improvement their
Quality factor. Other criticism emphases on the consequence of the
quality factor on performance of scholars, editors and other patrons.
Another reason that can undermine this system is that there is a
general inclination on the fragment of a mentioning individual to be
prejudiced by the already specified QF. Of any kind criticisms
about Quality Factor may not be true, public have to understand
benefit of the Quality Factor.

21. Categories Of Quality Factor Status
There are three classes of Quality factor Status defined by NOT-
2016/21, General, Special & Roster. These classes were the equivalent
of Category I, Category II & Roster status that were defined in NOT-
2016/21. Below are the current definitions.
General Status: A system with high quality factor (QF: 5 & above) is said
to be General Status Journal.
Special Status: A system with Intermediate quality factor (QF: 3.5 to 4.9)
is said to be Special Status Journal.
Roster Status: A system with Low quality factor (QF: Below 3.4) is said
to be Roster Status Journal.

22. Thank You !