• Save
What do we know about the h index?
Upcoming SlideShare
Loading in...5

What do we know about the h index?



Journal Club presentation of:

Journal Club presentation of:

Bornmann L, Daniel HD. What do we know about the h index? J Am Soc Inf Sci Technol. 2007;58(9):1381-1385.



Total Views
Views on SlideShare
Embed Views



2 Embeds 2

http://health.medicbd.com 1
http://www.slashdocs.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • h presumably for “Hirsch” – physicist at UCSD who first proposed the index in 2005Paper has been cited 267 times (last time I checked)h = 0: May have published papers—they just haven’t been cited
  • Hirsch: an individual’s h should increase more or less directly with time.vanRaan (2004):“A ‘Sleeping Beauty in Science’ is a publication that goes unnoticed (‘sleeps’) for a long time and then, almost suddenly, attracts a lot of attention (‘is awakened by a prince’).”
  • Data needed for calculation available in:Web of ScienceScopusGoogle ScholarChemical AbstractsIn WoS, do with author search in straight Search (not Cited Ref Search)*CLICK* Or, use the Citation Report creator…
  • … and read h directly off the report.
  • In Scopus, Starzlh=87 (diff DB, diff results)
  • Countries: in biology and biochemistry for the period 1996–2006, the US, the UK, and Germany have h indices of 400, 219 and 206 respectively (Csajbók et al, 2007).Complement to journal impact factors (Braun et al., 2005)Caveat: don’t include review journals b/c upper limit of a journal’s h is # of papers published, and they don’t publish very many.Banks (2006): h - b index for interesting topics and compounds: found by entering a topic (search string, like superstring or teleportation ) or compound (name or chemical formula) into the Web of Science database and then ordering the results in terms of citations, by largest first.how much work has already been done on certain topics or compounds, what the hot topics (or older topics ) of interest are, or what topic or compound is mainstream research at the present time
  • Convergent ValidityThe ability of a measurement scale to correlate (or converge) with other measures of the same variable. How does h relate to other standards for evaluating research performance?Other bibliometric indicators Outcomes of peer review
  • Boehringer Ingelheim Fonds (BIF) = an international foundation for promotion of basic biomedical research
  • van Raan’s actual conclusion is more nuanced:“Results show that the h-index and our bibliometric ‘crown indicator’ [a composite measure] both relate in a quite comparable way with peer judgments. But for smaller groups in fields with ‘less heavy citation traffic’ the crown indicator appears to be a more appropriate measure of research performance.” [emphasis mine]
  • Database = SPIRES (Stanford Physics Information Retrieval System database)
  • As derived from Web of Science records (same holds true for any other resource whose records don’t include a definitive author ID—which covers all products)From WoS documentation:“The h-index factor is based on the depth of your Web of Science subscription and your selected timespan. Items that do not appear on the Results page will not be factored into the calculation. If your subscription depth is 10 years, then the h-index value is based on this depth even though a particular author may have published articles more than 10 years ago.
  • h as alternative to inappropriate use on journal impact factors to report personal achievement in tenure and promotion decisions.My Dec 2006 HSLS Update article on this problem:http://www.hsls.pitt.edu/about/news/hslsupdate/2006/december/personal_cit_counting
  • But for every advantage to h cited by Hirsch and colleagues who see the glass half full, someone (sometimes the same person) points out a problem.Bornmann & Daniel (2007) citing Glänzel (2006)Hirsch (2005)Bornmann & Daniel (2007) citing Sidiropoulos et al. (2006)Bornmann & Daniel (2007) citing Sidiropoulos, Katsaros, & Manolopoulos (2006)Bornmann & Daniel (2007) citing Sidiropoulos et al. (2006)Bornmann & Daniel (2007) citing Sidiropoulos et al. (2006)
  • Bornmann & Daniel (2007) citing Kelly & Jennions (2006); Sidiropoulos et al. (2006)Hirsch (2005)Bornmann & Daniel (2007) citing Lehmann et al. (2008)Hirsch (2005)
  • Bornmann & Daniel (2007) citing Joint Committee on Quantitative Assessment of Research (2008)Hirsch (2005)
  • N.B.: Need for several suggested by Hirsch himselfm quotient: Corrects for bias toward researchers with longer careers/more papershI: More co-authors, more articles, and potentially more citations (?)

What do we know about the h index? What do we know about the h index? Presentation Transcript

  • What do we know aboutthe h index?
    Patricia Weiss, MLIS  Health Sciences Library System
    University of Pittsburgh
    © 2009-2010 Patricia M. Weiss
  • Today’s article
    Bornmann L, Daniel HD. What do we know about the h index? J Am Soc Inf Sci Technol. 2007;58(9):1381-1385.
    Review of research inspired by:
    Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 2005;102(46):16569-16572.
    Also will allude to:
    Bornmann L, Daniel HD. The state of h index research. EMBO Rep 2009;10(1):2-6.
  • What is the h Index?
    Proposed by Hirsch (2005)
    Quantifies scientific output of a single researcher as a single number:
    An h index of 40 means that a scientist has published 40 papers that each had at least 40 citations.
    “… original and simple new measure incorporating both quantity and visibility of publications”
  • h index and the Scientific Life Cycle
    A scientist's h index never decreases
    It increases as…
    New high-impact papers are published
    Sleeping Beauties come to life (van Raan, 2004)
    A scientist's papers attract citations
  • Advantages of h
    Estimates broad impact of cumulative research contributions (Hirsch, 2005)
    Insensitive to citation extremes
    Infrequently cited or noncited papers
    “One-hit wonders”
    It “favors enduring performers that publish a continuous stream of papers with lasting and above-average impact.”
  • Article #71 in results list
    Cited by 72 articles
     h=71
  • Article #87 in results list
    Cited by 87 articles
     h=87
  • Cited by 111 articles
     h=111
    Article #111 in results list
  • Applications of the h Index
    Micro vs. meso level applications
    Micro: single researcher
    Meso (intermediate): group of researchers
    Scientific facilities
    Countries (Csajbók et al, 2007)
    h - b index based on topic or compound search in Web of Science (Banks, 2006)
    How much work has already been done on topic?
    What topic is hot or mainstream now?
  • Convergent Validity of the h Index
    Hirsch computed h indices for:
    Nobel Prize physicists in the last 20 years
    Physicists and astronomers elected to National Academy of Sciences in 2005
    Highly cited scientists in biological and biomedical sciences
    Hirsch’s threshold values
    Successful scientist: h=20 after 20 years
    Outstanding scientist: h=40 after 20 years
    Truly unique individual: h=60 after 20 years, or 90 after 30 years
  • Correlates that Establish Convergent Validity
    • Bornmann and Daniel (2006)
    • Peer review committee for Boehringer Ingelheim Fonds postdoctoral fellowships
    • Average h indices of approved applicants higher than those of rejected applicants
    • Cronin and Meho (2006)
    • Rankings of influential US information scientists
    • Strong positive correlation between h and raw citation counts
  • Convergent Validity Correlates (2)
    • van Raan (2006): 147 Dutch university chemistry research groups (700 researchers, 18K publications)
    • Looked at correlations between h and other bibliometric indicators
    • h and other bibliometrics: both relate comparably to peer judgments
    • Kelly and Jennions (2006)
    • Looked at publication output of 187 editorial board members (ecologists and evolutionary biologists) of 7 journals
    • Closely correlated with h
  • Convergent Validity Correlates (3)
    • Lehmann, Jackson, and Lautrup (2005)
    • Used papers from high-energy physics database
    • Presented method for quantifying reliability of one-dimensional measures based on citation and publication data
    • Mean, median, maximum citations were reliable measures of scientific performances
    • h was not (lacks accuracy and precision)
  • Disadvantages of WoS-Derived h
    Publication and citation may actually be lower if multiple authors of same name
    Or, may be higher (Roediger, 2006)
    Only journals selected by Thomson Scientific are indexed
    Unclear citations (e.g., “in press”)
    Incorrect citations (e.g., wrong first page number)
    Only journal articles are counted
    (Also, noted in WoS documentation: subscription depth as factor)
  • Journal Impact Factors in Tenure and Promotion
  • Potato, Potahto
    BUTmeasures broad impact of one's work
    Can’t differentiate between active and inactive scientists
    BUT never decreases
    Can’t differentiate between no longer significant works and those now shaping scientific thinking
    BUTalso can't spotlight works which are "trendy”--this is a bad thing?
  • Different disciplines, different citation patterns: should be used to compare only scientists of similar professional age working in similar disciplines
    BUTHirsch himself noted that h in biosciences higher than in physics
    Combining publication and citation frequencies into one value "posits an equality between two quantities with no evident logical connection"
    BUT preferable to other single-number criteria (total number of papers; total number of citations; citations per paper)
  • "Think of two scientists, each with 10 papers with 10 citations, but one with an additional 90 papers with 9 citations each; or suppose one has exactly 10 papers of 10 citations and the other exactly 10 papers of 100 each. Would anyone think them equivalent?”
    BUT2 individuals with similar hs are comparable in terms of their overall scientific impact, even if their total number of papers or their total number of citations is very different.
  • Indices that Correct or Complement h
    One that corrects for self-citations*
    m quotient: Corrects for bias toward researchers with longer careers/more papers*
    Ones that look at a definite time period, such as c (for most recent calendar year)
    hI: Normalizes for different numbers of co-authors in different fields*
    a and g indexes: More sensitive to highly cited papers—the scientist’s “top performers”
    * Proposed or developed by Hirsch himself
  • Conclusion
    No thorough validation of the h index yet in its various applications
    Would need to entail… 
    Cross-discipline validation
    Broad statistical data
    So, h index and its derivatives and alternatives “should not (yet) be used as a criterion to inform decision making in science…”
  • What do we know aboutthe h index?
    Pat Weiss  HSLS Journal Club  March 4, 2009