• Save
The 'Perspective Shift' in bibliometrics and its consequences
Upcoming SlideShare
Loading in...5
×
 

The 'Perspective Shift' in bibliometrics and its consequences

on

  • 3,310 views

Wolfang Glanzel's presentation at InSciT2006.

Wolfang Glanzel's presentation at InSciT2006.

Statistics

Views

Total Views
3,310
Views on SlideShare
3,308
Embed Views
2

Actions

Likes
4
Downloads
0
Comments
0

1 Embed 2

http://www.slideshare.net 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The 'Perspective Shift' in bibliometrics and its consequences The 'Perspective Shift' in bibliometrics and its consequences Presentation Transcript

    • wolfgang glänzel
      • the 'perspective shift' in bibliometrics and its consequences
    k.u.leuven, steunpunt o&o statistieken, leuven (belgium) hungarian academy of sciences, ispr, budapest (hungary)
  • BACKGROUND In their discussion paper entitled “Little scientometrics, big scientometrics … and beyond? ” Glänzel & Schopeflin (1994) reported on symptoms of a crisis ( “teething troubles or agony?”). Among others, they mentioned lacking communication, drifting apart of sub-disciplines are and increasing commercialisation of the field. Although the field of bibliometrics/scientometrics became established, a certain ‘perspective shift’ has taken place. Pointing to some of its consequences will be the objective of this talk. glänzel: the ‘perspective shift’
    • Changes in methodology and objectives of the field
      • Worst Case Scenario: bibliometrics might cease to exist as a reaserch field (cf. Glänzel & Schoepflin , 1994)
    • Distorted behaviour of scientists based on policy use and misuse of bibliometric data
    glänzel: the ‘perspective shift’ Possible consequences of the perspective shift in terms of: View slide
  • glänzel: the ‘perspective shift’
    • ON BIBLIOMETRICS AND SCIENTOMETRICS
    • The two terms were introduced in 1969, and are nowadays used almost synonymously.
    • Pritchard
      • “ bibliometrics is the application of mathematical and statistical methods to books and other media of communication ”.
    • Nalimov & Mulchenko
      • Scientometrics is “the application of those quantitative methods which are dealing with the analysis of science viewed as an information process”.
    View slide
    • Bibliometrics for bibliometricians
      • The domain of bibliometric “basic research”.
    • Bibliometrics for scientific disciplines
      • A large but also the most diverse interest-group. Due to the scientists’ primary scientific orientation, their interests are strongly related to their speciality.
    • Bibliometrics for science policy and research management
      • The most important field of application. Here the assessment and comparative analysis of research performance are in the foreground.
    glänzel: the ‘perspective shift’ Three “components” of bibliometrics according to its main target-groups
  • glänzel: the ‘perspective shift’ Webometrics Informetrics Mathematics/Physics Library and Information Science Sociology of science History of science Economics Science information Services for Research in Librarianship Science policy Research management Life sciences LINKS WITH SCIENCE FIELDS AND APPLICATION SERVICES Computer science Scientometrics applied basic
  • glänzel: the ‘perspective shift’ THE “PERSPECTIVE SHIFT” IN USE OF BIBLIOMETRICS
  • glänzel: the ‘perspective shift’ Bibliometrics represented a statistical approach to master the flood of scientific information and to analyse and to understand the cognitive characteristics of “big science” by measuring quantitative aspects of communication in science and by providing the results to scientists and users outside the scientific community ( Price , 1963) . Monitoring, description and modelling of the production, dissemination and use of knowledge was originally in the foreground.
    • The growth of knowledge and scientific literature, international collaboration and competition, increasing specialisation as well as growing importance of cross- and interdisciplinarity in scientific research has reached a stage where funding systems based on personal knowledge and evaluations by peers became more and more difficult.
    • The ‘science indicators’ movement in the US in the 1970s resulted in a discussion about the possible use of bibliometrics for science policy around 1980.
    • In the following two decades, bibliometrics was characterised by a perceptible shift towards science-policy and research-management application.
    glänzel: the ‘perspective shift’
  • glänzel: the ‘perspective shift’
    • The need for supplementing the evaluation of research with quantitative methods and of linking funding to performance indicators was one consequence of this process.
    • Recently, Bibliometrics evolved from a discipline analysing all quantitative aspects and creating models of scientific communication to a tool for the benchmarking of knowledge (Glänzel & Schoepflin , 1994, van Raan , 1997, Weingart , 2003, Ohly, 2003).
    glänzel: the ‘perspective shift’
  • glänzel: the ‘perspective shift’
    • Sociology of science has laid the theoretical groundwork for this paradigmatic perspective shift.
    • In particular, sociologists recognised that communication in science is also characterised by the position scientists have reached and hold in the science system.
      • This is reflected by their publication behaviour, their status as co-authors (e.g. Kretschmer , 1985), their published results and the reception by their colleagues.
      • The Mertonian notion describes citation as a reward system, actually as the currency of science.
  • glänzel: the ‘perspective shift’ 1. Changes in methodology and objectives of the field In the paper “Through citations to marks and money” ( Frick , 2004) science policy and research management appears as the only acceptable application of bibliometrics. In the process of evaluation, benchmarking and funding, the role of bibliometrics is reduced to being an auxiliary tool. Narrowing down the discipline to one predominant service task and lacking methodological research might uncouple results from the original context and thus support uninformed use.
  • Research Services What happens if services dominate research? glänzel: the ‘perspective shift’
  • 2. Distorted behaviour based on policy use and misuse of bibliometric data The roots of changing behaviour of scientists can be detected in a time when bibliometrics not yet existed, but bibliometrics might act as catalysator. In particular, an additional issue concerns the changes in the publication, citation and collaboration behaviour of scientists (both positive and negative) that the consistent policy use of bibliometric indicators might potentially induce. glänzel: the ‘perspective shift’
  • Studies on the problem choice behaviour of academic scientists have revealed that both cognitive and social influences determine the manner in which scientists go about choosing the problems they work on ( Debackere & Rappa, 1994). Hence the issue should be raised to what extent the policy use of bibliometrics might or could affect this behaviour. Repercussions might be observed when bibliometric tools are used in decision-making in science policy and research management and the scientific community recognises the feedback in terms of their funding . glänzel: the ‘perspective shift’
  • glänzel: the ‘perspective shift’ Schematic visualisation of the feedback of policy use of bibliometrics on the scientific community Weingart, 2005
    • Publication activity:
      • Butler (2004) has shown on the example of Australia what might happen when funding is linked to publication counts. She found that the publications component of the Composite Index has stimulated an increased publication activity in the lower impact journals .
    glänzel: the ‘perspective shift’ EXAMPLES
    • Scientific collaboration:
      • According to deSolla Price (1963), massive funding is one of the characteristics of ‘big science’; team work is anther one.
      • A large share of co-authorship, above all of international co-authorship is almost considered a guarantee for success and impact.
      • The other side of the coin is a trend towards exaggerate co-authorship, even hyper-authorship ( Cronin , 2001) with inflationary features ( Persson et al, 2004). Collaboration is also used as means of compensation in the framework of national publication strategies ( Braun & Glänzel , 1996).
    glänzel: the ‘perspective shift’ EXAMPLES
    • Citations :
    • Some important notions of citation
      • Citations give a formalised account of the information use (Smith, 1981), and can be taken as a strong indicator of reception (Glänzel & Schoepflin, 1995) .
      • According to Merton, citations are intimately connected with the reward system of science and can be considered the currency of science.
      • According to Cozzens’ rhetoric-first model (1989), citation is only secondarily a reward system.
    glänzel: the ‘perspective shift’ EXAMPLES
      • Gross and Gross (1927) published their citation-based study in order to aid the decision which chemistry periodicals should best purchased by small college libraries.
      • Citations were considered documented use of information, and have consequently applied first in the context of librarianship, science information and information retrieval.
      • Bibliometrics analyses citation networks to depict the structure of science and its dynamics as well as the frequency of citations as a measure of the reception of research results.
    glänzel: the ‘perspective shift’ EXAMPLES
  • glänzel: the ‘perspective shift’ The co-citation based “Atlas of Science” developed and issued by the Institute for Scientific Information (ISI) was considered a new kind of “review literature” which is also suited to help students in choice of career in science ( Garfield, 1975 ). More recently Noyons observed that many issues brought up in domain mapping studies relate to policy-relevant questions, and consequently described the necessity of the evolution of bibliometric mapping towards a science policy tool. EXAMPLES
    • Author self-citations:
      • Re-interpreting underlying contexts such as the notion of citation shows author self-citations in an unfavourable light. Authors might thus be urged avoiding self-citations – a clear intervention into the mechanism of scientific communication.
    glänzel: the ‘perspective shift’ EXAMPLES
  • glänzel: the ‘perspective shift’ The process of re-interpreting the notion of citation and its consequences Information use Reward system (quality measure) Research evaluation/ Science policy Bibliometrics/ Information science citation uncitedness : unused information frequent cite : good reception self-cite : part of scient. communication interpretation re-interpretation repercussion (possible distortion of citation behaviour) uncitedness : low quality frequent cite : high quality self-cite : distortion of impact
  • glänzel: the ‘perspective shift’ Glänzel, Thijs, Schlemmer (2004) scientific commucation (18 cites) Schubert, Glänzel, Thijs (2006) fractional counting (0 cites) Glänzel et al. (2006) concise review (1 cites) Thijs & Glänzel (2005) meso indicators (0 cites) Glänzel & Thijs (2004b) co-authorship (4 cites) Glänzel & Thijs (2004a) macro indictors (5 cites) Example for a citation/self-citation network (Papers over author self-citations) Source: Web of Science, updated 14 October 2006)
  • Journal Impact Factor : The journal “Impact Factor” ( Garfield & Sher , 1963) was first used as measure for comparing journals independently of “size” and to help select journals for the Science Citation Index (SCI). Garfield recognised the power of the IF for journal evaluation and considered it also a journal performance indicator. glänzel: the ‘perspective shift’ EXAMPLES
  • “ American Journal of Transplantation is the leading journal in its field New impact factor for 2005 - 6.002 - Still the #1 transplantation journal (Ranked second in the surgery category)” Source: http://www.blackwellpublishing.com/journal.asp?ref=1600-6135&site=1 Accessed on 15 September 2006 glänzel: the ‘perspective shift’ Example for the role of the Impact Factor
  • Today, IF has become the “ common currency of scientific quality ” ( Sevinc , 2004) Consequence: Several journals have been accused of manipulating impact factor ( Smith , 1997, Weingart , 2005) Moreover, IF increasingly plays an important part in the evaluation of research groups and individuals (‘2 nd order perspective shift’ through uncoupling this measure from its original context). IF seems to be the only convertible currency in research evaluation, and has already influence on scientists’ funding and carrier. glänzel: the ‘perspective shift’ EXAMPLES
  • glänzel: the ‘perspective shift’
    • Typical questions addressed by scientists
    • “ What is the best output strategy for an individual researcher, cf. trade off between number of publications in top journals and number of publications in moderately ranked journals; trade off between investing time and energy in PhD-students (resulting in high number of co-authorships) or working alone (resulting in lower number of first authorships)?” ( Anon , 2005)
    • “ What is the weight of a co-authorship in a CV? What is the importance of author number and rank on the evaluation of a publication for a researcher?” ( Anon , 2005)
    EXAMPLES
  • glänzel: the ‘perspective shift’
    • Effects of policy use on the scientific community
    • Possible positive effects
      • Scientists might recognise that scientific collaboration and publishing in high-impact or even top journals pays off. Also their publication activity might be stimulated.
    • Possible negative effects
      • Exaggerated collaboration, even trends towards hyper-authorship, inflating publication output by splitting up publications to sequences, inflating citation impact by self-citations and forming citation cliques, etc.
  • glänzel: the ‘perspective shift’
      • Replacing targeting by visibility in individual and institutional publication strategies.
      • Trend towards replacing quality and recognition by visibility at any price or towards preferring journals as publication channels in social sciences and humanities might be among these effects.
      • A certain “champions league” mentality is spreading among scientists (“Shanghai ranking”, H-index, “IF Filter”).
    • CONCLUSIONS
    • Bibliometrics evolved from a sub-discipline of LIS, from an ‘extension’ of science information to an evaluation and benchmarking tool.
    • New fields of applications resulted in use of bibliographic databases, research methods and bibliometric measures for which these were originally not designed.
    • Beyond any doubt, growing policy application turned ‘little bibliometrics’ into ‘big bibliometrics’.
    • However, this evolution also resulted in a clear ‘perspective shift’.
    • This shift, in turn, has severe consequences for the field itself and the whole scientific community as well.
    glänzel: the ‘perspective shift’