Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Normalization of Mendeley reader impact on the reader- and paper-side

263 views

Published on

These are the slides for my talk presented at the STI 2016 in Valencia. The paper dx.doi.org/10.1016/j.joi.2016.04.015 contains additional information regarding this presentation.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Normalization of Mendeley reader impact on the reader- and paper-side

  1. 1. Normalization of Mendeley reader impact on the reader- and paper-side Robin Haunschild Lutz Bornmann 09/16/2016
  2. 2. Introduction Altmetrics Alternative metrics, closely related to article level metrics Facebook: posts, likes, ... Twitter: tweets, retweets, ... Mendeley, CiteULike, Zotero, ...: readers (reader counts, bookmarks, saves, ...) News outlets: stories, mentions Blogs: stories, mentions ... Two major fields of altmetrics research 1. Meaning of altmetrics counts 2. Normalization of altmetrics counts Mendeley reader impact Robin Haunschild et al. 2
  3. 3. Introduction Mendeley Online reference manager Desktop and mobile applications Social, academic networking component API for user statistics Global web traffic rank from Alexa Mendeley reader impact Robin Haunschild et al. 3
  4. 4. Introduction Research Questions 1. Is normalization important for Mendeley reader counts? 2. Which normalization procedures are possible? 3. Can analogous versions to citing-side and cited-side normalizations be done using Mendeley reader counts? 4. How do the methods differ? Mendeley reader impact Robin Haunschild et al. 4
  5. 5. Data set Data set DOIs from WoS papers from 2012 (Na = 1, 133, 224 articles and Nr = 64, 960 reviews) Search via Mendeley API for DOI in December 2014 Overall 94.8% of the articles and 96.6% of the reviews were found on Mendeley 9,352,424 Mendeley reader counts for the articles and 1,335,764 for the reviews. 0.05% of the article readers and 0.04% of the review readers did not share their (sub-)disciplinary information Mendeley reader impact Robin Haunschild et al. 5
  6. 6. Example data from Mendeley Data available from Mendeley Input DOI, PubMedID, ... (e.g., 10.1063/1.4769790, Insensitivity of the error of the minimally empirical hybrid functional revTPSSh to its parameters): Total reader count (here: 9) Reader count per academic status (here: 2 Researchers, 1 Other, 3 Professors, 2 PhD Students, and 1 Associate Professor) Reader count per Mendeley discipline (here: 1 in Materials Science, 1 in Physics, 5 in Chemistry, 1 in Social Sciences, and 1 in Economics) Reader count per country (here: 1 in USA, 1 in the Vatican, and 1 in South Korea) Mendeley reader impact Robin Haunschild et al. 6
  7. 7. Mendeley readers per document type Mendeley readers per article in the Mendeley disciplines Mendeley reader impact Robin Haunschild et al. 7
  8. 8. Mendeley readers per document type (cont’d) Mendeley readers per review in the Mendeley disciplines Mendeley reader impact Robin Haunschild et al. 8
  9. 9. Mendeley readers per document type (cont’d) Mendeley readers per article in the top WoS subject categories Mendeley reader impact Robin Haunschild et al. 9
  10. 10. Mendeley readers per document type (cont’d) Mendeley readers per review in the top WoS subject categories Mendeley reader impact Robin Haunschild et al. 10
  11. 11. Cited-side and citing-side methods Cited-side 1. Counting all citations a paper has received 2. Normalization of citation counts with respect to the scientific field of the cited paper Analogous version for Mendeley reader counts: paper-side Citing-side 1. Counting all citations a paper has received separately for each scientific field of the citing paper 2. Normalization of citation counts with respect to the scientific field of the citing paper Analogous version for Mendeley reader counts: reader-side Mendeley reader impact Robin Haunschild et al. 11
  12. 12. Method Paper-side Normalization Average number of readers per paper (ρ) in a scientific field (here: WoS subject category), document type, and publication year: ρc = 1 Nc Nc i=1 Ri (1) Nc: Number of papers in a WoS subject category, document type, and publication year Ri : Number of reader counts of paper i NRSi = Ri ρc (2) NRSi : Normalized Reader Score for paper i Multiplicative (or fractional or full counting) for papers with WoS subject categories Mendeley reader impact Robin Haunschild et al. 12
  13. 13. Method Paper-side Normalization (cont’d) Average over a set of papers of a specific unit: MNRS = 1 N N i=1 NRSi (3) N: Number of papers in a specific research unit Interpretation Analogous to MNCS: (M)NRS ≈ 1: average reader impact of paper (M)NRS < 1: below average reader impact of paper (M)NRS > 1: above average reader impact of paper Mendeley reader impact Robin Haunschild et al. 13
  14. 14. Method Reader-side Normalization Average number of readers per paper (ρ) in a scientific field (here: Mendeley discipline), document type, and publication year: ρd = 1 Nd Nd i=1 Rid (4) Nd : Number of papers in a Mendeley discipline, document type, and publication year Rid : Number of reader counts of paper i in Mendeley discipline d βid = Rid ρd (5) βid : Normalized Reader Score for paper i in Mendeley discipline d Mendeley reader impact Robin Haunschild et al. 14
  15. 15. Method Reader-side Normalization (cont’d) Sum over all Mendeley disciplines yields a paper-based reader impact value: DNRSi = D d=1 βid (6) D: Number of Mendeley disciplines where paper i has readers. DNRSi : Discipline Normalized Reader Score of paper i Average over a set of papers of a specific unit: MDNRS = 1 N N i=1 DNRSi (7) N: Number of papers in a specific research unit Mendeley reader impact Robin Haunschild et al. 15
  16. 16. Spearman correlation coefficients for journals MDNRS vs. MNRS for different OECD categories OECD category rs No. of journals Natural sciences 0.75 3337 Engineering and technology 0.81 1556 Medical and health sciences 0.82 2855 Agricultural sciences 0.89 385 Social sciences 0.83 1920 Humanities 0.42 563 Mendeley reader impact Robin Haunschild et al. 16
  17. 17. Additional Information More details DOI: 10.1016/j.joi.2016.04.015 Submission history 22 February, 2016: Upload of Manuscript (version 1) to Figshare and submission of revised version to JoI. 07 March, 2016: Submission of abstract for this contribution 14 March/29 March, 2016: Submission deadline for STI contributions 22 April, 2016: Manuscript accepted by JoI and upload of final manuscript version to Figshare. 30 May, 2016: Formal acceptance of this contribution to this conference. DOI: 10.1016/j.joi.2016.04.015, published in the August 2016 issue of JoI. Mendeley reader impact Robin Haunschild et al. 17
  18. 18. Summary and Conclusions Answering the Research Questions Yes, normalization is important for Mendeley reader counts. Raw reader counts should not be used for impact assessment. Paper-side and reader-side normalizations are possible. The paper-side normalization is the analogue of the cited-side normalization, and the reader-side normalization is the analogue of the citing-side normalization. The reader-side and paper-side normalization methods provide slightly different rankings, e.g. for journals. Mendeley reader impact Robin Haunschild et al. 18
  19. 19. Summary and Conclusions Normalization of Mendeley reader counts Two different methods for normalization of Mendeley reader counts were presented. Both methods correlate larger than expected for most journals. Outlook Normalization with respect to Mendeley disciplines has been done. Is it useful to normalize with respect to: academic status groups or country affiliations of Mendeley readers? Mendeley reader impact Robin Haunschild et al. 19
  20. 20. 53 23 1653 23 16 Mendeley reader impact Robin Haunschild et al. 20

×