Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Can we use altmetrics at the institutional level? A case study analysing the coverage by research areas of four Spanish universities

271 views

Published on

Presented at the STI Conference held in Valencia on September 15, 2016

Published in: Education
  • Be the first to comment

  • Be the first to like this

Can we use altmetrics at the institutional level? A case study analysing the coverage by research areas of four Spanish universities

  1. 1. Can we use altmetrics at the institutional level? A case study analysing the coverage by research areas of four Spanish universities
  2. 2. Can we use altmetrics at the institutional level? A case study analysing the coverage by research areas of four Spanish universities Daniel Torres-Salinas*, Nicolás Robinson-García** and Evaristo Jiménez-Contreras*** • * Univ. Navarra and Univ. Granada, EC3metrics & Medialab UGR - @torressalinas ** INGENIO (CSIC-UPV), Universitat Politècnica de València - @nrobinsongarcia *** Universidad de Granada, Grupo EC3 and EC3metrics To be published in: Proceedings of the 21st International Conference on Science and Technology Indicators València (Spain) | September 14-16, 2016 Full proceeding paper avaible at arxiv: https://arxiv.org/abs/1606.00232
  3. 3. Can we use altmetrics at institucional level? Introduction
  4. 4. Can we use altmetrics at institucional level? Introduction: altmetric studies Social media based indicators or altmetrics have been under scrutiny for the last seven years. 1.- The relation between citations and altmetric indicators 2.- Coverage and diversity of social media sources 3.- Their meaning as impact indicators Three lines of work can be observed
  5. 5. Can we use altmetrics at institucional level? Introduction: a gap in altmetric studies 1. Most studies are focus on the understanding of the nature and relation of altmetric indicators with citation data No study has been found profiling larger units of research such as institutions, especially universities 2.- Most papers are analysing altmetric indicators at paper (paper level metrics) or author level (author level metrics)
  6. 6. . Can we use altmetrics at institucional level? Introduction: information sources On the other hand, many data providers offering social media metrics have been developed Plum analytics Impact Story Altmetric.com
  7. 7. Can we use altmetrics at institucional level? Introduction: altmetric.com Altmetric.com has positioned itself as the most used source for altmetric studies 1.- Variety of social media platforms with regard to mentions, readership, etc. of scientific papers 2.- Offers an aggregated indicator or ‘altmetric’ score which is a weighted sum of values based of a given article in different social media 3.- Maintaining the ‘history’ of each altmetric indicator to an article Advantages of altmetric.com
  8. 8. Objetive 1: Analyse the coverage of Altmetric.com at the institutional and field level and verify its validity as evaluation tool Objetive 2: Analyse differences when obtaining subject bibliometric profiles from Altmetric.co and explore the possibility to make benchmarking studies at university level Can we use altmetrics at institucional level? Objetives: altmetrics and universities Our objectives are:
  9. 9. Can we use altmetrics at institucional level? Material & methods
  10. 10. Can we use altmetrics at institucional level? Material & methods: 4 universities y Size Staff Type and main field Foundation Year Region & City University of Granada 2399 Multidisciplinary 1531 Granada University Pompeu Fabra 288 Specialized Medicine/Biology 1990 Barcelona Polytechnic University Valencia 1847 Specialized Engineering 1971 Valencia University Carlos III 539 Specialized Social Sciences 1989 Madrid 1.- We will analyze 4 different Spanish universities 2.- Present different sizes and research publication specialization 3.- Different Foundation year and located in different regions Selection of spanish universities
  11. 11. Can we use altmetrics at institucional level? Material & methods: Web of Science 1.- Citable papers for the 2014 year indexed in the Web of Science for the 4 universities were retrieved (5922 records) 2.- Records from the Web of Science provide among other information the DOI number of each publication 3.- The DOI number is very important, as it will allow us to query the Altmetric.com API for each document 4.- A total of 93% of the total share from our data set included DOI Search for Web of Science papers and check DOI coverage 5.- Publications have been assigned to research areas
  12. 12. Can we use altmetrics at institucional level? Material & methods: Web of Science 1.- We retrieved altmetric data available for our set of papers with DOI. 2.- Then, the share of records with a score assigned by Altmetric.com has been calculated 3.- Altmetric.com covered 2149 records from the Web of Science, representing 36% of our data set Search for altmetric.com score using DOI
  13. 13. Can we use altmetrics at institutional level? Results
  14. 14. Can we use altmetrics at institucional level? Results: altmetric.com coverage Documents retrieved 4 universities 5922 93% with DOI 5922 web of Science Documents NOT Indexed in altmetric.com 3773 - 74% Indexed in altmetric.com 2149 -36% Indexed = at least with a value of 1 in altmetric score
  15. 15. Can we use altmetrics at institucional level? Results: altmetric.com coverage NOT Indexed in altmetric.com 3773 - 74% There are significance difference in the % of Web of Science documents indexed in altmetric.com
  16. 16. Can we use altmetrics at institucional level? Results: altmetric.com coverage Altmetric.com score are very concentrate just in Twitter and Mendeley Readers
  17. 17. Can we use altmetrics at institucional level? Results: altmetric.com coverage We can compare the “social media” patron for the four universities
  18. 18. Can we use altmetrics at institucional level? Results: altmetric.com coverage Important differences by university
  19. 19. Can we use altmetrics at institucional level? Results: Research field profile GranadaPompeuFabra P.ValenciaCarlosIII We have obtained the coverage profile (Web of science papers) of altmetric.com per research fields: • Significant differences in research field coverage (ej: Pompeu Fabra Vs Carlos III) • Best coverage in all fields in universities with S&T profile (ej: P. Valencia & Pompeu Fabra) • Dramatic coverage of Humanities (ej: Granada) and quite well in Social Science
  20. 20. Can we use altmetrics at institucional level? Results: Research field profile Totalaltmetric.comScore Nr of Web of Science Papers indexed in Altmetric.com Map of Web of Science Categories We can benchmark universities and compare altmetric performance at category level Multidisciplinary Sciences Pompeu Fabra Biochemistry Pompeu FabraBiochemistry Granada Multidisciplinary Sciences Granada
  21. 21. Can we use altmetrics at institucional level? Results: Research field profile – top10 Multidisciplinary Science (Science, Nature, Plos one…) have higher coverage and concentration of altmetric.com score. In the case of Pompeu Fabra 60% of altmetric.com score are related to papers published in multidisciplinary journals. We can see that social sciences and humanites are not well represented in top categories
  22. 22. Can we use altmetrics at institucional level? Results: Research field profile – top10 Top papers according Almetric.com Scores and journals where have been published We can observed the “Nature” and “Science” effect that determine altmetrics in these case studies Final results remarks
  23. 23. Can we use altmetrics at institucional level? Concluding remarks
  24. 24. Can we use altmetrics at institutional level? Concluding remarks Can we use altmetrics at institutional level? YES √ We can make a local bibliometric database combining bibliometric data from Web of Science and altmetric data from altmetric.com √ We can calculate altmetric indicators at university level and offer an altmetric research field profile in different levels (fields and categories) √ We can compare and make universities benchmarking studies
  25. 25. Can we use altmetrics at institutional level? Concluding remarks We observed a low coverage of altmetric indicators with only 36% of all documents have an ‘altmetric’ score To take into account: Low coverage of web of science papers: The DOI´s problem: One of the main factors to such low coverage could be the use of DOI in altmetric.com not all papers indexed in the Web of Science include DOI not all people that mention a paper in social media use the DOI So using DOI we have a loss of information
  26. 26. Can we use altmetrics at institutional level? Concluding remarks To take into account: Multidisciplinary Journals concentrate altmetrics The Nature and Science effect We observed that “multidisciplinary sciences” accumulate a big piece of the total altmetric score at university level. So universities publishing in nature, science, pnas, etc.… also have high levels of mentions in social media.
  27. 27. Can we use altmetrics at institutional level? Concluding remarks To take into account: Social science & humanities: low coverage and impact The national factor and language bias. It is necessary to verify if there is a national or linguistic factor influencing the coverage of altmetric indicators in the areas of Socials Sciences and Humanities & Arts. Another explanation could be that these areas are represented in social media by small communities, limiting their capacity to disseminate their research papers in social media.
  28. 28. Can we use altmetrics at the institutional level? Full proceeding paper avaible at arxiv: https://arxiv.org/abs/1606.00232

×