Bibliometrics in practice: how to generate reports for institutions.

Torres Salinas
Torres SalinasEvaluador investigación, CEO e investigador at Universidad de Navarra y Universidad de Granada (EC3metrics & Medialab UGR)
Daniel Torres-Salinas
Bibliometrics in practice: how to generate
reports for institutions.
Introduction
Objectives and index
★ Learn how to design bibliometric reports at the
institutional level (e.g., universities, hospitals, research
centers). We are going to concentrate in a specific report:
the annual memory (University of Granada - UGR).
★ Mission of the bibliometrics units
★ Talk Index
○ What information sources can we use
○ Content and selection of bibliometrics indicators
○ Making data available online
○ Examples of different reports
Type of bibliometric Reports
If you are working for a bibliometric unit or research evaluation unit
What kind of bibliometric reports can you offer to your institution?
Annual
memory
Reports on a
specific
topics
relevant to
the institution
Reports on
demand by
policy
managers
Type of bibliometric Reports
- Specific topics, for example gender, collaboration
- Specific areas, for example humanities
- Specific information sources, for example altmetrics
- Specific document type, for example books
- Reports on demand, examining a program
- Publication … Open Access
- Reports on new social media
Case Study: University of Granada
For this course we are going
to take as a case study the
annual memory from the
University of Granada
“Indicators and statistics at the
University of Granada”
Download at:
https://tinyurl.com/y8ywsdu3
Structure of a bibliometric report
For an annual memory we have to consider at least the following sections:
Main sections
1) Summary
2) Sources and indicators
3) General view
4) Fields and disciplines
5) Benchmarking and comparisons
6) Funding information
Information Sources
Types of sources
★ External database
○ Bibliographic databases and Citation Indexes: Scopus or
Web of Science Core Collection, ….
○ Bibliometric suites: Incites or Scival
★ Internal databases
○ Current Research Information Systems (CRIS)
○ Institutional administrative databases (Projects, Staff, etc…)
★ Other complementary sources
○ World University Rankings
○ Online academic profiles
○ ….
Types of sources
★ External database
○ In-Cites for bibliometric indicators (Around 60% of the
report)
★ Internal databases
○ e-proyecta: internal management database for projects
★ Other complementary sources
○ ARWU (Shanghai) - Benchmarking and verification of
scientific policies
○ Google Scholar Profiles to identify outstanding
researchers and promote internet visibility
Incites for reports
★ At the University of Granada we use InCites from Clarivate Analytics.
We recommend this bibliometric suite if:
○ you are working in large and multidisciplinary institutions
○ you don't have time for normalization and data cleaning
★ Advantages: indicators are already calculated, you can download
raw data and use them for your report
★ Disadvantages: non-normalization for authors, inaccurate information
for institutions due to errors in the organization enhanced field...
Bibliometric suites (Incites or Scival) are expensive (between 35.000-
60.000 Euros) depending institution´s size.
Incites for reports
The main advantage
of InCites is provided
useful datasets!!
Content and indicators
What indicators
- Which bibliometric indicators should you use?
- Easy interpretation. Complex indicators are difficult to
comprehend and should be avoided.
- Standard indicators approved by the international
community.
- They have to reflect the different dimensions of
research performance
What indicators
Useful indicators for reports. Four types of dimensions seven indicators
Lorem ipsum dolor sit
amet at nec at adipiscing
03
● Donec risus dolor porta venenatis
● Pharetra luctus felis
● Proin in tellus felis volutpat
Lorem ipsum dolor sit
amet at nec at adipiscing
02
● Donec risus dolor porta venenatis
● Pharetra luctus felis
● Proin in tellus felis volutpat
Lorem ipsum dolor sit
amet at nec at adipiscing
01
● Donec risus dolor porta venenatis
● Pharetra luctus felis
● Proin in tellus felis volutpat
Excellence04
● % Highly Cited Papers
● Papers in top journals (Science & Nature)
Impact03
● % of papers in first quartile journals
● Category Normalized Citation Impact
Collaboration02
● Number of documents with
international collaboration
Production01
● Number of documents
● Number of citable documents
Contextualization of performance
★ Bibliometric indicators make sense when comparing with
different aggregation levels, i.e. compare university with the
national average. How is our institution performing in comparison
with the national average?.
★ Some indicators for benchmarking are the Category Normalized
Citation Impact (InCites), relative indicators like percentage of
papers in first quartile journals or percentage of papers in
international collaboration
★ In Granada, we compare different indicators with three
geographical regions: Spain, European Union and USA
★ Without contextualization there is no meaning!
Contextualization of performance
★ Example for the publication profile in high impact factor journals
★ Indicator > Percentage of papers in first quartile journals
★ The publication profile of the University of Granada is quite similar to the
national and international standards
★ It is important to compare also our institution with other similar
institutions. We have to select a coherent and homogeneous
benchmarking group taking into account at least this variables:
○ Size of research output How many papers?
○ Size of staff How many faculty?
○ Same institutional objectives Focus on teaching or research?
○ Similar disciplinary profile Humanities, Life Sciences…?
★ At the University of Granada we compare our results with Spanish
historical universities with a multidisciplinary profile.
★ Data from other institutions is retrieved from the Shanghai Ranking
Benchmarking & Comparisons
Benchmarking & Comparisons
★ Example: benchmarking of the university of Granada with Spanish universities.
★ UGR performs well according to the number of citable papers, impact and
collaboration, but has to improve the share of papers in first Quartile journals
All universities
perform better than
Granada according
based on the
percentage of
papers in First
Quartile Journals
★ If we are working at universities it is important also to represent in a bivariate
graph the position of our institution within our national university system
Benchmarking & Comparisons
Nr of Citable Papers
Cat.NormalizedCitationImpact
UGR Impact
World Impact
★ Another question that we have to adress in our report is the
thematic profile of our institutions; detecting best fields fields and
disciplines
★ For this we must consider the use of different aggregation levels.
Select at least one general level for an overview of the fields (for
example Essential Science Indicators, 22 fields) and a more
specific discipline level (for example Web of Science Categories,
more than 200 categories).
★ Differentiate always between Science and Social Social
Sciences / Humanities
Fields and disciplines
Fields and disciplines
Cat.NormalizedCitationImpact
Nr of Citable Papers
1 Clinical Medicine, 2 Physics, 3 Social Sciences, 4 Chemistry, 5
Geosciences, 6 Engineering, 7 Mathematics, 8 Psychology, 9 Agricultural
Science, 10 Computer Sciences, 11 Ecology, 12 Biology, 13 Plant &
Animal Science, 14 Neuroscience, 15 Space Science, ...
We can see the general
scientific profile of the UGR
according ESI classification.
In this case we can identify
very productive but low
impact areas (1, 3, 4); very
productive and high impact
areas (2); relatively
productive but with high
impact areas (15) and not
productive and low impact
areas (14, 17, ….)
Take care when choosing a
classification, for example
ESI does not have an specific
field for the Humanities
Impact < World
Impact < World
Fields and disciplines
★ In this example we have the second level of presentation of the data at the
disciplinary level, the WoS categories applied to the social sciences and the
humanities.
Combining fields and institutions
★ Another interesting option to complete the report is to
compare the performance of our institution with other
universities in different scientific fields.
★ At University of Granada we check its position in five
different fields (Natural Science, Engineering, Health
Sciences, Social Sciences and Humanities) and compare its
results with the Spanish university system.
Combining fields and institutions
Nr of citable papers Category Normalized Citation Impact In this example we can
see the position of the
University of Granada in
the field of Natural
Sciences according to
two different bibliometric
indicators.
★ We use the number of projects and total funding in competitive
programmes to contextualize our results. We consider two calls a)
Spanish R&D National programmes and b) European funding from
the European Commission.
★ Nr. of project and total funding are important indicators but also
success rate (Nr of applications / Proposals selected for funding). It
is important also to compare these indicators with national or UE
average and do benchmarking with other institutions
Non bibliometric Indicators: funding
In order to complement the information
provided by bibliometric indicators we can
include information on other inputs
Non bibliometric Indicators: funding
★ In the University of Granada we focus on national research programmes. In this
example we can see the results of four Spanish universities in the last year. We
can see that the University of Granada is the second university in Spain with a
larger number of projects conceded
Indicators at author level
★ For authors we have included the researchers with the highest H-index in Web
of Science for different scientific categories
Indicators at author level
★ Finally, we include the most outstanding
researchers in "Google Scholar Profiles".
We review all profiles and only include
those with correct information. We have
analyzed a total of 2000 University of
Granada profiles and approved 1,700.
These profiles have been classified into five
different scientific areas. In the report we
present a summary of the 200 most cited
researchers
Link to google scholar profile
Making data available online
Transparency and availability of data
★ To promote institutional
transparency, reproducibility,
verification of results and data
reuse by other departments,
we share our data in an
standardized way.
★ Livemetrics Portal. We have
developed a portal that
includes a dynamic version of
the report with the main
indicators
Examples and resources
Institutional reports
Granada and Modena - Collaboration
Videoreports
Daniel Torres-Salinas
Bibliometrics in practice: how to generate
reports for institutions.
1 of 36

Recommended

Bibliometrics in practice: how to generate reports for institutions - v2.0 / ... by
Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...
Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...Torres Salinas
2K views48 slides
Bibliometric solutions for identifying potential collaborators by
Bibliometric solutions for identifying potential collaboratorsBibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaboratorsTorres Salinas
979 views57 slides
Unveiling the multiple faces of mobility: Towards a taxonomy of scientific mo... by
Unveiling the multiple faces of mobility: Towards a taxonomy of scientific mo...Unveiling the multiple faces of mobility: Towards a taxonomy of scientific mo...
Unveiling the multiple faces of mobility: Towards a taxonomy of scientific mo...Nicolas Robinson-Garcia
403 views19 slides
Disentangling gold open access by
Disentangling gold open accessDisentangling gold open access
Disentangling gold open accessNicolas Robinson-Garcia
1.7K views45 slides
Introduction to bibliometric data sources - Google Scholar by
Introduction to bibliometric data sources - Google ScholarIntroduction to bibliometric data sources - Google Scholar
Introduction to bibliometric data sources - Google ScholarNicolas Robinson-Garcia
1.7K views32 slides
Classifying and visualizing the disciplinary focus of universities: The invis... by
Classifying and visualizing the disciplinary focus of universities: The invis...Classifying and visualizing the disciplinary focus of universities: The invis...
Classifying and visualizing the disciplinary focus of universities: The invis...Nicolas Robinson-Garcia
12.3K views50 slides

More Related Content

What's hot

A scientometric perspective on university ranking by
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university rankingNees Jan van Eck
238 views54 slides
URAP 2012 by
URAP 2012URAP 2012
URAP 2012URAP Research Center
1.6K views46 slides
Brazil's university ranking a prediction study with machine learning 234 ifka... by
Brazil's university ranking a prediction study with machine learning 234 ifka...Brazil's university ranking a prediction study with machine learning 234 ifka...
Brazil's university ranking a prediction study with machine learning 234 ifka...IFSC
237 views12 slides
Can we use altmetric at institutional level? by
Can we use altmetric at institutional level?Can we use altmetric at institutional level?
Can we use altmetric at institutional level?Torres Salinas
1.2K views28 slides
Scientometrics for research assessment by
Scientometrics for research assessmentScientometrics for research assessment
Scientometrics for research assessmentLudo Waltman
1.8K views35 slides
The SSH conundrum: A matter of audiences by
The SSH conundrum: A matter of audiencesThe SSH conundrum: A matter of audiences
The SSH conundrum: A matter of audiencesNicolas Robinson-Garcia
444 views28 slides

What's hot(19)

A scientometric perspective on university ranking by Nees Jan van Eck
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university ranking
Nees Jan van Eck238 views
Brazil's university ranking a prediction study with machine learning 234 ifka... by IFSC
Brazil's university ranking a prediction study with machine learning 234 ifka...Brazil's university ranking a prediction study with machine learning 234 ifka...
Brazil's university ranking a prediction study with machine learning 234 ifka...
IFSC237 views
Can we use altmetric at institutional level? by Torres Salinas
Can we use altmetric at institutional level?Can we use altmetric at institutional level?
Can we use altmetric at institutional level?
Torres Salinas1.2K views
Scientometrics for research assessment by Ludo Waltman
Scientometrics for research assessmentScientometrics for research assessment
Scientometrics for research assessment
Ludo Waltman1.8K views
CWTS Leiden Ranking: An advanced bibliometric approach to university ranking by Nees Jan van Eck
CWTS Leiden Ranking: An advanced bibliometric approach to university rankingCWTS Leiden Ranking: An advanced bibliometric approach to university ranking
CWTS Leiden Ranking: An advanced bibliometric approach to university ranking
Nees Jan van Eck384 views
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1) by Murat KOÇAK
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
Murat KOÇAK2.3K views
A scientometric perspective on university ranking by Nees Jan van Eck
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university ranking
Nees Jan van Eck208 views
Research Evaluation - The current situation in Spain by Antonio Vallecillo
Research Evaluation - The current situation in SpainResearch Evaluation - The current situation in Spain
Research Evaluation - The current situation in Spain
Antonio Vallecillo512 views
Multiple bibliometric indicators approach to Croatian open access (OA) journals by Rudjer Boskovic Institute
Multiple bibliometric indicators approach to Croatian open access (OA) journalsMultiple bibliometric indicators approach to Croatian open access (OA) journals
Multiple bibliometric indicators approach to Croatian open access (OA) journals
Citation Analysis of Higher Education Texts in Selected Databases: A Comparis... by Che-Wei Lee
Citation Analysis of Higher Education Texts in Selected Databases: A Comparis...Citation Analysis of Higher Education Texts in Selected Databases: A Comparis...
Citation Analysis of Higher Education Texts in Selected Databases: A Comparis...
Che-Wei Lee486 views
European Trends in the Use of Evaluations and Indicators for Performance-Base... by ThomsonReutersMktg
European Trends in the Use of Evaluations and Indicators for Performance-Base...European Trends in the Use of Evaluations and Indicators for Performance-Base...
European Trends in the Use of Evaluations and Indicators for Performance-Base...
ThomsonReutersMktg209 views
SSH & the City. A network approach for tracing the societal contribution of t... by Nicolas Robinson-Garcia
SSH & the City. A network approach for tracing the societal contribution of t...SSH & the City. A network approach for tracing the societal contribution of t...
SSH & the City. A network approach for tracing the societal contribution of t...
Scientific information retrieval: Challenges and opportunities by Ludo Waltman
Scientific information retrieval: Challenges and opportunitiesScientific information retrieval: Challenges and opportunities
Scientific information retrieval: Challenges and opportunities
Ludo Waltman1.4K views

Similar to Bibliometrics in practice: how to generate reports for institutions.

Bibliometric solutions for identifying potential collaborators by
Bibliometric solutions for identifying potential collaboratorsBibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaboratorsNicolas Robinson-Garcia
1.4K views57 slides
Rankings isi at the university ranking round table un 2010 by
Rankings isi at the university ranking round table un 2010Rankings isi at the university ranking round table un 2010
Rankings isi at the university ranking round table un 2010Torres Salinas
1.7K views32 slides
Presentacion altmetrics Daniel Torres Salinas by
Presentacion altmetrics Daniel Torres SalinasPresentacion altmetrics Daniel Torres Salinas
Presentacion altmetrics Daniel Torres SalinasEC3metrics Spin-Off
255 views28 slides
Can we use altmetrics at the institutional level? A case study analysing the ... by
Can we use altmetrics at the institutional level? A case study analysing the ...Can we use altmetrics at the institutional level? A case study analysing the ...
Can we use altmetrics at the institutional level? A case study analysing the ...Nicolas Robinson-Garcia
493 views28 slides
World university rankings 2014 By Dinesh Gunathilaka by
World university rankings 2014 By  Dinesh GunathilakaWorld university rankings 2014 By  Dinesh Gunathilaka
World university rankings 2014 By Dinesh GunathilakaDinesh Gunathilaka
328 views9 slides
Journal and author impact measures Assessing your impact (h-index and beyond) by
Journal and author impact measures Assessing your impact (h-index and beyond)Journal and author impact measures Assessing your impact (h-index and beyond)
Journal and author impact measures Assessing your impact (h-index and beyond)Aboul Ella Hassanien
1.6K views81 slides

Similar to Bibliometrics in practice: how to generate reports for institutions.(20)

Bibliometric solutions for identifying potential collaborators by Nicolas Robinson-Garcia
Bibliometric solutions for identifying potential collaboratorsBibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaborators
Rankings isi at the university ranking round table un 2010 by Torres Salinas
Rankings isi at the university ranking round table un 2010Rankings isi at the university ranking round table un 2010
Rankings isi at the university ranking round table un 2010
Torres Salinas1.7K views
Can we use altmetrics at the institutional level? A case study analysing the ... by Nicolas Robinson-Garcia
Can we use altmetrics at the institutional level? A case study analysing the ...Can we use altmetrics at the institutional level? A case study analysing the ...
Can we use altmetrics at the institutional level? A case study analysing the ...
World university rankings 2014 By Dinesh Gunathilaka by Dinesh Gunathilaka
World university rankings 2014 By  Dinesh GunathilakaWorld university rankings 2014 By  Dinesh Gunathilaka
World university rankings 2014 By Dinesh Gunathilaka
Dinesh Gunathilaka328 views
Journal and author impact measures Assessing your impact (h-index and beyond) by Aboul Ella Hassanien
Journal and author impact measures Assessing your impact (h-index and beyond)Journal and author impact measures Assessing your impact (h-index and beyond)
Journal and author impact measures Assessing your impact (h-index and beyond)
The role of national university rankings in an international context the case... by Torres Salinas
The role of national university rankings in an international context the case...The role of national university rankings in an international context the case...
The role of national university rankings in an international context the case...
Torres Salinas1K views
The role of national university rankings in an international context: The cas... by Nicolas Robinson-Garcia
The role of national university rankings in an international context: The cas...The role of national university rankings in an international context: The cas...
The role of national university rankings in an international context: The cas...
The role of national university rankings in an international context the case... by EC3metrics Spin-Off
The role of national university rankings in an international context the case...The role of national university rankings in an international context the case...
The role of national university rankings in an international context the case...
FROM GRADUATE SCHOOL TO PROFESSIONAL LIFE PREPARING A LONG JOURNEY by Genoveva Vargas-Solar
FROM GRADUATE SCHOOL TO PROFESSIONAL LIFE PREPARING A LONG JOURNEYFROM GRADUATE SCHOOL TO PROFESSIONAL LIFE PREPARING A LONG JOURNEY
FROM GRADUATE SCHOOL TO PROFESSIONAL LIFE PREPARING A LONG JOURNEY
Morocco Top 1000 Scientists.pdf by darouich1
Morocco Top 1000 Scientists.pdfMorocco Top 1000 Scientists.pdf
Morocco Top 1000 Scientists.pdf
darouich1123 views
Scholarly Metrics in Specialized Settings by Elaine Lasda
Scholarly Metrics in Specialized SettingsScholarly Metrics in Specialized Settings
Scholarly Metrics in Specialized Settings
Elaine Lasda167 views
Universties global ranking systems for pub. by Ahmed Metwaly
Universties global ranking systems for pub.Universties global ranking systems for pub.
Universties global ranking systems for pub.
Ahmed Metwaly107 views
Doctoral Dissertation - Pablo Migliorini - July 2015 by Pablo Migliorini
Doctoral Dissertation - Pablo Migliorini - July 2015Doctoral Dissertation - Pablo Migliorini - July 2015
Doctoral Dissertation - Pablo Migliorini - July 2015
Pablo Migliorini224 views
EDI23-Challenges-in-evaluating-collaborative-postgraduate-WP-work-the-YCEDE-p... by UKCGE
EDI23-Challenges-in-evaluating-collaborative-postgraduate-WP-work-the-YCEDE-p...EDI23-Challenges-in-evaluating-collaborative-postgraduate-WP-work-the-YCEDE-p...
EDI23-Challenges-in-evaluating-collaborative-postgraduate-WP-work-the-YCEDE-p...
UKCGE5 views
Publishing and impact 20141028 by Hugo Besemer
Publishing and impact 20141028Publishing and impact 20141028
Publishing and impact 20141028
Hugo Besemer499 views
Characteristics of International versus Non-International Scientific Publicat... by Nadine Rons
Characteristics of International versus Non-International Scientific Publicat...Characteristics of International versus Non-International Scientific Publicat...
Characteristics of International versus Non-International Scientific Publicat...
Nadine Rons200 views

More from Torres Salinas

Journal impact measures: the Impact Factor by
Journal impact measures: the Impact FactorJournal impact measures: the Impact Factor
Journal impact measures: the Impact FactorTorres Salinas
4.1K views36 slides
Cómo seleccionar y publicar en revistas de impacto en Ciencias Sociales by
Cómo seleccionar y publicar en revistas de impacto en Ciencias SocialesCómo seleccionar y publicar en revistas de impacto en Ciencias Sociales
Cómo seleccionar y publicar en revistas de impacto en Ciencias SocialesTorres Salinas
3.1K views31 slides
La Segunda Guerra Mundial en los videojuegos by
La Segunda Guerra Mundial en los videojuegosLa Segunda Guerra Mundial en los videojuegos
La Segunda Guerra Mundial en los videojuegosTorres Salinas
1.5K views49 slides
Aportaciones presentadas en la I Reunión de Servicios de Evaluación Científica by
Aportaciones presentadas en la I Reunión de Servicios de Evaluación CientíficaAportaciones presentadas en la I Reunión de Servicios de Evaluación Científica
Aportaciones presentadas en la I Reunión de Servicios de Evaluación CientíficaTorres Salinas
657 views77 slides
Altmetric beauties. ¿Cuáles son los trabajos científicos con mayor impacto e... by
Altmetric beauties. ¿Cuáles son los trabajos científicos con  mayor impacto e...Altmetric beauties. ¿Cuáles son los trabajos científicos con  mayor impacto e...
Altmetric beauties. ¿Cuáles son los trabajos científicos con mayor impacto e...Torres Salinas
1.3K views21 slides
Cómo se evalúa y se progresa en la carrera científica [versión 2.0] by
Cómo se evalúa y se progresa en la  carrera científica [versión 2.0]Cómo se evalúa y se progresa en la  carrera científica [versión 2.0]
Cómo se evalúa y se progresa en la carrera científica [versión 2.0]Torres Salinas
1.1K views30 slides

More from Torres Salinas(20)

Journal impact measures: the Impact Factor by Torres Salinas
Journal impact measures: the Impact FactorJournal impact measures: the Impact Factor
Journal impact measures: the Impact Factor
Torres Salinas4.1K views
Cómo seleccionar y publicar en revistas de impacto en Ciencias Sociales by Torres Salinas
Cómo seleccionar y publicar en revistas de impacto en Ciencias SocialesCómo seleccionar y publicar en revistas de impacto en Ciencias Sociales
Cómo seleccionar y publicar en revistas de impacto en Ciencias Sociales
Torres Salinas3.1K views
La Segunda Guerra Mundial en los videojuegos by Torres Salinas
La Segunda Guerra Mundial en los videojuegosLa Segunda Guerra Mundial en los videojuegos
La Segunda Guerra Mundial en los videojuegos
Torres Salinas1.5K views
Aportaciones presentadas en la I Reunión de Servicios de Evaluación Científica by Torres Salinas
Aportaciones presentadas en la I Reunión de Servicios de Evaluación CientíficaAportaciones presentadas en la I Reunión de Servicios de Evaluación Científica
Aportaciones presentadas en la I Reunión de Servicios de Evaluación Científica
Torres Salinas657 views
Altmetric beauties. ¿Cuáles son los trabajos científicos con mayor impacto e... by Torres Salinas
Altmetric beauties. ¿Cuáles son los trabajos científicos con  mayor impacto e...Altmetric beauties. ¿Cuáles son los trabajos científicos con  mayor impacto e...
Altmetric beauties. ¿Cuáles son los trabajos científicos con mayor impacto e...
Torres Salinas1.3K views
Cómo se evalúa y se progresa en la carrera científica [versión 2.0] by Torres Salinas
Cómo se evalúa y se progresa en la  carrera científica [versión 2.0]Cómo se evalúa y se progresa en la  carrera científica [versión 2.0]
Cómo se evalúa y se progresa en la carrera científica [versión 2.0]
Torres Salinas1.1K views
El cv científico y su visibilidad: formatos, gestión y difusión en Internet by Torres Salinas
El cv científico y su visibilidad: formatos, gestión y difusión en InternetEl cv científico y su visibilidad: formatos, gestión y difusión en Internet
El cv científico y su visibilidad: formatos, gestión y difusión en Internet
Torres Salinas1.9K views
Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci... by Torres Salinas
Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...
Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...
Torres Salinas1.7K views
Cómo se evalúa y se progresa en la carrera científica by Torres Salinas
Cómo se evalúa y se progresa en la carrera científicaCómo se evalúa y se progresa en la carrera científica
Cómo se evalúa y se progresa en la carrera científica
Torres Salinas2.6K views
Inicitativas empresariales relacionadas con la evaluación de la ciencia el ... by Torres Salinas
Inicitativas empresariales relacionadas con la evaluación de la ciencia   el ...Inicitativas empresariales relacionadas con la evaluación de la ciencia   el ...
Inicitativas empresariales relacionadas con la evaluación de la ciencia el ...
Torres Salinas1.6K views
10 mejores apps para científicos #investigacionmovil by Torres Salinas
10 mejores apps para científicos #investigacionmovil10 mejores apps para científicos #investigacionmovil
10 mejores apps para científicos #investigacionmovil
Torres Salinas5.8K views
Diez reglas de oro para publicar en revistas de impacto by Torres Salinas
Diez reglas de oro para publicar en revistas de impactoDiez reglas de oro para publicar en revistas de impacto
Diez reglas de oro para publicar en revistas de impacto
Torres Salinas8.6K views
Pautas la elaboracion de proyectos: convocatoria retos y excelencia by Torres Salinas
Pautas la elaboracion de proyectos: convocatoria retos y excelenciaPautas la elaboracion de proyectos: convocatoria retos y excelencia
Pautas la elaboracion de proyectos: convocatoria retos y excelencia
Torres Salinas5.2K views
Cómo seleccionar una revista científica en comunicación by Torres Salinas
Cómo seleccionar una revista científica en comunicaciónCómo seleccionar una revista científica en comunicación
Cómo seleccionar una revista científica en comunicación
Torres Salinas2.3K views
Gestión y Monitorización de la información y el impacto científico: Perfiles,... by Torres Salinas
Gestión y Monitorización de la información y el impacto científico: Perfiles,...Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Torres Salinas3.3K views
La evaluación abierta de la investigación como elemento dinamizador by Torres Salinas
La evaluación abierta de la investigación como elemento dinamizadorLa evaluación abierta de la investigación como elemento dinamizador
La evaluación abierta de la investigación como elemento dinamizador
Torres Salinas1.8K views
Visiblegrinugr 150424024156-conversion-gate01 by Torres Salinas
Visiblegrinugr 150424024156-conversion-gate01Visiblegrinugr 150424024156-conversion-gate01
Visiblegrinugr 150424024156-conversion-gate01
Torres Salinas1.2K views
EC3metrics: Emprender y transferir desde las ciencias sociales by Torres Salinas
EC3metrics: Emprender y transferir desde las ciencias socialesEC3metrics: Emprender y transferir desde las ciencias sociales
EC3metrics: Emprender y transferir desde las ciencias sociales
Torres Salinas1.2K views
How many citations are there in the Data Citation Index? by Torres Salinas
How many citations are there in the Data Citation Index?How many citations are there in the Data Citation Index?
How many citations are there in the Data Citation Index?
Torres Salinas1.8K views
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014 by Torres Salinas
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Torres Salinas1.4K views

Recently uploaded

Gopal Chakraborty Memorial Quiz 2.0 Prelims.pptx by
Gopal Chakraborty Memorial Quiz 2.0 Prelims.pptxGopal Chakraborty Memorial Quiz 2.0 Prelims.pptx
Gopal Chakraborty Memorial Quiz 2.0 Prelims.pptxDebapriya Chakraborty
655 views81 slides
JiscOAWeek_LAIR_slides_October2023.pptx by
JiscOAWeek_LAIR_slides_October2023.pptxJiscOAWeek_LAIR_slides_October2023.pptx
JiscOAWeek_LAIR_slides_October2023.pptxJisc
96 views8 slides
Use of Probiotics in Aquaculture.pptx by
Use of Probiotics in Aquaculture.pptxUse of Probiotics in Aquaculture.pptx
Use of Probiotics in Aquaculture.pptxAKSHAY MANDAL
100 views15 slides
The Open Access Community Framework (OACF) 2023 (1).pptx by
The Open Access Community Framework (OACF) 2023 (1).pptxThe Open Access Community Framework (OACF) 2023 (1).pptx
The Open Access Community Framework (OACF) 2023 (1).pptxJisc
110 views7 slides
Create a Structure in VBNet.pptx by
Create a Structure in VBNet.pptxCreate a Structure in VBNet.pptx
Create a Structure in VBNet.pptxBreach_P
75 views8 slides
Narration lesson plan.docx by
Narration lesson plan.docxNarration lesson plan.docx
Narration lesson plan.docxTARIQ KHAN
112 views11 slides

Recently uploaded(20)

JiscOAWeek_LAIR_slides_October2023.pptx by Jisc
JiscOAWeek_LAIR_slides_October2023.pptxJiscOAWeek_LAIR_slides_October2023.pptx
JiscOAWeek_LAIR_slides_October2023.pptx
Jisc96 views
Use of Probiotics in Aquaculture.pptx by AKSHAY MANDAL
Use of Probiotics in Aquaculture.pptxUse of Probiotics in Aquaculture.pptx
Use of Probiotics in Aquaculture.pptx
AKSHAY MANDAL100 views
The Open Access Community Framework (OACF) 2023 (1).pptx by Jisc
The Open Access Community Framework (OACF) 2023 (1).pptxThe Open Access Community Framework (OACF) 2023 (1).pptx
The Open Access Community Framework (OACF) 2023 (1).pptx
Jisc110 views
Create a Structure in VBNet.pptx by Breach_P
Create a Structure in VBNet.pptxCreate a Structure in VBNet.pptx
Create a Structure in VBNet.pptx
Breach_P75 views
Narration lesson plan.docx by TARIQ KHAN
Narration lesson plan.docxNarration lesson plan.docx
Narration lesson plan.docx
TARIQ KHAN112 views
Dance KS5 Breakdown by WestHatch
Dance KS5 BreakdownDance KS5 Breakdown
Dance KS5 Breakdown
WestHatch79 views
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively by PECB
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks EffectivelyISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively
PECB 585 views
REPRESENTATION - GAUNTLET.pptx by iammrhaywood
REPRESENTATION - GAUNTLET.pptxREPRESENTATION - GAUNTLET.pptx
REPRESENTATION - GAUNTLET.pptx
iammrhaywood100 views
When Sex Gets Complicated: Porn, Affairs, & Cybersex by Marlene Maheu
When Sex Gets Complicated: Porn, Affairs, & CybersexWhen Sex Gets Complicated: Porn, Affairs, & Cybersex
When Sex Gets Complicated: Porn, Affairs, & Cybersex
Marlene Maheu67 views
Sociology KS5 by WestHatch
Sociology KS5Sociology KS5
Sociology KS5
WestHatch70 views
Classification of crude drugs.pptx by GayatriPatra14
Classification of crude drugs.pptxClassification of crude drugs.pptx
Classification of crude drugs.pptx
GayatriPatra1486 views
Are we onboard yet University of Sussex.pptx by Jisc
Are we onboard yet University of Sussex.pptxAre we onboard yet University of Sussex.pptx
Are we onboard yet University of Sussex.pptx
Jisc96 views

Bibliometrics in practice: how to generate reports for institutions.

  • 1. Daniel Torres-Salinas Bibliometrics in practice: how to generate reports for institutions.
  • 3. Objectives and index ★ Learn how to design bibliometric reports at the institutional level (e.g., universities, hospitals, research centers). We are going to concentrate in a specific report: the annual memory (University of Granada - UGR). ★ Mission of the bibliometrics units ★ Talk Index ○ What information sources can we use ○ Content and selection of bibliometrics indicators ○ Making data available online ○ Examples of different reports
  • 4. Type of bibliometric Reports If you are working for a bibliometric unit or research evaluation unit What kind of bibliometric reports can you offer to your institution? Annual memory Reports on a specific topics relevant to the institution Reports on demand by policy managers
  • 5. Type of bibliometric Reports - Specific topics, for example gender, collaboration - Specific areas, for example humanities - Specific information sources, for example altmetrics - Specific document type, for example books - Reports on demand, examining a program - Publication … Open Access - Reports on new social media
  • 6. Case Study: University of Granada For this course we are going to take as a case study the annual memory from the University of Granada “Indicators and statistics at the University of Granada” Download at: https://tinyurl.com/y8ywsdu3
  • 7. Structure of a bibliometric report For an annual memory we have to consider at least the following sections: Main sections 1) Summary 2) Sources and indicators 3) General view 4) Fields and disciplines 5) Benchmarking and comparisons 6) Funding information
  • 9. Types of sources ★ External database ○ Bibliographic databases and Citation Indexes: Scopus or Web of Science Core Collection, …. ○ Bibliometric suites: Incites or Scival ★ Internal databases ○ Current Research Information Systems (CRIS) ○ Institutional administrative databases (Projects, Staff, etc…) ★ Other complementary sources ○ World University Rankings ○ Online academic profiles ○ ….
  • 10. Types of sources ★ External database ○ In-Cites for bibliometric indicators (Around 60% of the report) ★ Internal databases ○ e-proyecta: internal management database for projects ★ Other complementary sources ○ ARWU (Shanghai) - Benchmarking and verification of scientific policies ○ Google Scholar Profiles to identify outstanding researchers and promote internet visibility
  • 11. Incites for reports ★ At the University of Granada we use InCites from Clarivate Analytics. We recommend this bibliometric suite if: ○ you are working in large and multidisciplinary institutions ○ you don't have time for normalization and data cleaning ★ Advantages: indicators are already calculated, you can download raw data and use them for your report ★ Disadvantages: non-normalization for authors, inaccurate information for institutions due to errors in the organization enhanced field... Bibliometric suites (Incites or Scival) are expensive (between 35.000- 60.000 Euros) depending institution´s size.
  • 12. Incites for reports The main advantage of InCites is provided useful datasets!!
  • 14. What indicators - Which bibliometric indicators should you use? - Easy interpretation. Complex indicators are difficult to comprehend and should be avoided. - Standard indicators approved by the international community. - They have to reflect the different dimensions of research performance
  • 15. What indicators Useful indicators for reports. Four types of dimensions seven indicators Lorem ipsum dolor sit amet at nec at adipiscing 03 ● Donec risus dolor porta venenatis ● Pharetra luctus felis ● Proin in tellus felis volutpat Lorem ipsum dolor sit amet at nec at adipiscing 02 ● Donec risus dolor porta venenatis ● Pharetra luctus felis ● Proin in tellus felis volutpat Lorem ipsum dolor sit amet at nec at adipiscing 01 ● Donec risus dolor porta venenatis ● Pharetra luctus felis ● Proin in tellus felis volutpat Excellence04 ● % Highly Cited Papers ● Papers in top journals (Science & Nature) Impact03 ● % of papers in first quartile journals ● Category Normalized Citation Impact Collaboration02 ● Number of documents with international collaboration Production01 ● Number of documents ● Number of citable documents
  • 16. Contextualization of performance ★ Bibliometric indicators make sense when comparing with different aggregation levels, i.e. compare university with the national average. How is our institution performing in comparison with the national average?. ★ Some indicators for benchmarking are the Category Normalized Citation Impact (InCites), relative indicators like percentage of papers in first quartile journals or percentage of papers in international collaboration ★ In Granada, we compare different indicators with three geographical regions: Spain, European Union and USA ★ Without contextualization there is no meaning!
  • 17. Contextualization of performance ★ Example for the publication profile in high impact factor journals ★ Indicator > Percentage of papers in first quartile journals ★ The publication profile of the University of Granada is quite similar to the national and international standards
  • 18. ★ It is important to compare also our institution with other similar institutions. We have to select a coherent and homogeneous benchmarking group taking into account at least this variables: ○ Size of research output How many papers? ○ Size of staff How many faculty? ○ Same institutional objectives Focus on teaching or research? ○ Similar disciplinary profile Humanities, Life Sciences…? ★ At the University of Granada we compare our results with Spanish historical universities with a multidisciplinary profile. ★ Data from other institutions is retrieved from the Shanghai Ranking Benchmarking & Comparisons
  • 19. Benchmarking & Comparisons ★ Example: benchmarking of the university of Granada with Spanish universities. ★ UGR performs well according to the number of citable papers, impact and collaboration, but has to improve the share of papers in first Quartile journals All universities perform better than Granada according based on the percentage of papers in First Quartile Journals
  • 20. ★ If we are working at universities it is important also to represent in a bivariate graph the position of our institution within our national university system Benchmarking & Comparisons Nr of Citable Papers Cat.NormalizedCitationImpact UGR Impact World Impact
  • 21. ★ Another question that we have to adress in our report is the thematic profile of our institutions; detecting best fields fields and disciplines ★ For this we must consider the use of different aggregation levels. Select at least one general level for an overview of the fields (for example Essential Science Indicators, 22 fields) and a more specific discipline level (for example Web of Science Categories, more than 200 categories). ★ Differentiate always between Science and Social Social Sciences / Humanities Fields and disciplines
  • 22. Fields and disciplines Cat.NormalizedCitationImpact Nr of Citable Papers 1 Clinical Medicine, 2 Physics, 3 Social Sciences, 4 Chemistry, 5 Geosciences, 6 Engineering, 7 Mathematics, 8 Psychology, 9 Agricultural Science, 10 Computer Sciences, 11 Ecology, 12 Biology, 13 Plant & Animal Science, 14 Neuroscience, 15 Space Science, ... We can see the general scientific profile of the UGR according ESI classification. In this case we can identify very productive but low impact areas (1, 3, 4); very productive and high impact areas (2); relatively productive but with high impact areas (15) and not productive and low impact areas (14, 17, ….) Take care when choosing a classification, for example ESI does not have an specific field for the Humanities Impact < World Impact < World
  • 23. Fields and disciplines ★ In this example we have the second level of presentation of the data at the disciplinary level, the WoS categories applied to the social sciences and the humanities.
  • 24. Combining fields and institutions ★ Another interesting option to complete the report is to compare the performance of our institution with other universities in different scientific fields. ★ At University of Granada we check its position in five different fields (Natural Science, Engineering, Health Sciences, Social Sciences and Humanities) and compare its results with the Spanish university system.
  • 25. Combining fields and institutions Nr of citable papers Category Normalized Citation Impact In this example we can see the position of the University of Granada in the field of Natural Sciences according to two different bibliometric indicators.
  • 26. ★ We use the number of projects and total funding in competitive programmes to contextualize our results. We consider two calls a) Spanish R&D National programmes and b) European funding from the European Commission. ★ Nr. of project and total funding are important indicators but also success rate (Nr of applications / Proposals selected for funding). It is important also to compare these indicators with national or UE average and do benchmarking with other institutions Non bibliometric Indicators: funding In order to complement the information provided by bibliometric indicators we can include information on other inputs
  • 27. Non bibliometric Indicators: funding ★ In the University of Granada we focus on national research programmes. In this example we can see the results of four Spanish universities in the last year. We can see that the University of Granada is the second university in Spain with a larger number of projects conceded
  • 28. Indicators at author level ★ For authors we have included the researchers with the highest H-index in Web of Science for different scientific categories
  • 29. Indicators at author level ★ Finally, we include the most outstanding researchers in "Google Scholar Profiles". We review all profiles and only include those with correct information. We have analyzed a total of 2000 University of Granada profiles and approved 1,700. These profiles have been classified into five different scientific areas. In the report we present a summary of the 200 most cited researchers Link to google scholar profile
  • 31. Transparency and availability of data ★ To promote institutional transparency, reproducibility, verification of results and data reuse by other departments, we share our data in an standardized way. ★ Livemetrics Portal. We have developed a portal that includes a dynamic version of the report with the main indicators
  • 34. Granada and Modena - Collaboration
  • 36. Daniel Torres-Salinas Bibliometrics in practice: how to generate reports for institutions.