SlideShare a Scribd company logo
1 of 43
Download to read offline
European Summer School| September 8, 2016
Societal Impact
Nicolas Robinson-Garcia Daniel Torres-Salinas
INGENIO (CSIC-UPV) Univ Navarra and Univ Granada
(ec3metrics & Medialab ugr)
www.ingenio.upv.es
Objectives
Understand the motivations that lead for
demands of societal impact indicators
1
Know the limitations and current problems
existing to meet such demands
2
Learn about potential uses of altmetrics to
showcase ‘aspects’ of societal impact
3
Index
Brief Intro
What is societal impact?
How can we measure societal impact?
Altmetrics and the ‘broader impacts’ of research
Potential uses of altmetrics
Food for thought
“The last 30 years have witnessed a slow, but
continuous process of reorganization of European
universities”
Geuna & Muscia, 2009
Brief intro
https://static.pexels.com/photos/30824/pexels-photo.jpg
Brief intro
Global competition and budget cuts
Accountability and transparency
Evaluation and funding
Brief intro
• Bibliometrics is seen as an answer to provide impact
measures within and even outside the scientific sphere
citation analysis – patent analysis – univ-industry collab
• In the recent years there has been a critical revision on
the limitations of these metrics
scientific impact ≠ societal impact
http://www.logic-canvas.com/wp-content/uploads/2013/11/Transparency-Glass.jpg
Brief intro
DORA Declaration
Against the use of journal based
indicators
Leiden Manifesto
Best practices on the use of
bibliometric indicators
The Metric Tide
UK Report on the role of bibliometric
indicators in research evaluation
What is societal impact?
Societal impact is usually referred as the
social, environmental, cultural or economic
benefits derived from academic activities
Teaching Research
Third
Mission
SOCIETAL IMPACT
What is societal impact?
Type of impact Output Activities
Scientific-
scholarly
Knowledge Growth Papers, books, reports,
cases, magazines
Increase and
organise the stock
of knowledgePublication outlets
Research networks
Workshops,
collaboration,
conferences,
committees
Build and convene
networks
Supply of human
capital
Societal
Cultural
Archives, museums,
festivals, exhibition
and performance,
events
Social Policy changes, social
benefits
Problem solving
Socioeconomic Patents, spin-offs,
start-ups, contracts,
consultancy
Support
entrepreneurialismTechnological
Source: Robinson-Garcia, van Leeuwen & Rafols, in progress
How can we measure
societal impact?
Socioeconomic & Technological Impact
• Largest body of literature
• Synergies between industry and university
• Tracking of knowledge transfer
Social Impact
• Translational research
• Environmental research
Cultural Impact
• Field-specific
• Largely unexplored
• Affects mainly the Social Sciences & Humanities
How can we measure
societal impact?
The attribution problem
Researcher
publishes a paper
Researcher gives
lectures?
Media coverage?
Patent protection?
Advocacy?
Local council
implements new
policy
Increase on
tourism rates
Employment rates
increase in industry
Creates a spin-off?
How can we measure
societal impact?
The attribution problem
Researcher
publishes a paper
undertakes a
research project?
Local council
implements new
policy
Researcher gives
lectures?
Media coverage?
Patent protection?
Advocacy?
Increase on
tourism rates
Employment rates
increase in industry
Creates a spin-off?
How can we measure
societal impact?
An example. El Cabanyal
The district
•Old village, now a district
•Architectonic value
The sting
•Access to the beach
•Salvem el Cabanyal
The link?
•Few papers (National journals)
•Not all researchers involved are authors
The attribution problem
GOALS
How can we measure
societal impact? UK REF
Selective allocation of research funding to Higher
Education Institutions
Produce evidence of benefits of public investment in
research
Provide benchmarking information and reputational
yardsticks
King’s College London & Digital Science, 2015
First attempt to allocate funding based on non-
academic impact
How can we measure
societal impact? UK REF
King’s College London & Digital Science, 2015
“Impact is defined as ‘any effect on, change or
benefit to the economy, society, culture, public
policy or services, health, the environment or
quality of life, beyond academia’ (REF, 2011)”
How can we measure
societal impact? UK REF
PANEL A
Biomedical & Health Sciences
6 UoA – 1586 case studies
PANEL D
Humanities & Arts
10 UoA – 1617 case studies
PANEL C
Social Sciences
10 UoA – 1965 case studies
PANEL B
Life & Earth Sciences,
Engineering, Physics, Maths
9 UoA – 1469 case studies
Assessing societal impact through impact case
studies
How can we measure
societal impact? UK REF
IMPACT CASE STUDY TEMPLATE
1. Summary of the impact
2. Description of the underpinning research
3. References of the research
4. Details of the impact
5. Sources to corroborate the impact
How can we measure
societal impact? UK REF
Case studies by type of impact
How can we measure
societal impact? UK REF
Case studies by type of impact
How can we measure
societal impact? UK REF
Ratings by type of impact
How can we measure
societal impact? UK REF
Wilsdon et al., 2015
“[…] for the impact component
of the REF, it is not currently
feasible to use quantitative
indicators in place of narrative
impact case studies, or the
impact template”
Altmetrics and the ‘broader’
impacts of science
Wilsdon et al., 2015
“[…] wider use of quantitative
indicators, and the emergence of
alternative metrics for societal
impact, could support the
transition to a more open,
accountable and outward-facing
research system”
Altmetrics and the ‘broader’
impacts of science
Classic perspective Author perspective
Two perspectives
Case: societal impact
indicators at UGR
Multidimensional evaluation using biplot.
Research at the University of Granada
Source: Milanes-Guisado, Y. (2015). Multidimensional evaluation of research. Micro analysis
at the university of Granada during the period 2009-2013 [doctoral thesis]. Universidad de
Granada. Supervisors: Daniel Torres-Salinas, José Navarrete, Francisco Solis
Research questions:
• ¿What is the metric
profile of research
groups taking into
account bibliometric
and non-bibliometric
indicators?
Methodology:
• Time frame: 2009-2013
• Analysis unit:
161 research groups = 2216 res.
• Methods:
Correlations
biplot analysis
Case: societal impact
indicators at UGR
We calculated
46 indicators
and classified them into
9 different dimensions
We reduced and make a biplot analysis
Multidimensional evaluation using biplot.
Research at the University of Granada
Case: societal impact
indicators at UGR
Altmetrics
Funding and projects
Scientific output
Knowledge transfer
Media and news impact
Scientific impact – Citations
Teaching dissemination
9 Dimensions of academic
activity and impact
Case: societal impact
indicators at UGR
Altmetrics
Funding and projects
Scientific output
Knowledge transfer
Media and news impact
Scientific impact – Citations
Teaching dissemination
Societal impact related dimensions
Case: societal impact
indicators at UGR
Altmetrics
Google +
Mentions
Facebook
Likes
Twitter
Indicators
Knowledge
Transfer
Patents
Contracts
Spin offs
Media and
News Impact
Newspapers
contributions
Public
exposure
Mentions in
Newspapers
Teaching
Dissemination
Teaching
Material
Innovation
Projects
Some examples of “societal impact” indicators
Case: societal impact
indicators at UGR
Some examples of the information sources
Case: societal impact
indicators at UGR
Method remarks
1) Calculation of 46 indicators for research
groups in four areas
2) Normalization and standarization of
indicators values using z-scores
3) Representation of dimensions and groups
into two dimensions using biplot analysis
4) Reducing the biplot with clockwise
representation
Now we are going to see an example in
Exact Sciences
Potential applications
of altmetrics
These are the
dimensions
(blue)
These are the
research groups
(green)
BIPLOT: “EXACT SCIENCES 52” Research groups
Potential applications
of altmetrics
Teaching Dissemination
Media and News Impact
Knowledge Transfers
Altmetrics These are cluster and
dimensions used to identify
SOCIETAL IMPACT
Potential applications
of altmetrics
Altmetrics
What we see is that most of the research
groups are in dimensions related to
traditional publications and the impact
system (ie: Web of Science)
Scientific Impact
Scientific output
Potential applications
of altmetrics
Altmetrics
DOES ALTMETRICS
MEASURE
SOCIETAL IMPACT?
We see that altmetrics is
more related to scientific
impact and scientific output
(at the UGR)
Scientific Impact
Scientific output
Potential applications
of altmetrics
Teaching Dissemination
Media and News Impact
Knowledge Transfers
So, for Exact Sciences at the University of
Granada we have a really clear quadrant
related to SOCIETAL IMPACT
Potential applications
of altmetrics
We have identified four groups with a
clear Societal Impact orientation
FQM325
FQM171
FQM338
FQM325
FQM324
Teaching dissemination
Media and news impact
Funding and projects
National scientific output
Knowledge
transfer
Scientific impact - Citations
International scientific output
Altmetrics and webmetrics
Finally this is the general and multidimensional
overview of the exact sciences at the UGR
Potential applications
of altmetrics
Social
engagement
through Twitter
1.Select a researcher
2.Identify its follower/followee network
3.Identify different communities of
contacts
4.Identify geographical/cognitive/…
proximities
Potential applications
of altmetrics
Social
engagement
through Twitter
L Waltman’s Twitter
network
Potential applications
of altmetrics
L Waltman’s Twitter
network
Social
engagement
through Twitter
• Threats derived from a purely qualitative
assessment
Peripheries and prestige
• Threats derived from purely quantitative
assessment
Biases by type and scope of impacts
Visualizations as a balanced solution
Food for thought
European Summer School | September 8, 2016
Questions?
Nicolas Robinson-Garcia Daniel Torres-Salinas
INGENIO (CSIC-UPV) EC3Metrics SL
www.ingenio.upv.es
European Summer School | September 8, 2016
Especial thanks to Yusnelkis Milanés
for providing data from her PhD thesis
www.ingenio.upv.es
Source: Milanes-Guisado, Y. (2015). Multidimensional evaluation of research. Micro analysis
at the university of Granada during the period 2009-2013 [doctoral thesis]. Universidad de
Granada. Supervisors: Daniel Torres-Salinas, José Navarrete, Francisco Solis

More Related Content

What's hot

The 2030 Agenda for Sustainable Development: From Commitment to Delivery
The 2030 Agenda for Sustainable Development: From Commitment to DeliveryThe 2030 Agenda for Sustainable Development: From Commitment to Delivery
The 2030 Agenda for Sustainable Development: From Commitment to Delivery
SDGsPlus
 
Sustainable development powerpoint
Sustainable development powerpointSustainable development powerpoint
Sustainable development powerpoint
Pamela Hill
 

What's hot (20)

csr - current trends and opportunities
csr - current trends and opportunitiescsr - current trends and opportunities
csr - current trends and opportunities
 
Corporate Social Responsibility
Corporate Social ResponsibilityCorporate Social Responsibility
Corporate Social Responsibility
 
Environmental Protection and Corporate Social Responsibility PPT 2019
Environmental Protection and Corporate Social Responsibility PPT 2019Environmental Protection and Corporate Social Responsibility PPT 2019
Environmental Protection and Corporate Social Responsibility PPT 2019
 
Carbon footprint
Carbon footprintCarbon footprint
Carbon footprint
 
Latest trends in information technology
Latest trends in information technologyLatest trends in information technology
Latest trends in information technology
 
IoT presentation
IoT presentationIoT presentation
IoT presentation
 
Green Business Practices
Green Business PracticesGreen Business Practices
Green Business Practices
 
Carbon foot print.ppt
Carbon  foot print.pptCarbon  foot print.ppt
Carbon foot print.ppt
 
Corporate social responsibility: How economic is it to be socially responsible??
Corporate social responsibility: How economic is it to be socially responsible??Corporate social responsibility: How economic is it to be socially responsible??
Corporate social responsibility: How economic is it to be socially responsible??
 
Internet of Things and its applications
Internet of Things and its applicationsInternet of Things and its applications
Internet of Things and its applications
 
Community service for college students
Community service for college studentsCommunity service for college students
Community service for college students
 
The 2030 Agenda for Sustainable Development: From Commitment to Delivery
The 2030 Agenda for Sustainable Development: From Commitment to DeliveryThe 2030 Agenda for Sustainable Development: From Commitment to Delivery
The 2030 Agenda for Sustainable Development: From Commitment to Delivery
 
Sustainable development goal 12
Sustainable development goal 12Sustainable development goal 12
Sustainable development goal 12
 
Presentation on GREEN TECHNOLOGY
Presentation on GREEN TECHNOLOGYPresentation on GREEN TECHNOLOGY
Presentation on GREEN TECHNOLOGY
 
Internet of things startup basic
Internet of things  startup basicInternet of things  startup basic
Internet of things startup basic
 
FUTURE OF IOT
FUTURE OF IOTFUTURE OF IOT
FUTURE OF IOT
 
Internet of things (IoT)- Introduction, Utilities, Applications
Internet of things (IoT)- Introduction, Utilities, ApplicationsInternet of things (IoT)- Introduction, Utilities, Applications
Internet of things (IoT)- Introduction, Utilities, Applications
 
IoT Sensing and Actuation.pdf
 IoT Sensing and Actuation.pdf IoT Sensing and Actuation.pdf
IoT Sensing and Actuation.pdf
 
IoT Solutions for Smart Energy Smart Grid and Smart Utility Applications
IoT Solutions for Smart Energy Smart Grid and Smart Utility ApplicationsIoT Solutions for Smart Energy Smart Grid and Smart Utility Applications
IoT Solutions for Smart Energy Smart Grid and Smart Utility Applications
 
Sustainable development powerpoint
Sustainable development powerpointSustainable development powerpoint
Sustainable development powerpoint
 

Viewers also liked

Bibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaboratorsBibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaborators
Torres Salinas
 
Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Gestión y Monitorización de la información y el impacto científico: Perfiles,...Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Torres Salinas
 

Viewers also liked (16)

Inicitativas empresariales relacionadas con la evaluación de la ciencia el ...
Inicitativas empresariales relacionadas con la evaluación de la ciencia   el ...Inicitativas empresariales relacionadas con la evaluación de la ciencia   el ...
Inicitativas empresariales relacionadas con la evaluación de la ciencia el ...
 
Cómo se evalúa y se progresa en la carrera científica
Cómo se evalúa y se progresa en la carrera científicaCómo se evalúa y se progresa en la carrera científica
Cómo se evalúa y se progresa en la carrera científica
 
Visiblegrinugr 150424024156-conversion-gate01
Visiblegrinugr 150424024156-conversion-gate01Visiblegrinugr 150424024156-conversion-gate01
Visiblegrinugr 150424024156-conversion-gate01
 
Can we use altmetric at institutional level?
Can we use altmetric at institutional level?Can we use altmetric at institutional level?
Can we use altmetric at institutional level?
 
Cómo se evalúa y se progresa en la carrera científica [versión 2.0]
Cómo se evalúa y se progresa en la  carrera científica [versión 2.0]Cómo se evalúa y se progresa en la  carrera científica [versión 2.0]
Cómo se evalúa y se progresa en la carrera científica [versión 2.0]
 
La evaluación abierta de la investigación como elemento dinamizador
La evaluación abierta de la investigación como elemento dinamizadorLa evaluación abierta de la investigación como elemento dinamizador
La evaluación abierta de la investigación como elemento dinamizador
 
Bibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaboratorsBibliometric solutions for identifying potential collaborators
Bibliometric solutions for identifying potential collaborators
 
Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...
Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...
Actas de la Jornada #appugr: aplicaciones móviles orientadas a la investigaci...
 
Pautas la elaboracion de proyectos: convocatoria retos y excelencia
Pautas la elaboracion de proyectos: convocatoria retos y excelenciaPautas la elaboracion de proyectos: convocatoria retos y excelencia
Pautas la elaboracion de proyectos: convocatoria retos y excelencia
 
Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Gestión y Monitorización de la información y el impacto científico: Perfiles,...Gestión y Monitorización de la información y el impacto científico: Perfiles,...
Gestión y Monitorización de la información y el impacto científico: Perfiles,...
 
Aportaciones presentadas en la I Reunión de Servicios de Evaluación Científica
Aportaciones presentadas en la I Reunión de Servicios de Evaluación CientíficaAportaciones presentadas en la I Reunión de Servicios de Evaluación Científica
Aportaciones presentadas en la I Reunión de Servicios de Evaluación Científica
 
El cv científico y su visibilidad: formatos, gestión y difusión en Internet
El cv científico y su visibilidad: formatos, gestión y difusión en InternetEl cv científico y su visibilidad: formatos, gestión y difusión en Internet
El cv científico y su visibilidad: formatos, gestión y difusión en Internet
 
Altmetric beauties. ¿Cuáles son los trabajos científicos con mayor impacto e...
Altmetric beauties. ¿Cuáles son los trabajos científicos con  mayor impacto e...Altmetric beauties. ¿Cuáles son los trabajos científicos con  mayor impacto e...
Altmetric beauties. ¿Cuáles son los trabajos científicos con mayor impacto e...
 
Cómo seleccionar una revista científica en comunicación
Cómo seleccionar una revista científica en comunicaciónCómo seleccionar una revista científica en comunicación
Cómo seleccionar una revista científica en comunicación
 
10 mejores apps para científicos #investigacionmovil
10 mejores apps para científicos #investigacionmovil10 mejores apps para científicos #investigacionmovil
10 mejores apps para científicos #investigacionmovil
 
Diez reglas de oro para publicar en revistas de impacto
Diez reglas de oro para publicar en revistas de impactoDiez reglas de oro para publicar en revistas de impacto
Diez reglas de oro para publicar en revistas de impacto
 

Similar to Societal Impact

Oecd uni indcollaboration_ch3_website
Oecd uni indcollaboration_ch3_websiteOecd uni indcollaboration_ch3_website
Oecd uni indcollaboration_ch3_website
slideshow19
 
Webinar series: Public engagement, education and outreach for carbon capture ...
Webinar series: Public engagement, education and outreach for carbon capture ...Webinar series: Public engagement, education and outreach for carbon capture ...
Webinar series: Public engagement, education and outreach for carbon capture ...
Global CCS Institute
 
Oecd uni indcollaboration_ch1_website
Oecd uni indcollaboration_ch1_websiteOecd uni indcollaboration_ch1_website
Oecd uni indcollaboration_ch1_website
slideshow19
 

Similar to Societal Impact (20)

Societal Impact
Societal ImpactSocietal Impact
Societal Impact
 
20190527_Marc Vanholsbeeck_Open Science monitoring and the notion of research...
20190527_Marc Vanholsbeeck_Open Science monitoring and the notion of research...20190527_Marc Vanholsbeeck_Open Science monitoring and the notion of research...
20190527_Marc Vanholsbeeck_Open Science monitoring and the notion of research...
 
Salazar - Towards more inclusive science and innovations indicators
Salazar - Towards more inclusive science and innovations indicatorsSalazar - Towards more inclusive science and innovations indicators
Salazar - Towards more inclusive science and innovations indicators
 
Aligning scientific impact and societal relevance: The roles of academic enga...
Aligning scientific impact and societal relevance: The roles of academic enga...Aligning scientific impact and societal relevance: The roles of academic enga...
Aligning scientific impact and societal relevance: The roles of academic enga...
 
Seminari CRICC : Avaluació de la recerca.
Seminari CRICC : Avaluació de la recerca. Seminari CRICC : Avaluació de la recerca.
Seminari CRICC : Avaluació de la recerca.
 
R. Malagrida: Responsible Research and Innovation, a new paradigm in Horizon ...
R. Malagrida: Responsible Research and Innovation, a new paradigm in Horizon ...R. Malagrida: Responsible Research and Innovation, a new paradigm in Horizon ...
R. Malagrida: Responsible Research and Innovation, a new paradigm in Horizon ...
 
Towards indicators for 'opening up' science and technology policy
Towards indicators for 'opening up' science and technology policyTowards indicators for 'opening up' science and technology policy
Towards indicators for 'opening up' science and technology policy
 
SIOR: recognition of the social impact of research through open access resources
SIOR: recognition of the social impact of research through open access resourcesSIOR: recognition of the social impact of research through open access resources
SIOR: recognition of the social impact of research through open access resources
 
Oecd uni indcollaboration_ch3_website
Oecd uni indcollaboration_ch3_websiteOecd uni indcollaboration_ch3_website
Oecd uni indcollaboration_ch3_website
 
Webinar series: Public engagement, education and outreach for carbon capture ...
Webinar series: Public engagement, education and outreach for carbon capture ...Webinar series: Public engagement, education and outreach for carbon capture ...
Webinar series: Public engagement, education and outreach for carbon capture ...
 
Rafols - Towards more inclusive STI indicators
Rafols - Towards more inclusive STI indicatorsRafols - Towards more inclusive STI indicators
Rafols - Towards more inclusive STI indicators
 
Citizen Science in Open Science context: measuring & understanding impacts of...
Citizen Science in Open Science context: measuring & understanding impacts of...Citizen Science in Open Science context: measuring & understanding impacts of...
Citizen Science in Open Science context: measuring & understanding impacts of...
 
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&IData4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
Data4Impact expert workshop. The Conceptual Framework of Societal Impact of R&I
 
Why responsible innovation vonschomberg-ext.[6689]
Why responsible innovation vonschomberg-ext.[6689]Why responsible innovation vonschomberg-ext.[6689]
Why responsible innovation vonschomberg-ext.[6689]
 
Oecd uni indcollaboration_ch1_website
Oecd uni indcollaboration_ch1_websiteOecd uni indcollaboration_ch1_website
Oecd uni indcollaboration_ch1_website
 
Making a difference: How can we demonstrate the impact of learning technology...
Making a difference: How can we demonstrate the impact of learning technology...Making a difference: How can we demonstrate the impact of learning technology...
Making a difference: How can we demonstrate the impact of learning technology...
 
Liam Cleere University College Dublin’s Senior Manager for Research Analytics...
Liam Cleere University College Dublin’s Senior Manager for Research Analytics...Liam Cleere University College Dublin’s Senior Manager for Research Analytics...
Liam Cleere University College Dublin’s Senior Manager for Research Analytics...
 
Research to Innovation
Research to InnovationResearch to Innovation
Research to Innovation
 
SSH & the City. From measuring societal impact to mapping social engagement
SSH & the City. From measuring societal impact to mapping social engagementSSH & the City. From measuring societal impact to mapping social engagement
SSH & the City. From measuring societal impact to mapping social engagement
 
Oxford_ImpactConference_2023_Rets_et_al.pptx
Oxford_ImpactConference_2023_Rets_et_al.pptxOxford_ImpactConference_2023_Rets_et_al.pptx
Oxford_ImpactConference_2023_Rets_et_al.pptx
 

More from Torres Salinas

Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Torres Salinas
 
Curso 4º ed, cómo publicar en revistas científicas de impacto consejos y reg...
Curso 4º ed, cómo publicar en revistas científicas de impacto  consejos y reg...Curso 4º ed, cómo publicar en revistas científicas de impacto  consejos y reg...
Curso 4º ed, cómo publicar en revistas científicas de impacto consejos y reg...
Torres Salinas
 
Analyzing the citation characteristics of books edited books, book series and...
Analyzing the citation characteristics of books edited books, book series and...Analyzing the citation characteristics of books edited books, book series and...
Analyzing the citation characteristics of books edited books, book series and...
Torres Salinas
 

More from Torres Salinas (14)

Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...
Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...
Bibliometrics in practice: how to generate reports for institutions - v2.0 / ...
 
Journal impact measures: the Impact Factor
Journal impact measures: the Impact FactorJournal impact measures: the Impact Factor
Journal impact measures: the Impact Factor
 
Bibliometrics in practice: how to generate reports for institutions.
Bibliometrics in practice: how to generate reports for institutions.Bibliometrics in practice: how to generate reports for institutions.
Bibliometrics in practice: how to generate reports for institutions.
 
Cómo seleccionar y publicar en revistas de impacto en Ciencias Sociales
Cómo seleccionar y publicar en revistas de impacto en Ciencias SocialesCómo seleccionar y publicar en revistas de impacto en Ciencias Sociales
Cómo seleccionar y publicar en revistas de impacto en Ciencias Sociales
 
La Segunda Guerra Mundial en los videojuegos
La Segunda Guerra Mundial en los videojuegosLa Segunda Guerra Mundial en los videojuegos
La Segunda Guerra Mundial en los videojuegos
 
EC3metrics: Emprender y transferir desde las ciencias sociales
EC3metrics: Emprender y transferir desde las ciencias socialesEC3metrics: Emprender y transferir desde las ciencias sociales
EC3metrics: Emprender y transferir desde las ciencias sociales
 
How many citations are there in the Data Citation Index?
How many citations are there in the Data Citation Index?How many citations are there in the Data Citation Index?
How many citations are there in the Data Citation Index?
 
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
Evaluación editorial: Filosofía y Procedimiento. CRECS 2014
 
Traspasando el muro acceso abierto a la ciencia
Traspasando el muro acceso abierto a la cienciaTraspasando el muro acceso abierto a la ciencia
Traspasando el muro acceso abierto a la ciencia
 
Torres salinas. egociencia reputación on-line para científicos
Torres salinas. egociencia reputación on-line para científicosTorres salinas. egociencia reputación on-line para científicos
Torres salinas. egociencia reputación on-line para científicos
 
Altmetrics indicadores, utilidades y limitaciones
Altmetrics indicadores, utilidades y limitacionesAltmetrics indicadores, utilidades y limitaciones
Altmetrics indicadores, utilidades y limitaciones
 
Curso 4º ed, cómo publicar en revistas científicas de impacto consejos y reg...
Curso 4º ed, cómo publicar en revistas científicas de impacto  consejos y reg...Curso 4º ed, cómo publicar en revistas científicas de impacto  consejos y reg...
Curso 4º ed, cómo publicar en revistas científicas de impacto consejos y reg...
 
Most borrowed is most cited? Library loan statistics as a proxy for monograph...
Most borrowed is most cited? Library loan statistics as a proxy for monograph...Most borrowed is most cited? Library loan statistics as a proxy for monograph...
Most borrowed is most cited? Library loan statistics as a proxy for monograph...
 
Analyzing the citation characteristics of books edited books, book series and...
Analyzing the citation characteristics of books edited books, book series and...Analyzing the citation characteristics of books edited books, book series and...
Analyzing the citation characteristics of books edited books, book series and...
 

Recently uploaded

Neurulation and the formation of the neural tube
Neurulation and the formation of the neural tubeNeurulation and the formation of the neural tube
Neurulation and the formation of the neural tube
SaadHumayun7
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
heathfieldcps1
 
IATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdffIATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdff
17thcssbs2
 

Recently uploaded (20)

Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
Basic Civil Engineering notes on Transportation Engineering, Modes of Transpo...
 
Matatag-Curriculum and the 21st Century Skills Presentation.pptx
Matatag-Curriculum and the 21st Century Skills Presentation.pptxMatatag-Curriculum and the 21st Century Skills Presentation.pptx
Matatag-Curriculum and the 21st Century Skills Presentation.pptx
 
Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17
 
UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...
UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...
UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...
 
Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).
 
Championnat de France de Tennis de table/
Championnat de France de Tennis de table/Championnat de France de Tennis de table/
Championnat de France de Tennis de table/
 
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdfPost Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
 
[GDSC YCCE] Build with AI Online Presentation
[GDSC YCCE] Build with AI Online Presentation[GDSC YCCE] Build with AI Online Presentation
[GDSC YCCE] Build with AI Online Presentation
 
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
Operations Management - Book1.p  - Dr. Abdulfatah A. SalemOperations Management - Book1.p  - Dr. Abdulfatah A. Salem
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
 
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
 
Neurulation and the formation of the neural tube
Neurulation and the formation of the neural tubeNeurulation and the formation of the neural tube
Neurulation and the formation of the neural tube
 
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
 
Mbaye_Astou.Education Civica_Human Rights.pptx
Mbaye_Astou.Education Civica_Human Rights.pptxMbaye_Astou.Education Civica_Human Rights.pptx
Mbaye_Astou.Education Civica_Human Rights.pptx
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
 
IATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdffIATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdff
 
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdfPost Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
 
The Benefits and Challenges of Open Educational Resources
The Benefits and Challenges of Open Educational ResourcesThe Benefits and Challenges of Open Educational Resources
The Benefits and Challenges of Open Educational Resources
 
size separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceuticssize separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceutics
 
“O BEIJO” EM ARTE .
“O BEIJO” EM ARTE                       .“O BEIJO” EM ARTE                       .
“O BEIJO” EM ARTE .
 
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdfDanh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
Danh sách HSG Bộ môn cấp trường - Cấp THPT.pdf
 

Societal Impact

  • 1. European Summer School| September 8, 2016 Societal Impact Nicolas Robinson-Garcia Daniel Torres-Salinas INGENIO (CSIC-UPV) Univ Navarra and Univ Granada (ec3metrics & Medialab ugr) www.ingenio.upv.es
  • 2. Objectives Understand the motivations that lead for demands of societal impact indicators 1 Know the limitations and current problems existing to meet such demands 2 Learn about potential uses of altmetrics to showcase ‘aspects’ of societal impact 3
  • 3. Index Brief Intro What is societal impact? How can we measure societal impact? Altmetrics and the ‘broader impacts’ of research Potential uses of altmetrics Food for thought
  • 4. “The last 30 years have witnessed a slow, but continuous process of reorganization of European universities” Geuna & Muscia, 2009 Brief intro https://static.pexels.com/photos/30824/pexels-photo.jpg
  • 5. Brief intro Global competition and budget cuts Accountability and transparency Evaluation and funding
  • 6. Brief intro • Bibliometrics is seen as an answer to provide impact measures within and even outside the scientific sphere citation analysis – patent analysis – univ-industry collab • In the recent years there has been a critical revision on the limitations of these metrics scientific impact ≠ societal impact http://www.logic-canvas.com/wp-content/uploads/2013/11/Transparency-Glass.jpg
  • 7. Brief intro DORA Declaration Against the use of journal based indicators Leiden Manifesto Best practices on the use of bibliometric indicators The Metric Tide UK Report on the role of bibliometric indicators in research evaluation
  • 8. What is societal impact? Societal impact is usually referred as the social, environmental, cultural or economic benefits derived from academic activities Teaching Research Third Mission SOCIETAL IMPACT
  • 9. What is societal impact? Type of impact Output Activities Scientific- scholarly Knowledge Growth Papers, books, reports, cases, magazines Increase and organise the stock of knowledgePublication outlets Research networks Workshops, collaboration, conferences, committees Build and convene networks Supply of human capital Societal Cultural Archives, museums, festivals, exhibition and performance, events Social Policy changes, social benefits Problem solving Socioeconomic Patents, spin-offs, start-ups, contracts, consultancy Support entrepreneurialismTechnological Source: Robinson-Garcia, van Leeuwen & Rafols, in progress
  • 10. How can we measure societal impact? Socioeconomic & Technological Impact • Largest body of literature • Synergies between industry and university • Tracking of knowledge transfer Social Impact • Translational research • Environmental research Cultural Impact • Field-specific • Largely unexplored • Affects mainly the Social Sciences & Humanities
  • 11. How can we measure societal impact? The attribution problem Researcher publishes a paper Researcher gives lectures? Media coverage? Patent protection? Advocacy? Local council implements new policy Increase on tourism rates Employment rates increase in industry Creates a spin-off?
  • 12. How can we measure societal impact? The attribution problem Researcher publishes a paper undertakes a research project? Local council implements new policy Researcher gives lectures? Media coverage? Patent protection? Advocacy? Increase on tourism rates Employment rates increase in industry Creates a spin-off?
  • 13. How can we measure societal impact? An example. El Cabanyal The district •Old village, now a district •Architectonic value The sting •Access to the beach •Salvem el Cabanyal The link? •Few papers (National journals) •Not all researchers involved are authors The attribution problem
  • 14. GOALS How can we measure societal impact? UK REF Selective allocation of research funding to Higher Education Institutions Produce evidence of benefits of public investment in research Provide benchmarking information and reputational yardsticks King’s College London & Digital Science, 2015
  • 15. First attempt to allocate funding based on non- academic impact How can we measure societal impact? UK REF King’s College London & Digital Science, 2015 “Impact is defined as ‘any effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’ (REF, 2011)”
  • 16. How can we measure societal impact? UK REF PANEL A Biomedical & Health Sciences 6 UoA – 1586 case studies PANEL D Humanities & Arts 10 UoA – 1617 case studies PANEL C Social Sciences 10 UoA – 1965 case studies PANEL B Life & Earth Sciences, Engineering, Physics, Maths 9 UoA – 1469 case studies
  • 17. Assessing societal impact through impact case studies How can we measure societal impact? UK REF IMPACT CASE STUDY TEMPLATE 1. Summary of the impact 2. Description of the underpinning research 3. References of the research 4. Details of the impact 5. Sources to corroborate the impact
  • 18. How can we measure societal impact? UK REF Case studies by type of impact
  • 19. How can we measure societal impact? UK REF Case studies by type of impact
  • 20. How can we measure societal impact? UK REF Ratings by type of impact
  • 21. How can we measure societal impact? UK REF Wilsdon et al., 2015 “[…] for the impact component of the REF, it is not currently feasible to use quantitative indicators in place of narrative impact case studies, or the impact template”
  • 22. Altmetrics and the ‘broader’ impacts of science Wilsdon et al., 2015 “[…] wider use of quantitative indicators, and the emergence of alternative metrics for societal impact, could support the transition to a more open, accountable and outward-facing research system”
  • 23. Altmetrics and the ‘broader’ impacts of science Classic perspective Author perspective Two perspectives
  • 24. Case: societal impact indicators at UGR Multidimensional evaluation using biplot. Research at the University of Granada Source: Milanes-Guisado, Y. (2015). Multidimensional evaluation of research. Micro analysis at the university of Granada during the period 2009-2013 [doctoral thesis]. Universidad de Granada. Supervisors: Daniel Torres-Salinas, José Navarrete, Francisco Solis Research questions: • ¿What is the metric profile of research groups taking into account bibliometric and non-bibliometric indicators? Methodology: • Time frame: 2009-2013 • Analysis unit: 161 research groups = 2216 res. • Methods: Correlations biplot analysis
  • 25. Case: societal impact indicators at UGR We calculated 46 indicators and classified them into 9 different dimensions We reduced and make a biplot analysis Multidimensional evaluation using biplot. Research at the University of Granada
  • 26. Case: societal impact indicators at UGR Altmetrics Funding and projects Scientific output Knowledge transfer Media and news impact Scientific impact – Citations Teaching dissemination 9 Dimensions of academic activity and impact
  • 27. Case: societal impact indicators at UGR Altmetrics Funding and projects Scientific output Knowledge transfer Media and news impact Scientific impact – Citations Teaching dissemination Societal impact related dimensions
  • 28. Case: societal impact indicators at UGR Altmetrics Google + Mentions Facebook Likes Twitter Indicators Knowledge Transfer Patents Contracts Spin offs Media and News Impact Newspapers contributions Public exposure Mentions in Newspapers Teaching Dissemination Teaching Material Innovation Projects Some examples of “societal impact” indicators
  • 29. Case: societal impact indicators at UGR Some examples of the information sources
  • 30. Case: societal impact indicators at UGR Method remarks 1) Calculation of 46 indicators for research groups in four areas 2) Normalization and standarization of indicators values using z-scores 3) Representation of dimensions and groups into two dimensions using biplot analysis 4) Reducing the biplot with clockwise representation Now we are going to see an example in Exact Sciences
  • 31. Potential applications of altmetrics These are the dimensions (blue) These are the research groups (green) BIPLOT: “EXACT SCIENCES 52” Research groups
  • 32. Potential applications of altmetrics Teaching Dissemination Media and News Impact Knowledge Transfers Altmetrics These are cluster and dimensions used to identify SOCIETAL IMPACT
  • 33. Potential applications of altmetrics Altmetrics What we see is that most of the research groups are in dimensions related to traditional publications and the impact system (ie: Web of Science) Scientific Impact Scientific output
  • 34. Potential applications of altmetrics Altmetrics DOES ALTMETRICS MEASURE SOCIETAL IMPACT? We see that altmetrics is more related to scientific impact and scientific output (at the UGR) Scientific Impact Scientific output
  • 35. Potential applications of altmetrics Teaching Dissemination Media and News Impact Knowledge Transfers So, for Exact Sciences at the University of Granada we have a really clear quadrant related to SOCIETAL IMPACT
  • 36. Potential applications of altmetrics We have identified four groups with a clear Societal Impact orientation FQM325 FQM171 FQM338 FQM325 FQM324
  • 37. Teaching dissemination Media and news impact Funding and projects National scientific output Knowledge transfer Scientific impact - Citations International scientific output Altmetrics and webmetrics Finally this is the general and multidimensional overview of the exact sciences at the UGR
  • 38. Potential applications of altmetrics Social engagement through Twitter 1.Select a researcher 2.Identify its follower/followee network 3.Identify different communities of contacts 4.Identify geographical/cognitive/… proximities
  • 39. Potential applications of altmetrics Social engagement through Twitter L Waltman’s Twitter network
  • 40. Potential applications of altmetrics L Waltman’s Twitter network Social engagement through Twitter
  • 41. • Threats derived from a purely qualitative assessment Peripheries and prestige • Threats derived from purely quantitative assessment Biases by type and scope of impacts Visualizations as a balanced solution Food for thought
  • 42. European Summer School | September 8, 2016 Questions? Nicolas Robinson-Garcia Daniel Torres-Salinas INGENIO (CSIC-UPV) EC3Metrics SL www.ingenio.upv.es
  • 43. European Summer School | September 8, 2016 Especial thanks to Yusnelkis Milanés for providing data from her PhD thesis www.ingenio.upv.es Source: Milanes-Guisado, Y. (2015). Multidimensional evaluation of research. Micro analysis at the university of Granada during the period 2009-2013 [doctoral thesis]. Universidad de Granada. Supervisors: Daniel Torres-Salinas, José Navarrete, Francisco Solis