This document provides an overview of metrics and tools used to assess scholarly impact and discusses current issues and trends. It covers journal-level metrics like journal impact factor and eigenfactor scores, article-level metrics including citations and altmetrics, and author-level metrics like the h-index. It explains how and where to find these various metrics, and highlights some alternative new metrics and changing expectations around research assessment and demonstrating broader impact.
Updated 30/01/2015
This session included discussions around the value of bibliometrics for individual performance management/promotion and the REF.
What are bibliometrics?
Journal metrics
Personal metrics
Article level metrics and altmetrics
Presented to members of the Psychology department as part of the New Tricks Seminar series (February 2016)
• journal metrics using WoS and Scopus
• article level metrics in WoS, Scopus and Google Scholar, and from publishers and the differences in each. Touch on altmetrics.
• author metrics in the above. Touch on Publish or Perish
Tanya Williamson, Academic Liaison Librarian
Reputation and bibliometric approaches to identifying the most influential journals to which a scholar should submit his or her research for maximum impact and influence.
A presentation delivered online to the Mountain Plains Management Conference at Cedar City, UT on Oct. 18, 2013.
Presented by: Jon Ritterbush of the Calvin T. Ryan Library at the University of Nebraska-Kearney.
Quality Assurance for Journal GuidanceSmriti Arora
Definitions
What is the need for quality assurance in journals ?
Type of journals
Bibliometric indicators
How to identify credible journals ?
Predatory/cloned journals
Updated 30/01/2015
This session included discussions around the value of bibliometrics for individual performance management/promotion and the REF.
What are bibliometrics?
Journal metrics
Personal metrics
Article level metrics and altmetrics
Presented to members of the Psychology department as part of the New Tricks Seminar series (February 2016)
• journal metrics using WoS and Scopus
• article level metrics in WoS, Scopus and Google Scholar, and from publishers and the differences in each. Touch on altmetrics.
• author metrics in the above. Touch on Publish or Perish
Tanya Williamson, Academic Liaison Librarian
Reputation and bibliometric approaches to identifying the most influential journals to which a scholar should submit his or her research for maximum impact and influence.
A presentation delivered online to the Mountain Plains Management Conference at Cedar City, UT on Oct. 18, 2013.
Presented by: Jon Ritterbush of the Calvin T. Ryan Library at the University of Nebraska-Kearney.
Quality Assurance for Journal GuidanceSmriti Arora
Definitions
What is the need for quality assurance in journals ?
Type of journals
Bibliometric indicators
How to identify credible journals ?
Predatory/cloned journals
Durham Researcher Development Programme 2015-16: Bibliometric Research Indica...Jamie Bisset
There is an ever-increasing need to make your research more visible as you establish your career, and metrics to measure your research performance when it comes to thinking about promotion and probation.
This session will focus on bibliometric research indicators (such as the Journal Impact Factor and SCImago, author metrics such as the h-index and g-index) and sources for accessing citation data (Web of Science, Journal Citation Reports and Google Scholar). These may be one of several factors to consider when thinking about where to submit an article manuscript for publication to maximise the potential academic impact of the research, and tools useful to be familiar with if they form part of any research evaluation you and your authored journal papers may be subject to.
An additional section will also look at tips to consider when writing an article abstract to maximise its discoverability and cite-ability.
Learning Outcomes:
• Understanding of meaning and intended uses of bibliometric research indicators
• Understanding of how some key indicators (JIF, H-index) are calculated
• Ability to make a judgement as to the appropriateness and limitations of such indicators
• Ability to use online datasets to view and calculate key bibliometric measures
• Awareness of some factors which can increase the visibility and discoverability of your own research in bibliographic databases.
Previous participants have said:
"The session has helped provide me with the basic information on Journal Impact and where to find information such as an author's h-index. It will be useful for future journal submission consideration."
"This session was very useful for me to become familiar with the topic."
Publishing a paper is a vital step in the academic and scientific journey, playing a pivotal role in advancing knowledge and establishing one's professional reputation. The process of learning how to publish a paper is crucial because it not only disseminates research findings to a wider audience but also ensures the work undergoes rigorous scrutiny through peer review. Through publication, researchers contribute to the collective understanding of their field, fostering a collaborative and dynamic academic environment. Understanding the nuances of manuscript preparation, journal selection, and submission protocols is essential for navigating the competitive world of academic publishing. Successful publication not only validates the credibility of the research but also opens avenues for career progression, securing research funding, and influencing the direction of scientific discourse. Learning how to publish equips researchers with the skills to communicate effectively, share their discoveries, and actively contribute to the growth and evolution of their respective fields.
A presentation on 'Publishing in Academic Journals – Tips to help you succeed' presented at the 2015 University of South Africa (Unisa) Authors Workshop organised by Unisa’s College of Graduate Studies and the Unisa Library
This slide aims to help and guide students on how to start finding literature review through WOS and SCOPUS. The content is excerpted from various sources available from the internet. This is solely meant for education purpose.
Early Career Tactics to Increase Scholarly ImpactElaine Lasda
Workshp for Ph.D. candidates, postdocs and faculy on how bilbiometrics, altmetrics, open access, ORCID, and other resources enable greater visibility of research output.
Showcasing your Research Impact using BibliometricsCiarán Quinn
Seminar to make academics aware of the bibliometric resources available to them and how to use them to improve their research impact. The session looked at
• What are Bibliometrics and Altmetrics
• Why they are important for you
• How to identify your research impact
and research profile
• How to improve your citations
• How to identify potential research collaborations
These slides were shared at a Measuring Impact presentation in ECU's Office for Faculty Excellence. It is licensed under a CC-BY-NC 4.0 Creative Commons License.
Beyond the Journal Impact Factor: Altmetrics; New Ways of Measuring Impactsbeas1
A powerpoint presentation given at Portland State University Library as part of the Library's workshop series for faculty. Download the file to see the notes for each slide.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Durham Researcher Development Programme 2015-16: Bibliometric Research Indica...Jamie Bisset
There is an ever-increasing need to make your research more visible as you establish your career, and metrics to measure your research performance when it comes to thinking about promotion and probation.
This session will focus on bibliometric research indicators (such as the Journal Impact Factor and SCImago, author metrics such as the h-index and g-index) and sources for accessing citation data (Web of Science, Journal Citation Reports and Google Scholar). These may be one of several factors to consider when thinking about where to submit an article manuscript for publication to maximise the potential academic impact of the research, and tools useful to be familiar with if they form part of any research evaluation you and your authored journal papers may be subject to.
An additional section will also look at tips to consider when writing an article abstract to maximise its discoverability and cite-ability.
Learning Outcomes:
• Understanding of meaning and intended uses of bibliometric research indicators
• Understanding of how some key indicators (JIF, H-index) are calculated
• Ability to make a judgement as to the appropriateness and limitations of such indicators
• Ability to use online datasets to view and calculate key bibliometric measures
• Awareness of some factors which can increase the visibility and discoverability of your own research in bibliographic databases.
Previous participants have said:
"The session has helped provide me with the basic information on Journal Impact and where to find information such as an author's h-index. It will be useful for future journal submission consideration."
"This session was very useful for me to become familiar with the topic."
Publishing a paper is a vital step in the academic and scientific journey, playing a pivotal role in advancing knowledge and establishing one's professional reputation. The process of learning how to publish a paper is crucial because it not only disseminates research findings to a wider audience but also ensures the work undergoes rigorous scrutiny through peer review. Through publication, researchers contribute to the collective understanding of their field, fostering a collaborative and dynamic academic environment. Understanding the nuances of manuscript preparation, journal selection, and submission protocols is essential for navigating the competitive world of academic publishing. Successful publication not only validates the credibility of the research but also opens avenues for career progression, securing research funding, and influencing the direction of scientific discourse. Learning how to publish equips researchers with the skills to communicate effectively, share their discoveries, and actively contribute to the growth and evolution of their respective fields.
A presentation on 'Publishing in Academic Journals – Tips to help you succeed' presented at the 2015 University of South Africa (Unisa) Authors Workshop organised by Unisa’s College of Graduate Studies and the Unisa Library
This slide aims to help and guide students on how to start finding literature review through WOS and SCOPUS. The content is excerpted from various sources available from the internet. This is solely meant for education purpose.
Early Career Tactics to Increase Scholarly ImpactElaine Lasda
Workshp for Ph.D. candidates, postdocs and faculy on how bilbiometrics, altmetrics, open access, ORCID, and other resources enable greater visibility of research output.
Showcasing your Research Impact using BibliometricsCiarán Quinn
Seminar to make academics aware of the bibliometric resources available to them and how to use them to improve their research impact. The session looked at
• What are Bibliometrics and Altmetrics
• Why they are important for you
• How to identify your research impact
and research profile
• How to improve your citations
• How to identify potential research collaborations
These slides were shared at a Measuring Impact presentation in ECU's Office for Faculty Excellence. It is licensed under a CC-BY-NC 4.0 Creative Commons License.
Beyond the Journal Impact Factor: Altmetrics; New Ways of Measuring Impactsbeas1
A powerpoint presentation given at Portland State University Library as part of the Library's workshop series for faculty. Download the file to see the notes for each slide.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
2. Topics Covered
• Why assess scholarly impact of research
• Overview of metrics and tools
• Journal Metrics
• Article Metrics
• Author Metrics
• What are altmetrics
• Current issues and future considerations
3. What are we talking about when
we talk about impact? (Typically)
Extent to which research is:
• Read
• Discussed
• Used
….And
• Disseminated
Inside and outside academe
4. Why we measure
• For Promotion and Tenure
• Determine research Quality
• Assess potential for grant funding
• Job Market
• Prove productivity
• Why else?
5. What are we measuring?
• Journal Quality
• Article reach
• Article impact/Influence
• Dissemination
• Researcher Impact
6. How are we measuring?
• Journal-Level Metrics
• Measure quality of Journal using citation formulas
• Article-Level Metrics
• Citation Based or altmetrics
• Author-Level Metrics
• Measure bibliographic impact of individual authors
• Altmetrics
• Measures and monitors reach and impact of research
through online interactions
7. Journal-Level Metrics
• Most journal-level metrics are calculated from the
pool of journals indexed in two citation indexing
databases
• Web of Science(WOS) (Clarivate Analytics)
Over 11,549 journals from the Science Citation Index Expanded
(SCIE) and Social Science Citation Index (SSCI). The Library has a
subscription to Web of Science, does not include access to
Journal Citation Reports (JCR)
• SCOPUS (Elsevier)
Covers nearly 22,000 titles in the scientific, technical, medical and
social sciences (including arts and humanities. The Library does
not have a subscription to SCOPUS. Journal Metrics are freely
available.
9. Journal Impact Factor (JIF)
• Most well known and especially influential in STEM
disciplines
• JIF is a measure of the frequency with which the
"average article" published in a given scholarly
journal has been cited in a particular year or period
• JIF is not normalized for discipline. Can use Quartile
position of title in category: Q1, Q2, Q3, Q4
• 5-year indicators are also available
• The JIF can be found independently of Journal
Citation Reports, usually on the journal home page
10. Eigenfactor Scores® and Article
Influence ®
• The Eigenfactor Project is sponsored by the West
Lab at the Information School and the Bergstrom
Lab in the Department of Biology at the University
of Washington
• It uses information from the entire citation network
to measure the importance of each journal, much
as Google's PageRank algorithm measures the
importance of websites on the world wide web
• Find Eigenfactor metrics at Eigenfactor.Org
11. Scopus-Based Metrics
• CiteScore (Elsevier)
• Calculates the average number of citations received in a
calendar year by all items published in that journal in
the preceding three years
• CiteScore counts all documents since they all have the
potential to attract citations, and the Impact Factor
counts the documents considered most likely to attract
citations. CiteScore is independent of the document-
type classification
• See About CiteScore and its derivative metrics
• Find CiteScore at https://journalmetrics.scopus.com/
12. Scopus-Based Metrics
• SNIP (Source Normalized Impact per paper)
• Defined as the ratio of the raw Impact per Publication divided
by the Relative Database Citation Potential
• Corrects for subject-specific characteristics of the field
someone is publishing in so any two journal can be compared
• Find at CWTS Journal Indicators
http://www.journalindicators.com/
• SJR Indictor (SciMago)
• Citations are weighted, depending on the rank of the citing
journal
• A citation from an important journal will count as more than
one citation; a citation coming from a less important journal
will count as less than one citation.
• Find at www.scimagojr.com
13. Article-Level Metrics
Citation-based and altmetric measures can show
impact of individual research publication
• How many times was an article cited
• How is it tracking in social media
• What is the geographic distribution of citing papers
• What is the disciplinary distribution of citing papers
• What is the impact outside of the scholarly
Community
14. Web of Science Citations
http://ezproxy.fau.edu/login?url=http://isiknowledge.com/wos
15. Google Scholar
• Useful for authors and publications in disciplines
less well covered by the commercial services
• Author search in Google Scholar will produce a list
of publications with citation and a link to Google
Scholar Profile
• Indexes varied content
• Pro – Includes useful cited by literature such as policies
• Con – Can inflate citation counts
16. Google Scholar Author Search
https://scholar.google.com/citations?view_op=search_authors
17. Author Metrics
• H-Index is the best known. Attempts to measure
both productivity and impact of the published
work.
• A scientist has an index h if h of his/her Np papers
has at least h citations each, and the other (Np h)
papers have no more than h citations each
• To have an h-index of 5, an author has to have
5 publications, each receiving at least 5 citations
• Variants include g-index and m-index
• Account for highly cited papers or author career span
18. Where to Find H-Index
Value of the index may vary depending on the source of
information (number of indexed publications, time span, etc.)
• Web of Science
• To identify all publications by an author you can use Author Analyze
search function or enter author's ORCID or ResearcherID identifier
(if known)
• Google Scholar
• Requires creation of Google Scholar profile before providing metrics
• H-index tends to be higher than what is calculated by Web of
Science
• Publish or Perish http://www.harzing.com/pop.htm
• Freely accessible software program
• Uses Google Scholar data and includes h-index, g-index, m-index
and other metrics
19. Altmetrics
“Altmetrics expand our view of what impact looks
like, but also of what’s making the impact. This
matters because expressions of scholarship are
becoming more diverse.”
From http://altmetrics.org/manifesto/
Simply, altmetrics are metrics beyond traditional
citations.
20. What altmetrics address
• How many times something is downloaded
• Who is reading the work
• Has is it been covered by news outlets
• Who is commenting on the work
• How is it being shared
• Which countries are looking at my work
http://pitt.libguides.com/altmetrics/introduction
From University of Pittsburgh Library System
23. What’s Next?
• The end of the Journal Impact Factor?
• Maybe…slowly
• 2012 San Francisco Declaration on Research Assessment
(DORA)
• New metrics such as the Relative Citation Ratio
• Changing expectations of funding agencies
• Acceptance of altmetrics
• Social Impact