Brace for Impact: New Means for Measuring Research Metrics

M
Mary Ellen SloaneScholarly Communications Librarian for Basic and Applied Science
BRACE FOR IMPACT: NEW
MEANS OF MEASURING
RESEARCH METRICS
MARY ELLEN SLOANE,
SCHOLARLY COMMUNICATIONS LIBRARIAN
FOR BASIC AND APPLIED SCIENCES,
MIDDLE TENNESSEE STATE UNIVERSITY
OUTLINE
• Impact Factor system
• Evolution of Scholarly Communication
• Altmetrics sources
• Open access platforms
• Sharing platforms
• Saving platforms
• Hacking altmetrics
• Impact of altmetrics
SUMMARY
• As open access journals and repositories gain a foothold
in scholarly communication, researchers are finding that
the traditional impact factor and citation count metrics
only reflect a portion of the dissemination of scholarly
works.
• New technology, research, and citation tools aid our
ability to measure the influence of research. A matrix of
tools and initiatives, like PLoS Article-Level Metrics,
BePress’ Author Dashboard, Mendeley, Altmetrics, and
ImpactStory are providing a more robust picture of
scholarly communication today.
• This presentation will provide an overview of the impact
factor system and new tools for gathering metrics and
their relevance for librarians and researchers.
IMPACT FACTOR SYSTEM
• The impact factor system measures the number of
citations for a given journal over a period of time.
IMPACT FACTOR SYSTEM
IMPACT FACTOR SYSTEM
•
•
•
•
•
•
•
•
•

Not always a good indicator of quality research
Journal ‘impact’ is relative to an article’s value
Not necessarily meaningful to authors or researchers
Self-citations can skew results
Focuses on overall impact, as opposed to articlelevel measures
Review journals have more citations
San Francisco Declaration on Research Assessment –
concerns with impact system
Open access sources may not be represented
Tenure and promotion criteria tied to publishing
EIGENFACTOR
EVOLUTION OF SCHOLARLY
COMMUNICATION
New means of publishing
Institutional repositories
Sharing data sets
Sharing code
Sharing presentations
Sharing video
Article-level metrics can help encourage publishing
in open-access journals and repositories
• Open access publishing and sharing is increasing
• Social networks can evolve organically and include
open access and other sources of sharing
•
•
•
•
•
•
•
EVOLUTION OF SCHOLARLY
COMMUNICATION
• Ability to measure use by readers, researchers
• Earlier indicators of research impact
• Web of Science:

• Mendeley:
EVOLUTION OF SCHOLARLY
COMMUNICATION
METRICS SOURCES
•
•
•
•
•
•
•
•
•

Social media
Blog posts, Wikipedia
Sharing links to publisher full text
Open access views, downloads, citations, followers
Bookmarks, favorites
Citation saving and groups
Worldcat holdings
Standard citation counts
PlumX - http://www.plumanalytics.com/metrics.html
METRICS SOURCES
• Open access sources:
•
•
•
•
•
•

Dryad
Figshare
GitHub
BePress
dSpace
PLOS

• Saves

• Mendeley,
• citeulike
• Delicious

• Uses

• Youtube, Vimeo
• Slideshare

• Shares

• Twitter
• Facebook
• Bit.ly

• Citations
•
•
•
•

Web of Science
Scopus
Google Scholar
PubMed

• Aggregate metrics
• Altmetrics, and
• ImpactStory
• Plum X
WEB OF SCIENCE ARTICLE METRICS
• C&RL Impact Factor:
• Citation map:
OPEN ACCESS - PLOS
OPEN ACCESS - FIGSHARE
OPEN ACCESS - DRYAD
INSTITUTIONAL REPOSITORIES
IF YOU CAN’T BEAT THEM, JOIN THEM
• Articles Metrics from publishers
ACCURACY PROBLEMS
IMPACT STORY
ALTMETRIC
PLUMX
• PlumX Tracks Groups
•
•
•
•
•
•
•
•
•
•
•

University
Institute
Museum
Department
Lab
Company
Publisher
Journal
Volume
Issue
Collection
SO WHAT

?

• How many are using social media and
reference managers in meaningful ways?
• How meaningful are the metrics?
• Links between related items, article and data
set
• Can easily be gamed
• Debate over early downloads as indicator of
later citation
• Limited to DOI in some cases
• Disciplines may use different platforms and tools
HACKING METRICS
• Does this milk smell bad?
IMPACT OF ALTMETRICS
•
•
•
•
•
•
•
•

New ways to perform bibliometrics
New ways to do research
New ways to share and connect
Recent articles will have more altmetrics
Discipline-specific measures of impact
Tenure and promotion considerations
Researcher’s dissemination will evolve
At some point will become customary
COMING SOON
• NISO Alternative Assessment Metrics (Altmetrics)
Project

• Here Comes the Sunburst: Measuring and
Visualizing Scholarly Impact – Sunday 11/10 – 9:00
am
1 of 26

Recommended

Article Impact (ALM and altmetrics) by
Article Impact (ALM and altmetrics)Article Impact (ALM and altmetrics)
Article Impact (ALM and altmetrics)Richard Cave
3.3K views22 slides
Altmetrics Overview by
Altmetrics OverviewAltmetrics Overview
Altmetrics OverviewPaul Groth
2.9K views29 slides
Overview of the altmetrics landscape by
Overview of the altmetrics landscapeOverview of the altmetrics landscape
Overview of the altmetrics landscapeRichard Cave
10.6K views36 slides
Altmetrics: how librarians can support researchers in improving their impact by
Altmetrics: how librarians can support researchers in improving their impactAltmetrics: how librarians can support researchers in improving their impact
Altmetrics: how librarians can support researchers in improving their impactGIDIF-RBM
1.7K views122 slides
Introduction to Altmetrics by
Introduction to Altmetrics Introduction to Altmetrics
Introduction to Altmetrics Linda Galloway
3.4K views28 slides
Altmetrics by
AltmetricsAltmetrics
AltmetricsMicah Vandegrift
12.4K views27 slides

More Related Content

What's hot

Joining the ‘buzz’ : the role of social media in raising research visibility by
Joining the ‘buzz’ : the role of social media in raising research visibilityJoining the ‘buzz’ : the role of social media in raising research visibility
Joining the ‘buzz’ : the role of social media in raising research visibility Eileen Shepherd
2.9K views64 slides
Altmetrics 2014-4-15-slideshare by
Altmetrics 2014-4-15-slideshareAltmetrics 2014-4-15-slideshare
Altmetrics 2014-4-15-slideshareCMHSL
9.3K views46 slides
Altmetrics apples and oranges by
Altmetrics apples and orangesAltmetrics apples and oranges
Altmetrics apples and orangesPat Loria
2.5K views20 slides
Measuring research impact with bibliometrics by
Measuring research impact with bibliometricsMeasuring research impact with bibliometrics
Measuring research impact with bibliometricsLancaster University Library
2.8K views24 slides
Altmetrics Day Workshop - Internet Librarian International 2014 by
Altmetrics Day Workshop - Internet Librarian International 2014Altmetrics Day Workshop - Internet Librarian International 2014
Altmetrics Day Workshop - Internet Librarian International 2014Andy Tattersall
2.1K views91 slides
Niso Article Level Metrics Presentation For Online 2 by
Niso Article Level Metrics Presentation For Online 2Niso Article Level Metrics Presentation For Online 2
Niso Article Level Metrics Presentation For Online 2Liz Allen
1.4K views30 slides

What's hot(20)

Joining the ‘buzz’ : the role of social media in raising research visibility by Eileen Shepherd
Joining the ‘buzz’ : the role of social media in raising research visibilityJoining the ‘buzz’ : the role of social media in raising research visibility
Joining the ‘buzz’ : the role of social media in raising research visibility
Eileen Shepherd2.9K views
Altmetrics 2014-4-15-slideshare by CMHSL
Altmetrics 2014-4-15-slideshareAltmetrics 2014-4-15-slideshare
Altmetrics 2014-4-15-slideshare
CMHSL9.3K views
Altmetrics apples and oranges by Pat Loria
Altmetrics apples and orangesAltmetrics apples and oranges
Altmetrics apples and oranges
Pat Loria2.5K views
Altmetrics Day Workshop - Internet Librarian International 2014 by Andy Tattersall
Altmetrics Day Workshop - Internet Librarian International 2014Altmetrics Day Workshop - Internet Librarian International 2014
Altmetrics Day Workshop - Internet Librarian International 2014
Andy Tattersall2.1K views
Niso Article Level Metrics Presentation For Online 2 by Liz Allen
Niso Article Level Metrics Presentation For Online 2Niso Article Level Metrics Presentation For Online 2
Niso Article Level Metrics Presentation For Online 2
Liz Allen1.4K views
When will librarians start research support with altmetrics to their research... by Keita Bando
When will librarians start research support with altmetrics to their research...When will librarians start research support with altmetrics to their research...
When will librarians start research support with altmetrics to their research...
Keita Bando3.3K views
Enhancing the Visibility and Impact of Your Research by Karen Gutzman
Enhancing the Visibility and Impact of Your ResearchEnhancing the Visibility and Impact of Your Research
Enhancing the Visibility and Impact of Your Research
Karen Gutzman1.5K views
Overview of Bibliometrics - IAP Course version 1.1 by Micah Altman
Overview of Bibliometrics - IAP Course version 1.1Overview of Bibliometrics - IAP Course version 1.1
Overview of Bibliometrics - IAP Course version 1.1
Micah Altman4.6K views
Altmetric: Getting Started with Article-Level Metrics by Altmetric
Altmetric: Getting Started with Article-Level MetricsAltmetric: Getting Started with Article-Level Metrics
Altmetric: Getting Started with Article-Level Metrics
Altmetric6.7K views
The Right Metrics for Generation Open [Open Access Week 2014] by Impactstory Team
The Right Metrics for Generation Open [Open Access Week 2014]The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]
Impactstory Team6.1K views
Altmetrics for research: impact measurement &#hcsm by Robin Featherstone
Altmetrics for research: impact measurement &#hcsmAltmetrics for research: impact measurement &#hcsm
Altmetrics for research: impact measurement &#hcsm
Robin Featherstone3.5K views
The Impact Factor, Eigenfactor, and Altmetrics: From Theory to Analysis by Molly Keener
The Impact Factor, Eigenfactor, and Altmetrics: From Theory to AnalysisThe Impact Factor, Eigenfactor, and Altmetrics: From Theory to Analysis
The Impact Factor, Eigenfactor, and Altmetrics: From Theory to Analysis
Molly Keener2.2K views
Using ImpactStory: an Introduction by SarahG_SS
Using ImpactStory: an IntroductionUsing ImpactStory: an Introduction
Using ImpactStory: an Introduction
SarahG_SS5.8K views
Citation Analysis: From Publication to Impact - Anne-Wil Harzing by Charlies1000
Citation Analysis: From Publication to Impact - Anne-Wil HarzingCitation Analysis: From Publication to Impact - Anne-Wil Harzing
Citation Analysis: From Publication to Impact - Anne-Wil Harzing
Charlies100043.6K views

Similar to Brace for Impact: New Means for Measuring Research Metrics

Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact... by
Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...
Stephanie Faulkner - Using Metrics Beyond Citations to Demonstrate the Impact...SciELO - Scientific Electronic Library Online
396 views32 slides
Introduction to Altmetrics for Medical and Special Librarians by
Introduction to Altmetrics for Medical and Special LibrariansIntroduction to Altmetrics for Medical and Special Librarians
Introduction to Altmetrics for Medical and Special LibrariansLinda Galloway
2K views49 slides
From Tweetations to Citations: Social Media and the Researcher by
From Tweetations to Citations: Social Media and the ResearcherFrom Tweetations to Citations: Social Media and the Researcher
From Tweetations to Citations: Social Media and the ResearcherSharon Karasmanis
2.3K views24 slides
Practical Uses of Altmetrics - Hear librarians talk about how they use new me... by
Practical Uses of Altmetrics - Hear librarians talk about how they use new me...Practical Uses of Altmetrics - Hear librarians talk about how they use new me...
Practical Uses of Altmetrics - Hear librarians talk about how they use new me...plumanalytics
801 views73 slides
Shifting ground: scholarly communication in geography by
Shifting ground: scholarly communication in geographyShifting ground: scholarly communication in geography
Shifting ground: scholarly communication in geographyElizabeth Yates
1.3K views34 slides
Altmetricon 2015 by
Altmetricon 2015Altmetricon 2015
Altmetricon 2015Jonathan Tennant
937 views17 slides

Similar to Brace for Impact: New Means for Measuring Research Metrics(20)

Introduction to Altmetrics for Medical and Special Librarians by Linda Galloway
Introduction to Altmetrics for Medical and Special LibrariansIntroduction to Altmetrics for Medical and Special Librarians
Introduction to Altmetrics for Medical and Special Librarians
Linda Galloway2K views
From Tweetations to Citations: Social Media and the Researcher by Sharon Karasmanis
From Tweetations to Citations: Social Media and the ResearcherFrom Tweetations to Citations: Social Media and the Researcher
From Tweetations to Citations: Social Media and the Researcher
Sharon Karasmanis2.3K views
Practical Uses of Altmetrics - Hear librarians talk about how they use new me... by plumanalytics
Practical Uses of Altmetrics - Hear librarians talk about how they use new me...Practical Uses of Altmetrics - Hear librarians talk about how they use new me...
Practical Uses of Altmetrics - Hear librarians talk about how they use new me...
plumanalytics801 views
Shifting ground: scholarly communication in geography by Elizabeth Yates
Shifting ground: scholarly communication in geographyShifting ground: scholarly communication in geography
Shifting ground: scholarly communication in geography
Elizabeth Yates1.3K views
Altmetrics in Practice - wssf -- Montreal - Oct 14, 2013 by plumanalytics
Altmetrics in Practice  - wssf -- Montreal - Oct 14, 2013Altmetrics in Practice  - wssf -- Montreal - Oct 14, 2013
Altmetrics in Practice - wssf -- Montreal - Oct 14, 2013
plumanalytics2K views
Use of social media by scholarly authors by Kudos
Use of social media by scholarly authorsUse of social media by scholarly authors
Use of social media by scholarly authors
Kudos2.4K views
Early Career Tactics to Increase Scholarly Impact by Elaine Lasda
Early Career Tactics to Increase Scholarly ImpactEarly Career Tactics to Increase Scholarly Impact
Early Career Tactics to Increase Scholarly Impact
Elaine Lasda131 views
How practising open research can benefit you by UoLResearchSupport
How practising open research can benefit youHow practising open research can benefit you
How practising open research can benefit you
UoLResearchSupport354 views

Recently uploaded

Collective Bargaining and Understanding a Teacher Contract(16793704.1).pptx by
Collective Bargaining and Understanding a Teacher Contract(16793704.1).pptxCollective Bargaining and Understanding a Teacher Contract(16793704.1).pptx
Collective Bargaining and Understanding a Teacher Contract(16793704.1).pptxCenter for Integrated Training & Education
95 views57 slides
Recap of our Class by
Recap of our ClassRecap of our Class
Recap of our ClassCorinne Weisgerber
88 views15 slides
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx by
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptxPharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptxMs. Pooja Bhandare
113 views51 slides
S1_SD_Resources Walkthrough.pptx by
S1_SD_Resources Walkthrough.pptxS1_SD_Resources Walkthrough.pptx
S1_SD_Resources Walkthrough.pptxLAZAROAREVALO1
64 views57 slides
REPRESENTATION - GAUNTLET.pptx by
REPRESENTATION - GAUNTLET.pptxREPRESENTATION - GAUNTLET.pptx
REPRESENTATION - GAUNTLET.pptxiammrhaywood
138 views26 slides
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively by
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks EffectivelyISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks EffectivelyPECB
623 views18 slides

Recently uploaded(20)

Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx by Ms. Pooja Bhandare
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptxPharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx
Ms. Pooja Bhandare113 views
S1_SD_Resources Walkthrough.pptx by LAZAROAREVALO1
S1_SD_Resources Walkthrough.pptxS1_SD_Resources Walkthrough.pptx
S1_SD_Resources Walkthrough.pptx
LAZAROAREVALO164 views
REPRESENTATION - GAUNTLET.pptx by iammrhaywood
REPRESENTATION - GAUNTLET.pptxREPRESENTATION - GAUNTLET.pptx
REPRESENTATION - GAUNTLET.pptx
iammrhaywood138 views
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively by PECB
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks EffectivelyISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively
ISO/IEC 27001 and ISO/IEC 27005: Managing AI Risks Effectively
PECB 623 views
How to empty an One2many field in Odoo by Celine George
How to empty an One2many field in OdooHow to empty an One2many field in Odoo
How to empty an One2many field in Odoo
Celine George87 views
AUDIENCE - BANDURA.pptx by iammrhaywood
AUDIENCE - BANDURA.pptxAUDIENCE - BANDURA.pptx
AUDIENCE - BANDURA.pptx
iammrhaywood117 views
EIT-Digital_Spohrer_AI_Intro 20231128 v1.pptx by ISSIP
EIT-Digital_Spohrer_AI_Intro 20231128 v1.pptxEIT-Digital_Spohrer_AI_Intro 20231128 v1.pptx
EIT-Digital_Spohrer_AI_Intro 20231128 v1.pptx
ISSIP386 views
Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant... by Ms. Pooja Bhandare
Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant...Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant...
Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant...
Ms. Pooja Bhandare133 views
Narration lesson plan by TARIQ KHAN
Narration lesson planNarration lesson plan
Narration lesson plan
TARIQ KHAN61 views
Ch. 8 Political Party and Party System.pptx by Rommel Regala
Ch. 8 Political Party and Party System.pptxCh. 8 Political Party and Party System.pptx
Ch. 8 Political Party and Party System.pptx
Rommel Regala54 views

Brace for Impact: New Means for Measuring Research Metrics

  • 1. BRACE FOR IMPACT: NEW MEANS OF MEASURING RESEARCH METRICS MARY ELLEN SLOANE, SCHOLARLY COMMUNICATIONS LIBRARIAN FOR BASIC AND APPLIED SCIENCES, MIDDLE TENNESSEE STATE UNIVERSITY
  • 2. OUTLINE • Impact Factor system • Evolution of Scholarly Communication • Altmetrics sources • Open access platforms • Sharing platforms • Saving platforms • Hacking altmetrics • Impact of altmetrics
  • 3. SUMMARY • As open access journals and repositories gain a foothold in scholarly communication, researchers are finding that the traditional impact factor and citation count metrics only reflect a portion of the dissemination of scholarly works. • New technology, research, and citation tools aid our ability to measure the influence of research. A matrix of tools and initiatives, like PLoS Article-Level Metrics, BePress’ Author Dashboard, Mendeley, Altmetrics, and ImpactStory are providing a more robust picture of scholarly communication today. • This presentation will provide an overview of the impact factor system and new tools for gathering metrics and their relevance for librarians and researchers.
  • 4. IMPACT FACTOR SYSTEM • The impact factor system measures the number of citations for a given journal over a period of time.
  • 6. IMPACT FACTOR SYSTEM • • • • • • • • • Not always a good indicator of quality research Journal ‘impact’ is relative to an article’s value Not necessarily meaningful to authors or researchers Self-citations can skew results Focuses on overall impact, as opposed to articlelevel measures Review journals have more citations San Francisco Declaration on Research Assessment – concerns with impact system Open access sources may not be represented Tenure and promotion criteria tied to publishing
  • 8. EVOLUTION OF SCHOLARLY COMMUNICATION New means of publishing Institutional repositories Sharing data sets Sharing code Sharing presentations Sharing video Article-level metrics can help encourage publishing in open-access journals and repositories • Open access publishing and sharing is increasing • Social networks can evolve organically and include open access and other sources of sharing • • • • • • •
  • 9. EVOLUTION OF SCHOLARLY COMMUNICATION • Ability to measure use by readers, researchers • Earlier indicators of research impact • Web of Science: • Mendeley:
  • 11. METRICS SOURCES • • • • • • • • • Social media Blog posts, Wikipedia Sharing links to publisher full text Open access views, downloads, citations, followers Bookmarks, favorites Citation saving and groups Worldcat holdings Standard citation counts PlumX - http://www.plumanalytics.com/metrics.html
  • 12. METRICS SOURCES • Open access sources: • • • • • • Dryad Figshare GitHub BePress dSpace PLOS • Saves • Mendeley, • citeulike • Delicious • Uses • Youtube, Vimeo • Slideshare • Shares • Twitter • Facebook • Bit.ly • Citations • • • • Web of Science Scopus Google Scholar PubMed • Aggregate metrics • Altmetrics, and • ImpactStory • Plum X
  • 13. WEB OF SCIENCE ARTICLE METRICS • C&RL Impact Factor: • Citation map:
  • 15. OPEN ACCESS - FIGSHARE
  • 16. OPEN ACCESS - DRYAD
  • 18. IF YOU CAN’T BEAT THEM, JOIN THEM • Articles Metrics from publishers
  • 22. PLUMX • PlumX Tracks Groups • • • • • • • • • • • University Institute Museum Department Lab Company Publisher Journal Volume Issue Collection
  • 23. SO WHAT ? • How many are using social media and reference managers in meaningful ways? • How meaningful are the metrics? • Links between related items, article and data set • Can easily be gamed • Debate over early downloads as indicator of later citation • Limited to DOI in some cases • Disciplines may use different platforms and tools
  • 24. HACKING METRICS • Does this milk smell bad?
  • 25. IMPACT OF ALTMETRICS • • • • • • • • New ways to perform bibliometrics New ways to do research New ways to share and connect Recent articles will have more altmetrics Discipline-specific measures of impact Tenure and promotion considerations Researcher’s dissemination will evolve At some point will become customary
  • 26. COMING SOON • NISO Alternative Assessment Metrics (Altmetrics) Project • Here Comes the Sunburst: Measuring and Visualizing Scholarly Impact – Sunday 11/10 – 9:00 am