Librarians & altmetrics: Tools, tips and use cases

Webinar
Librarians & altmetrics: Tools, tips and use cases
February 20, 2014
Mike Taylor, Research Specialist, Elsevier Labs (@herrison)
Mike's current areas of work include altmetrics, contributorship,
research networks, the future of scholarly communications and other
identity issues. He has worked in various capacities within the ORCID
initiative.
Jenny Delasalle, Freelance Consultant/Librarian (@JennyDelasalle)
Jenny's interests include bibliometrics and altmetrics, the changing
landscape of scholarly communication, and how researchers can and
do share and promote their research while also protecting their
professional image. Jenny has worked in a number of academic library
roles at various UK higher education institutions, including most
recently at the University of Warwick, managing the library's support
of researchers.
Kristi Holmes, Bioinformaticist, Washington University in St. Louis
(@kristiholmes)
Kristi's professional interests include open science, support and
training in genomic medicine, and understanding the impact of
research efforts. She serves as the outreach lead for the research
discovery platform VIVO and is a member of the ORCID Outreach
Steering Group.
#LCwebinar

Webinar
#LCwebinar
Research Specialist, Elsevier Labs
http://orcid.org/0000-0002-8534-5985
mi.taylor@elsevier.com
Mike
Taylor
Altmetrics: a primer
Where does the data come from?
Can it be gamed?
Buy in or build your own?

Webinar
Altmetrics: a primer Mike Taylor
What is the data?
• A set of altmetric data is about a common document
and represents usage, recommendation, shares, re-
usage
• Identified by DOI, URL, shortened URL, other ID
(e.g., arXiv, PubMed)
• It does not show common intent: a tweet is not the
same as a Mendeley share is not the same as a Data
Dryad data download is not the same as mass media
coverage or a blog

Webinar
Various providers…
• Altmetric.com
• Impactstory.org
• Plum Analytics
• PLOS / PLOS code
• Altmetrics is not Altmetric.com
Each has strengths and weaknesses, no canonical source

Webinar
Different data have different characteristics
• Example from 13,500 papers:
• Highly tweeted stories focus on policy, gender,
funding, “contentious science” issues, mostly
summaries on Nature News
• Highly shared papers in Mendeley are hard core
original research
• Different platforms have discipline bias
• Scholarly blogs both lead interest and respond
• Data from Altmetric.com

Webinar
Bringing together sources…
• Altmetrics isn’t one thing, so attempting to express it
as one thing will fail
• We favour intelligent clusters of data: social activity,
mass media, scholarly activity, scholarly comment,
re-use
• Elsevier believes that more research is needed, and
that best indicators are scholarly activity and
scholarly comment

Webinar
Gaming / cheating
• If people take this data seriously, will they cheat?
• E.g., Brazilian citation scandal, strategies used by people to
increase IF of journals
• Expertise in detecting fraudulent downloads (e.g., SSRN),
self-tweeting – when is “normal” corrupt?
• One thing to buy 1,000 tweets, another to buy 10 blogs, or
mass media coverage
• Do those Twitter accounts have scholarly followers?
• Pattern analysis, usage analysis, network analysis
• Public data = public analysis = public response

Webinar
Other criticisms
• Biggest criticisms are when people try and conflate
all the data into a single thing
• Easy point of attack – tweets are all about “sex drugs
and rock ‘n’ roll papers”*
• Using clusters is more intelligible to academic
community – e.g., re-use, scholarly activity, scholarly
comment (blogs, reviews, discussions)
• * this isn’t true anyway

Webinar
Buy-in, or bake-your-own
• Buy-in: Altmetric.com and PLUM from Ebsco
• Free-to-use: Impactstory.org, platforms that use
PLOS article-level-metrics code
• Bake-your-own: Impactstory.org, PLOS
• Or a root-and-branch build

Webinar
Topics covered
• Data sources
• Providers
• Different types of data, differences and similarities
• Criticisms, weaknesses and strategies
• Your next steps

Webinar
#LCwebinar
Freelance Consultant/Librarian
Jenny
Delasalle
Altmetrics: why it's relevant to a librarian
http://jennydelasalle.wordpress.com/

Webinar
Altmetrics : why it's relevant to a librarian Jenny Delasalle
Consider the potential of altmetrics
• They are appearing in library subscription
products… consequences
• Should they appear in institutional
repositories? How?
• Possibly use when considering crowd
funding/citizen science projects
• Spaces to watch

Webinar
Appearing in library subscription &
library recommended products
1. User education/
information
skills/literacy
2. Appropriate use of
data & tools by
institution
3. Marketing and
outreach potential?

Webinar
What the public sees

Webinar
Should they appear in
institutional repositories?
1. A showcase: “Bring a paper to life” : track
related discussions and materials
2. For authors & admin staff to see attention
from newspapers, blogs, social media
3. Use by repository manager: collection
development, depositor encouragement &
proof of the IR’s contribution

Webinar
Which article level measures in an IR?
• Pageviews, downloads, citations
• Saves on Mendeley, Delicious, CiteULike, etc.
• Shares through Twitter, Facebook, blog posts,
Mendeley, etc.
• Choose a source & tool, e.g.,
Altmetric.com/Impactstory/Plum/PLOS
• Allow authors control?

Webinar
Plum
Analytics:
University
of
Pittsburgh

Webinar
Altmetrics: University of Warwick

Webinar
Impactstory
& figshare

Webinar
Crowdsourcing/citizen science
1. Benefits of crowdsourcing include
investment of cash, expertise or resource
2. It is both a means of engaging with the
public, and also relies upon being successful
at such engagement
3. By measuring interest in your research, you
can consider whether there would be
interest in participation/investment

Webinar
Spaces to watch
1. Kudos, growkudos.com “helps authors and
institutions maximize the impact and visibility of
their publications” (Also publishers & societies)
2. ORCID aggregates an individual’s work: authors can
use with impactstory.org : are they using it?
3. COUNTER compliant institutional repositories
4. Services (not only tools) that allow universities “ to
use analytics to develop actionable insights on how
to improve their research, student experience and
efficiency, and effectiveness.” – Times HigherEd

Webinar
References & Further Reading
• Priem, Costello, Tyler (2012) Prevalence and use of Twitter among scholars -
http://figshare.com/articles/Prevalence_and_use_of_Twitter_among_scholars/10
4629
• The San Francisco Declaration on Research Assessment (DORA declaration),
initiated by the American Society for Cell Biology (ASCB) -
http://am.ascb.org/dora/
• Konkiel, S (2013) Altmetrics in Institutional repositories, ASIS&T 2013, Montreal
https://scholarworks.iu.edu/dspace/bitstream/handle/2022/17122/2013_ASIST_
Altmetrics%20%26%20Libraries.pdf?sequence=1
• Smith, D (2013) What is Kudos? An Interview with David Sommer, Co-Founder,
Scholarly Kitchen blog - http://scholarlykitchen.sspnet.org/2013/12/17/what-is-
kudos-an-interview-with-david-sommer-co-founder/
• Brody et al (2009) Publisher and Institutional Repository Usage Statistics (PIRUS)
final report -
http://www.jisc.ac.uk/media/documents/programmes/pals3/pirus_finalreport.pd
f
• Parr, C (2014) Bett show: four areas of technology that could transform
universities, THE - http://www.timeshighereducation.co.uk/news/bett-show-four-
areas-of-technology-that-could-transform-universities/4/2011070.article

Webinar
#LCwebinar
Bioinformaticist
Washington University in St. Louis
Kristi
Holmes
Going beyond counts: understanding impact

Webinar
Going beyond counts: understanding impact Kristi Holmes
Translational Research
The bench to the bedside and beyond
http://icts.wustl.edu/

Webinar
Why do we need to think about research
impact?
https://becker.wustl.edu/impact-assessment/model
• Quantify and document research
impact
• Justify future requests for funding
• Quantify return on research
investment
• Discover how research findings are
being used
• Identify similar research projects
• Identify possible collaborators
• Determine if research findings are
duplicated, confirmed, corrected,
improved or repudiated
• Determine if research findings were
extended (different human
populations, different animal
models/species, etc.)
• Confirm that research findings were
properly attributed/credited
• Demonstrate that research findings
are resulting in meaningful health
outcomes
• Discover community benefit as a
result of research findings
• Progress reports
• Tenure
• Promotion dossiers

Webinar
How do we think
about productivity?
http://www.philnel.com/2010/10/14/procrastigrading/
Don’t knock publications
• Citations
– Rates and rankings
– Networks
– Dissemination patterns
• Collaboration
• Dissemination
• Grant funding
• Research trends

Webinar
…but there’s so
much more!
http://www.philnel.com/2010/10/14/procrastigrading/

Webinar
Research Impact
Wells R, Whitworth A. 2007. Assessing outcomes of health and medical research: do we measure what
counts or count what we can measure? Australia and New Zealand Health Policy, 4:14
“It is no longer enough to measure what we
can – we need to measure what matters.”
So how do we measure what
matters?

Webinar
Going beyond the counts to find meaningful
impact
In everyone’s CV, we’ll find
• Reviews
• New funding awarded
• New types of funding
mechanisms
• New research studies
• Invited lectures, new focus areas
at conferences
• Membership on committees
• Awards
Consider specific pathways*
to uncover meaningful
impact…
Advancement of Knowledge
Clinical Implementation
Legislation and Policy Enactment
Economic Benefit
Community Benefit
*can be adapted to reflect any number of disciplines

Webinar
Going beyond the counts to find meaningful
impact • New diagnostic criteria
• New standard of care
• Curriculum guidelines
• Measurement instruments
• Continuing education materials
• Clinical/practice guidelines
• Quality measure guidelines
• Private healthcare benefit plans
• Cost-effective intervention
• Consensus development conferences
American Medical Association
Current Procedural Terminology
(CPT) codes
• Change in delivery of healthcare
services

Webinar
Slide title
https://becker.wustl.edu/impact-assessment

Webinar
The Becker Model
• Provides a supplement to publication analysis to provide a more
robust and comprehensive perspective of biomedical research
impact.
– reporting templates, glossary of resources and terms, examples of relevant indicators
of impact across the research process, and readings
• Straightforward framework for tracking diffusion of research
outputs and activities to locate indicators that demonstrate evidence
of biomedical research impact
– individual, core, and institutional-level; modify for different disciplines
• Guidance for quantifying and documenting research impact as well
as resources for locating evidence of impact.
• Strategies for enhancing the impact of research

Webinar
Strategies for enhancing the impact
of research
https://becker.wustl.edu/impact-assessment/strategies
Repetition, consistency, and an
awareness of the intended
audience form the basis of most
the strategies.
Suggestions for researchers and
recommendations to reach out to their
library for assistance.
The strategies focus upon
Preparing for Publication,
Dissemination, and Keeping Track
of Your Research.
Optimizing discoverability and
access of your research is the
surest way to enhance its
visibility and impact.

Webinar
Understanding the impact of a single paper can
be challenging.
How do we understand the impact of a person?
A research center?
A university?

Webinar
Research networking and discovery systems
• Enable more efficient means of
collecting & representing
meaningful outputs en masse
• Showcase achievements and
expertise
• Facilitate diffusion of research
products
• Support team-based science and
collaboration
• Allow better understanding of the
research enterprise
– Peer comparisons
– Strategic planning
– Emerging trends

Webinar
What is VIVO?
1. An open source semantic
web application
2. An information model
3. An open community*
* A big, welcoming OS community! Let us know if you have
questions or need information, connections, or materials:
http://vivoweb.org/contact

Webinar
Tying it all together
http://vivo.ufl.edu/display/n86607
Continue to represent
typical profile
information –
absolutely!
Enhance profiles by
incorporating
meaningful outputs, as
described by the Becker
Model or other
frameworks, to enhance
their content and value
Can facilitate the
dissemination of
scholarship and more

Webinar
http://blog.plumanalytics.com/post/57
707501083/plumx-now-supports-vivo
PlumX supports VIVO
1. Aggregate
information about our
researchers
2. Harvest research
outputs from the
profile
3. Plum’s harvesting
engine calculates
metrics and provides
both an analytics
dashboard as well as
visualizations.
4. Widgets can be
embedded back into
the VIVO profile!

Webinar
http://blog.plumanalytics.com/post/57707501083/plumx-
now-supports-vivo
PlumX supports VIVO
1. Aggregate
information about our
researchers
2. Harvest research
outputs from the
profile
3. Plum’s harvesting
engine calculates
metrics and provides
both an analytics
dashboard as well as
visualizations.
4. Widgets can be
embedded back into
the VIVO profile!
http://vivo.ufl.edu/display/n86607
FigShare, lab notebooks,
project outputs,
SlideShare, etc.

Webinar
Acknowledgements
Funding:
• Washington University Institute
of Clinical and Translational
Sciences, NIH award UL1
RR024992
• VIVO - DuraSpace
Thanks:
• Cathy Sarli, MLS, AHIP
• Karen Gutzman – NLM Fellow
• Jae Allen
• Washington University ICTS and
WU ICTS Tracking & Evaluation
Team
• Andrea Michalek at Plum
@amichalek
• VIVO Community @VIVOcollab
• Becker Medical Library

Webinar
Questions & Thank You!
Mike Taylor, Research Specialist, Elsevier Labs
@herrison
mi.taylor@elsevier.com
http://orcid.org/0000-0002-8534-5985
Jenny Delasalle, Freelance Consultant/Librarian
@JennyDelasalle
jennydelasalle@gmail.com
http://jennydelasalle.wordpress.com/
Kristi Holmes, Bioinformaticist, Washington University in St. Louis
@kristiholmes
holmeskr@wustl.edu
#LCwebinar

Librarians & altmetrics: Tools, tips and use cases

More Related Content

What's hot

Similar to Librarians & altmetrics: Tools, tips and use cases

More from Library_Connect

Recently uploaded

Librarians & altmetrics: Tools, tips and use cases