ALTMETRICS : A HASTY PEEP INTO NEW SCHOLARLY MEASUREMENT

Warm Welcome
ALTMETRICS : A HASTY PEEP INTO NEW SCHOLARLY MEASUREMENT
Saptarshi Ghosh
NBU
Nov.26 – Dec-16, 2015

“Altmetrics: FUN or No Fun at all…”
Overview:
• Research, Impact and it’s Evaluation
• Traditional Metrics
• What is Altmetrics?
• What Altmetrics measure?
• Metrics and Data Sources in Altmetrics
• Service Providers for Altmetrics
• Few Examples of it’s Implementations
• Advantages of Altmetrics
• Limitations of Altmetrics
• Altmetrics v/s Traditional Metrics
• Role of Librarians in the age of Altmetrics
• References
5/22/2020 Copy-Left from Saptarshi Ghosh 2

* Research:
• Study.com: Research is a careful and detailed study into a specific problem,
concern, or issue using the scientific method.
• Defined in its simplest terms, research is searching for and gathering information,
usually to answer a particular question or problem.
• Oxford Advanced Learner's Dictionary: The systematic investigation into and study
of materials and sources in order to establish facts and reach new conclusions.
• The simplest & shortest is just “Re+Search”.
* Research Impact:
 According to Australian Research Council (ARC): Research impact is the
demonstrable contribution that research makes to the economy, society, culture,
national security, public policy or services, health, the environment, or quality of
life, beyond contributions to academia.
 Research Councils UK (RCUK) defines research impact as 'the demonstrable
contribution that excellent research makes to society and the economy‘.
 Academic Impact: demonstrate contribution to scientific advances, across
and within disciplines; its understanding, method, theory and application.
 Economic and Societal Impact: demonstrate contribution towards society
and the economy, of benefit to individuals, organisations and nations.
35/22/2020 Copy-Left from Saptarshi Ghosh

* Benefits of Evaluation of Research Impact:
• Quantify and document research impact;
• Justify future requests for funding;
• Quantify return on research investment;
• Discover how research findings are being used;
• Identify similar research projects;
• Identify possible collaborators;
• Determine if research findings are duplicated, confirmed, corrected, or improved;
• Determine if research findings were extended (different human populations,
different animal models/species, etc.);
• Confirm that research findings were properly attributed/credited;
• Demonstrate that research findings are resulting in meaningful outcomes;
• Discover community benefit as a result of research findings;
• Progress reports;
• Tenure;
• Promotion dossiers;
(https://becker.wustl.edu/impact-assessment/model)

* Research Impact Evaluation:
Impact assessment is one of the major drivers in research and scholarly
communication:
 Research < Academic < Peer-review & Bibliometrics etc.
 Research < Societal < Case-study approach.
• Traditional Metrics:
* Journal level metrics
 Impact Factor-
 Eigen factor-
 SCImago Journal & Country Ranking (SJR)-
* Author and article level metrics
 Citation count/analysis-
 h-index-
 i-10 Index-
 g-Index-

* In-efficiency of Traditional Metrics:
 ‘Citation metrics’ and ‘impact factor are solely based on ‘citation’.
 Data sources are limited: Web of Science (ISI-Thomson Reuters), Scopus
(Elsevier) and Google Scholar.
 Citations took years to occur.
 Mostly, relevant to Academic Impact not the Societal Impact as a whole.
 “A sea change in Scholarship Model”: As like the publishing industry
changed from hand-written to type, than to print, now to the digital, e-
book and open access publishing; the scholarly communication process is
too moving away from the platforms of conferences and print journals
publication into the open access, web and network-based venues like
blogs, IRs, social forums and social medias which is altogether termed as
“Digital Scholarship”.
 Traditional metrics doesn’t support and consider these new and emerging
venues of digital scholarship for evaluation.

• “Were print journals to disappear…I am confident that a new impact factor
would be invented. Information scientists are already computing web impact
factors. It would be more relevant to use the actual impact (citation
frequency) of individual papers in evaluating the work of individual scientists
rather than using the journal impact factor as a surrogate.”
– “Impact factors, and why they won't go away,”
Eugene Garfield, 2001.
• Here it comes...Altmetrics!
What is Altmetrics?
* The term ‘Altmetrics’ was proposed by Jason Priem, a PhD student at the School of
Information and Library Science at University of North Carolina, Chapel Hill through a
tweet [https://twitter.com/asnpriem/status/25844968813].
* Altmetrics is the combination of two words such as: ‘Alternative’ and ‘Metrics’ in
which the ‘alt-‘part refers to alternative types of metrics (that is alternative to
traditional metrics such as citation analysis, impact factor, downloads & usage data
etc.).
* Altmetrics is the creation and study of new metrics based on the Social Web for
analyzing, and informing scholarship (http://altmetrics.org/about/). It is the study of
new indicators for the analysis of academic activity based on Web 2.0.

What is Altmetrics?
* According to Adie, Euan (2013) the founder of Altmetric.com; “Altmetrics indicate
the quantity and quality of online attention in multiple channels, including social
media, blog posts, and news coverage”.
* Australian Open Access Support Group (AOASG) (2013). Altmetrics are quantitative
indicators of public reach and influence (It provides a more comprehensive
understanding of impact across sectors, including public impact.
* Altmetrics is the study of new metrics for analyzing and informing scholarship based
on the social web.
* Alternative metrics (called Altmetrics to distinguish them from bibliometrics) are
considered an interesting option for assessing the societal impact of research, as
they offer new ways to measure (public) engagement with research output
(Piwowar, 2013).
* “Altmetrics . . . is a term to describe web-based metrics for the impact of scholarly
material, with an emphasis on social media outlets as sources of data” (Shema, Bar-
Ilan, & Thelwall, 2014).
“All of these above definitions covers the impact beyond academia which is
tracked by Altmetrics unlike bibliometrics.”

• What Altmetrics measure?
Altmetrics measure the number of times a research output gets cited, tweeted about,
liked, shared, bookmarked, viewed, downloaded, mentioned, favourited, reviewed, or
discussed on various kind of web platforms. It harvests those web influence data from
a wide variety of web sources and platforms including open access journal platforms,
scholarly citation databases, web-based research sharing services, and social media.
The numbers are harvested almost in real time, providing researchers with fast
evidence that their research has made an impact or generated a conversation in the
public forum.
(Altmetrics.org, 2010)

• Metrics and Data Sources in Altmetrics
Altmetrics capture and assess the broad ranges of online influence a paper or work
can have from various web sources which includes citations; views, mentions, shares,
bookmarks and many more.
a. Usage Data: Page views: HTML & PDF; and document downloads: PDF, etc. to assess
scholarly impact. Data can be counted from various databases and repositories like
Dryad, Figshare, GitHub, SlideShare etc.
b. Citations: Apart from citations tracked from Web of Science and Scopus; Altmetrics
takes into count the non-scholarly citations from sources like Google Scholar, CrossRef,
PubMed, ScienceSeeker, Wikipedia, Scholarpedia etc.
c. Captures: Altmetrics capture Social bookmarking data of a research paper in
platforms like Delicious, CiteULike, Connotea; saves in EndNote, Zotero & Mendeley;
favorites in SlideShare and YouTube; followed in GitHub; and number of Mendeley
readers of that specific paper which helps to figure out the impact on scholarly
community.
d. Mentions: This metrics counts the number of comments & mentions in Facebook,
Twitter, LinkedIn, SlideShare; mentioned in blog, e-news & medias, on-line forums;
and linked & mentioned in Wikipedia etc.
e. Social Media: Online discussions of a research article in social media like Facebook,
Twitter, Google+, LinkedIn, and Reditt.com etc.

• Data Sources in Altmetrics
115/22/2020 Copy-Left from Saptarshi Ghosh

 Web of Science:
WoS is the ISI-Thomson Reuters’s flagship product to facilitate a research platform. It is
the single destination to the world’s largest collection of research data, books,
journals, proceedings, publications and patents covering: 100+ years of abstracts; over
90 million records covering 5,300 social science publications in 55 disciplines; 800
million+ cited references; 8.2 million records across 160,000 conference proceedings
across regions, all disciplines and content types.

 Scopus:
Scopus is the largest abstract and citation database of peer-reviewed literature:
scientific journals, books and conference proceedings. Delivering a comprehensive
overview of the world's research output in the fields of science, technology,
medicine, social sciences, and arts and humanities, Scopus features smart tools to
track, analyze and visualize research. It covers 55 million records, 21,915 titles and
5,000 publishers.

 Google Scholar:
• Google Scholar provides a simple way to
broadly search for scholarly literature.
From one place, we can search across
many disciplines and sources: articles,
theses, books, abstracts and court
opinions, from academic publishers,
professional societies, online
repositories, universities and other web
sites. Google Scholar helps us find
relevant work across the world of
scholarly research.
• Google Scholar Citations provide a
simple way for authors to keep track of
citations to their articles. We can check
who is citing our publications, graph
citations over time, and compute several
citation metrics.
• Google Scholar Metrics provide an easy
way for authors to quickly gauge the
visibility and influence of recent articles
in scholarly publications. Scholar Metrics
summarize recent citations to many
publications, to help authors as they
consider where to publish their new
research.

 Mendeley:
Mendeley is one most widely used Altmetrics services - the number of articles with
Mendeley bookmarks is similar to the number of articles that have citations. Mendeley
provides information about the number of readers and groups. In contrast to CiteULike
no usernames for readers are provided, but Mendeley provides basic information
regarding demographics such as country and academic position. Mendeley is a social
bookmarking tool used by scholars and the metrics probably reflect an important
scholarly activity - adding a downloaded article to a reference manager.

 CiteULike:
CiteULike is another social bookmarking tool, not as widely used as Mendeley and
without reference manager functionality. One advantage over Mendeley is that
usernames and dates for all sharing events are publicly available, making it easier to
explore the bookmarking activity over time.

• ResearchGate:
• ResearchGate was built by scientists, for scientists. It was founded in 2008 by
physicians Dr. Ijad Madisch and Dr. Sören Hofmayer, and computer scientist Horst
Fickenscher. ResearchGate today has more than 6 million members. It helps
researchers to connect, collaborate and make it easy for them to share and access
scientific output, knowledge, and expertise.

 Twitter:
Collecting tweets linking to scholarly papers is challenging, because they are only
stored for short periods of time (typically around 7 days). There is a lot of Twitter
activity around papers, and only a small fraction is from the authors and/or journal.
With some journals up to 90% of articles are tweeted, the number for new PLOS
journal articles is currently at about 50%. The Twitter activity typically peeks a few
days after publication, and probably reflects attention rather than impact.

 Facebook:
Facebook is almost as popular as Twitter with regards to scholarly content, and
provides a wider variety of interactions (likes, shares and comments). Facebook
activity is a good indicator for public interest in a scholarly article and correlates more
with HTML views than PDF downloads.

• Wikipedia:
Wikipedia is the most popular Free Encyclopedia in English available on web. Its web
pages are written collaboratively by anonymous internet volunteers. Scholarly content
is frequently linked from Wikipedia. As per a study, in the English Wikipedia the most
frequently cited publisher is Elsevier with close to 35,000 links. In addition to
Wikipedia pages, links to scholarly articles are also found on user and file pages.

A small number of service providers are currently providing comprehensive Altmetrics
service:
 Public Library of Science (PLOS): The open access publisher Public Library of
Science (PLOS) was the first organization to routinely provide Altmetrics on a large
number of scholarly articles published on its own publishing platform. The first version
of their article-level metrics service was started in March 2009. PLOS currently
provides usage data, citations and social web activity from 13 different data sources.
The article-level metrics data are provided via an open and free API “PLOS Article-
Level Metrics (ALMs)” (http://github.com/articlemetrics/alm/wiki/API) and as monthly
public data dump. Its Altmetrics data on articles includes such as: views, citations,
save, discussions and recommendation etc. (http://article-level-metrics.plos.org/)

 Altmetric.com: Altmetric, a London-based commercial start-up started by Euan Adie
in 2011. Its mission is to track and analyse the online activity around scholarly
literature by maintaining a cluster of servers that watch social media sites, blogs,
newspapers and magazines etc. for any mentions of scholarly articles. Altmetric
Explorer, their first standalone version of Altmetric was released in February 2012.
Altmetric is being funded by Digital Science. It collects mentions of scholarly articles
from all across the web by gathering attention from newspapers, blogs, social media,
and more. For non-commercial use, the Altmetric Application Programming Interface
(API) is free for all. (http://www.altmetric.com)
Altmetric system tracks the “attention” that scholarly articles and datasets receive
online. It does this by pulling in data from three main sources:
Social media like Twitter, Facebook, Google+, Pinterest, Reddit, and LinkedIn;
Mainstream media - The Guardian, New York Times and science specific (New
Scientist, Scientific American) and blogs.
Online reference managers like Mendeley and CiteULike.
Altmetric can optionally harvest download counts from publishers.

 ImpactStory: ImpactStory is an open-source, web-based tool that helps researchers
explore and share the diverse impacts of all their research products - from traditional
ones like journal articles, to emerging products like blog posts, datasets, and software
etc. ImpactStory was being started as “Total-Impact”, a hackathon project at the
Beyond Impact Workshop by Jason Priem and Heather Piwowar in 2011. It is being
funded by the National Science Foundation (NSF) and the Alfred P. Sloan Foundation;
and a non-profit service provider. (https://impactstory.org/about)

 Plum Analytics: Plum™ Analytics is a profit-making organization founded by Andrea
Michalek and Mike Buschman in late 2011. Its product “PlumX” is the impact
dashboard for measuring research output. Plum Analytics is now a part of EBSCO
Information Services which major aim is to provide Altmetrics data to universities and
libraries. It provides Altmetrics data for assessing research by analyzing the five
categories of metrics which are: Usage, Captures, Mentions, Social media activities,
and Citations. (http://www.plumanalytics.com).
Metrics Data Sources
Usage downloads, views, book
holdings, ILL, document
delivery
DSpace, ePrints, PLoS, bit.ly, Facebook, Dryad,
Figshare, SlideShare, GitHub, IRs, WorldCat.
Captures favorites, bookmarks, saves,
readers, groups, and watchers
CiteULike, Delicious, SlideShare, YouTube,
Mendeley
Mentions blog posts, news stories,
Wikipedia articles, comments,
reviews
Facebook, Reddit, SlideShare, Vimeo, YouTube,
GitHub, Wikipedia, Research blogging,
ScienceSeeker, Amazon
Social
Media
Recommendations, tweets,
likes, shares, ratings
SourceForge, Figshare, Reddit, Facebook, Twitter
Citations Citations count PubMed, Scopus, USPTO

Many major publishers have already incorporated Altmetrics applications into their
publishing platform.

• Advantages of Altmetrics:
Priem, Taraborelli, Groth, and Neylon (2010) noted the limitations and slowness of peer
review and citations process in the post “Altmetrics: a Manifesto”, and suggested that
the speed with which Altmetrics data are available could potentially lead to real-time
recommendation and collaborative filtering systems” to assess the quality of research.
* Broadness: Potential for measuring the broader impact of research, that is, beyond
science; can deliver more transparent descriptions of the interest, usage and reach of
scholarly products; provide more diverse and nuanced forms of impact analyses than
traditional metrics; and offer access to the opinions of a wider audience, such as
professionals, undergraduates, government and – as a whole – the interested general
public.
* Speed: Often more timely data, showing evidence of impact in days or weeks instead of
years. As soon after publication, a paper is read, bookmarked, saved, annotated and
discussed within academic circles and by the public.
* Diversity: Altmetrics are not only more diverse in kinds of data and accordingly numbers
of data sources (whereas for traditional citations only the cited references in journals
serve as data source), but also allow for evaluation of a greater diversity of products, i.e.,
not just publications, also products might be datasets, software, copyrights, algorithms,
grey literature, and slides etc.; a diverse audiences too.
* Openness: Indications of societal impact as Altmetrics data are based on web and open
access platforms.

• Limitations of Altmetrics:
Despite being considered as a alternative metrics to traditional metrics in recent time,
Altmetrics has many flaws in it:
* It may takes time to clearly understand the quality and applicability of a research in a
broader approach.
* Subject to gaming: Social media and usage statistics are vulnerable as these can be
manipulated and subject to gaming too which will directly effect the Altmetrics data and
its result.
* Self-promotion: It may be possible that a same researcher may use various platforms
extensively to promote his/her research article. Thus, Altmetrics data should not be
necessarily refers to a greater influence or impact of research.
* Altmetrics is more interested in things that can be measured like ‘attentions’.
* Altmetrics has lack of presence in ‘dark social’.
* Technology is constantly changing in nature, so as the social web platforms. The
platforms, the sources which are being used for Altmetrics data today may be obsolete
or would not even exist in future in the web. So, there would not be accuracy in data
and stability in data sources.
* Till date, there is no such defined and clear indicators to measure Altmetrics data.
* World Wide Web is the base to have Altmetrics data which needs much more social
media literacy among the Altmetrics researchers. At time, researchers may not be
aware of all available web sources.

• Altmetrics v/s Traditional Metrics:
* Time: Citations took years to occur; but Altmetrics data like tweets, blog
posts, tags, face book shares, comments and bookmarks tend to occur much
more quickly after publications.
* Impact: Citation measures and Impact factors neglect impact outside the
academic. But, Altmetrics provides fast evidence of public reach and influence
of research.
* Sources: Citation data sources were recognized and universally accepted by
the research community i.e. Web of Science and Scopus. Whereas, Altmetrics
data sources are very much diverse in the web and also not always known
and used by the research community uniformly.
* Indicators: Citation metrics have well-developed theories and methods such
as Impact Factor, H-Index, JIF etc. to measure; whereas in Altmetrics there is
no such defined and clear theories and indicators to measure the data
sources till date.

 Librarians can use Altmetrics to explore a certain topic, measure public attention to
their institutional research output produced by its own faculty members and
researchers. They can use Altmetrics as a value-added application by integrating it
with their IRs which would help them to track the use of research articles.

 Barbaro, Gentili, and Rebufi (2014) suggested that with the help of Altmetrics data
like readership data from Mendeley; librarians can track, collect and inform emerging
research conversations penetrating in the web with its research community.
 Librarians can engage themselves in early Altmetrics education for their academic
community and outreach too.

• References:
 AOASG. (2013). Altmetrics and open access: a measure of public interest. Retrieved from
http://aoasg.org.au/altmetrics-and-open-access-a-measure-of-public-interest/
 Barbaro, A., Gentili, D., & Rebufi, C. (2014). Altmetrics as a new indicators of scientific impact. Journal of the
European Association for Health Information and Libraries, 10 (1), 3-6.
 Buschman, M., & Michalek, A. (2013). Are Alternative Metrics Still Alternative?. Bulletin of the American Society
for Information Science and Technology, 39(4), 5-3. Retrieved from http://www.asis.org/Bulletin/Apr-
13/AprMay13_Buschman_Michalek.html
 Kwok, Roberta (2013). Research impact: Altmetrics make their mark. Nature, 500, 491-493. doi:10.1038/nj7463-
491a
 Lagace, N. (2013, June 20). NISO to Develop Standards and Recommended Practices for Altmetrics. Retrieved from
http://www.niso.org/news/
 Liu, J., Adie, E., Bishop, L., & Venis, S. (2013). Poster: Using altmetrics and citation counts to assess the social and
academic impact of Medecins Sans Frontieres publications. figshare.
http://dx.doi.org/10.6084/m9.figshare.684926
 Mounce, R. (2013). Open Access and Altmetrics: Distinct but Complementary. Bulletin of the American Society for
Information Science and Technology, 39(4), 14-17. Retrieved from http://www.asis.org/Bulletin/Apr-
13/AprMay13_Mounce.html
 Piwowar, H. (2013). Altmetrics: What, Why and Where?. Bulletin of the American Society for Information Science
and Technology, 39(4), 8-9, Retrieved from http://www.asis.org/Bulletin/Apr-13/AprMay13_Piwowar.html
 Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved from
http://altmetrics.org/manifesto/
 William. (2013, June 2012). What flavor is your research? Take our survey on grant review and tenure decision
making. [Web log post]. Retrieved from http://blog.mendeley.com/mendeley-use-case/what-flavor-is-your-
research-take-our-survey-on-grant-review-and-tenure-decision-making/

"WHAT questions do you have for me?”
Thank You!

ALTMETRICS : A HASTY PEEP INTO NEW SCHOLARLY MEASUREMENT

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to ALTMETRICS : A HASTY PEEP INTO NEW SCHOLARLY MEASUREMENT

Similar to ALTMETRICS : A HASTY PEEP INTO NEW SCHOLARLY MEASUREMENT (20)

More from Saptarshi Ghosh

More from Saptarshi Ghosh (20)

Recently uploaded

Recently uploaded (20)

ALTMETRICS : A HASTY PEEP INTO NEW SCHOLARLY MEASUREMENT