Feature Analysis of Research Metrics Systems Authors: Aarthy Nagarajan, Mojisola Erdt, Yin-Leng Theng Workshop Website: http://www.altmetrics.ntuchess.com/AROSIM2018/

1. www.altmetrics.ntuchess.com
Feature Analysis of Research Metrics Systems
Abstract
Feature Analysis of
Systems Identify presence of
Key Features
Limitations
Inter-Rater Reliability Method (IRR)
Unsure as per one or both coders; 0.5
Present; 2
Not present as per one of the coders; 1
0 5 10 15 20 25 30 35
Verification of metrics provided
DOI or URL provided to the research work
ORCID
DOIS
Google Scholar ID
ResearcherID
ScopusID
Offers own API
Bookmarklet
Plugins
Profile/ Logo link to profile embedding
Supports Search
Supports Sorting
Supports Filtering
Allows import
Allows export
RSS feed
Supports article sharing to third party platform
Novel in-house metrics offered
Publically known factors contributing to metric
The context of the metrics is provided
Artifact level metrics
Article level metrics
Journal level metrics
Author/ Researcher level metrics
Institutional/ Organisational level metrics
Country level
Registration
Free access
Subscription/License Needed
Time-series (Temporal) analysis
Spatial analysis
Cross-metric Analysis
Informational details shown
Google Scholar Citations
WoS Citations
Scopus Citations
H-index
Impact Factor
Number of publication/articles
Recommendations
Dashboard provided
No of full text downloads
No. of views
Altmetric.com score
Twitter
Facebook
Mendeley
DataQualityUsercontrol&EmbeddingMetricsrelated
User
AccessVisualizationBibliometricSources
User-
friendl
y
interfa
ceAltmetricSources
Coverage per system
Features
ImpactStory KUDOS Aminer Publish or Perish Figshare ORCID
Mendeley Altmetric.com Plum Analytics Scholarometer ResearchGate Academia.Edu
Elsevier ScienceDirect Elsevier Scopus Springer (Springer Nature) Emerald Insight Wiley Online Library
Inter-Rater
Reliability was
found to be
the highest
(κ = 0.675,
p<0.001) for
features
related to
altmetric
sources
Inter-Rater
Reliability was
found to be
the lowest
(κ = 0.406,
p<0.001) for
features
related to
metric
validation
User-control &
embedding features
are covered by
maximum number of
systems
Visualization
is the least
covered feature
Mendeley and
Elsevier Scopus
show presence of
maximum number
of features
Since the advent of “Altmetrics”, the number of research metrics systems that have been developed has increased. As
part of this study, we have analysed the presence of some prominent features in 10 altmetrics systems, 2 academic
social networking systems, and 6 other systems covering digital libraries, databases and publisher systems. We
considered a total of 48 individual features under 14 broad categories for the analysis. We found that user-control and
embedding features were present in most of the systems, whereas visualization features were present in few of the
systems. The results of this study give insights into features that could be improved and developed in future.
Authors
Aarthy Nagarajan Dr Mojisola Erdt Prof Theng Yin-Leng
Research Engineer Research Fellow Professor
Nanyang Technological University Nanyang Technological University Nanyang Technological University
This research is supported by the National Research Foundation, Prime Minister's Office, Singapore under its Science of Research, Innovation and Enterprise
programme (SRIE Award No. NRF2014-NRF-SRIE001-019). Any opinions, findings, conclusions and/or recommendations expressed in this material are those of
the author(s) and do not necessarily reflect the views of the National Research Foundation Singapore.