Evolution and state-of-the art of Altmetric research: Insights from network analysis and altmetric analysis Authors: Hiran Lathabai, Thara Prabhakaran, Manoj Changat Workshop Website: http://www.altmetrics.ntuchess.com/AROSIM2018/

1. Evolution and state-of-the art of Altmetric research:
Insights from network analysis and altmetric analysis
Hiran H. Lathabai,Thara Prabhakaran, Manoj Changat
hiranhl007@gmail.com, thara.dfs@gmail.com , mchangat@gmail.com
Department of Futures Studies, University of Kerala, Thiruvananthapuram-695581, India
Abstract: Altmetrics or alternative metrics, that includes a growing set of proxy measures of the
visibility, popularity, impact, etc. of scholarly documents, is advocated by many researchers in the
scientific community as a potential driver of paradigm shift in scientometrics that could complement the
traditional approach to scholarly research assessment. In this background, it is the right time to assess
the relatively new field –Altmetric research, with an objective to track important milestones in its
evolution and to draw insights about the present state-of-the-art of Altmetric research. Path analysis,
one of the effective tools in network scientometric approach is used for the evolutionary assessment.
Once the key papers that form the backbone of altmetric research are identified, these are evaluated in
an altmetric perspective. ResearchGate (RG) and Mendeley readership are used for the altemtric
evaluation of the key papers. It is found that key papers exhibit a fair performance in terms of of the
chosen altmetric scores. From a correlation analysis of readership and citation in both the sources, RG
readership is found to correlate less with citations than Mendeley readership. An interesting direction
that can be taken by altmetric research is the network approach to altmetric analysis and we envision
the realization of ‘altmetric networks’ or ‘altnets’ as a blend of both worlds that could deliver best.
Keywords: Altmetrics, Network scientometrics, Path analysis, Main path, Critical path, Altmetric
networks
Introduction: Altmetrics or alternative metrics are non-traditional metrics for research impact
assessment. Despite its potential to effectively assess progress and impact of research in some fields and
non-standard research outputs, altmetric field is gaining accord as a complementary extension of
traditional fields than a competing one. Therefore, need to identify and retain ‘best of both worlds’ is
prevailing.
Objectives: Our main objective is the identification of evolutionary trajectories of the field ‘altmetrics’.
Evaluation of key papers in the important evolutionary trajectories using altmetrics is performed.
Network Scientometrics and path analysis: Establishment of Science Citation Index (SCI) [1] by
Garfield et al. and Price’s groundbreaking works [2] marked the rise of modern scientometrics and
network approach. Hummon and Doreian introduced path analysis [3] along with two methods to assign
traversal weights- Search Path Link Count (SPLC) and Search Path Node Pair (SPNP). Batagelj’s
introduction of Search Path Count (SPC) method [4] as an improved way to compute SPLC and/or
SPNP weights led to the implementation of path analysis tools in the software PAJEK [5]. Two kinds of
paths- main path and critical path were proposed in the same.
Path method
fig. 1 A schematic diagram of path retrieval in citation network
Data : Collected from WoS (Web of Science) on 18th January, 2018 using keyword ‘Altmetrics’. 315
papers were retrieved and network created consists of 315 papers and 892 links. Network of altmetric
literature is shown in fig. 2. Using path method, main path and critical path of the citation network are
obtained, shown in fig. 3 (left) and (right).
fig. 2 citation network of altmetric during 2012-2018 (January 18)
SPC
weights
Local search
Global search
Main path
Critical path
Content analysis revealed that among altmetric sources, F1000 (Faculty of 1000) is reputed for its post
publication peer review and field/subject classification and useful for funding or policy purposes. It is apt for
scientific disciplines such as medical field and others.
In terms of coverage, Mendeley is found to be a reliable source. Social media sites like twitter, facebook can be
depended to assess wider impact of research in humanities and social sciences. Goodreads is one of the good
sources for impact assessment of books.
Need for mean normalized altmetrics for assessment, need for benchmark to test reliability of altmetrics etc., are
discussed by state-of-the-art works.
fig. 3 (left ) Main path and (right) critical path of citation network of altmetrics
There are 18 papers in main path, 11 in critical path. 24 unique papers and 5 common papers.
Table 1. Key papers found in both paths, their almetric and citation scores in two sources
Correlation analysis: Among 24 key papers, 281 Thelwall M, 2017 is found to be an outlier.
It’s title is ResearchGate vs. Google Scholar: Which finds more early citations?
RG reads= 2336, RG citations=3, Mendeley reads= 46, Mendeley citations= 3.
Is this work a Potential game changer? Or is there a reliability problem for RG reads?
Read count in RG is showing almost zero correlation to RG citations as well as Mendeley reads and citations.
Read count in Mendeley shows high correlation to RG citations (0.9) and Mendeley citations (0.81). Disregarding
281 Thelwall M, 2017, RG read count improves its association with others (0.25 -0.33 range, which is still weak).
Among the two, RG is found to be best for its citation tracking and Mendeley is more reliable on readership counts
(cannot be confirmed by analysis of present dataset, extensive study required).
Conclusion: Network methods such as path analysis can be used for early assessment of scholarly publications.
Combination of network approach and altmetric approach can lead to powerful assessment methods and hence the
concept of altmetric networks or altnets is envisioned.
Acknowledgement: We gratefully acknowledge the AROSIM 2018 Organizers for supporting us with travel
grant.
References:
1. Garfield, E. (1955). Citation indexes for science. Science, 122, 108-111.
2. Price, D. J. D. S. (1965). Networks of scientific papers. Science, 510-515.
3. Hummon, N. P., & Dereian, P. (1989). Connectivity in a citation network: The development of
DNA theory. Social networks, 11(1), 39-63.
4. Batagelj, V. (2003). Efficient algorithms for citation network analysis. arXiv preprint cs/0309023.
5. Batagelj, V., & Mrvar, A. (1998). Pajek-program for large network analysis. Connections, 21(2),
47-57.
Altmetrics for Research Outputs Measurement and Scholarly Information Management
AROSIM 2018 (January 26, 2018)
Label Title
RG
Reads
RG
cites
Mendeley
Reads
Mendeley
cites
14 Mohammadi E,
2013
Assessing non-standard article impact
using F1000 labels 235 38 86 33
18 Sud P, 2014 Evaluating altmetrics 127 73 273 60
34 Bornmann L, 2014
Do altmetrics point to the broader
impact of research? An overview of
benefits and disadvantages of altmetrics 277 95 380 76
40 Hammarfelt B,
2014
Using altmetrics for assessing research
impact in the humanities 114 70 238 55
313 Correia A, 2018
Scientometric analysis of scientific
publications in CSCW 47 0 5 0