A Comparative Investigation on Citation Counts and Altmetrics between Papers Authored by Universities and Companies in the Research Field of Artificial Intelligence Authors: Feiheng Luo, Han Zheng, Mojisola Erdt, Aravind Sesagiri Raamkumar, Yin-Leng Theng Workshop Website: http://www.altmetrics.ntuchess.com/AROSIM2018/

1. A Comparative Investigation on Citation Counts and
Altmetrics between Papers Authored by Top Universities
and Companies in the Research Field of Artificial
Intelligence
26 January 2018
Feiheng Luo, Han Zheng, Mojisola Erdt, Aravind Sesagiri Raamkumar, Yin-Leng Theng
Wee Kim Wee School of Communication and Information

2. Introduction
• Artificial Intelligence (AI) is a most popular topic
• Academia and industry have made a lot of efforts in AI
• Impressive results have been achieved
• In our current study, we evaluate the research outputs
produced by academia and industry in AI

3. Introduction
• For research outputs, we only focus on scientific papers
• How to evaluate scientific papers?
– Citation counts
– Altmetrics
• Comparative investigation on papers published by top
academic universities and industrial companies in AI

4. Data Collection
• Data collection was completed in August 2017
• Data sources:
– Scopus
– Google Scholar
– Altmetric.com
– PlumX

5. Data – Identification of Papers
• Identified AI related publication venues
• Scopus
– Provides a publication venue ranking for computer science
– Selected top 10 journals and top 10 conferences based on the Scopus CiteScore
• Google Scholar
– Provides a list of top 20 publication venues in AI
– arXiv venues were excluded, since arXiv papers are mostly preprints
• Excluded venues in which paper DOIs were not available

6. Data – Identification of Papers
• 30 publication venues without overlaps were identified
• 14,647 research papers published in the 30 venues in
2015 and 2016 were retrieved from Scopus

7. Data: Summary of Selected Venues
S/N Venue Source # of papers
1 Expert Systems with Applications GS 1,393
2 IEEE Transactions on Neural Networks and Learning Systems GS 542
3 IEEE Transactions on Fuzzy Systems GS 330
4 Knowledge-Based Systems GS 68
5 Applied Soft Computing GS 1,198
6 Neurocomputing GS 2,901
7 Neural Networks GS 93
8 Engineering Applications of Artificial Intelligence GS 385
9 Neural Computing and Applications GS 321
10 Robotics and Autonomous Systems GS 249

8. Data: Summary of Selected Venues
S/N Venue Source # of papers
11 IEEE Transactions on Pattern Analysis and Machine Intelligence Scopus 386
12 IEEE Transactions on Evolutionary Computation Scopus 124
13 Wiley Interdisciplinary Reviews: Computational Molecular Science Scopus 32
14 ACM Computing Surveys Scopus 111
15 International Journal of Computer Vision Scopus 150
16 IEEE Communications Magazine Scopus 526
17 IEEE Internet of Things Journal Scopus 162
18 IEEE Journal on Selected Areas in Communications Scopus 477
19 IEEE Wireless Communications Scopus 181
20 IEEE Signal Processing Magazine Scopus 125

9. Data: Summary of Selected Venues
S/N Venue Source # of papers
21 Proceedings of the IEEE Computer Society Conference on Computer Vision
and Pattern Recognition
Scopus 1,245
22 Proceedings of the IEEE International Conference on Computer Vision Scopus 600
23 International Conference on Architectural Support for Programming
Languages and Operating Systems
Scopus 102
24 Proceedings of the ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining
Scopus 455
25 Proceedings of the Annual ACM Symposium on Theory of Computing Scopus 185
26 Proceedings of the ACM SIGMOD International Conference on Management
of Data
Scopus 390
27 Proceedings - International Conference on Software Engineering Scopus 582
28 Proceedings of the Annual International Conference on Mobile Computing
and Networking, MOBICOM
Scopus 223
29 Proceedings - IEEE INFOCOM Scopus 1,004
30 Proceedings of the ACM SIGPLAN Conference on Programming Language
Design and Implementation
Scopus 107

10. Data – Identification of Affiliations
• Identified prestigious affiliations as a part of this step
• We sorted the affiliations by the number of affiliated papers,
and identified the top 10 universities and the top 10 companies
• In general, academic universities are more engaged in the
research community, and thus we additionally included the top
10 universities on the QS World University Rankings 2015 for
Computer Science
• Since there was an overlap between the two lists of top 10
universities, we finally obtained 19 unique universities and 10
companies

11. Data: Summary of Identified Universities and Companies
S/N Affiliation Type # of papers
1 MIT University 175
2 Stanford University University 181
3 University of Oxford University 47
4 Carnegie Mellon University University 210
5 Harvard University University 58
6 University of California, Berkeley University 211
7 University of Cambridge University 39
8 The Hong Kong University of Science and Technology University 108
9 ETH Zurich - Swiss Federal Institute of Technology University 119
10 Princeton University University 75

12. S/N Affiliation Type # of papers
11 Tsinghua University University 450
12 Harbin Institute of Technology University 229
13 Southeast University University 198
14 Nanyang Technological University University 203
15 Huazhong University of Science and Technology University 205
16 Xidian University University 194
17 Shanghai Jiao Tong University University 210
18 Zhejiang University University 180
19 National University of Singapore University 173
Data: Summary of Identified Universities and Companies

13. S/N Affiliation Type # of papers
20 Microsoft Research Company 318
21 IBM Research Company 136
22 Google Company 121
23 Huawei Technologies Company 88
24 NEC Laboratories America Company 61
25 Ericsson Research Company 59
26 Nokia Company 59
27 Samsung Electronics Company 57
28 Yahoo Labs Company 51
29 Adobe Research Company 49
Data: Summary of Identified Universities and Companies

14. Data – Metrics Collection
• Citation counts retrieved from Scopus
• Altmetrics data retrieved from PlumX
– Usage, e.g., views and downloads
– Captures, e.g., Mendeley reader count
– Mentions, e.g., blog mention count
– Social media metrics, e.g., Tweet count

15. Data – Metrics Collection
• Altmetrics data retrieved from Altmetric.com
– For some papers, both final version and arXiv preprint version were
indexed separately on the Altmetric.com, and we collected the data for
both versions and extracted the higher values for each metric
– Altmetric Attention Score
– Mendeley reader count
– Tweet count

16. Paper Classification for Results
• Papers were classified into three groups:
– University-authored papers: the associated affiliations of papers involved
the identified universities
– Company-authored papers: the associated affiliations of papers involved
the identified companies
– Co-authored papers: the associated affiliations of papers involved the
identified universities and companies

17. Results
Comparisons of citation counts

18. Results
Comparisons of data collected from PlumX

19. Results
Comparisons of data collected from Altmetric.com

20. Results
• Comparisons of the most impactful paper counts
University Company Co-authored Total
Highly cited papers
(citation counts >= 100)
9 10 1 20
Highly discussed papers
(Altmetric Attention Scores
>= 100)
3 8 1 12

21. Conclusions
• University-authored papers received slightly higher citation
counts than company-authored papers
• Company-authored papers and co-authored papers obtained
considerably more attention online than university-authored
papers
• Limitations:
– Comparison results could be affected by the unbalanced numbers of the
top universities and companies
– Some important venues in AI such as the International Conference on
Machine Learning (ICML) were not included in the current study, due to
the lack of DOIs of the papers published in these venues

22. THANK YOU