This document discusses automatically linking software packages to GitHub repositories. It describes linking package manager data to GitHub data to determine repository relationships (good). However, there are also challenges (bad, ugly) including irrelevant, invalid, or ambiguous links. The document proposes an automated process to classify links as good, bad, or ugly based on checking for repository existence and renaming. It provides an example analysis of the RubyGems ecosystem where most links were good but some fell into the bad or ugly categories.

1. Automatic link extraction:
The good, the bad and the ugly in
software ecosystem mining
Eleni Constantinou
Tom Mens

2. Software ecosystem health
Prof. Dr. Tom Mens
Software Engineering Lab
University of Mons
• Ecosystem evolution
• Ecosystem health
www.secohealth.org
2

3. Software ecosystem health
Sustainability
Longevity Growth
Success
Resilience Survival
Diversity Popularity
3

4. Software ecosystem health
4

5. Software ecosystem evolution
Combine data from different sources
• Package manager data
• Repository activity
5

6. Combine Information
Package manager GitHub repositories
6

7. Combine Information
Package manager GitHub repositories
GOAL
Link packages to GitHub
repositories
6

8. Combine Information
Package manager GitHub repositories
6

9. Combine Information
7

10. Combine Information
7

11. Combine Information
7

12. Combine Information
8

13. Combine Information
8

14. The good
9

15. The good
10

16. The bad
11

17. The Bad
Irrelevant links
12

18. The bad - Irrelevant links
13

19. The bad - Irrelevant links
14

20. The Bad
Invalid GitHub links
15

21. The bad - Invalid GitHub links
16

22. The bad - Invalid GitHub links
16

23. The bad - Invalid GitHub links
17

24. The Bad
GitHub pages
18

25. The bad - GitHub pages
19

26. The bad - GitHub pages
https://github.com/krmbzds/qf
19

27. The Bad
GitHub user profiles
20

28. The bad - GitHub user profiles
21

29. The bad - GitHub user profiles
https://github.com/ghaki/[GEM_NAME]
21

30. The Bad
Sub-directory GitHub links
22

31. The bad - Sub-directory GitHub links
23

32. The bad - Sub-directory GitHub links
23

33. The ugly
24

34. The ugly
25

35. The ugly
25
Company Webpage

36. The ugly
25
Company Webpage
GitHub organization profile

37. The ugly
25
Company Webpage
GitHub organization profile
GEM_NAME
GEM_
NAME
GEM_REPOSITORY

38. How to automate
this process?
GitHub link?
Repository
link?
No
Yes
GitHub page?
Link exists?
Yes
No
Link to
GitHub
repository?
IRRELEVANT
Repository
found?
Yes
No
DELETED
REPOSITORY
Same repository
name (GitHub
API)?Yes
No
RENAMED
REPOSITORY
GOODYes
No
IRRELEVANT
User/project
page?
Yes
No IRRELEVANT
USER PROFILE/
PROJECT PAGE
Yes
No
IRRELEVANT
COMPANY
PAGE
Yes
No
26

39. How to automate this process?
Package
manager
External
Source
Software
repository
27
GitHub links
Investigate external sources

40. How to automate this process?
Package
manager
External
Source
Software
repository
27
GitHub links
Investigate external sources

41. The RubyGems ecosystem
Category Packages
Empty Fields 19,645
Good 69,740
Bad
Irrelevant links 8,650
Invalid GitHub links
(deleted)
11,153
Invalid GitHub links
(renamed)
7,069
GitHub pages 401
GitHub user profiles 858
GitHub sub-directories 172
Ugly (candidates) 3,016
28
120,704
packages

42. Improve and fully automate this process
Verify our process with other ecosystems
Provide support for additional platforms
Merge multi-platform activity
What next?
29

43. http://www.econst.eu
@eleni_const