AndroLib is a search-based approach for recommending libraries for Android apps. It uses NSGA-II to generate a set of non-dominated library recommendation solutions based on three objectives: maximizing recommended library co-usage, maximizing library functional diversity, and maximizing reuse from successful apps. This addresses limitations of existing approaches which do not customize recommendations for Android or consider libraries together. An evaluation compares AndroLib to state-of-the-art techniques to show it achieves more accurate recommendations.

1. Third-Party Software Library Reuse :
From Adoption to Migration
1 ETS Montreal, University of Quebec, Montreal, QC, Canada
2 Rochester Institute of Technology, Rochester, NY, USA
3 Microsoft Corporation, WA, USA
International Conference on Software and Systems Reuse (ICSR), 1-4 December 2020
Ali Ouni 1 Mohamed Wiem Mkaouer 2 Hussein AlRubaye 2,3

2. ● Associate Professor, ETS Montreal, University of Quebec, QC, Canada
● He received his PhD degree from the University of Montreal in 2015
● Software Technology and Intelligence (STI) Research Group: https://ouniali.github.io/
● Research interests
○ Software Re-engineering and modernization
○ Software Maintenance and Evolution
○ Software Quality
○ Empirical Software Engineering
○ Search-based Software Engineering
○ Application of AI to Software Engineering
Who is Ali Ouni?

3. ● Assistant Professor in the Software Engineering Department, Rochester Institute of
Technology, NY, USA.
● He received his PhD in 2016 from the University of Michigan-Dearborn.
● His research interests include
○ software quality
○ systems refactoring
○ model-driven engineering
○ software testing
○ Use computational search and evolutionary algorithms to address several software
engineering problems
Who is Mohamed Wiem Mkaouer?

4. ● Senior Software Engineer at Microsoft Corporation.
● PhD from Rochester Institute of Technology, 2020
● He has 8 successful apps in Google and iOS stores with millions of users who
are using his apps.
● He is working as software developer for 9 years with .NET, C++, and web security
● He have a successfully paid tutorial in Udemy.
● He got Microsoft Most Valuable Professional.
● He is working on research to build tools and algorithms that automate the
process of API Migration.
Who is Hussein AlRubaye?

5. ● What is third party software library?
● Software library adoption
● Software library update
● Software library migration
● Demonstration: Recommending library migration
● Summary and challenges
Outline
5

6. In 1995, . . ., 2018, 2019, 2020, somewhere out there …
A developer is in need of a function, feature ...
My code
My code without
Third-party
libraries
[Credit to @Raula Gaikovina Kula]
6

7. Why write code when you can use a library?
Library
Dependencies
Source Code
Library/package
manager
7
[Credit to @Raula Gaikovina Kula]

8. What is a library API?
Application Program Interface (API) : a set of commands, functions, and
protocols, offered by third-party libraries, which programmers can (re)use
when building software
8

9. Libraries are…
● Useful
○ provide useful services
● Open
○ intended for open-ended set of diverse projects
● Stable
○ provide stability across multiple releases
● Growing
○ evolve from release to release to better meet client needs
Characteristics of libraries
9
Slide reference: https://www.unit5.org

10. ● The Java standard class library is part of any Java development environment
● Its classes are not part of the Java language per se, but we rely on them heavily
● Various classes we've already used (System , Scanner, String) are part of the Java standard class library
● Other class libraries can be obtained through third party vendors, or you can create them yourself
Example: J2SE
Package
java.lang
java.applet
java.awt
javax.swing
java.net
java.util
javax.xml.parsers
Purpose
General support
Creating applets for the web
Graphics and graphical user interfaces
Additional graphics capabilities
Network communication
Utilities
XML document processing
10
Slide reference: https://www.unit5.org

11. ● Consider L the set of libraries, each library 𝑙 ∈ 𝐿 is a standalone library with multiple versions, each version is
implemented and made available for use through function(s). These functions are publicly available in the form
of an API. Let F be the set of functions provided by the interface. i.e., for each 𝑙 ∈ 𝐿, there exist a set of functions
𝑓 ∈ 𝐹 such that 𝑎𝑝𝑖 𝑙 = 𝑓.
Library
11
Slide reference: https://www.unit5.org

12. ● It serves the purpose of providing unique identifiers to libraries. In this
chapter, we study the evolution of libraries that are managed by Apache
Maven, so we adopt its naming scheme:
< 𝐺𝑟𝑜𝑢𝑝𝐼𝐷 >. < 𝐴𝑟𝑡𝑖𝑓𝑎𝑐𝑡𝐼𝐷 >. < 𝑉𝑒𝑟𝑠𝑖𝑜𝑛 >
● e.g., org.apache.maven.1.0.1
● Versioning varies from one organization to another, for instance, Maven uses
the following strategy:
< 𝑚𝑎𝑗𝑜𝑟 >. < 𝑚𝑖𝑛𝑜𝑟 >. < 𝑝𝑎𝑡𝑐ℎ > −< 𝑡𝑦𝑝𝑒 >−< 𝑎𝑡𝑡𝑒𝑚𝑝𝑡 >
Naming Scheme
12

13. ● Let D be the set of dependencies and calls of all functions belonging to F and linking L to the software S. Formally,
for 𝑙 ∈ 𝐿, we define a set of functions 𝑓 ∈ 𝐹 such that, if 𝑙 𝑝 is used by 𝑠 𝑞 (p and q are arbitrary numbers
representing respectively the library and software versions) then we define a set of dependencies 𝑑 ∈ 𝐷 | 𝐷 ⊆
𝐹 | 𝑑𝑒𝑝 𝑙 𝑝, 𝑠 𝑞 = 𝑑 where dep(𝑙 𝑝, 𝑠 𝑞) returns the set of dependencies of 𝑙 𝑝 in 𝑠 𝑞. Note that we assume that a
software 𝑠 𝑞 depends on a library 𝑙 𝑝 if there exists at least one function, offered by 𝑙 𝑝, which is called by 𝑠 𝑞.
Dependency
13

14. ● Changing the library version, with another, typically more recent, one of the same library. Typically, a newer version of a library
maintains its core functionality while extending it by either updating its existing functions or/and by adding new functions. For a
given software S, let 𝑙 𝑝 ∈ 𝐿 be a library belonging to the version 𝑠 𝑞, the upgrade, denoted 𝑙 𝑝 → 𝑙 𝑝+1 requires that all functions of
𝑙 𝑝 are located 𝑙 𝑝+1 unless deprecated. Formally: if 𝑓𝑝 (respectively 𝑓𝑝+1) the functions offered by 𝑙 𝑝 (respectively 𝑙 𝑝+1) then 𝑓𝑝 ,
𝑓𝑝+1 ∈ 𝐹 | 𝑓𝑝 ⊆ 𝑓𝑝+1, expect for when some functions from 𝑓𝑝 might be deprecated or merged.
● 2 Methods:
○ Incremental. The library’s upgrade is reflected by incrementing its version.
○ Repackaging. Just like any regular Java project, libraries are subject to design improvement. This process includes
restructuring class files into multiple packages based on various criteria including their coupling and cohesion, semantic
similarity, collaboration in the functionality.
Library Upgrade
14

15. Incremental
Incremental. Upgrading log4j from 1.2.13 to 1.2.15 in the docx4j project [1].
[1] https://github.com/plutext/docx4j/commit/d1e28e6fbb3b5ed8974572920c0899254abbd49b 15

16. Repackaging
Upgrading jetty-all library in the Apache hive project[1].
[1] https://github.com/apache/hive/commit/191302cf4f9eae5ef51964bdab8d8e859292aa17
16

17. ● Defines the act of replacing one existing library with another newly introduced one(s). During the
migration, all the functions of the retired library are removed or replaced. For a given software S,
let 𝑙 𝑝 ∈ 𝐿 be a library belonging to the software version 𝑠 𝑞 to be replaced by another library 𝑙′ 𝑟 ∈
𝐿 in the next version 𝑠 𝑞+1. The migration process, denoted 𝑙 𝑝 → 𝑙′ 𝑟, is defined by the following
condition:
𝑑𝑒𝑝 𝑙 𝑝, 𝑠 𝑞 ≠ ∅ ∧ 𝑑𝑒𝑝 𝑙′ 𝑟, 𝑠 𝑞 = ∅ ∧ 𝑑𝑒𝑝 𝑙 𝑝, 𝑠 𝑞 ≠ ∅ ∧ 𝑑𝑒𝑝 𝑙′ 𝑟, 𝑠 𝑞 = ∅.
● 2 Methods:
○ Instant. The migration process is considered instant if the replacing library is added and
replaced library is removed simultaneously.
○ Delayed. The migration is considered delayed if, in contrast with the instant migration, the
retired library remains in the project POM file even if it no longer contributes to the
project.
Library Migration
17

18. Instant
Migrating from junit to testng and fest-assert-core in the ps2-parser project[1].
[1] https://github.com/ssindelar/ps2-parser/commit/4e76b35f32011159db321e8a1540d03e004d25e8
18

19. Instant
Replacing junit functions by fest-assert-core functions in the ps2-parser project.
Replacing junit functions by to testng functions in the ps2-parser project.
[1] https://github.com/ssindelar/ps2-parser/commit/4e76b35f32011159db321e8a1540d03e004d25e8
19

20. Delayed
Replacing org.redmine.ta.internal.json functions by to org.jason:jason functions in the redmine-java-api project. 20

21. • Evolving with frameworks is costly
– Struts 2.0.9 and 2.2.3.1
• Upgrading an authentication framework
• A whole week of work
– Linux Debian Distribution
• Upgrading Perl from 5.10 to 5.12 took
• Seven weeks to complete
API Evolution Challenges
Raemaekers et al. (ICSM 2012)
● Once API is released there’s no going back
● Breaking changes are bad news
● Either extend API in next release
● Or provide improved API alongside old API
21

22. Framework API
API Evolution challenges: Platform API
22
Wu et al. (SCAM 2014)

23. default Query<R> setHibernateFlushMode(FlushMode flushMode) {
setFlushMode( flushMode );
return this;
}
default org.hibernate.query.Query<R> setHibernateFlushMode(FlushMode
flushMode) {
setFlushMode( flushMode );
return (org.hibernate.query.Query) this;
}
hibernate/hibernate-orm
Version 5.2.0
API Evolution challenges: Platform API
23
Wu et al. (SCAM 2014)

24. default org.hibernate.query.Query<R> setHibernateFlushMode(FlushMode
flushMode) {
setFlushMode( flushMode );
return (org.hibernate.query.Query) this;
}
Version 5.2.0
API Evolution challenges: Platform API
24

25. Breaking
Changes
Elements removal
Method signature changes
Changes in deprecated elements
Elements addition
Elements deprecationField changes
(Dig and Johnson, ICSM, 2006)
Non Breaking
Changes
API Evolution challenges: Platform API
25(Dig and Johnson, ICSM, 2006)

26. Third-party Software Library reuse life-cycle
Adoption
How to find a useful
library to your
current software
system?
Update
Libraries are in
constant evolution.
How do developers
update their library
dependencies?
Migration
Other competing
libraries can take
the place of a given
library
Retirement
At a given point in time,
some libraries are no
longer maintained and
evolved by its original
devlopers
26

27. Libraries.io monitors 5.5 millionopen source packages
across 37 different package managers (Dec. 2020)

28. Finding a library for my project can be so …
28

29. Need for efficient library recommendation tools!

30. Android
3.04 million Android apps available on
Google Play store
>1k apps published everyday
>1.5 billion apps downloaded every
month
https://www.statista.com [Malavolta et al.]

31. 31
Android apps are eating the world!
31

32. Android Arsenal : 7026 free libraries for Android
Android apps development!
32
60% of code in Android apps constitutes sub-
packages from external libraries
90% of Android apps use 5 libraries or more
Android Arsonal
[Q. He et al., TSE 2020]

33. High user expectations
New feature requests
Rapid technology changes
Continuous feedback
Why finding efficient Android libraries is so important?
33

34. ● LibRec (Thung et al., 2013)
○ Association rule mining and collaborative filtering
○ Historic software artifacts usage to determine commonly used libraries
● LibFinder (Ouni et al., 2017)
○ Search-based approach by leverage libraries usage and source code semantic similarity
● LibCUP (Saied et al., 2018)
○ Clustering technique to mine and recommend library co-usage patterns
● CrossRec (Nguyen et al., 2020)
○ Collaborative filtering by relying on the set of dependenciesincluded in the project being developed
● LibSeek (He et al., 2020)
○ Collaborative filtering based on adaptive weighting mechanism to predict Android libraries by
leveraging information about similar third-party libraries and systems.
How researchers tackled this?
34

35. Limitations – problem statement
Not customized
for Android
Despite being good in finding
libraries in desktop software
systems, existing techniques
are not adapted to mobile apps
Individual
recommendations
The main limitation
of these existing
approaches resides in the
fact that libraries are
recommended individually
Library poularity
dilema!
A small fraction of popular
libraries dominates the market while
most other libraries are ill-served. No
chance for new emerging libraries to
be recommended!
✗
35

36. How would you address this problem?
?
? ? ?
?
??
?
?
? ? ??
?
?
?
?

37. OSS Android Apps
& Third-Party Libraries
AndroLib
Data Encoding
Module Library Recommendation Module Libraries Ranking
Module
Android developer
Multi-objective Search-based
Library Recommendation
(NSGA-II)
A B C
AndroLib: proposed approach
37

38. Search algorithm: NSGA-II adaptation
• Encoding of candidate solutions
• Definition of fitness function
• Definition of genetic operators
• Generate initial population
• NSGA-II: Non-dominated Sorting Genetic Algorithm (K. Deb et al., 2002)
38

39. Lib 1 Lib 2 Lib 3 Lib 4 Lib 5
App 1 1 0 0 0 0
App 2 1 1 0 0 0
App 3 0 0 0 1 1
App 4 0 1 0 1 1
App 5 1 0 1 0 0
App 6 1 1 1 0 1
…. …
App 1
App 2
App 5
App 3
App 4
App 6
Lib 1
Lib 2
Lib 3
Lib 5
Lib 4
Graph encoding
Data and solution encoding
MPAndroidChart Picasso Facebook CAMView Dagger
Dataset encoding
NSGA-II solution encoding
0 1 2 3 4
39

40. Three objective functions
1. Maximize Recommended Library Co-Usage (RLC)
○ Discover third-party libraries that are commonly used together by other Android apps
2. Maximize Library Functional Diversity (LFD)
○ Diversify the recommended libraries in terms of their functionality based on the library
categories, e.g., image loader, animation, graphics, advertisement, camera, etc.
3. Maximize Reuse from Successful Apps (RSA)
○ Increase the chance of recommending commonly used libraries in apps having high user
rating, while decreasing the chance to recommend those from low rated apps.
Fitness function
40

41. RQ1 (State-of-the-art comparison) How does AndroLib compare with the
state-of-the-art approaches in terms of recommendation accuracy?
Experimental dataset : https://doi.org/10.6084/m9.figshare.9366341
State-of-the-art comparison
● CrossRec (Nguyen et al., 2020): Collaborative filtering and relying on library dependencies
● LibCUP (Saied et al., 2018) : Clustering technique to mine and recommend library co-usage
patterns
● LibRec (Thung et al., 2013) : Association rule mining and collaborative filtering
Evaluation
Statistics Value
# of Android apps 2,752
# of third-party libraries 979
# of library categories 76
Average number of libraries per app 4.62
41

42. ● Ground truth: the experimental dataset
○ 10 fold cross-validation
○ for each testing app, a half of its libraries are randomly taken out and saved as
ground truth data.
● Evaluation method for RQ1: state-of-the art comparison
Success rate
○ SuccessRate@k: at least one of recommended library is part of the ground truth
Accuracy
○ Precision@k
○ Recall@k
Evaluation method and metrics (RQ1)
42

43. Results for RQ1: Success rate and Accuracy
43
AndroLib correctly recommends Android
libraries with an average of 78%
of precision and 81% of recall

44. 44
[Credit to @Raula Gaikovina Kula]

45. Third-party Software Library reuse life-cycle
Adoption
How to find a useful
library to your
current software
system?
Update
Libraries are in
constant evolution.
How do developers
update their library
dependencies?
Migration
Other competing
libraries can take
the place of a given
library
Retirement
At a given point in time,
some libraries are no
longer maintained and
evolved by its original
devlopers

47. ● Large dataset
○ 4,600 software projects from Github (Java)
○ 2,700 library dependencies from Maven Central Repository
Empirical study on Maven Central Repository
47

48. Library migration in practice
48
RQ1: To what extent are developers updating their library dependencies?
“Systems are less likely to upgrade
their library dependencies, with 81.5%
of systems remaining with a popular
older version”.
“Although system heavily depend
on libraries, most systems rarely
update their libraries”

49. Effectiveness of Awarness Mechanisms
49
RQ2: What is the Response to Important Awareness Mechanisms such as a New
Release Announcement and a Security Advisory on Library Updates?
★ 3 new releases of popular libraries
★ 5 security vulnerabilities
“New release of a popular library there
exist :
(i) patterns of consistent migration, and
(ii) patterns where an older popular
library version is still preferred.”
“For a security advisory disclosure we find
cases of developer non responsiveness to
security advisory disclosure, which is
sometimes due to an incomplete patch or a
latent security advisory.”

50. Library tracking model
Simple example of the LU-based
metrics.
We show the Peak LU at time t1,
current LU at time t2 and library
residue (Peak LU / Current LU).

51. Visualization of Library Usage
A Library Migration Plot. In this example, the
release of a related security advisory CVE-2014-
0114 (black dashed line) that affects beanutils
versions 1.9.1 (marked with crossbones).

52. Effectiveness of Awarness Mechanisms
52
RQ3: Why are developers non responsive to a security advisory?
★ Vulnerable projects contacted for feedback
★ Understand feedback
“69% of developers were unaware of
their vulnerable dependencies and
proceeded to immediately migrate to
a safer dependency.”
★ Developers evaluate based on project
specific priorities.
★ Developers cite migration as a practice
that requires extra effort and added
responsibility.

53. Awareness and Motivation is needed for Updates
53
But ..
Libraries
Age
R. G. Kula et al, 2018]

54. New mechanisms for library dependency security
54
https://snyk.io/
R. G. Kula et al, 2018]

55. Third-party Software Library reuse life-cycle
Adoption
How to find a useful
library to your
current software
system?
Update
Libraries are in
constant evolution.
How do developers
update their library
dependencies?
Migration
Other competing
libraries can take
the place of a given
library
Retirement
At a given point in time,
some libraries are no
longer maintained and
evolved by its original
devlopers

56. ● Migrate to new library offer better services and price?
● Need to replace old library with new library “API Migration”
● Ex, json -> gson, testing->junit.
● Developer face problems .
● Ex, how this process could be done manually.
Library API Migration
json gson

57. Solution
Methods Of Library A Function Mapping
- String toJSONString()
+ String toString()
- void add( String)
+ void add(String, JsonElement)
- String get(int)
+ long getAsLong()
- String getString(String)
+ String getAsString()
- String toJSONString()
- String getString(String)
- void add( String)
- String get(int)
+ String toString()
+ String getAsString()
+ void add(String, JsonElement)
+ long getAsLong()
Solution
Methods Of Library B

58. Solution White
boxed

59. MigrationMiner
59
Dataset available for the community:
http://migrationlab.net/index.php?cf=icpc2019

60. Third-party Software Library reuse life-cycle
Adoption
How to find a useful
library to your
current software
system?
Update
Libraries are in
constant evolution.
How do developers
update their library
dependencies?
Migration
Other competing
libraries can take
the place of a given
library
Retirement
At a given point in time,
some libraries are no
longer maintained and
evolved by its original
devlopers

61. https://github.com/storj/java-libstorj

62. https://github.com/stellar-deprecated/java-stellar-base

63. DEMO!

66. Third-Party Software Library Reuse : From Adoption to Migration
Ali Ouni 1 Mohamed Wiem Mkaouer 2 Hussein AlRubaye 2,3
Thank you!
Question? Discussion? Suggestions?
ali.ouni@etsmtl.ca mwmvse@rit.edu hat6622@rit.edu
1 ETS Montreal, University of Quebec, Montreal, QC, Canada
2 Rochester Institute of Technology, Rochester, NY, USA
3 Microsoft Corporation, WA, USA