This document discusses a study on predicting the compatibility of Eclipse third-party plugins (ETPs) with new Eclipse releases. The researchers built prediction models using logistic regression based on ETPs' usage of Eclipse APIs versus non-APIs. They trained and tested 36 models and found high accuracy, precision, and recall over 80% in many cases. The models show promise for predicting compatibility and could be developed into a tool to help ETP users and developers.

1. Compatibility Prediction of
Eclipse Third-party Plug-ins in
new Eclipse Releases
John Businge, Alexander Serebrenik,
Mark van den Brand

2. The Eclipse Framework
Software Engineering and Technology (SET) PAGE 19-6-2014
P1 P4P3P2
Eclipse
Framework
Eclipse Third-
party Plug-ins
(ETPs)

3. The Eclipse Framework …
Software Engineering and Technology (SET) PAGE 29-6-2014
P1 P4P3P2
Eclipse
Framework
Eclipse Third-
party Plug-ins
(ETPs)
Eclipse APIs(“good”)
• no “internal”
• stable,
• supported
Eclipse non-APIs (“bad”)
• “internal”
• unstable,
• discouraged,
• unsupported

4. The Eclipse Framework …
Software Engineering and Technology (SET) PAGE 39-6-2014
P1 P4P3P2
Eclipse
Framework
Eclipse Third-
party Plug-ins
(ETPs)
Eclipse APIs(“good”)
• no “internal”
• stable,
• supported
Eclipse non-APIs (“bad”)
• “internal”
• unstable,
• discouraged,
• unsupported
 P3 – ETP-APIs
 P1, P2 and P4 – ETP-non-APIs

5. Motivation
• Previous study (Survival of ETPs – ICSM 2012), we
tested compatibility of 345 ETP-APIs and 288 ETP-non-
APIs with different Eclipse releases.
• Our observations:
1. ETP-APIs always compatible in new Eclipse releases.
2. Bad interfaces are the main cause of incompatibilities.
3. Informally, found old bad interfaces stable.
• Formally verified observation 2.
• Trained prediction models
• Tested the prediction models.
Software Engineering and Technology (SET) PAGE 49-6-2014

6. Motivation
• Previous study (Survival of ETPs – ICSM 2012), we
tested compatibility of 345 ETP-APIs and 288 ETP-non-
APIs with different Eclipse releases.
• Our observations:
1. ETP-APIs always compatible in new Eclipse releases.
2. Bad interfaces are the main cause of incompatibilities.
3. Informally, found old bad interfaces stable.
• Formally verified observation 2.
• Trained prediction models
• Tested the prediction models.
Software Engineering and Technology (SET) PAGE 59-6-2014

7. Compatibility prediction
• Requirements of compatibility
prediction:
− Current SDK compatible with ETP
− Later SDK to make prediction
• We built 36 prediction models in total
• Models are bases on bad interfaces
used by ETPs
Software Engineering and Technology (SET) PAGE 79-6-2014

8. ETP-non-APIs supported in Eclipse Releases
Software Engineering and Technology (SET) PAGE 89-6-2014
Eclipse 2.1 3.0 3.1 3.2 3.3 3.4 3.5 3.6 Total
# ETPs 29 48 34 40 38 36 33 30 288

9. Model Training
 Statistics – Binary Logistic
Regression
𝑷 𝑒𝑣𝑒𝑛𝑡 = 1/(1 + 𝑒−𝒛
)
𝒛 = 𝑏0 + 𝑏1 𝑋1 + … + 𝑏 𝑝 𝑋 𝑝
𝑷 𝑒𝑣𝑒𝑛𝑡 −Dependent variable
p # Independent variables
24-09-2012
9

10. Model Training Example
In put
P1 P2 P3
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 10

11. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1 P2 P3
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 11

12. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1 P2 P3
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 12

13. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 13

14. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 14

15. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 15
Dependent variable
Independent variables

16. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
Logistic
Regression
Machine
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 16
Dependent variable
Independent variables

17. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
Logistic
Regression
Out put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
𝑏2𝑏1 𝑏3 𝑏4
Machine
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 17
Dependent variable
Independent variables

18. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
Logistic
Regression
Out put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
𝑏2𝑏1 𝑏3 𝑏4
Machine
𝑷 𝑋 = 1/(1 + 𝑒−𝒛
)
𝒛 = 𝑏0 + 𝑏1 ∗ 𝑺𝑫𝑲1.0 + 𝑏4 ∗ 𝑺𝑫𝑲3.0
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 18
Dependent variable
Independent variables

19. Model Training Example
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P4
20
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 19

20. Model Training Example
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P4
20
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 20
𝑷 𝑋 = 1/(1 + 𝑒−𝒛)
𝒛 = 𝑏0 + 𝑏1 ∗ 8 + 𝑏4 ∗ 5
𝑷 𝑋 <0.5 – incompatibility
𝑷 𝑋 >=0.5 – compatibility

21. Results
• In both model training and testing: High
Precision, Accuracy, and Recall, where
some were 80% and more
24-09-2012 21
Model Testing Error Analysis
3.5 3.6 3.7
A P R A P R A P R
3.4 94 100 94 93 100 93 93 100 93
3.5 91 94 96 88 91 96

22. Conclusion and Future Work
• Mining interface usage from ETPs
to detect or predict compatibility
shows good results.
• Next, develop a domain specific
tool to make predictions.
• Who can use the tool? users and
developers of ETPs
•
Software Engineering and Technology (SET) PAGE 229-6-2014

23. Thank you for listening
Software Engineering and Technology (SET) PAGE 239-6-2014