Analyzing the Eclipse API Usage: Putting the Developer in the Loop

404 views

Published on

John's talk at CSMR 2013

Published in: Science, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
404
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Technical support from framework developers, ease of maintenance, and
  • Analyzing the Eclipse API Usage: Putting the Developer in the Loop

    1. 1. Analyzing the Eclipse API Usage: Putting the Developer in the Loop John Businge, Alexander Serebrenik, Mark van den Brand
    2. 2. Introduction Software Engineering and Technology (SET) PAGE 19-6-2014 Software engineering researchers obtaining convincing evidence
    3. 3. Introduction Software Engineering and Technology (SET) PAGE 29-6-2014 Software engineering researchers obtaining convincing evidence Few examples of convincing evidence
    4. 4. Introduction Software Engineering and Technology (SET) PAGE 39-6-2014 Software engineering researchers obtaining convincing evidence Few examples of convincing evidence Researchers generate evidence about A, C, D and E yet developers care about B
    5. 5. Introduction Software Engineering and Technology (SET) PAGE 49-6-2014 Software engineering researchers obtaining convincing evidence Few examples of convincing evidence Researchers generate evidence about A, C, D and E yet developers care about B Include developer in the loop
    6. 6. Terms used Software Engineering and Technology (SET) PAGE 59-6-2014 P1 P4P3P2 Eclipse Framework Eclipse Third- party Plug-ins (ETPs)
    7. 7. Terms used Software Engineering and Technology (SET) PAGE 69-6-2014 P1 P4P3P2 Eclipse Framework Eclipse Third- party Plug-ins (ETPs) Eclipse APIs(“good”) • no “internal” • stable, • supported Eclipse non-APIs (“bad”) • “internal” • unstable, • discouraged, • unsupported  P3 – good ETPs  P1, P2 and P4 – bad ETPs
    8. 8. Terms used • Eclipse provisional API guidelines – document provided by Eclipse describing the different interfaces Software Engineering and Technology (SET) PAGE 79-6-2014 Embryonic interfaces
    9. 9. Terms used • Eclipse provisional API guidelines – document provided by Eclipse describing the different interfaces Software Engineering and Technology (SET) PAGE 89-6-2014 Embryonic interfaces Battle-headed interfaces
    10. 10. Motivation of our study • Our previous studies about Eclipse API Usage were based on source code analysis. • Our observations: 1. 44% of ETPs on SourceForge use bad interfaces. 2. bad ETPs are larger than good ETPs. 3. Good interfaces are indeed very stable in new framework releases. 4. Bad interfaces are indeed unstable in new framework releases. Software Engineering and Technology (SET) PAGE 99-6-2014
    11. 11. Motivation of our study • Omitting developer in the loop in previous studies • Why bad ETPs are larger than good ETPs???? • Why developers use bad interfaces???? Software Engineering and Technology (SET) PAGE 109-6-2014 Include developer in the loop
    12. 12. Survey to include developer in loop • Goal – obtain the state-of-practice Eclipse interface usage 1. Reasons for use of bad interfaces 2. Difference in characteristics of bad ETPs and good ETPs • Research questions RQ1: Can we observe some relationships in factors related to Eclipse interface usage in the survey? RQ2: What are the differences in characteristics between bad and good ETPs? RQ3: What are the differences in characteristics between proprietary and open-source ETPs? Software Engineering and Technology (SET) PAGE 119-6-2014
    13. 13. Survey to include developer in loop • Goal – obtain the state-of-practice Eclipse interface usage 1. Reasons for use of bad interfaces 2. Difference in characteristics of bad ETPs and good ETPs • Research questions RQ1: Can we observe some relationships in factors related to Eclipse interface usage in the survey? RQ2: What are the differences in characteristics between bad and good ETPs? RQ3: What are the differences in characteristics between proprietary and open-source ETPs? Software Engineering and Technology (SET) PAGE 129-6-2014
    14. 14. Survey factors of investigation Software Engineering and Technology (SET) PAGE 139-6-2014 No Factors 1 Years of Education 2 Experience as a software engineer 3 Experience as a Java developer 4 Experience as an ETP developer 5 ETP development team size 6 Importance of updating ETP with new Eclipse releases 7 Number of versions of the ETP 8 Number of files of the ETP 9 Awareness of Eclipse guidelines 10 Following Eclipse guidelines
    15. 15. Sample Questions in the survey i. How many years of experience as an Eclipse product/solution developer do you have? ii. Are you aware of the “Eclipse Provisional API Guidelines”? iii. Do you use non-APIs? iv. If answer was “NO” in (iii). Do you deliberately avoid using Eclipse non-APIs? • Survey Response − 30 ETP developers fully answered the questionnaire. Software Engineering and Technology (SET) PAGE 149-6-2014
    16. 16. RQ2 and RQ3 - groups • RQ2 – Difference between good and bad ETPs Software Engineering and Technology (SET) PAGE 159-6-2014 Use of bad interfaces Group Respondents Developers who use bad interfaces 21 Developers who do not use bad interfaces 9 Open-source vs proprietary ETPs Group Respondents Developers who develop open-source ETPs 16 Developers who develop proprietary ETPs 14 • RQ3 – Difference between proprietary and open- source ETPs
    17. 17. RQ2 and RQ3 - groups • The two groups of RQ2 and RQ3 can be compared using the 10 identified factors. Software Engineering and Technology (SET) PAGE 169-6-2014
    18. 18. Differences between bad and good ETPs – RQ2 • Hypotheses: • H0: Developers who use bad interfaces and those who do not use bad interfaces have the same average values on each of the identified factors. • Ha: There is a difference on the average values of the identified factors between the developers that use and do not use bad interfaces. Software Engineering and Technology (SET) PAGE 179-6-2014
    19. 19. Differences between developers that use and those that do not use bad interfaces Factors Difference Years of Education NO Experience as a software engineer YES Experience as a Java developer YES Experience as an Eclipse plug-in developer YES ETP development team size YES Importance of updating the plug-in with Eclipse NO Number of versions NO Number of files YES Awareness of Eclipse guidelines NO Following Eclipse guidelines NO Software Engineering and Technology (SET) PAGE 189-6-2014
    20. 20. Differences between developers that use and those that do not use bad interfaces Factors Difference Years of Education NO Experience as a software engineer YES Experience as a Java developer YES Experience as an Eclipse plug-in developer YES ETP development team size YES Importance of updating the plug-in with Eclipse NO Number of versions NO Number of files YES Awareness of Eclipse guidelines NO Following Eclipse guidelines NO Software Engineering and Technology (SET) PAGE 199-6-2014 We reject H0 that Developers who use bad interfaces and those who do not use bad interfaces have the same average values on each of the identified factors
    21. 21. How different are the two groups of developers? • We discovered that developers that use bad interfaces have higher values on these factors compared to those that do not use bad interfaces Software Engineering and Technology (SET) PAGE 209-6-2014 Factors Experience as a software engineer Experience as a Java developer Experience as an Eclipse plug-in developer ETP development team size Number of files
    22. 22. How different are the two groups of developers? • We discovered that developers that use bad interfaces have higher values on these factors compared to those that do not use bad interfaces Software Engineering and Technology (SET) PAGE 219-6-2014 Factors Experience as a software engineer Experience as a Java developer Experience as an Eclipse plug-in developer ETP development team size Number of files 1. More experiences developers 2. Larger ETP development teams Previously unanswered question “Why bad ETPs are larger than good ETPs?” Human factors Developers of bad ETPs
    23. 23. Differences between open-source and proprietary ETPs • Motivation: • Previous studies we only considered open-source software potentially putting generalizability to proprietary software in jeopardy. • Hypotheses: • H0: Developers of open-source and proprietary ETPs have the same average values on each of the identified factors. • Ha: There is a difference on the average values of the identified factors between the developers of open- source and proprietary ETPs. Software Engineering and Technology (SET) PAGE 229-6-2014
    24. 24. How different are the two groups of open- source and proprietary developers? Software Engineering and Technology (SET) PAGE 239-6-2014 Factors Difference Years of Education NO Experience as a software engineer NO Experience as a Java developer NO Experience as an Eclipse plug-in developer NO ETP development team size NO Importance of updating the plug-in with Eclipse NO Number of versions NO Number of files NO Awareness of Eclipse guidelines NO Following Eclipse guidelines NO
    25. 25. How different are the two groups of open- source and proprietary developers? • We accept H0 that there is no difference between open- source and proprietary ETPs in terms of the factors investigated Software Engineering and Technology (SET) PAGE 249-6-2014 Factors Difference Years of Education NO Experience as a software engineer NO Experience as a Java developer NO Experience as an Eclipse plug-in developer NO ETP development team size NO Importance of updating the plug-in with Eclipse NO Number of versions NO Number of files NO Awareness of Eclipse guidelines NO Following Eclipse guidelines NO
    26. 26. How different are the two groups of open- source and proprietary developers? • Current findings clear our doubts on generalizability of our previous studies. Software Engineering and Technology (SET) PAGE 259-6-2014
    27. 27. Use of bad interfaces • Asked why developers deliberately avoid using bad interfaces. − Answerers were related to, knowing/assuming that the bad interfaces are unstable. • Asked why developers deliberately use bad interfaces − No good interface with necessary functionality. − Avoid reinventing the wheel. − To expose these bad interfaces. − Old bad interfaces are unlikely to disappear Software Engineering and Technology (SET) PAGE 269-6-2014
    28. 28. Use of bad interfaces • Asked why developers deliberately avoid using bad interfaces. − Answerers were related to, knowing/assuming that the bad interfaces are unstable. Less experienced developers • Asked why developers deliberately use bad interfaces − No good interface with necessary functionality. − Avoid reinventing the wheel. − To expose these bad interfaces. − Old bad interfaces are unlikely to disappear More experienced developers Software Engineering and Technology (SET) PAGE 279-6-2014
    29. 29. Conclusion Software Engineering and Technology (SET) PAGE 289-6-2014 Survey was conducted to understand why developers use bad interfaces.
    30. 30. Conclusion Software Engineering and Technology (SET) PAGE 299-6-2014 Less experienced developers Survey was conducted to understand why developers use bad interfaces. Instability overshadows benefits
    31. 31. Conclusion Software Engineering and Technology (SET) PAGE 309-6-2014 Less experienced developers More experienced developers Survey was conducted to understand why developers use bad interfaces. Instability overshadows benefits Enjoy benefits despite the instability
    32. 32. Conclusion Software Engineering and Technology (SET) PAGE 319-6-2014 Less experienced developers More experienced developers Survey was conducted to understand why developers use bad interfaces. Instability overshadows benefits Enjoy benefits despite the instability Discovered that there is no difference between proprietary and open-source ETPs based on the factors we studied.
    33. 33. Thank you for listening Software Engineering and Technology (SET) PAGE 329-6-2014 • http://www.win.tue.nl/~jbusinge/CSMR13/ Researchers should generate evidence about topics developers care about Include developer in the loop by

    ×