Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Extracting Examples for API Usage Patterns

1,437 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Extracting Examples for API Usage Patterns

  1. 1. Extracting Examples for API Usage Patterns Hudson Silva Borges (Msc Student) Marco Túlio Valente (Advisor) March 20th, 2014
  2. 2. Agenda (1) Introduction (2) Proposed Solution (3) Evaluation (4) Conclusions 2March 20th, 2014 Extracting Examples for API Usage Patterns
  3. 3. Introduction 3March 20th, 2014 Extracting Examples for API Usage Patterns
  4. 4. Introduction ● Systems often reuse functionality provided by APIs ● Reusing APIs generally requires a non-trivial effort ● Learning resources are a major obstacle ● Documentation is the most severe obstacle 4March 20th, 2014 Extracting Examples for API Usage Patterns
  5. 5. 5March 20th, 2014 Extracting Examples for API Usage Patterns Motivation ● Rely on client usage data to improve APIs learning resources
  6. 6. 6March 20th, 2014 Extracting Examples for API Usage Patterns Why Examples? Robillard e DeLine, Empirical Software Engineering – 2011 “code examples are an essential element of API learning”
  7. 7. 7March 20th, 2014 Extracting Examples for API Usage Patterns APIMiner 1.0 ● Examples for single API methods
  8. 8. 8March 20th, 2014 Extracting Examples for API Usage Patterns Problem Robillard e DeLine, Empirical Software Engineering – 2011 “Small examples that nevertheless demonstrate API usage patterns involving more than one method call will be more useful than single-call examples”
  9. 9. Proposed Solution 9March 20th, 2014 Extracting Examples for API Usage Patterns
  10. 10. APIMiner 2.0 – Overview 10March 20th, 2014 Extracting Examples for API Usage Patterns Extends APIMiner 1.0 to consider usage patterns and readability improvements to provide better source code examples
  11. 11. Usage Patterns ● Set of API methods frequently called together ● Example: open() ⇒ close() 11March 20th, 2014 Extracting Examples for API Usage Patterns
  12. 12. APIMiner 2.0 – Architecture 12March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients
  13. 13. Architecture: Source Code Analyzer 13March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients
  14. 14. Architecture: Source Code Analyzer 14March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients Client System Method API Calls A ma1 m1 , m10 A ma2 ... B mb1 m22 B mb2 m2 , m3 , m11 ...
  15. 15. Architecture: Patterns Analyzer 15March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● API Usage Pattern ● Implications of the form X Y⇒ open() ⇒ close()
  16. 16. Patterns Analyzer – Association Rules 16March 20th, 2014 Extracting Examples for API Usage Patterns Client System Method API Calls A ma1 m1 , m10 A ma2 ... B mb1 m22 B mb2 m2 , m3 , m11 ... ● Usage Data
  17. 17. Patterns Analyzer – Association Rules 17March 20th, 2014 Extracting Examples for API Usage Patterns Transaction Items t1 m1 , m10 t2 ... tn m22 tn+1 m2 , m3 , m11 ... ● Input to the algorithm
  18. 18. Patterns Analyzer – Association Rules 18March 20th, 2014 Extracting Examples for API Usage Patterns Transaction Items t1 m1 , m3 , m10 t2 m1 , m10 t3 m2 , m3 , m10 t4 m1 t5 m1 , m5 , m8 , m10 t6 m10 ● m1, m10 ● m1 m⇒ 10 ● Algorithm
  19. 19. Patterns Analyzer – Association Rules 19March 20th, 2014 Extracting Examples for API Usage Patterns Transaction Items t1 m1 , m3 , m10 t2 m1 , m10 t3 m1 , m3 , m9 t4 m2 t5 m1 , m5 , m8 , m10 t6 m10 ● m1, m10 ● 3 transactions ● m1 m⇒ 10 ● Support = 3 ● Confidence = ( 3 / 4 ) = 75% ● Algorithm
  20. 20. Architecture: Example Extractor 20March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● For each client's method: 1. Identifies calls to API methods 2. Extracts examples for single API calls 3. Extracts examples for usage patterns 4. Removes similar examples
  21. 21. Architecture: Example Extractor 21March 20th, 2014 Extracting Examples for API Usage Patterns ● For each client's method: 1. Identifies calls to API methods 2. Extracts examples for single API calls 3. Extracts examples for usage patterns 4. Removes similar examples Slicing algorithm Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients
  22. 22. Extracting Examples – Readability improvements 22March 20th, 2014 Extracting Examples for API Usage Patterns (1) Variables not identified
  23. 23. Extracting Examples – Readability improvements 23March 20th, 2014 Extracting Examples for API Usage Patterns (1) Variables not identified
  24. 24. Extracting Examples – Readability improvements 24March 20th, 2014 Extracting Examples for API Usage Patterns (1) Variables not identified Original Extraction Improved
  25. 25. Extracting Examples – Readability improvements 25March 20th, 2014 Extracting Examples for API Usage Patterns (2) Empty blocks
  26. 26. Extracting Examples – Readability improvements 26March 20th, 2014 Extracting Examples for API Usage Patterns (2) Empty blocks ?
  27. 27. Extracting Examples – Readability improvements 27March 20th, 2014 Extracting Examples for API Usage Patterns (2) Empty blocks
  28. 28. Architecture: Example Extractor 28March 20th, 2014 Extracting Examples for API Usage Patterns JavaDoc ● For each client's method: 1. Identifies calls to API methods 2. Extracts examples for single API calls 3. Extracts examples for usage patterns 4. Removes similar examples Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients
  29. 29. Architecture: Ranking Engine 29March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● Ranking examples ● Completeness ● Lines of Code ● Users Feedback ● Ranking usage patterns ● Lift ● Number of methods in the association rule
  30. 30. Architecture: JavaDoc Weaver 30March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients
  31. 31. Architecture: JavaDoc Weaver 31March 20th, 2014 Extracting Examples for API Usage Patterns
  32. 32. Architecture: JavaDoc Weaver 32March 20th, 2014 Extracting Examples for API Usage Patterns
  33. 33. Evaluation 33March 20th, 2014 Extracting Examples for API Usage Patterns
  34. 34. Android APIMiner 2.0 34March 20th, 2014 Extracting Examples for API Usage Patterns www.apiminer.org
  35. 35. Android APIMiner 2.0 ● Version 4.1.2-r1 ● 26,540 methods ● 2,968 classes ● 17,303 (65.2%) methods with JavaDoc annotation ● 2,575 (86.76%) classes with JavaDoc annotation 35March 20th, 2014 Extracting Examples for API Usage Patterns
  36. 36. Mining Dataset ● Script to automatically search and download in Github ● Criteria: ● Java Programming language ● Minimum of 50 commits ● Not a fork ● Some projects inserted manually 36March 20th, 2014 Extracting Examples for API Usage Patterns
  37. 37. ● Numbers: ● 396 projects ● 57,658 classes ● 450,762 methods ● 362,900 calls to API methods. ● Highlight: ● CyanogenMod ● Astrid ● Wordpress 37March 20th, 2014 Extracting Examples for API Usage Patterns Mining Dataset
  38. 38. ● Rigid strategy ● Clients selected based on their popularity among the Android developers community ● Example Dataset Mining Dataset⊆ ● Results: ● 151 projects ● 287,263 examples 38March 20th, 2014 Extracting Examples for API Usage Patterns Example Dataset
  39. 39. Transactions ● Number of method calls in the useful transactions 39March 20th, 2014 Extracting Examples for API Usage Patterns 0102030 API methods calls %ofusefultransactions 2 4 6 8 10 14 18 22 26 30 34 38 Median of 3 Mean of 4.6
  40. 40. Transactions ● Intra-class and Inter-class calls in useful transactions 40March 20th, 2014 Extracting Examples for API Usage Patterns
  41. 41. Association Rules ● Problem: What values to use for support and confidence? ● Our strategy: ● A not very large number of rules ● Maximize the API methods coverage ● Tests varying the values of support and confidence 41March 20th, 2014 Extracting Examples for API Usage Patterns
  42. 42. Association Rules ● Number of rules by varying the minimum support value 42March 20th, 2014 Extracting Examples for API Usage Patterns
  43. 43. Association Rules ● API methods coverage by varying the minimum support value 43March 20th, 2014 Extracting Examples for API Usage Patterns
  44. 44. Association Rules ● Used values: ● Support = 21 ● Confidence = 70% ● Results: ● 38,661 rules ● 1,952 rules with antecedent size equal one ● Coverage of 624 API methods; 191 API classes 44March 20th, 2014 Extracting Examples for API Usage Patterns
  45. 45. Examples 45March 20th, 2014 Extracting Examples for API Usage Patterns ● 151 Android client systems ● 287,263 examples ● 102,442 examples for single API methods ● 184,821 examples for the usage patterns ● Coverage: ● 3,915 Android API methods (14.75%) ● 549 Android API classes (18.5%)
  46. 46. Examples 46March 20th, 2014 Extracting Examples for API Usage Patterns ● Size of the examples ● For single API methods ● 50% have at most three lines of code ● For usage patterns ● 50% have at most 14 lines of code
  47. 47. Ranking 47March 20th, 2014 Extracting Examples for API Usage Patterns ● Activity.findViewById(int) ● API method with more examples ● Top 20 examples analyzed ● Examples size equal 1 ● No compilation problems
  48. 48. Activity.findViewById(int) 48March 20th, 2014 Extracting Examples for API Usage Patterns ● To return a value ● In an assignment ● As a target ● In a casting
  49. 49. Usage Patterns 49March 20th, 2014 Extracting Examples for API Usage Patterns ● MediaRecorder.prepare() ● First usage patterns recommended: MediaRecorder.prepare() MediaRecorder.start() MediaRecorder.setAudioSource(int) MediaRecorder.setOutputFile(String)
  50. 50. MediaRecorder.prepare() 50March 20th, 2014 Extracting Examples for API Usage Patterns
  51. 51. MediaRecorder.prepare() 51March 20th, 2014 Extracting Examples for API Usage Patterns
  52. 52. MediaRecorder.prepare() 52March 20th, 2014 Extracting Examples for API Usage Patterns
  53. 53. MediaRecorder.prepare() 53March 20th, 2014 Extracting Examples for API Usage Patterns
  54. 54. MediaRecorder.prepare() 54March 20th, 2014 Extracting Examples for API Usage Patterns ● http://developer.android.com/reference/android/media/MediaRecorder.html
  55. 55. MediaRecorder.prepare() 55March 20th, 2014 Extracting Examples for API Usage Patterns ● http://developer.android.com/reference/android/media/MediaRecorder.html
  56. 56. Field Study 56March 20th, 2014 Extracting Examples for API Usage Patterns
  57. 57. Field Study 57March 20th, 2014 Extracting Examples for API Usage Patterns ● Period of 13th May 2013 to 11th October 2013 (five months) ● No promotions ● Total of 32,335 visits
  58. 58. Field Study 58March 20th, 2014 Extracting Examples for API Usage Patterns ● Top ten countries in visits
  59. 59. Field Study 59March 20th, 2014 Extracting Examples for API Usage Patterns ● 5,545 requests for single API examples ● 2,601 (46.9%) clicks on “Example” button ● 176 requests for usage patterns ● 135 (out of 536) API methods with usage pattern were requested
  60. 60. User Study 60March 20th, 2014 Extracting Examples for API Usage Patterns
  61. 61. User Study 61March 20th, 2014 Extracting Examples for API Usage Patterns ● 29 subjects ● Two corrective maintenance tasks in a small Android app Do the recommendations provided by APIMiner 2.0 help developers to implement maintenance tasks with less effort? Do the recommendations provided by APIMiner 2.0 help developers to implement maintenance tasks with less effort?
  62. 62. More Aqui 62March 20th, 2014 Extracting Examples for API Usage Patterns
  63. 63. Maintenance Tasks 63March 20th, 2014 Extracting Examples for API Usage Patterns ● Designed to represent real maintenance activities ● Tasks with requirements well defined ● Tasks: ● Obtaining location (Tutorial) ● Inserting new properties ● System messages ● Automated tests to check ● Crossover methodology
  64. 64. 64March 20th, 2014 Extracting Examples for API Usage Patterns ● Number of subjects who completed the tasks User Study – Results
  65. 65. 65March 20th, 2014 Extracting Examples for API Usage Patterns ● Number of subjects who completed the tasks User Study – Results
  66. 66. 66March 20th, 2014 Extracting Examples for API Usage Patterns ● Number of subjects who completed the tasks User Study – Results
  67. 67. 67March 20th, 2014 Extracting Examples for API Usage Patterns ● Time spent in the tasks User Study – Results
  68. 68. 68March 20th, 2014 Extracting Examples for API Usage Patterns ● Examples Requests ● 686 examples requests for single API methods ● 182 examples requests for usage patterns ● 23 examples for single API methods by subject ● 6 examples for usage patterns examples by subject User Study – Results
  69. 69. Conclusions 69March 20th, 2014 Extracting Examples for API Usage Patterns
  70. 70. Contributions 70March 20th, 2014 Extracting Examples for API Usage Patterns ● Usage patterns examples ● Readability improvements ● Android APIMiner 2.0 ● 1,952 usage patterns for 624 API methods ● 287,263 examples ● www.apiminer.org ● Evaluation ● Data produced when creating the instance ● Usage data obtained from real Android developers ● Controlled experiment with 29 subjects
  71. 71. Future Work 71March 20th, 2014 Extracting Examples for API Usage Patterns ● Detect different usage types among the examples ● Ranking criteria based on readability ● Other static slicing implementations
  72. 72. Publications 72March 20th, 2014 Extracting Examples for API Usage Patterns ● Montandon J. E. et al. Documenting APIs with Examples: Lessons Learned with the APIMiner Platform. In 20th Working Conference on Reverse Engineering (WCRE), 2013. ● Borges H. S. et al. APIMiner 2.0: Recomendaçao de exemplos de uso de APIs usando regras de associaçao. IV Congresso Brasileiro de Software: Teoria e Prática (Sessão de Ferramentas), 2013. ● Borges H. S.. APIMiner 2.0: Recomendação de Exemplos de Uso de APIs usando Regras de Associação. Poster, ELA-ES, 2013.
  73. 73. 73March 20th, 2014 Extracting Examples for API Usage Patterns Thanks! hsborges@dcc.ufmg.br
  74. 74. 74March 20th, 2014 Extracting Examples for API Usage Patterns Thanks! hsborges@dcc.ufmg.br
  75. 75. Architecture: API and Client systems 75March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● API Source Code API’s source code ● API Clients to Mining and Selected API Clients Repository with API client systems
  76. 76. Architecture: Mining Database 76March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● Database that stores the data generated by the Source Code Analyzer
  77. 77. Architecture: Examples Database 77March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● Store examples and usage patterns produced
  78. 78. Architecture: JavaDoc 78March 20th, 2014 Extracting Examples for API Usage Patterns Mining Database Patterns Analyzer API Source Code API Clients to Mining Souce Code Analyzer Examples Extractor Examples Database JavaDoc Ranking Engine JavaDoc Weaver Selected API Clients ● Output of the proposed approach
  79. 79. ● Ratio of methods with at least an API call among the systems ● Considering all client methods ● In median, for each 3 methods are called 4 API methods ● 75% of the systems had a ratio between 0.78 and 4.40 API calls/method ● Considering only client methods with API method calls ● In median, for each 3 methods are called 9 API methods ● 50% of the projects have between 2.72 and 3.89 API calls/method 79March 20th, 2014 Extracting Examples for API Usage Patterns Mining Dataset
  80. 80. Transactions ● Transaction = Client method ● Input to the association rules algorithm ● Not all clients' methods call an API method ● Transactions with a single call are not relevant 80March 20th, 2014 Extracting Examples for API Usage Patterns
  81. 81. Transactions ● Characterization of the all 512,993 transactions 81March 20th, 2014 Extracting Examples for API Usage Patterns 77.42% 9.19% 13.38% Without calls With a single calls With more than one call
  82. 82. Transactions ● Characterization of the all 512,993 transactions 82March 20th, 2014 Extracting Examples for API Usage Patterns 77.42% 9.19% 13.38% Without calls With a single calls With more than one call
  83. 83. Transactions ● Percentage of useful transactions per project 83March 20th, 2014 Extracting Examples for API Usage Patterns Average of 30.49% Median of 26.53%
  84. 84. Transactions ● Top five most common transaction items 84March 20th, 2014 Extracting Examples for API Usage Patterns
  85. 85. Association Rules 85March 20th, 2014 Extracting Examples for API Usage Patterns ● Top five rules with higher support
  86. 86. Examples 86March 20th, 2014 Extracting Examples for API Usage Patterns ● Number of API method calls in the examples
  87. 87. Examples 87March 20th, 2014 Extracting Examples for API Usage Patterns ● API methods with more examples
  88. 88. Examples 88March 20th, 2014 Extracting Examples for API Usage Patterns ● Classes that most appeared in all examples
  89. 89. Usage Patterns 89March 20th, 2014 Extracting Examples for API Usage Patterns ● Usage patterns without examples ● 280 association rules (14% of 1,952) ● 88 API methods out of the 624 distinct API methods
  90. 90. Field Study 90March 20th, 2014 Extracting Examples for API Usage Patterns ● 31 evaluations of examples by end-users ● Examples of 26 API methods ● Methods with more evaluations: ● Color.rgb(int, int, int) ● ViewPager.OnPageChangeListener.onPageSelected(int) ● Intent.getStringArrayExtra(String) ● NotificationCompat.Builder.setVibrate(long[])
  91. 91. NotificationCompat.Builder.setVibrate(long[]) 91March 20th, 2014 Extracting Examples for API Usage Patterns
  92. 92. User Study – Setup 92March 20th, 2014 Extracting Examples for API Usage Patterns ● Subject characterization ● Crossover methodology ● Group A and Group B ● Restrictions: ● Subjects should have skills on the Java ● Subjects should not be experts in the Android platform
  93. 93. User Study – Execution 93March 20th, 2014 Extracting Examples for API Usage Patterns ● Brief introduction ● More Aqui App ● Task Obtaining location as tutorial ● Subjects: 1. Read the description 2. Register the starting time 3. Run the automated test 4. Perform the task 5. Register the ending time
  94. 94. User Study – Results 94March 20th, 2014 Extracting Examples for API Usage Patterns ● Knowledge of the subjects in the Java
  95. 95. 95March 20th, 2014 Extracting Examples for API Usage Patterns ● Knowledge of the subjects in the Android User Study – Results
  96. 96. 96March 20th, 2014 Extracting Examples for API Usage Patterns ● Professional experience of the subjects (in years) User Study – Results
  97. 97. Related Work 97March 20th, 2014 Extracting Examples for API Usage Patterns ● IDE-based Recommendation Systems ● Vertical Code Completion (VCC) ● Mining API Code Snippets (MACs) ● API Documentation ● APIMiner 1.0 ● ExoaDocs ● APIMiner 2.0

×