Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Recommending and localizing change requests for mobile apps based on user reviews

433 views

Published on

Researchers have proposed several approaches to extract information from user reviews useful for maintaining and evolving mobile apps. However, most of them just perform automatic classification of user reviews according to specific keywords (e.g., bugs, features). Moreover, they do not provide any support for linking user feedback to the source code components to be changed, thus requiring a manual, time-consuming, and error-prone task. In this paper, we introduce ChangeAdvisor, a novel approach that analyzes the structure, semantics, and sentiments of sentences contained in user reviews to extract useful (user) feedback from maintenance perspectives and recommend to developers changes to software artifacts. It relies on natural language processing and clustering algorithms to group user reviews around similar user needs and suggestions for change. Then, it involves textual based heuristics to determine the code artifacts that need to be maintained according to the recommended software changes. The quantitative and qualitative studies carried out on 44 683 user reviews of 10 open source mobile apps and their original developers showed a high accuracy of ChangeAdvisor in (i) clustering similar user change requests and (ii) identifying the code components impacted by the suggested changes. Moreover, the obtained results show that ChangeAdvisor is more accurate than a baseline approach for linking user feedback clusters to the source code in terms of both precision (+47%) and recall (+38%).

  • Be the first to comment

  • Be the first to like this

Recommending and localizing change requests for mobile apps based on user reviews

  1. 1. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews Delft University of Technology ICSE 2017 International Conference on Software Engineering UNIVERSITÀ DEGLI STUDI DI SALERNO Recommending and Localizing Change Requests for Mobile Apps based on User Reviews Fabio Palomba1, Pasquale Salza2, Adelina Ciurumelea3, Sebastiano Panichella3 Harald Gall3, Filomena Ferrucci2, Andrea De Lucia2 1Delft University of Technology, 2University of Salerno, 3University of Zurich
  2. 2. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Apps everywhere Over 2 billions people rely on 5 millions of mobile apps for social and emergency connectivity
  3. 3. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Apps are software too Short period releases User reviews on the stores
  4. 4. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 User reviews are useful… Other than non informative content, they contain important pieces of information
  5. 5. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Reading all of them is an effort-prone task …but they are too many!
  6. 6. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 The State-of-the-Art But, it is not possible to: 1. Extract only the useful information hidden behind different user reviews 2. Group together fine-grained information 3. Understand the actual impact of each change request SURF CLAP Summarizes user reviews Prioritizes user reviews
  7. 7. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 ChangeAdvisor Transforms user reviews in change requests and localises them within the source code
  8. 8. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 ChangeAdvisor JAVA JAVA JAVA JAVA Parser ARdoc HDP-LDADice Indexer Source code Feedback clusters Code components Filter Feedback preprocessing Source code preprocessing Problem discovery and Feature requests Classified feedback Ranked list (cluster/component) Reviews May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017
  9. 9. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 JAVA JAVA JAVA JAVA Parser ARdoc Source code Code components Filter Source code preprocessing Classified feedbackReviews May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Extraction of a bag of words for each class
  10. 10. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 JAVA JAVA JAVA JAVA Parser ARdoc Dice Indexer Source code Code components S pr Classified feedbackReviews May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Extraction and classification of user feedback
  11. 11. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 HDice Indexer Code components Filter F pre Source code preprocessing Problem discovery and Feature requests Classified feedback May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Filtering of change requests
  12. 12. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 HDP-LDAexer Feedback clusters Filter Feedback preprocessing Source code preprocessing Problem discovery and Feature requests May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Grouping similar user needs
  13. 13. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 ARdoc Dice Indexer Feedback clusters Filter Problem discovery and Feature requests Classified feedback Ranked list (cluster/component) iews May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Code components Linking feedback to components
  14. 14. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Empirical Study 10 open source apps 13,510 feedback 4,138 classes
  15. 15. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 RQ1 Does ChangeAdvisor identify cohesive user feedback clusters representing related change requests?
  16. 16. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Clusters evaluation We asked the experts to rate the cohesiveness of clusters using a Likert scale 5+ years experience 4 3
  17. 17. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Results Overall median = 4 Overall max = 5 8.2 clusters/app
  18. 18. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 RQ2 Does ChangeAdvisor correctly link user feedback clusters to source code components and how does it compare with the state-of-the-art?
  19. 19. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Oracle definition We asked the experts to define the link between clusters and classes
  20. 20. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Links evaluation We measured the accuracy of ChangeAdvisor using precision and recall
  21. 21. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Comparison We exploited BLUiR that links bug reports to classes using the Vector Space Model BLUiR
  22. 22. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Results ChangeAdvisor Precision = 81 % Recall = 70 % BLUiR Precision = 34 % Recall = 32 % May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017
  23. 23. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Overlap analysis ChangeAdvisor BLUiR May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Number of links correctly identified by one approach and missed by the other
  24. 24. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Overlap analysis ChangeAdvisor BLUiR 72 % 5 %23 % May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017
  25. 25. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Are the suggestions provided by ChangeAdvisor actually useful for developers? RQ3
  26. 26. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Surveying developers Cohesiveness. How well are the user reviews grouped according to the number of source code components that need to be modified? Precision. How well do the proposed classes match the actual set of those needed to be changed in order to satisfy the user requests? Completeness. Evaluate the completeness of the set of classes suggested compared to the actual set of those needed to be modified
  27. 27. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Results Cohesiveness = high Precision = very high Completeness = very high
  28. 28. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Results I know what changes I have to make in my app when implementing a change. However, a tool like this may help in quantifying the number of classes to be modified.
  29. 29. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Future directions • Extend the study involving more apps and developers • Improve our approach to prioritize user feedback based on the number of classes to be modified
  30. 30. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 One more thing… http://bit.ly/2r5GzMq ChangeAdvisor
  31. 31. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 NLP steps 1. Spelling correction 2. Contractions expansion 3. Nouns and verbs filtering 4. Tokenization 5. Singularization 6. Stopword removal 7. Stemming 8. Repetitions removal 9. Short tokens removal 10.Short tokens removal 11.Short documents removal
  32. 32. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Dice similarity
  33. 33. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Characteristics of the apps in the dataset
  34. 34. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Evaluation of the clusters cohesiveness
  35. 35. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 ChangeAdvisor vs BLUiR
  36. 36. May 25th, 2017Recommending and Localizing Change Requests for Mobile Apps based on User Reviews ICSE 2017 Surveys results

×