Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ARdoc: App Reviews Development
Oriented Classifier
Sebastiano Andrea Emitza Corrado Gerardo Harald
Panichella Di Sorbo Guzm...
App User Reviews
2
Reviews Include Useful
Information for Developers
Pagano et al. – RE2013 Chen et al. – ICSE2014 Galvis Carreno et al. – IC...
Users Submit Many
Reviews Regularly
iOS apps receive on average 23
reviews per day
Facebook for iOS receive more
than 4000...
Past Work
Chen et al – ICSE 2014
ARMiner: an approach to help
app developers discover the
most informative user
reviews
i....
6
Non
Informati
ve
Informative
Reviews
PROBLE
M?
Identifying Useful Reviews
i. The awful button in the page doesn’t work
ii. A button in the page should be added
7
Identifying Useful Reviews
i. The awful button in the page doesn’t work
ii. A button in the page should be added
8
BUG DES...
Available Sources for identifying Useful Reviews
i. The awful button in the page doesn’t work
ii. A button in the page sho...
10
ARdoc: App Reviews
Development Oriented
Classifier
ARdoc’s Architecture
11
Stanford CoreNLP
Apache Lucene API
ARdoc’s Architecture
12
Stanford CoreNLP
Apache Lucene API
WEKA
Taxonomy & Examples
14
Panichella et al. “How can I improve my app? Classifying user reviews for
software maintenance and ...
ARdoc’s DEMO
15
Stanford CoreNLP
Apache Lucene API
WEKA
http://www.ifi.uzh.ch/seal/people/panichella/tools/ARdoc.html
ARdoc Classification Accuracy?
17
ARdoc Classification Accuracy?
18
3 Apps
ARdoc Classification Accuracy?
19
3 Apps
Minesweeper
PowernAPP
Picturex
ARdoc Classification Accuracy?
20
3 Apps
Minesweeper
PowernAPP
Picturex
https://www.scribd.com/document/323048838/ARdoc-Ap...
ARdoc Classification Accuracy
21
Minesweeper
PowernAPP
Picturex
https://www.scribd.com/document/323048838/ARdoc-Appendix
3...
ARdoc Classification Accuracy
24
Minesweeper
PowernAPP
Picturex
https://www.scribd.com/document/323048838/ARdoc-Appendix
3...
Conclusion & Future Work
25
1) ARdoc a novel tool able to mine relevant feedback for
real world developers interested in a...
Conclusion & Future Work
26
1) ARdoc a novel tool able to mine relevant feedback for
real world developers interested in a...
Thanks for the Attention!
27
Stanford CoreNLP
Apache Lucene API
WEKA
Questions?
Upcoming SlideShare
Loading in …5
×

ARdoc: App Reviews Development Oriented Classifier

334 views

Published on

Google Play, Apple App Store and Windows Phone Store are well known distribution platforms where users can download mobile apps, rate them and write review comments about the apps they are using. Previous research studies demonstrated that these reviews contain important information to help developers improve their apps. However, analyzing reviews is challenging due to the large amount of reviews posted every day, the unstructured nature of reviews and its varying quality. In this demo we present
ARdoc, a tool which combines three techniques: (1) Natural Language Parsing,(2) Text Analysis and (3) Sentiment Analysis to automatically classify useful feedback contained in app reviews important for performing software maintenance and evolution tasks. Our quantitative and qualitative analysis (involving mobile professional developers) demonstrates that ARdoc correctly classifies feedback useful for maintenance perspectives in user reviews with high precision (ranging between84% and 89%), recall (ranging between 84% and 89%), and an F-Measure (ranging between 84% and 89%). While evaluating our tool developers of our study confirmed the use-fulness of ARdoc in extracting important maintenance tasks for their mobile applications.

  • Be the first to comment

  • Be the first to like this

ARdoc: App Reviews Development Oriented Classifier

  1. 1. ARdoc: App Reviews Development Oriented Classifier Sebastiano Andrea Emitza Corrado Gerardo Harald Panichella Di Sorbo Guzman Visaggio Canfora Gall
  2. 2. App User Reviews 2
  3. 3. Reviews Include Useful Information for Developers Pagano et al. – RE2013 Chen et al. – ICSE2014 Galvis Carreno et al. – ICSE2013 3
  4. 4. Users Submit Many Reviews Regularly iOS apps receive on average 23 reviews per day Facebook for iOS receive more than 4000 reviews per day [ Pagano et al. - RE 2013 ] 4
  5. 5. Past Work Chen et al – ICSE 2014 ARMiner: an approach to help app developers discover the most informative user reviews i. text analysis and machine learning to filter out non- informative reviews ii. topic analysis to recognize topics treated in the reviews classified as informative 5
  6. 6. 6 Non Informati ve Informative Reviews PROBLE M?
  7. 7. Identifying Useful Reviews i. The awful button in the page doesn’t work ii. A button in the page should be added 7
  8. 8. Identifying Useful Reviews i. The awful button in the page doesn’t work ii. A button in the page should be added 8 BUG DESCRIPTION
  9. 9. Available Sources for identifying Useful Reviews i. The awful button in the page doesn’t work ii. A button in the page should be added 9 sentiment lexicon structure Natural Language Parsing Sentiment Analysis Text Analysis
  10. 10. 10 ARdoc: App Reviews Development Oriented Classifier
  11. 11. ARdoc’s Architecture 11 Stanford CoreNLP Apache Lucene API
  12. 12. ARdoc’s Architecture 12 Stanford CoreNLP Apache Lucene API WEKA
  13. 13. Taxonomy & Examples 14 Panichella et al. “How can I improve my app? Classifying user reviews for software maintenance and evolution” – ICSME 2015
  14. 14. ARdoc’s DEMO 15 Stanford CoreNLP Apache Lucene API WEKA
  15. 15. http://www.ifi.uzh.ch/seal/people/panichella/tools/ARdoc.html
  16. 16. ARdoc Classification Accuracy? 17
  17. 17. ARdoc Classification Accuracy? 18 3 Apps
  18. 18. ARdoc Classification Accuracy? 19 3 Apps Minesweeper PowernAPP Picturex
  19. 19. ARdoc Classification Accuracy? 20 3 Apps Minesweeper PowernAPP Picturex https://www.scribd.com/document/323048838/ARdoc-Appendix
  20. 20. ARdoc Classification Accuracy 21 Minesweeper PowernAPP Picturex https://www.scribd.com/document/323048838/ARdoc-Appendix 3 Apps 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89%
  21. 21. ARdoc Classification Accuracy 24 Minesweeper PowernAPP Picturex https://www.scribd.com/document/323048838/ARdoc-Appendix 3 Apps 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89%
  22. 22. Conclusion & Future Work 25 1) ARdoc a novel tool able to mine relevant feedback for real world developers interested in accomplishing software maintenance and evolution tasks. 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89%
  23. 23. Conclusion & Future Work 26 1) ARdoc a novel tool able to mine relevant feedback for real world developers interested in accomplishing software maintenance and evolution tasks. 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% & Di Sorbo et al. “What Would Users Change in My App? Summarizing App Reviews for Recommending Software Changes” – FSE 16/11//2016 (Session 11)
  24. 24. Thanks for the Attention! 27 Stanford CoreNLP Apache Lucene API WEKA Questions?

×