Why People Hate Your App: Making Sense of User Feedback in a Mobile App Store, at KDD 2013

  • 105 views
Uploaded on

User review is a crucial component of open mobile app markets such as the Google Play Store. How do we automatically summarize millions of user reviews and make sense out of them? Unfortunately, …

User review is a crucial component of open mobile app markets such as the Google Play Store. How do we automatically summarize millions of user reviews and make sense out of them? Unfortunately, beyond simple summaries such as histograms of user ratings, there are few analytic tools that can provide insights into user reviews. In this paper, we propose WisCom, a system that can analyze tens of millions user ratings and comments in mobile app markets at three di erent levels of detail. Our system is able to (a) discover inconsistencies in reviews; (b) identify reasons why users like or dislike a given app, and provide an interactive, zoomable view of how users' reviews evolve over time; and (c) provide valuable insights into the entire app market, identifying users' major concerns and preferences of di fferent types of apps. Results using our techniques are reported on a 32GB dataset consisting of over 13 million user reviews of 171,493 Android apps in the Google Play Store. We discuss how the techniques presented herein can be deployed to help a mobile app market operator such as Google as well as individual app developers and end-users.

Bin Fu, Jialiu Lin, Lei Li, Jason Hong, Christos Faloutsos, Norman Sadeh

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
105
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
4
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Distinct Features of App ReviewsShorter in length More typos, slangs, not structured sentence Reviews of the same app can target to different versions
  • Detect inconsistent comments/ratings;Identify root causes of users’ negative reviewsTrack the evolving patterns of user reviewsDiscover market trends
  • Observation 1: spikes corresponds to versions of apps
  • Arcade & Action Brain & Puzzle Cards & Casino RacingSports Games CasualBusiness: e-wallet, square
  • 100 most reviewed free applications and 100 most reviewed free games, paid and paidApplication received more unified complains. Left…. , game can attribute to multiple reasons Horizontally, 1 higher quality in generalUsers more tolerant to cost of games than applications
  • 100 most reviewed free applications and 100 most reviewed free games, paid and paidApplication received more unified complains. Left…. , game can attribute to multiple reasons Horizontally, 1 higher quality in generalUsers more tolerant to cost of games than applications
  • Sentiment analysis, extract the vocabulary that can be used for further analysisDoes this review make sense?Sample 8% of the commentsRemove HTML tags and delete non-English reviewsSegment comments into wordsRemove rarely used words.- Yield 19K distinct words, from 988K comments.
  • Before we go that direction, there is another small application we want to talk about Might attributed to careless mistakes or intentional manipulationInconsistent reviews were removed for later analysis
  • Objective function of the regression model
  • However, most of the words only express strong feeling, but not telling us the reasons. Let’s see another set of words that we picked up from the model
  • This is also motivate us to proceed to the next level, to find the root cause of bad reviews
  • Q1: What are the advantages and defects for a particular app?Q2: What are main complaints from users? What are main complaints from my competitors? Automatic summarizationQ3: Is there any anomalies or spam comments? Is there any difference among different categories of apps? What are the market trends?
  • Some contains features, like , picture, media, telephon, some relate to performance such as stability, compativility, etc. Others talking about cost. These 10 topics are the most significant ones that detected cross the whole market

Transcript

  • 1. Why People Hate Your App Making Sense of User Feedback in a Mobile App Store Bin Fu, Jialiu Lin, Lei Li, Jason Hong, Christos Faloutsos, Norman Sadeh Carnegie Mellon University, University of California Berkeley KDD’13 Industry Track
  • 2. 2 32.4GB data: 180k Android app until Nov 2012. 13M user reviews ......
  • 3. awesome good but needs improving crash, pls fix!!! What are users complaining about? 8/15/2013 Lei Li @ KDD 2013 3 awesome good but needs improving crash, pls fix!!!
  • 4. What are market trends? 8/15/2013 Lei Li @ KDD 2013 4
  • 5. All build upon a single question 8/15/2013 Lei Li @ KDD 2013 5 How to summarize millions of comments with ratings? Beyond histogram
  • 6. WisCom to rescue • S1: Per review analysis –Inconsistent reviews • S2: Per app analysis –Root causes –Dynamic view • S3: Whole market analysis –Trends and insights 8/15/2013 Lei Li @ KDD 2013 6 Unstable Costly Unattractive
  • 7. S2: Meso Analysis: Dynamic View of Root Causes 7 • Summarization of all comments / app • Root causes over time 8/15/2013 Lei Li @ KDD 2013
  • 8. S2: Dynamic View: Life Story of App 8Negative reviews Positive reviews “What’s in it?” J.Kleinberg 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments …
  • 9. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 9 stability cost Stability Cost Connectivity Compatibility
  • 10. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 10 12/30/2011 fix it! It keeps force closing on stage 1, need an update, please!!! stability cost Stability Cost Connectivity Compatibility
  • 11. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 11 stability cost Stability Cost Connectivity Compatibility “What’s in it?”
  • 12. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 12 stability cost Stability Cost Connectivity Compatibility stability cost connectivity
  • 13. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 13 stability cost Stability Cost Connectivity Compatibility stability cost connectivity 05/30/2012 Would give 0 stars if I could. Server error.
  • 14. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 14 stability cost Stability Cost Connectivity Compatibility stability cost connectivity 05/30/2012 Would give 0 stars if I could. Server error. 06/06/2012 Finally fixed. Hooray >:)
  • 15. S2: Dynamic View: Life Story of App 15 Stability Cost Connectivity Compatibility #ofcomments day
  • 16. S3: Macro Analysis: Discovery of Market Trends 16 • User prefs for different types of apps? 8/15/2013 Lei Li @ KDD 2013
  • 17. S3:Outstanding prefs in market categories • Games, same top complaints • Non-games: different top complaints 10% 18% 55% 0% 10% 20% 30% 40% 50% 60% cost stability attractiveness 31% 56% 0% 10% 20% 30% 40% 50% 60% connectivity accuracy weather business
  • 18. Users’ Reaction to Free and Paid Apps 18 100 most reviewed applications and games equally scattered complaints Unstable Costly Unattractive Free apps G: Game A: other apps Paid apps ?? Unstable Costly Unattractiv Paid apps8/15/2013 Lei Li @ KDD 2013
  • 19. Users’ Reaction to Free and Paid Apps 19 100 most reviewed applications and games equally scattered complaints Unstable Costly Unattractive Free apps G: Game A: other apps Unstable Costly UnattractivPaid apps8/15/2013 Lei Li @ KDD 2013 Users more tolerant to cost of games than applications
  • 20. Conclusions • WisCom discovers –Inconsistency –Root causes –Market trends • leili@cs.berkeley.edu • Stay tuned for online version 208/15/2013 Lei Li @ KDD 2013 Unstable Costly Unattractive