0
Why People Hate Your App
Making Sense of User Feedback in a
Mobile App Store
Bin Fu, Jialiu Lin, Lei Li,
Jason Hong, Chris...
2
32.4GB data:
180k Android app
until Nov 2012.
13M user reviews
......
awesome
good but
needs
improving
crash, pls
fix!!!
What are users complaining about?
8/15/2013 Lei Li @ KDD 2013 3
awesome...
What are market trends?
8/15/2013 Lei Li @ KDD 2013 4
All build upon a single question
8/15/2013 Lei Li @ KDD 2013 5
How to
summarize
millions of
comments
with ratings?
Beyond ...
WisCom to rescue
• S1: Per review analysis
–Inconsistent reviews
• S2: Per app analysis
–Root causes
–Dynamic view
• S3: W...
S2: Meso Analysis:
Dynamic View of Root Causes
7
• Summarization of all comments / app
• Root causes over time
8/15/2013 L...
S2: Dynamic View: Life Story of App
8Negative reviews Positive reviews
“What’s in it?” J.Kleinberg
0
10
20
30
40
50
60
70
...
0
10
20
30
40
50
60
70
80
90
100
day
#ofcomments
S2: Dynamic View: Life Story of App
9
stability cost
Stability Cost
Conne...
0
10
20
30
40
50
60
70
80
90
100
day
#ofcomments
S2: Dynamic View: Life Story of App
10
12/30/2011 fix
it! It keeps force
...
0
10
20
30
40
50
60
70
80
90
100
day
#ofcomments
S2: Dynamic View: Life Story of App
11
stability cost
Stability Cost
Conn...
0
10
20
30
40
50
60
70
80
90
100
day
#ofcomments
S2: Dynamic View: Life Story of App
12
stability cost
Stability Cost
Conn...
0
10
20
30
40
50
60
70
80
90
100
day
#ofcomments
S2: Dynamic View: Life Story of App
13
stability cost
Stability Cost
Conn...
0
10
20
30
40
50
60
70
80
90
100
day
#ofcomments
S2: Dynamic View: Life Story of App
14
stability cost
Stability Cost
Conn...
S2: Dynamic View: Life Story of App
15
Stability Cost
Connectivity Compatibility
#ofcomments
day
S3: Macro Analysis:
Discovery of Market Trends
16
• User prefs for different types of apps?
8/15/2013 Lei Li @ KDD 2013
S3:Outstanding prefs in market categories
• Games, same top complaints
• Non-games: different top complaints
10%
18%
55%
0...
Users’ Reaction to Free and Paid Apps
18
100 most reviewed applications and games
equally
scattered
complaints
Unstable
Co...
Users’ Reaction to Free and Paid Apps
19
100 most reviewed applications and games
equally
scattered
complaints
Unstable
Co...
Conclusions
• WisCom discovers
–Inconsistency
–Root causes
–Market trends
• leili@cs.berkeley.edu
• Stay tuned for online
...
Upcoming SlideShare
Loading in...5
×

Why People Hate Your App: Making Sense of User Feedback in a Mobile App Store, at KDD 2013

137

Published on

User review is a crucial component of open mobile app markets such as the Google Play Store. How do we automatically summarize millions of user reviews and make sense out of them? Unfortunately, beyond simple summaries such as histograms of user ratings, there are few analytic tools that can provide insights into user reviews. In this paper, we propose WisCom, a system that can analyze tens of millions user ratings and comments in mobile app markets at three di erent levels of detail. Our system is able to (a) discover inconsistencies in reviews; (b) identify reasons why users like or dislike a given app, and provide an interactive, zoomable view of how users' reviews evolve over time; and (c) provide valuable insights into the entire app market, identifying users' major concerns and preferences of di fferent types of apps. Results using our techniques are reported on a 32GB dataset consisting of over 13 million user reviews of 171,493 Android apps in the Google Play Store. We discuss how the techniques presented herein can be deployed to help a mobile app market operator such as Google as well as individual app developers and end-users.

Bin Fu, Jialiu Lin, Lei Li, Jason Hong, Christos Faloutsos, Norman Sadeh

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
137
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Distinct Features of App ReviewsShorter in length More typos, slangs, not structured sentence Reviews of the same app can target to different versions
  • Detect inconsistent comments/ratings;Identify root causes of users’ negative reviewsTrack the evolving patterns of user reviewsDiscover market trends
  • Observation 1: spikes corresponds to versions of apps
  • Arcade & Action Brain & Puzzle Cards & Casino RacingSports Games CasualBusiness: e-wallet, square
  • 100 most reviewed free applications and 100 most reviewed free games, paid and paidApplication received more unified complains. Left…. , game can attribute to multiple reasons Horizontally, 1 higher quality in generalUsers more tolerant to cost of games than applications
  • 100 most reviewed free applications and 100 most reviewed free games, paid and paidApplication received more unified complains. Left…. , game can attribute to multiple reasons Horizontally, 1 higher quality in generalUsers more tolerant to cost of games than applications
  • Sentiment analysis, extract the vocabulary that can be used for further analysisDoes this review make sense?Sample 8% of the commentsRemove HTML tags and delete non-English reviewsSegment comments into wordsRemove rarely used words.- Yield 19K distinct words, from 988K comments.
  • Before we go that direction, there is another small application we want to talk about Might attributed to careless mistakes or intentional manipulationInconsistent reviews were removed for later analysis
  • Objective function of the regression model
  • However, most of the words only express strong feeling, but not telling us the reasons. Let’s see another set of words that we picked up from the model
  • This is also motivate us to proceed to the next level, to find the root cause of bad reviews
  • Q1: What are the advantages and defects for a particular app?Q2: What are main complaints from users? What are main complaints from my competitors? Automatic summarizationQ3: Is there any anomalies or spam comments? Is there any difference among different categories of apps? What are the market trends?
  • Some contains features, like , picture, media, telephon, some relate to performance such as stability, compativility, etc. Others talking about cost. These 10 topics are the most significant ones that detected cross the whole market
  • Transcript of "Why People Hate Your App: Making Sense of User Feedback in a Mobile App Store, at KDD 2013"

    1. 1. Why People Hate Your App Making Sense of User Feedback in a Mobile App Store Bin Fu, Jialiu Lin, Lei Li, Jason Hong, Christos Faloutsos, Norman Sadeh Carnegie Mellon University, University of California Berkeley KDD’13 Industry Track
    2. 2. 2 32.4GB data: 180k Android app until Nov 2012. 13M user reviews ......
    3. 3. awesome good but needs improving crash, pls fix!!! What are users complaining about? 8/15/2013 Lei Li @ KDD 2013 3 awesome good but needs improving crash, pls fix!!!
    4. 4. What are market trends? 8/15/2013 Lei Li @ KDD 2013 4
    5. 5. All build upon a single question 8/15/2013 Lei Li @ KDD 2013 5 How to summarize millions of comments with ratings? Beyond histogram
    6. 6. WisCom to rescue • S1: Per review analysis –Inconsistent reviews • S2: Per app analysis –Root causes –Dynamic view • S3: Whole market analysis –Trends and insights 8/15/2013 Lei Li @ KDD 2013 6 Unstable Costly Unattractive
    7. 7. S2: Meso Analysis: Dynamic View of Root Causes 7 • Summarization of all comments / app • Root causes over time 8/15/2013 Lei Li @ KDD 2013
    8. 8. S2: Dynamic View: Life Story of App 8Negative reviews Positive reviews “What’s in it?” J.Kleinberg 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments …
    9. 9. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 9 stability cost Stability Cost Connectivity Compatibility
    10. 10. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 10 12/30/2011 fix it! It keeps force closing on stage 1, need an update, please!!! stability cost Stability Cost Connectivity Compatibility
    11. 11. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 11 stability cost Stability Cost Connectivity Compatibility “What’s in it?”
    12. 12. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 12 stability cost Stability Cost Connectivity Compatibility stability cost connectivity
    13. 13. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 13 stability cost Stability Cost Connectivity Compatibility stability cost connectivity 05/30/2012 Would give 0 stars if I could. Server error.
    14. 14. 0 10 20 30 40 50 60 70 80 90 100 day #ofcomments S2: Dynamic View: Life Story of App 14 stability cost Stability Cost Connectivity Compatibility stability cost connectivity 05/30/2012 Would give 0 stars if I could. Server error. 06/06/2012 Finally fixed. Hooray >:)
    15. 15. S2: Dynamic View: Life Story of App 15 Stability Cost Connectivity Compatibility #ofcomments day
    16. 16. S3: Macro Analysis: Discovery of Market Trends 16 • User prefs for different types of apps? 8/15/2013 Lei Li @ KDD 2013
    17. 17. S3:Outstanding prefs in market categories • Games, same top complaints • Non-games: different top complaints 10% 18% 55% 0% 10% 20% 30% 40% 50% 60% cost stability attractiveness 31% 56% 0% 10% 20% 30% 40% 50% 60% connectivity accuracy weather business
    18. 18. Users’ Reaction to Free and Paid Apps 18 100 most reviewed applications and games equally scattered complaints Unstable Costly Unattractive Free apps G: Game A: other apps Paid apps ?? Unstable Costly Unattractiv Paid apps8/15/2013 Lei Li @ KDD 2013
    19. 19. Users’ Reaction to Free and Paid Apps 19 100 most reviewed applications and games equally scattered complaints Unstable Costly Unattractive Free apps G: Game A: other apps Unstable Costly UnattractivPaid apps8/15/2013 Lei Li @ KDD 2013 Users more tolerant to cost of games than applications
    20. 20. Conclusions • WisCom discovers –Inconsistency –Root causes –Market trends • leili@cs.berkeley.edu • Stay tuned for online version 208/15/2013 Lei Li @ KDD 2013 Unstable Costly Unattractive
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×