0
AppFunnel: A Framework for Usage-centricEvaluation of Recommender Systemsthat Suggest Mobile ApplicationsMatthias BöhmerLy...
Introduction‣ Number of mobile applications steadily increasing‣ Finding good apps can become a difficult task‣ Recommender ...
Our paper...(1) Introduces a usage-centric evaluation framework(2) Presents results of a case study                       ...
‣ AppFunnel              4
Concept of AppFunnel ‣ Idea: evaluate recommendation based on app engagement ‣ Concept of funnels adopted from advertising...
‣ Case Study               6
Testbed: appazaar‣ Recommender system for Android applications‣ Available for end-users on Google Play Market for free‣ 6,...
Recommender Engines under Test                     Non-personalized         Personalized   Context-less   - App popularity...
Conversion Stages                                                                          AppFunnel stage                ...
Conversion Rates                                                                             Conversion                   ...
Results of Case Study‣ AppFunnel shows difference in engines‘ performances, e.g.   - Context-less engines: more views   - C...
Limitations‣ Recommender engines under test are rather simplistic‣ Updates and removal not taken into account   - Update d...
Discussion‣ Choosing from AppFunnel‘s metrics   - Metrics might counteract (e.g., direct vs. long-term usage)   - Metrics ...
‣ Conclusion               14
Conclusion‣ We presented AppFunnel  - Usage-centric evaluation of recommender engines  - Recommendation of mobile applicat...
Upcoming SlideShare
Loading in...5
×

AppFunnel: A Framework for Usage-centric Evaluation of Recommender Systems that Suggest Mobile Applications

1,424

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,424
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "AppFunnel: A Framework for Usage-centric Evaluation of Recommender Systems that Suggest Mobile Applications"

  1. 1. AppFunnel: A Framework for Usage-centricEvaluation of Recommender Systemsthat Suggest Mobile ApplicationsMatthias BöhmerLyubomir GanevAntonio Krüger
  2. 2. Introduction‣ Number of mobile applications steadily increasing‣ Finding good apps can become a difficult task‣ Recommender systems can help - Mobile apps special type of items - Context is important‣ „Dead apps“ on smartphones - Less than 1/2 of apps are actively used - Installation counts good for evaluation?
  3. 3. Our paper...(1) Introduces a usage-centric evaluation framework(2) Presents results of a case study 3
  4. 4. ‣ AppFunnel 4
  5. 5. Concept of AppFunnel ‣ Idea: evaluate recommendation based on app engagement ‣ Concept of funnels adopted from advertising domain ‣ Apps can reach different stages after recommendation LONG-TERM VIEW INSTALLATION DIRECT USAGE USAGERECOMMENDATION ‣ Tracking engagement with apps along stages
  6. 6. ‣ Case Study 6
  7. 7. Testbed: appazaar‣ Recommender system for Android applications‣ Available for end-users on Google Play Market for free‣ 6,680+ users of appazaar, worldwide distribution‣ 45 users contributed to case study over three months
  8. 8. Recommender Engines under Test Non-personalized Personalized Context-less - App popularity - Usage-based CF Context-aware - App-aware filtering - Location-aware CF - Time-aware CF
  9. 9. Conversion Stages AppFunnel stage view view market installation direct usage long−term usageAverage number of occurrences per recommendation list 1.0 0.8 0.6 0.4 0.2 0.0 rity App−popularity CF Usage−based a App−awarere CF CF Time−aware Location−aware a ed aw e e ul Collaborative Collaborative Collaborative p- ar ar asFiltering Filtering Filtering op b p aw Filtering -aw Filtering p -p e- A e- n p g m io A sa Ti at U L oc
  10. 10. Conversion Rates Conversion view to installation installation to direct usage installation to long−term usage 100Conversion rates in percent 90 80 70 60 50 40 30 20 10 0 ity F Usage−based C re CF Time−aware Location−aware CF r wa App−popularity App−aware u la d Collaborative e a re re Collaborative Collaborative as Filtering pp- waFiltering waFiltering p Filtering Filtering o -a -a -p -b A e n pp ge m io A sa Ti at U L oc
  11. 11. Results of Case Study‣ AppFunnel shows difference in engines‘ performances, e.g. - Context-less engines: more views - Context-aware engines: better from installation to direct use‣ AppFunnel enables validation of engines‘ design goals, e.g. - App-aware engine: high installation to direct-usage rate - Context-less engines: higher rates for long-term usage‣ AppFunnel helped us to trace other issues of engines, e.g. - Location-aware engine: many views but no installations
  12. 12. Limitations‣ Recommender engines under test are rather simplistic‣ Updates and removal not taken into account - Update does not tell much about app engagement - Removal events needs further refinement of AppFunnel‣ Future work: - Improve recommender engines by incorporating results of case study
  13. 13. Discussion‣ Choosing from AppFunnel‘s metrics - Metrics might counteract (e.g., direct vs. long-term usage) - Metrics should reflect design goal (context -> direct usage)‣ Usage-centric evaluation valuable for other domains - When engagement with item can be traced - Changing view from „selling items“ to UX of items
  14. 14. ‣ Conclusion 14
  15. 15. Conclusion‣ We presented AppFunnel - Usage-centric evaluation of recommender engines - Recommendation of mobile applications‣ We presented a case study of AppFunnel in the wild - Traces down performance beyond installations - Shows differences between engines LONG-TERM VIEW INSTALLATION DIRECT USAGE USAGERECOMMENDATION
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×