Downloaded 23 times
![CONTEXTUAL BANDITS
[0.1, 0.6]
[0.6, 0.4]
[0.7, 0.1]
[0.4, 0.2]](https://image.slidesharecdn.com/hml-bandits-190623230755/75/Practical-contextual-bandits-for-business-15-2048.jpg)
Uploaded byYan Xu
Practical contextual bandits for business
This document summarizes Yan Xu's presentation on practical applications of multi-armed bandits. Bandits can be used for personalized recommendation, such as recommending news articles, by balancing exploration of new articles with exploitation of known good articles. Amazon's bandit algorithm allows for real-time optimization of multiple variables by modeling interactions between variables. The algorithm was able to increase website conversion by 21% after a single week of optimization.
In this document
Powered by AI
Overview of the presentation by Yan Xu on practical bandit algorithms and applications.
Summary of topics including bandit problems and algorithms for personalized recommendations.
Introduction to multi-armed bandits, a foundational concept in decision-making.
Discussion on the trade-off between exploring new options and exploiting known preferences.
Techniques to resolve the exploitation vs exploration trade-off: Epsilon Greedy, UCB, Thompson Sampling.
Examination of reward and regret in the context of multi-armed bandits.
Mathematical formulation and understanding of multi-armed bandit problems.
Real-world applications of bandit algorithms in various decision-making scenarios.
How bandit algorithms can be used in personalized content and article recommendations.
Application of contextual bandits in recommending news articles to users.
Explanation of contextual bandits, a more advanced version of the multi-armed bandit model.
Detailed discussion on the LINUCB algorithm tailored for contextual bandits.
Presentation on linear disjoint models and their relevance to bandit problems.
Illustration of upper bounds related to bandit algorithms.
Differentiation between feature-free and linear contextual bandit approaches.
Methods and criteria for evaluating the performance of bandit algorithms.
Strategies to manage challenges related to high dimensionality in contextual bandits.
Results indicating effectiveness in personalized news article recommendations.
Summary of key insights and applications discussed throughout the presentation.
Insights into how Amazon applies bandit algorithms for multivariate optimization.
Use of bandit algorithms to optimize user web experiences by assessing layout.
Step-by-step explanation of problem formulation and applying bandit algorithms.
Presentation of simulation results showing conversion increase from bandit optimization.
Final thoughts on contextual bandits, applications, and links to detailed papers.
More Related Content
Similar to Practical contextual bandits for business
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Practical contextual bandits for business
- 1. PRACTICAL BANDITS FOR BUSINESS YanXu Houston Machine Learning Meetup June 22, 2019
- 2. OUTLINE - Recap onBandit Problem - A Contextual-Bandit Approach to Personalized News Article Recommendation http://rob.schapire.net/papers/www10.pdf - An efficient bandit algorithm for realtime multivariate optimization https://www.kdd.org/kdd2017/papers/view/an- efficient-bandit-algorithm-for-realtime-multivariate- optimization
- 3.
- 4. DILEMMA: EXPLORATION VS. EXPLOITATION Theexploration/exploitation trade-off is a dilemma we frequently face in choosing between options. Stay the same route to drive home, or try a new route? Choose your favorite restaurant, or the new one? Listen to your favorite music channel, or try a new artist? Attend a new meetup?
- 5. HOW TO RESOLVETHE DILEMMA https://pavlov.tech/2019/03/02/animated-multi- armed-bandit-policies/ Epsilon Greedy UCB (Upper Confidence Bound) Thompson Sampling
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15. CONTEXTUAL BANDITS [0.1, 0.6] [0.6,0.4] [0.7, 0.1] [0.4, 0.2]
- 16.
- 17.
- 18.
- 19.
- 20.
- 21. FEATURE FREE VSLINEAR CONTEXTUAL BANDIT
- 22.
- 23.
- 24.
- 25.
- 26. DEALING WITH HIGH DIMENSIONALITY ～1000binary features per user; ~100 binary feature per article
- 27.
- 28.
- 29. RESULT: PERSONALIZED NEWS Omniscient: alwayschooses the article with highest empirical
- 30.
- 31. AMAZON: BANDITS FOR MULTIVARIATEOPTIMIZATION Published at KDD 2017, KDD 2019 is in Alaska!
- 32. AMAZON: BANDITS FOR MULTIVARIATEOPTIMIZATION
- 33.
- 34.
- 35. STEP 1: PROBITREGRESSION
- 36.
- 37. STEP 3: HILLING-CLIMBINGTO DECIDE
- 38.
- 39. SIMULATION RESULT Control widgetinteraction in simulation through alpha_2.
- 40. EXPERIMENT ON REAL TRAFFIC •After only a single week of online optimization, we saw a 21% conversion increase compared to the median layout
- 41. SUMMARY Contextual bandits Linearpayoff Add interaction components UCB: Variance estimation of expected rewards Thompson sampling: Sample weights from posterior distribution Applications Recommendation Multi-variate optimization For more details - A Contextual-Bandit Approach to Personalized News Article Recommendation http://rob.schapire.net/papers/www10.pdf - An efficient bandit algorithm for realtime multivariate optimization https://www.kdd.org/kdd2017/papers/view/an-efficient- bandit-algorithm-for-realtime-multivariate-optimization