Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Website optimisation with Multi Armed Bandit algorithms

121 views

Published on

A talk I gave at 99designs about website optimisation using Multi Armed Bandit algorithms.

It's based on a talk and ebook by John Miles White — "Bandit Algorithms for Website Optimization"

Published in: Data & Analytics
  • Be the first to comment

Website optimisation with Multi Armed Bandit algorithms

  1. 1. A/B testing
  2. 2. Multi Armed Bandit Algorithms
  3. 3. Why do we run A/B tests?
  4. 4. Before During After Logo 1 Logo 1 + Logo 2 Logo 2 ConversionRate Time
  5. 5. Before During After Logo 1 Logo 2 ConversionRate Time
  6. 6. Explore or exploit
  7. 7. Objectively best Option that will be the best in the future
  8. 8. Subjectively best Option that has been the best in the past
  9. 9. Explore Choose any option Exploit Choose the subjectively best option
  10. 10. Regret
  11. 11. Some classic Multi Armed Bandits...
  12. 12. Epsilon greedy
  13. 13. ((1-e) * 100)% to subjectively best (e/2 * 100)% to subjectively best (e/2 * 100)% to subjectively worst
  14. 14. Monte Carlo Run random simulations 1,000’s of times
  15. 15. Weaknesses of ε-greedy Situation 1: A: 99% B: 0.001% Situation 2: A: 0.001% B: 0.002%
  16. 16. Softmax
  17. 17. P(A) = 0.1 P(B) - 0.2
  18. 18. Weaknesses of softmax Situation 1: A: 0.01% after 100 trials B: 0.02% after 100 trials Situation 2: A: 0.01% after 100,000,000 trials A: 0.02% after 100,000,000 trials
  19. 19. UCB Upper Confidence Bound
  20. 20. Weakness of UCB1 Gotcha: rewards have to be between 0.0 and 1.0 Works best on conversion rates. Not as well on arbitrary dollar rewards.
  21. 21. Further reading..
  22. 22. • Other UCB* algorithms • LinUCB / GLM-UCB • Exp3 and other Exp* algorithms
  23. 23. Thanks!

×