Successfully reported this slideshow.
Your SlideShare is downloading. ×

Learn Like a Human: Taking Machine Learning from Batch to Real-Time

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 20 Ad

Learn Like a Human: Taking Machine Learning from Batch to Real-Time

Elad Rosenheim, our Chief Software Architect, gave a talk about machine learning and conversion optimization at eXelate's "Big Data eXposed 2015" event in Tel-Aviv. In his presentation, Elad touched on some of the most common problems encountered when trying to optimize websites for better conversion rates, and covered a wide range of solutions - starting with classic A/B testing (unsuitable for rapidly changing content), through Multi-arm Bandits, and finally Contextual Bandits – those online machine learning algorithms that allow for true personalization in real time.

Elad Rosenheim, our Chief Software Architect, gave a talk about machine learning and conversion optimization at eXelate's "Big Data eXposed 2015" event in Tel-Aviv. In his presentation, Elad touched on some of the most common problems encountered when trying to optimize websites for better conversion rates, and covered a wide range of solutions - starting with classic A/B testing (unsuitable for rapidly changing content), through Multi-arm Bandits, and finally Contextual Bandits – those online machine learning algorithms that allow for true personalization in real time.

Advertisement
Advertisement

More Related Content

Similar to Learn Like a Human: Taking Machine Learning from Batch to Real-Time (20)

Advertisement
Advertisement

Learn Like a Human: Taking Machine Learning from Batch to Real-Time

  1. 1. Learn Like a Human: Taking Machine Learning from Batch to Real-Time By Elad Rosenheim, Senior Software Architect delivered in
  2. 2. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • The Industry’s Problem • Old School Solutions • Meet the Contextual Bandits On the Agenda:
  3. 3. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Publishers, retailers, SaaS all share a common problem • They know their domain but not how to optimize for each user • Screen real-estate is limited yet everyone sees the same thing The Industry’s Problem
  4. 4. Learn Like a Human: Taking Machine Learning from Batch to Real-Time Commonly asked questions • What articles should be shown on the homepage of an online publisher? • What titles and images would attract the most clicks? • What should the order and layout of the articles be? • Should every user see the “Top Videos” section? • What is the best layout for the homepage?
  5. 5. Learn Like a Human: Taking Machine Learning from Batch to Real-Time Commonly asked questions • Which product sorting layout would yield the highest revenue? • Should the layout differ significantly between different user segments?
  6. 6. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • What types of products should be shown, and to whom? Commonly asked questions
  7. 7. Learn Like a Human: Taking Machine Learning from Batch to Real-Time In the Beginning: • First came the educated guess • Then came the A/B test • "Data Beats Opinion“ • Freedom to experiment (with nice tools) • Hopefully: Less fear of change, less politics • How does it work? • Split traffic between baseline and alternative variations • In theory: Sit and wait for significant results • In practice: Peek at the numbers ‘til “95% confidence” is reached
  8. 8. Learn Like a Human: Taking Machine Learning from Batch to Real-Time While you wait, you're bleeding clicks! Clicks = Money What about the really dynamic stuff? Campaigns, headlines, products on sale A/B Tests: Already Old School?
  9. 9. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • A Single-arm Bandit • Suppose I have multiple arms in front of me, each with its unknown mean reward… • How do I optimize income from multiple machines? • Caution or Haste? • Explore vs. Exploit • In our context: How do I optimize multiple variations? Enter the Multi-arm Bandits
  10. 10. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • (Very) Simple Solutions • ε-greedy, ε–decreasing • First 100% random explore, then ~90% exploit? • Magic numbers, built-in revenue loss • Bayesian-based approaches • Smoother curve from explore to exploit • “Winner” is now a less relevant term Bandits - A Classic Problem
  11. 11. Learn Like a Human: Taking Machine Learning from Batch to Real-Time We want to find the variation “best on average“ …but we’re not improving the conversion rate of any single variation Bandits work well when… 0.4%1.7%2.4%
  12. 12. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Each of us is a beautiful and unique feature vector! • By showing the right variation to the right people, we can improve conversions per variation and beat the best variation • Machine learning challenge accepted! Enter Personalization
  13. 13. Learn Like a Human: Taking Machine Learning from Batch to Real-Time The Usual Suspects • Collaborative Filtering? • Very big, very sparse matrix • Cold start, batch process • Not suitable in this case • Classifiers? • Logistic Regression, Random Forest et al. • Periodically learn over all converters so far • More data = more time, bigger model • “Partial Feedback” problem: • Users aren’t of class “Variation A” or “Variation B”. • Rather, for any given user, we’re placing our bets on a variation and don’t know what the reward would be for any other alternative.
  14. 14. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Like a bandit, we need to learn as we go (not in batch), but this time with “context” - the user’s data • Incremental Learning over the stream of impressions & rewards (“Partial Fit”) • We’re looking to… • Start learning from the first impression • Handle the explore-exploit curve • Run fast (enough) • Worst case: Converge on the best variation like a bandit What We Need
  15. 15. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • They “eat” the data stream • They demand fast access to user data • Historical or immediate • Their model is always ready for action • In the Papers • Linear Bayes, LinUCB • What we do: Per-Variation Logistic Regression • A variant supporting updates in “mini-batches” • Exploration-on-top • Worst case: “Garbage In Multi Arm Bandit Out” • Light on memory, compact output Meet the Contextual Bandits
  16. 16. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Online should be fast & scaled • Offline: A test-bed for iteratively testing new ideas • New algorithms • Tweaked parameters • Feature transformations - last but not least! How We Do It: Online & Offline
  17. 17. Learn Like a Human: Taking Machine Learning from Batch to Real-Time Persist Model The Online Flow DY Web Servers: a. Get our script b. Log impressions, conversions Learn Workers User DB Load to Predict Server Queue Per Test A B C A B C A B C Predictions
  18. 18. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Test, Improve, Iterate • Using real-world data • Using generated data • From easy to hard The Offline Evaluator
  19. 19. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Learn in the center site, predict quickly in each geo. How? • Push models via local Redis slaves • Compressed SSH tunnel • User data - daily aggregation • Store into LMDB (simple, fast memory-mapped K/V DB) • Sync via S3 (LZ4 compressed), read from local SSD • Learn & Predict services • Python as ML lingua franca: NumPy, SciPy, scikit-learn Going Global
  20. 20. Learn Like a Human: Taking Machine Learning from Batch to Real-Time • Better data beats better algorithms • Reduce aggressively Keep It Simple, Smart! By Elad Rosenheim, Senior Software Architect Final Word: * Thanks to Idan Michaeli, Lead Data Scientist

Editor's Notes

  • איך ניסו עד היום לתקוף את הבעיות האלה?
  • בואו ניקח את הפיתרון שלנו שלב אחד קדימה
  • בואו ניקח את הפיתרון שלנו שלב אחד קדימה
  • בואו ניקח את הפיתרון שלנו שלב אחד קדימה
  • בקיצור, העולם לא כ"כ מופלא כפי שמוכרים לנו
  • בואו ניקח את הפיתרון שלנו שלב אחד קדימה
  • לבעיה הזו אין פיתרון אופטימלי, אבל יש בהחלט גישות שונות ברמות שונות של מורכבות
  • עכשיו, בנדיטים הם לא כ"כ רעים למעשה...
  • והמגבלה הזו מביאה אותנו לשלב הבא בחיפוש, והוא: פרסונליזציה
  • יפה, אז קיבלנו את האתגר. על איזה אלגוריתמים אנחנו חושבים?
  • אז בעצם, אנחנו מחפשים משהו אחר – משפחה חדשה של אלגוריתמים
  • אז בואו ונכיר את הגיבורים החדשים שלנו...
  • אלגוריתם כשלעצמו זה דבר נחמד, אבל איך בונים את כל המעטפת לפרודקשן?
  • בואו ונבין טוב יותר את הזרימה

×