Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

3,423 views

Published on

Published in:
Technology

No Downloads

Total views

3,423

On SlideShare

0

From Embeds

0

Number of Embeds

2,003

Shares

0

Downloads

46

Comments

0

Likes

8

No embeds

No notes for slide

- 1. Real-time Learning©MapR Technologies - Confidential 1
- 2. whoami – Ted Dunning Chief Application Architect, MapR Technologies Committer, member, Apache Software Foundation – particularly Mahout, Zookeeper and Drill (we’re hiring) Contact me at tdunning@maprtech.com tdunning@apache.com ted.dunning@gmail.com @ted_dunning©MapR Technologies - Confidential 2
- 3. Slides and such (available late tonight): – http://www.mapr.com/company/events/devoxx-3-29-2013 Hash tags: #mapr #devoxxfr©MapR Technologies - Confidential 3
- 4. Agenda What is real-time learning? A sample problem Philosophy, statistics and the nature of the knowledge A solution System design©MapR Technologies - Confidential 4
- 5. What is Real-time Learning? Training data arrives one record at a time The system improves a mathematical model based on a small amount of training data We retain at most a fixed amount of state Each learning step takes O(1) time and memory©MapR Technologies - Confidential 5
- 6. We have a product to sell … from a web-site©MapR Technologies - Confidential 6
- 7. What tag- What line? picture? Bogus Dog Food is the Best! Now available in handy 1 ton bags! Buy 5! What call to action?©MapR Technologies - Confidential 7
- 8. The Challenge Design decisions affect probability of success – Cheesy web-sites don’t even sell cheese The best designers do better when allowed to fail – Exploration juices creativity But failing is expensive – If only because we could have succeeded – But also because offending or disappointing customers is bad©MapR Technologies - Confidential 8
- 9. A Quick Diversion You see a coin – What is the probability of heads? – Could it be larger or smaller than that? I flip the coin and while it is in the air ask again I catch the coin and ask again I look at the coin (and you don’t) and ask again Why does the answer change? – And did it ever have a single value?©MapR Technologies - Confidential 9
- 10. A Philosophical Conclusion Probability as expressed by humans is subjective and depends on information and experience©MapR Technologies - Confidential 10
- 11. So now you understand Bayesian probability©MapR Technologies - Confidential 11
- 12. Another Quick Diversion Let’s play a shell game This is a special shell game It costs you nothing to play The pea has constant probability of being under each shell (trust me) How do you find the best shell? How do you find it while maximizing the number of wins?©MapR Technologies - Confidential 12
- 13. Pause for short con-game©MapR Technologies - Confidential 13
- 14. Conclusions Can you identify winners or losers without trying them out? No Can you ever completely eliminate a shell with a bad streak? No Should you keep trying apparent losers? Yes, but at a decreasing rate©MapR Technologies - Confidential 14
- 15. So now you understand multi-armed bandits©MapR Technologies - Confidential 15
- 16. Is there an optimum strategy?©MapR Technologies - Confidential 16
- 17. Thompson Sampling Select each shell according to the probability that it is the best Probability that it is the best can be computed using posterior é ùP(i is best) = ò I êE[ri | q ] = max E[rj | q ]ú P(q | D) dq ë j û But I promised a simple answer©MapR Technologies - Confidential 17
- 18. Thompson Sampling – Take 2 Sample θ q ~ P(q | D) Pick i to maximize reward i = argmax E[r | q ] j Record result from using i©MapR Technologies - Confidential 18
- 19. Nearly Forgotten until Recently Citations for Thompson sampling©MapR Technologies - Confidential 19
- 20. Bayesian Bandit for the Shells Compute distributions based on data so far Sample p1, p2 and p3 from these distributions Pick shell i where i = argmaxi pi Lemma 1: The probability of picking shell i will match the probability it is the best shell Lemma 2: This is as good as it gets©MapR Technologies - Confidential 20
- 21. And it works! 0.12 0.11 0.1 0.09 0.08 0.07 regret 0.06 ε- greedy, ε = 0.05 0.05 0.04 Bayesian Bandit with Gam m a- Norm al 0.03 0.02 0.01 0 0 100 200 300 400 500 600 700 800 900 1000 1100 n©MapR Technologies - Confidential 21
- 22. Video Demo©MapR Technologies - Confidential 22
- 23. The Basic Idea We can encode a distribution by sampling Sampling allows unification of exploration and exploitation Can be extended to more general response models©MapR Technologies - Confidential 23
- 24. The Original Problem x2 x1 Bogus Dog Food is the Best! Now available in handy 1 ton bags! Buy 5! x3©MapR Technologies - Confidential 24
- 25. Mathematical Statement Logistic or probit regression P(conversion) = w (å x q ) i ij 1 w(x) = 1+ e- x erf(x) +1 w(x) = 2©MapR Technologies - Confidential 25
- 26. Same Algorithm Sample θ q ~ P(q | D) Pick design x to maximize reward x* = argmax E[rx | q ] = argmax å xiqij x x©MapR Technologies - Confidential 26
- 27. Context Variables x2 x1 Bogus Dog Food is the Best! Now available in handy 1 ton bags! Buy 5! x3 y1=user.geo y2=env.time y3=env.day_of_week y4=env.weekend©MapR Technologies - Confidential 27
- 28. Two Kinds of Variables The web-site design - x1, x2, x3 – We can change these – Different values give different web-site designs The environment or context – y1, y2, y3, y4 – We can’t change these – They can change themselves Our model should include interactions between x and y©MapR Technologies - Confidential 28
- 29. Same Algorithm, More Greek Letters Sample θ, π, φ (q, P, F)~ P(q, P, F | D) Pick design x to maximize reward, y’s are constant x* = argmax E[rx | q ] x = argmax å xiqi + å xi y j p ij + å yij i x i i, j i This looks very fancy, but is actually pretty simple©MapR Technologies - Confidential 29
- 30. Surprises We cannot record a non-conversion until we wait We cannot record a conversion until we wait for the same time Learning from conversions requires delay We don’t have to wait very long©MapR Technologies - Confidential 30
- 31. ©MapR Technologies - Confidential 31
- 32. ©MapR Technologies - Confidential 32
- 33. ©MapR Technologies - Confidential 33
- 34. ©MapR Technologies - Confidential 34
- 35. Required Steps Learn distribution of parameters from data – Logistic regression or probit regression (can be on-line!) – Need Bayesian learning algorithm Sample from posterior distribution – Generally included in Bayesian learning algorithm Pick design – Simple sequential search Record data©MapR Technologies - Confidential 35
- 36. Required system design©MapR Technologies - Confidential 36
- 37. Hadoop is Not Very Real-time Unprocessed now Data t Fully Latest full Hadoop job processed period takes this long for this data©MapR Technologies - Confidential 37
- 38. Real-time and Long-time together Blended now View view t Hadoop works Storm great back here works here©MapR Technologies - Confidential 38
- 39. Traditional Hadoop Design Can use Kafka cluster to queue log lines Can use Storm cluster to do real time learning Can host web site on NAS Can use Flume cluster to import data from Kafka to Hadoop Can record long-term history on Hadoop Cluster How many clusters?©MapR Technologies - Confidential 39
- 40. HDFS Data Flume Hadoop Users Kafka Kafka Kafka Cluster Cluster Kafka Cluster API Storm Kafka Web Site Design Targeting Web Service NAS©MapR Technologies - Confidential 40
- 41. That is a lot of moving parts!©MapR Technologies - Confidential 41
- 42. Alternative Design Can host log catcher on MapR via NFS Storm can read data directly from queue Can host web server directly on cluster Only one cluster needed – Total instances drops by 3x – Admin burden massively decreased©MapR Technologies - Confidential 42
- 43. Users http Web-server Catcher Storm Topic Web Queue Data MapR©MapR Technologies - Confidential 43
- 44. You can do this yourself!©MapR Technologies - Confidential 44
- 45. Contact Me! We’re hiring at MapR in US and Europe MapR software available for research use Contact me at tdunning@maprtech.com or @ted_dunning Share news with @apachemahout Tweet #devoxxfr #mapr #mahout @ted_dunning©MapR Technologies - Confidential 45

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment