Downloaded 483 times
More Related Content
Similar to Building multi-modal recommendation engines using search engines
Building multi-modal recommendation engines using search engines
- 1. Introduction to Mahout AndHow To Build a Recommender ©MapR Technologies 2013- Confidential 1
- 2. Topic For Today Whatis recommendation? What makes it different? What is multi-model recommendation? How can I build it using common household items? ©MapR Technologies 2013- Confidential 2
- 3. Oh … AlsoThis Detailed break-down of a live machine learning system running with Mahout on MapR With code examples ©MapR Technologies 2013- Confidential 3
- 4. I may haveto summarize ©MapR Technologies 2013- Confidential 4
- 5. I may haveto summarize just a bit ©MapR Technologies 2013- Confidential 5
- 6. Part 1: 5 minutesof background ©MapR Technologies 2013- Confidential 6
- 7. Part 2: 5 minutes:I want a pony ©MapR Technologies 2013- Confidential 7
- 8. ©MapR Technologies 2013-Confidential 8
- 9. Part 1: 5 minutesof background ©MapR Technologies 2013- Confidential 9
- 10. What Does MachineLearning Look Like? ©MapR Technologies 2013- Confidential 10
- 11. What Does MachineLearning Look Like? é T ù T é A A ù é A A ù = ê A1 úé 2 û ë 1 2 û ë 1 ê AT úë ë 2 û é T A1 A1 =ê T ê A 2 A1 ë é r ù é AT A ê 1 ú=ê 1 1 ê r2 ú ê AT A1 ë û ë 2 k3 O(k2 k3 O(κ k d + d) = d log n + d) for small k, high quality O(κ d log k) or O(d log κ log k) for larger k, looser quality A1 A2 ù û ù T A1 A 2 ú AT A 2 ú 2 û ù T A1 A 2 úé h1 ù ê ú T úê h 2 ú A 2 A 2 ûë û é T T r1 = ê A1 A1 A1 A 2 ë é ù ùê h1 ú úê h ú û ë 2 û But tonight we’re going to show you how to keep it simple yet powerful… ©MapR Technologies 2013- Confidential 11
- 12. Recommendations as MachineLearning Recommendation: – – – Involves observation of interactions between people taking action (users) and items for input data to the recommender model Goal is to suggest additional appropriate or desirable interactions Applications include: movie, music or map-based restaurant choices; suggesting sale items for e-stores or via cash-register receipts ©MapR Technologies 2013- Confidential 12
- 13. ©MapR Technologies 2013-Confidential 13
- 14. ©MapR Technologies 2013-Confidential 14
- 15. Part 2: How recommenderswork (I still want a pony) ©MapR Technologies 2013- Confidential 15
- 16. Recommendations Recap: Behavior of acrowd helps us understand what individuals will do ©MapR Technologies 2013- Confidential 16
- 17. Recommendations Alice Charles ©MapR Technologies 2013-Confidential Alice got an apple and a puppy Charles got a bicycle 17
- 18. Recommendations Alice Bob Charles ©MapR Technologies 2013-Confidential Alice got an apple and a puppy Bob got an apple Charles got a bicycle 18
- 19. Recommendations Alice Bob ? What else wouldBob like? Charles ©MapR Technologies 2013- Confidential 19
- 20. Recommendations Alice Bob A puppy, ofcourse! Charles ©MapR Technologies 2013- Confidential 20
- 21. You get theidea of how recommenders work… (By the way, like me, Bob also wants a pony) ©MapR Technologies 2013- Confidential 21
- 22. Recommendations Alice What if everybodygets a pony? Bob Amelia ? What else would you recommend for Amelia? Charles ©MapR Technologies 2013- Confidential 22
- 23. Recommendations Alice Bob Amelia ? If everybody getsa pony, it’s not a very good indicator of what to else predict... Charles ©MapR Technologies 2013- Confidential 23
- 24. Problems with RawCo-occurrence Very popular items co-occur with everything (or why it’s not very helpful to know that everybody wants a pony…) – Very widespread occurrence is not interesting as a way to generate indicators – Examples: Welcome document; Elevator music Unless you want to offer an item that is constantly desired, such as razor blades (or ponies) What we want is anomalous co-occurrence – This is the source of interesting indicators of preference on which to base recommendation ©MapR Technologies 2013- Confidential 24
- 25. Get Useful Indicatorsfrom Behaviors Use log files to build history matrix of users x items 1. – Remember: this history of interactions will be sparse compared to all potential combinations 2. Transform to a co-occurrence matrix of items x items 3. Look for useful co-occurrence by looking for anomalous cooccurrences to make an indicator matrix – – Log Likelihood Ratio (LLR) can be helpful to judge which co-occurrences can with confidence be used as indicators of preference RowSimilarityJob in Apache Mahout uses LLR ©MapR Technologies 2013- Confidential 25
- 26. Log Files Alice Charles Charles Alice Alice Bob Bob ©MapR Technologies2013- Confidential 26
- 27. Log Files u1 u2 t4 u2 t3 u1 t2 u1 t3 u3 t3 u3 ©MapR Technologies2013- Confidential t1 t1 27
- 28. Log Files andDimensions u1 t1 u2 t4 u2 t3 u1 t2 t1 u1 t3 t2 u3 t3 u3 t1 ©MapR Technologies 2013- Confidential Things Users u1 Alice u2 Charles u3 Bob 28 t3 t4
- 29. History Matrix: Usersby Items Alice ✔ Bob ✔ Charles ©MapR Technologies 2013- Confidential ✔ ✔ ✔ ✔ 29 ✔
- 30. Co-occurrence Matrix: Itemsby Items How do you tell which co-occurrences are useful?. 1 2 1 1 2 ©MapR Technologies 2013- Confidential 1 0 - 0 1 1 30 0 0
- 31. Co-occurrence Matrix: Itemsby Items Use LLR test to turn co-occurrence into indicators… 1 2 1 1 2 ©MapR Technologies 2013- Confidential 1 0 - 0 1 1 31 0 0
- 32. Co-occurrence Binary Matrix not not ©MapRTechnologies 2013- Confidential 1 1 32 1
- 33. Spot the Anomaly Whatconclusion do you draw from each situation? A not A B 13 1000 not B 1000 100,000 A not A B 1 0 not B 0 10,000 ©MapR Technologies 2013- Confidential A B 1 0 not B 0 2 A not A B 10 0 not B 33 not A 0 100,000
- 34. Spot the Anomaly Whatconclusion do you draw from each situation? A not A B 13 1000 not B 1000 100,000 A not A B 1 0 not B 0 10,000 0.90 4.52 A not A B 1 0 not B 0 2 A not A B 10 0 not B 0 100,000 1.95 14.3 Root LLR is roughly like standard deviations In Apache Mahout, RowSimilarityJob uses LLR ©MapR Technologies 2013- Confidential 34
- 35. Co-occurrence Matrix Recap: UseLLR test to turn co-occurrence into indicators 1 2 1 1 2 ©MapR Technologies 2013- Confidential 1 0 - 0 1 1 35 0 0
- 36. Indicator Matrix: AnomalousCo-Occurrence Result: The marked row will be added to the indicator field in the item document… ✔ ✔ ©MapR Technologies 2013- Confidential 36
- 37. Indicator Matrix That onerow from indicator matrix becomes the indicator field in the Solr document used to deploy the recommendation engine. ✔ id: t4 title: puppy desc: The sweetest little puppy ever. keywords: puppy, dog, pet indicators: (t1) Note: data for the indicator field is added directly to meta-data for a document in Solr index. You don’t need to create a separate index for the indicators. ©MapR Technologies 2013- Confidential 37
- 38. Internals of theRecommender Engine 38 ©MapR Technologies 2013- Confidential 38
- 39. Internals of theRecommender Engine 39 ©MapR Technologies 2013- Confidential 39
- 40. Looking Inside LucidWorks Real-timerecommendation query and results: Evaluation What to recommend if new user listened to 2122: Fats Domino & 303: Beatles? Recommendation is “1710 : Chuck Berry” 40 ©MapR Technologies 2013- Confidential 40
- 41. Search-based Recommendations Sample document – – – – – MerchantId Field for text description Phone Address Location ©MapR Technologies 2013- Confidential 41
- 42. Search-based Recommendations Sample document – – – – – – – – – – MerchantId Field for text description Phone Address Location Indicator merchant id’s Indicator industry (SIC) id’s Indicator offers Indicator text Local top40 ©MapR Technologies 2013- Confidential 42
- 43. Search-based Recommendations Sample document – – – – – MerchantId Field for text description Phone Address Location Sample query – – – – – – – – – – – Indicator merchant id’s Indicator industry (SIC) id’s Indicator offers Indicator text Local top40 ©MapR Technologies 2013- Confidential 43 Current location Recent merchant descriptions Recent merchant id’s Recent SIC codes Recent accepted offers Local top40
- 44. Search-based Recommendations Original data Sampledocument and meta-data – Merchant Id – – – – Sample query – Field for text description Phone Address Location – – – – – – – – – – Current location Recent merchant descriptions Recent merchant id’s Recent SIC codes Recent accepted offers Local top40 Indicator merchant id’s Recommendation Indicator industry (SIC) id’s query Indicator offers Indicator text Derived from cooccurrence Local top40 and cross-occurrence analysis ©MapR Technologies 2013- Confidential 44
- 45. For example Users enterqueries (A) – Users view videos (B) – (actor = user, item=video) ATA gives query recommendation – (actor = user, item=query) “did you mean to ask for” BTB gives video recommendation – “you might like these videos” ©MapR Technologies 2013- Confidential 45
- 46. The punch-line BTA recommendsvideos in response to a query – – (isn’t that a search engine?) (not quite, it doesn’t look at content or meta-data) ©MapR Technologies 2013- Confidential 46
- 47. Real-life example Query: “Pacode Lucia” Conventional meta-data search results: – – “hombres del paco” times 400 not much else Recommendation based search: – – – Flamenco guitar and dancers Spanish and classical guitar Van Halen doing a classical/flamenco riff ©MapR Technologies 2013- Confidential 47
- 48. Real-life example ©MapR Technologies2013- Confidential 48
- 49. Hypothetical Example Want anavigational ontology? Just put labels on a web page with traffic – Remember viewing history – This gives B = users x items Cross recommend – This gives A = users x label clicks B’A = label to item mapping After several users click, results are whatever users think they should be ©MapR Technologies 2013- Confidential 49
- 50. Nice. But we cando better? ©MapR Technologies 2013- Confidential 50
- 51. A Quick Simplification Userswho do h (a vector of things a user has done) Ah A translates things into users Also do r A ( Ah) User-centric recommendations (transpose translates back to things) ( A A) h Item-centric recommendations (change the order of operations) T T ©MapR Technologies 2013- Confidential 51
- 52. Symmetry Gives CrossRecommentations ( A A) h T ( ) BT A h ©MapR Technologies 2013- Confidential Conventional recommendations with off-line learning Cross recommendations 52
- 53. things users ©MapR Technologies 2013-Confidential A 53
- 54. thing thing type 1type 2 users ©MapR Technologies 2013- Confidential é A A ù 2 û ë 1 54
- 55. é A ë 1 é A2 ù é A1 A 2 ù = ê û ë û ê ë é =ê ê ë é r ù é ê 1 ú=ê ê r2 ú ê ë û ë T T ù A1 úé A1 T úë A2 û A2 ù û ù T T A1 A1 A1 A 2 ú AT A1 AT A 2 ú 2 2 û ù T T A1 A1 A1 A 2 úé h1 ê T T A 2 A1 A 2 A 2 úê h 2 ûë é h é T ù 1 T r1 = ê A1 A1 A1 A 2 úê ë ûê h 2 ë ©MapR Technologies 2013- Confidential 55 ù ú ú û ù ú ú û
- 56. Part 3: What aboutthat worked example? ©MapR Technologies 2013- Confidential 56
- 57. History collector (6) User behavior generator(1) Presentation tier (2) Diagnostic browsing (9) Cooccurrence analysis (7) Post to search engine (8) Search engine (4) Session collector (3) http://bit.ly/18vbbaT ©MapR Technologies 2013- Confidential 57 Metrics and logs (5)
- 58. Analyze with Map-Reduce Complete history SolR SolR Indexer Solr Indexer indexing Cooccurrence (Mahout) Itemmetadata ©MapR Technologies 2013- Confidential Index shards 58
- 59. Deploy with ConventionalSearch System User history SolR SolR Indexer Solr Indexer search Web tier Item metadata ©MapR Technologies 2013- Confidential Index shards 59
- 60. Me, Us Ted Dunning,Chief Application Architect, MapR Committer PMC member, Mahout, Zookeeper, Drill Bought the beer at the first HUG MapR Distributes more open source components for Hadoop Adds major technology for performance, HA, industry standard API’s Info Hash tag - #mapr See also - @ApacheMahout @ApacheDrill @ted_dunning and @mapR ©MapR Technologies 2013- Confidential 60
Editor's Notes
- #11 Note to speaker: Move quickly through 1st two slides just to set the tone of familiar use cases but somewhat complicated under-the-covers math and algorithms… You don’t need to explain or discuss these examples at this point… just mention one or twoTalk track: Machine learning shows up in many familiar everyday examples, from product recommendations to listing news topics to filtering out that nasty spam from email….
- #12 Talk track: Under the covers, machine learning looks very complicated. So how do you get from here to the familiar examples? Tonight’s presentation will show you some simple tricks to help you apply machine learning techniques to build a powerful recommendation engine.
- #17 Note to trainers: the next series of slides start with a cartoon example just to set the pattern of how to find co-occurrence and use it to find indicators of what to recommend. Of course, real examples require a LOT of data of user-item interaction history to actually work, so this is just an analogy to get the idea across…
- #18 * A history of what everybody has done. Obviously this is just a cartoon because large numbers of users and interactions with items would be required to build a recommender* Next step will be to predict what a new user might like…
- #19 *Bob is the “new user” and getting apple is his history
- #20 *Here is where the recommendation engine needs to go to work…Note to trainer: you might see if audience calls out the answer before revealing next slide…
- #21 Now you see the idea of co-occurrence as a basis for recommendation…
- #23 *Now we have a new user, Amelia. Like everybody else, she gets a pony… what should the recommender offer her based on her history?
- #24 * Pony not interesting because it is so widespread that it does not differentiate a pattern
- #27 Note to trainer: This is the situation similar to that in which we started, with three users in our history. The difference is that now everybody got a pony. Bob has apple and pony but not a puppy…yet
- #30 *Binary matrix is stored sparsely
- #31 *Convert by MapReduce into a binary matrixNote to trainer: Whether consider apple to have occurred with self is open question
- #32 *Convert by MapReduce into a binary matrixNote to trainer: diagonal gives total occurrence for each item (self to self) and is a distraction/ not helpful, so the diagonal here is left blank
- #33 Old joke: all the world can be divided into 2 categories: Scotch tape and non-Scotch tape… This is a way to think about the co-occurrence
- #34 Note to trainer: Give students time to offer comments. There’s a lot to discuss here.*Upper left: In context of A, B occurs the largest number of times, 13 times out of 1013 appearances with over 100,000 samples. But that’s only ~1.3% as co-occurrence with A out of of all times B appears.*Upper right: B occurs in context of A 33% of time, but counts so small as to be of concern.*Lower right: most significant anomaly in that B still occurs a small number of times of over 100,000 samples, but it ALWAYS co-occurs with A when it does appear.
- #35 *The test Mahout uses for this is Log Likelihood Ration (LLR)* Red circle marks the choice that displays highest confidenceNote to trainer: Slide animates with click to show LLR results. SECOND Click animates the choice that has highest confidence.
- #36 Note to trainer: we go back to the earlier matrix as a reminder…
- #37 Only important co-occurrence is puppy follows apple
- #38 *Take that row of matrix and combine with all the meta data we might have…*Important thing to get from the co-occurrence matrix is this indicator..Cool thing: analogous to what a lot of recommendation engines do*This row forms the indicator field in a Solr document containing meta-data (you do NOT have to build a separate index for the indicators)Find the useful co-occurrence and get rid of the rest. Sparsify and get the anomalous co-occurrence
- #39 Note to trainer: take a little time to explore this here and on the next couple of slides. Details enlarged on next slide
- #40 *This indicator field is where the output of the Mahout recommendation engine are stored (the row from the indicator matrix that identified significant or interesting co-occurrence. *Keep in mind that this recommendation indicator data is added to the same original document in the Solr index that contains meta data for the item in question
- #41 This is a diagnostics window in the LucidWorksSolr index (not the web interface a user would see). It’s a way for the developer to do a rough evaluation (laugh test) of the choices offered by the recommendation engine.In other words, do these indicator artists represented by their indicator Id make reasonable recommendations Note to trainer: artist 303 happens to be The Beatles. Is that a good match for Chuck Berry?
- #42 Here we recap what we have in the different components of the recommenderWe start with the meta data for an item stored in the Solr index
- #43 *Here we’ve added examples of indicator data for the indicator field(s) of the document
- #44 *Here we show you what information might be in the sample query
- #45 Note to trainer: you could ask the class to consider which data is related… for example, the first 3 bullets of the query relate to meta data for the item, not to data produced by the recommendation algorithm. The last 3 bullets refer to data in the sample query related to data in the indicator field(s) that were produced by the Mahout recommendation engine.