Building multi-modal recommendation engines using search engines
Upcoming SlideShare
Loading in...5
×
 

Building multi-modal recommendation engines using search engines

on

  • 8,582 views

This is my strata NY talk about how to build recommendation engines using common items. In particular, I show how multi-modal recommendations can be built using the same framework.

This is my strata NY talk about how to build recommendation engines using common items. In particular, I show how multi-modal recommendations can be built using the same framework.

Statistics

Views

Total Views
8,582
Views on SlideShare
8,239
Embed Views
343

Actions

Likes
17
Downloads
292
Comments
0

4 Embeds 343

http://strataconf.com 282
https://twitter.com 59
http://www.conferize.com 1
http://www.google.co.kr 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Note to speaker: Move quickly through 1st two slides just to set the tone of familiar use cases but somewhat complicated under-the-covers math and algorithms… You don’t need to explain or discuss these examples at this point… just mention one or twoTalk track: Machine learning shows up in many familiar everyday examples, from product recommendations to listing news topics to filtering out that nasty spam from email….
  • Talk track: Under the covers, machine learning looks very complicated. So how do you get from here to the familiar examples? Tonight’s presentation will show you some simple tricks to help you apply machine learning techniques to build a powerful recommendation engine.
  • Note to trainers: the next series of slides start with a cartoon example just to set the pattern of how to find co-occurrence and use it to find indicators of what to recommend. Of course, real examples require a LOT of data of user-item interaction history to actually work, so this is just an analogy to get the idea across…
  • * A history of what everybody has done. Obviously this is just a cartoon because large numbers of users and interactions with items would be required to build a recommender* Next step will be to predict what a new user might like…
  • *Bob is the “new user” and getting apple is his history
  • *Here is where the recommendation engine needs to go to work…Note to trainer: you might see if audience calls out the answer before revealing next slide…
  • Now you see the idea of co-occurrence as a basis for recommendation…
  • *Now we have a new user, Amelia. Like everybody else, she gets a pony… what should the recommender offer her based on her history?
  • * Pony not interesting because it is so widespread that it does not differentiate a pattern
  • Note to trainer: This is the situation similar to that in which we started, with three users in our history. The difference is that now everybody got a pony. Bob has apple and pony but not a puppy…yet
  • *Binary matrix is stored sparsely
  • *Convert by MapReduce into a binary matrixNote to trainer: Whether consider apple to have occurred with self is open question
  • *Convert by MapReduce into a binary matrixNote to trainer: diagonal gives total occurrence for each item (self to self) and is a distraction/ not helpful, so the diagonal here is left blank
  • Old joke: all the world can be divided into 2 categories: Scotch tape and non-Scotch tape… This is a way to think about the co-occurrence
  • Note to trainer: Give students time to offer comments. There’s a lot to discuss here.*Upper left: In context of A, B occurs the largest number of times, 13 times out of 1013 appearances with over 100,000 samples. But that’s only ~1.3% as co-occurrence with A out of of all times B appears.*Upper right: B occurs in context of A 33% of time, but counts so small as to be of concern.*Lower right: most significant anomaly in that B still occurs a small number of times of over 100,000 samples, but it ALWAYS co-occurs with A when it does appear.
  • *The test Mahout uses for this is Log Likelihood Ration (LLR)* Red circle marks the choice that displays highest confidenceNote to trainer: Slide animates with click to show LLR results. SECOND Click animates the choice that has highest confidence.
  • Note to trainer: we go back to the earlier matrix as a reminder…
  • Only important co-occurrence is puppy follows apple
  • *Take that row of matrix and combine with all the meta data we might have…*Important thing to get from the co-occurrence matrix is this indicator..Cool thing: analogous to what a lot of recommendation engines do*This row forms the indicator field in a Solr document containing meta-data (you do NOT have to build a separate index for the indicators)Find the useful co-occurrence and get rid of the rest. Sparsify and get the anomalous co-occurrence
  • Note to trainer: take a little time to explore this here and on the next couple of slides. Details enlarged on next slide
  • *This indicator field is where the output of the Mahout recommendation engine are stored (the row from the indicator matrix that identified significant or interesting co-occurrence. *Keep in mind that this recommendation indicator data is added to the same original document in the Solr index that contains meta data for the item in question
  • This is a diagnostics window in the LucidWorksSolr index (not the web interface a user would see). It’s a way for the developer to do a rough evaluation (laugh test) of the choices offered by the recommendation engine.In other words, do these indicator artists represented by their indicator Id make reasonable recommendations Note to trainer: artist 303 happens to be The Beatles. Is that a good match for Chuck Berry?
  • Here we recap what we have in the different components of the recommenderWe start with the meta data for an item stored in the Solr index
  • *Here we’ve added examples of indicator data for the indicator field(s) of the document
  • *Here we show you what information might be in the sample query
  • Note to trainer: you could ask the class to consider which data is related… for example, the first 3 bullets of the query relate to meta data for the item, not to data produced by the recommendation algorithm. The last 3 bullets refer to data in the sample query related to data in the indicator field(s) that were produced by the Mahout recommendation engine.

Building multi-modal recommendation engines using search engines Building multi-modal recommendation engines using search engines Presentation Transcript

  • Introduction to Mahout And How To Build a Recommender ©MapR Technologies 2013- Confidential 1
  • Topic For Today  What is recommendation?  What makes it different?  What is multi-model recommendation?  How can I build it using common household items? ©MapR Technologies 2013- Confidential 2
  • Oh … Also This  Detailed break-down of a live machine learning system running with Mahout on MapR  With code examples ©MapR Technologies 2013- Confidential 3
  • I may have to summarize ©MapR Technologies 2013- Confidential 4
  • I may have to summarize just a bit ©MapR Technologies 2013- Confidential 5
  • Part 1: 5 minutes of background ©MapR Technologies 2013- Confidential 6
  • Part 2: 5 minutes: I want a pony ©MapR Technologies 2013- Confidential 7
  • ©MapR Technologies 2013- Confidential 8
  • Part 1: 5 minutes of background ©MapR Technologies 2013- Confidential 9
  • What Does Machine Learning Look Like? ©MapR Technologies 2013- Confidential 10
  • What Does Machine Learning Look Like? é T ù T é A A ù é A A ù = ê A1 úé 2 û ë 1 2 û ë 1 ê AT úë ë 2 û é T A1 A1 =ê T ê A 2 A1 ë é r ù é AT A ê 1 ú=ê 1 1 ê r2 ú ê AT A1 ë û ë 2 k3 O(k2 k3 O(κ k d + d) = d log n + d) for small k, high quality O(κ d log k) or O(d log κ log k) for larger k, looser quality A1 A2 ù û ù T A1 A 2 ú AT A 2 ú 2 û ù T A1 A 2 úé h1 ù ê ú T úê h 2 ú A 2 A 2 ûë û é T T r1 = ê A1 A1 A1 A 2 ë é ù ùê h1 ú úê h ú û ë 2 û But tonight we’re going to show you how to keep it simple yet powerful… ©MapR Technologies 2013- Confidential 11
  • Recommendations as Machine Learning  Recommendation: – – – Involves observation of interactions between people taking action (users) and items for input data to the recommender model Goal is to suggest additional appropriate or desirable interactions Applications include: movie, music or map-based restaurant choices; suggesting sale items for e-stores or via cash-register receipts ©MapR Technologies 2013- Confidential 12
  • ©MapR Technologies 2013- Confidential 13
  • ©MapR Technologies 2013- Confidential 14
  • Part 2: How recommenders work (I still want a pony) ©MapR Technologies 2013- Confidential 15
  • Recommendations Recap: Behavior of a crowd helps us understand what individuals will do ©MapR Technologies 2013- Confidential 16
  • Recommendations Alice Charles ©MapR Technologies 2013- Confidential Alice got an apple and a puppy Charles got a bicycle 17
  • Recommendations Alice Bob Charles ©MapR Technologies 2013- Confidential Alice got an apple and a puppy Bob got an apple Charles got a bicycle 18
  • Recommendations Alice Bob ? What else would Bob like? Charles ©MapR Technologies 2013- Confidential 19
  • Recommendations Alice Bob A puppy, of course! Charles ©MapR Technologies 2013- Confidential 20
  • You get the idea of how recommenders work… (By the way, like me, Bob also wants a pony) ©MapR Technologies 2013- Confidential 21
  • Recommendations Alice What if everybody gets a pony? Bob Amelia ? What else would you recommend for Amelia? Charles ©MapR Technologies 2013- Confidential 22
  • Recommendations Alice Bob Amelia ? If everybody gets a pony, it’s not a very good indicator of what to else predict... Charles ©MapR Technologies 2013- Confidential 23
  • Problems with Raw Co-occurrence  Very popular items co-occur with everything (or why it’s not very helpful to know that everybody wants a pony…) –  Very widespread occurrence is not interesting as a way to generate indicators –  Examples: Welcome document; Elevator music Unless you want to offer an item that is constantly desired, such as razor blades (or ponies) What we want is anomalous co-occurrence – This is the source of interesting indicators of preference on which to base recommendation ©MapR Technologies 2013- Confidential 24
  • Get Useful Indicators from Behaviors Use log files to build history matrix of users x items 1. – Remember: this history of interactions will be sparse compared to all potential combinations 2. Transform to a co-occurrence matrix of items x items 3. Look for useful co-occurrence by looking for anomalous cooccurrences to make an indicator matrix – – Log Likelihood Ratio (LLR) can be helpful to judge which co-occurrences can with confidence be used as indicators of preference RowSimilarityJob in Apache Mahout uses LLR ©MapR Technologies 2013- Confidential 25
  • Log Files Alice Charles Charles Alice Alice Bob Bob ©MapR Technologies 2013- Confidential 26
  • Log Files u1 u2 t4 u2 t3 u1 t2 u1 t3 u3 t3 u3 ©MapR Technologies 2013- Confidential t1 t1 27
  • Log Files and Dimensions u1 t1 u2 t4 u2 t3 u1 t2 t1 u1 t3 t2 u3 t3 u3 t1 ©MapR Technologies 2013- Confidential Things Users u1 Alice u2 Charles u3 Bob 28 t3 t4
  • History Matrix: Users by Items Alice ✔ Bob ✔ Charles ©MapR Technologies 2013- Confidential ✔ ✔ ✔ ✔ 29 ✔
  • Co-occurrence Matrix: Items by Items How do you tell which co-occurrences are useful?. 1 2 1 1 2 ©MapR Technologies 2013- Confidential 1 0 - 0 1 1 30 0 0
  • Co-occurrence Matrix: Items by Items Use LLR test to turn co-occurrence into indicators… 1 2 1 1 2 ©MapR Technologies 2013- Confidential 1 0 - 0 1 1 31 0 0
  • Co-occurrence Binary Matrix not not ©MapR Technologies 2013- Confidential 1 1 32 1
  • Spot the Anomaly What conclusion do you draw from each situation? A not A B 13 1000 not B 1000 100,000 A not A B 1 0 not B 0 10,000 ©MapR Technologies 2013- Confidential A B 1 0 not B 0 2 A not A B 10 0 not B 33 not A 0 100,000
  • Spot the Anomaly What conclusion do you draw from each situation? A not A B 13 1000 not B 1000 100,000 A not A B 1 0 not B 0 10,000 0.90 4.52 A not A B 1 0 not B 0 2 A not A B 10 0 not B 0 100,000 1.95 14.3 Root LLR is roughly like standard deviations  In Apache Mahout, RowSimilarityJob uses LLR  ©MapR Technologies 2013- Confidential 34
  • Co-occurrence Matrix Recap: Use LLR test to turn co-occurrence into indicators 1 2 1 1 2 ©MapR Technologies 2013- Confidential 1 0 - 0 1 1 35 0 0
  • Indicator Matrix: Anomalous Co-Occurrence Result: The marked row will be added to the indicator field in the item document… ✔ ✔ ©MapR Technologies 2013- Confidential 36
  • Indicator Matrix That one row from indicator matrix becomes the indicator field in the Solr document used to deploy the recommendation engine. ✔ id: t4 title: puppy desc: The sweetest little puppy ever. keywords: puppy, dog, pet indicators: (t1) Note: data for the indicator field is added directly to meta-data for a document in Solr index. You don’t need to create a separate index for the indicators. ©MapR Technologies 2013- Confidential 37
  • Internals of the Recommender Engine 38 ©MapR Technologies 2013- Confidential 38
  • Internals of the Recommender Engine 39 ©MapR Technologies 2013- Confidential 39
  • Looking Inside LucidWorks Real-time recommendation query and results: Evaluation What to recommend if new user listened to 2122: Fats Domino & 303: Beatles? Recommendation is “1710 : Chuck Berry” 40 ©MapR Technologies 2013- Confidential 40
  • Search-based Recommendations  Sample document – – – – – Merchant Id Field for text description Phone Address Location ©MapR Technologies 2013- Confidential 41
  • Search-based Recommendations  Sample document – – – – – – – – – – Merchant Id Field for text description Phone Address Location Indicator merchant id’s Indicator industry (SIC) id’s Indicator offers Indicator text Local top40 ©MapR Technologies 2013- Confidential 42
  • Search-based Recommendations  Sample document – – – – –  Merchant Id Field for text description Phone Address Location Sample query – – – – – – – – – – – Indicator merchant id’s Indicator industry (SIC) id’s Indicator offers Indicator text Local top40 ©MapR Technologies 2013- Confidential 43 Current location Recent merchant descriptions Recent merchant id’s Recent SIC codes Recent accepted offers Local top40
  • Search-based Recommendations  Original data Sample document and meta-data – Merchant Id – – – –  Sample query – Field for text description Phone Address Location – – – – – – – – – – Current location Recent merchant descriptions Recent merchant id’s Recent SIC codes Recent accepted offers Local top40 Indicator merchant id’s Recommendation Indicator industry (SIC) id’s query Indicator offers Indicator text Derived from cooccurrence Local top40 and cross-occurrence analysis ©MapR Technologies 2013- Confidential 44
  • For example  Users enter queries (A) –  Users view videos (B) –  (actor = user, item=video) ATA gives query recommendation –  (actor = user, item=query) “did you mean to ask for” BTB gives video recommendation – “you might like these videos” ©MapR Technologies 2013- Confidential 45
  • The punch-line  BTA recommends videos in response to a query – – (isn’t that a search engine?) (not quite, it doesn’t look at content or meta-data) ©MapR Technologies 2013- Confidential 46
  • Real-life example  Query: “Paco de Lucia”  Conventional meta-data search results: – –  “hombres del paco” times 400 not much else Recommendation based search: – – – Flamenco guitar and dancers Spanish and classical guitar Van Halen doing a classical/flamenco riff ©MapR Technologies 2013- Confidential 47
  • Real-life example ©MapR Technologies 2013- Confidential 48
  • Hypothetical Example  Want a navigational ontology?  Just put labels on a web page with traffic –  Remember viewing history –  This gives B = users x items Cross recommend –  This gives A = users x label clicks B’A = label to item mapping After several users click, results are whatever users think they should be ©MapR Technologies 2013- Confidential 49
  • Nice. But we can do better? ©MapR Technologies 2013- Confidential 50
  • A Quick Simplification  Users who do h (a vector of things a user has done) Ah  A translates things into users Also do r A ( Ah) User-centric recommendations (transpose translates back to things) ( A A) h Item-centric recommendations (change the order of operations) T T ©MapR Technologies 2013- Confidential 51
  • Symmetry Gives Cross Recommentations ( A A) h T ( ) BT A h ©MapR Technologies 2013- Confidential Conventional recommendations with off-line learning Cross recommendations 52
  • things users ©MapR Technologies 2013- Confidential A 53
  • thing thing type 1 type 2 users ©MapR Technologies 2013- Confidential é A A ù 2 û ë 1 54
  • é A ë 1 é A 2 ù é A1 A 2 ù = ê û ë û ê ë é =ê ê ë é r ù é ê 1 ú=ê ê r2 ú ê ë û ë T T ù A1 úé A1 T úë A2 û A2 ù û ù T T A1 A1 A1 A 2 ú AT A1 AT A 2 ú 2 2 û ù T T A1 A1 A1 A 2 úé h1 ê T T A 2 A1 A 2 A 2 úê h 2 ûë é h é T ù 1 T r1 = ê A1 A1 A1 A 2 úê ë ûê h 2 ë ©MapR Technologies 2013- Confidential 55 ù ú ú û ù ú ú û
  • Part 3: What about that worked example? ©MapR Technologies 2013- Confidential 56
  • History collector (6) User behavior generator (1) Presentation tier (2) Diagnostic browsing (9) Cooccurrence analysis (7) Post to search engine (8) Search engine (4) Session collector (3) http://bit.ly/18vbbaT ©MapR Technologies 2013- Confidential 57 Metrics and logs (5)
  • Analyze with Map-Reduce Complete history SolR SolR Indexer Solr Indexer indexing Cooccurrence (Mahout) Item metadata ©MapR Technologies 2013- Confidential Index shards 58
  • Deploy with Conventional Search System User history SolR SolR Indexer Solr Indexer search Web tier Item metadata ©MapR Technologies 2013- Confidential Index shards 59
  • Me, Us  Ted Dunning, Chief Application Architect, MapR Committer PMC member, Mahout, Zookeeper, Drill Bought the beer at the first HUG  MapR Distributes more open source components for Hadoop Adds major technology for performance, HA, industry standard API’s  Info Hash tag - #mapr See also - @ApacheMahout @ApacheDrill @ted_dunning and @mapR ©MapR Technologies 2013- Confidential 60