Recommendations 101

RECKED Jim Shur Amsterdam, The Netherlands 26 Jan 2009 `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ N

^ÖÉåÇ~ • qçé=päáÅÉW=tÜ~í=áë=píê~åÇë\\ • qÜÉ=jÉ~í=Eçê=îÉÖÖáÉëFW= ====J=_ìáäÇáåÖ=~=oÉÅçããÉåÇÉê • _çííçã=päáÅÉW=píê~åÇë=pçÅá~ä=oÉÅçããÉåÇÉê `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ O

tÜ~í=áë=píê~åÇë\\ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ P

tÜ~í=áë=píê~åÇë\\ píê~åÇë=ÇÉîÉäçéë= êÉÅçããÉåÇ~íáçå=íÉÅÜåçäçÖáÉë=íç= ÜÉäé=éÉçéäÉ=ÇáëÅçîÉê=åÉï=íÜáåÖë aÉîÉäçéÉêëW= _ìëáåÉëëÉëW= cêÉÉ=~åÇ=É~ëó=íç=~ÇÇ= båÇ=ìëÉêëW= qÜêçìÖÜ=~=ÜçëíÉÇ=ëçäìíáçåI= êÉÅçããÉåÇ~íáçå= píê~åÇëKÅçã píê~åÇë=ÜÉäéë=ÄìëáåÉëëÉë= ïáÇÖÉíë=ïáíÜ=çìê=^mfë= ãçåÉóKpíê~åÇëKÅçã= ÇÉäáîÉê=íÜÉ=êáÖÜí=ÅçåíÉåí=íç= íÜÉ=êáÖÜí=éÉçéäÉ=~í=íÜÉ= cçê=ÇÉîÉäçéÉêë=~åÇ= êáÖÜí=íáãÉ ëã~ää=éêçàÉÅíë `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ Q

mêÉëÉåíáåÖ qÜÉ=låÉJäáåÉJçÑJ`çÇÉ= oÉÅçããÉåÇÉê `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ R

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ T

^ic • ^ëëçÅá~íáçå=iáëí=cçêã~í • iáëíë=çÑ=áíÉãë=Ñçê=ïÜáÅÜ=óçì=Ü~îÉ=ëçãÉ=êÉ~ëçå= íç=ÄÉäáÉîÉ=íÜ~í=É~ÅÜ=áíÉã=ÅçìäÇ=ëÉêîÉ=~ë=~= êÉÅçããÉåÇ~íáçå=Ñçê=íÜÉ=çíÜÉê • _ÉÜ~îáçê~ä • jìëáÅ=éä~óäáëíëI=éä~ó=ëíêÉ~ãëI=Ñ~îçêáíÉëI=KKK • mìêÅÜ~ëÉ=Ä~ëâÉíëI=é~ÖÉ=îáëáíëI=KKK • `çããçå=ãÉí~Ç~í~=EÅçäÇ=ëí~êíF `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ U

labs:~/cgi-bin/data/movie$ wc -l alf_*.txt 214711 alf_movie_imdb_actors.txt 129865 alf_movie_imdb_actresses.txt 24970 alf_movie_imdb_cinematographers.txt 19018 alf_movie_imdb_composers.txt 7120 alf_movie_imdb_costume-designers.txt 47145 alf_movie_imdb_directors.txt 28048 alf_movie_imdb_distributors.txt 24544 alf_movie_imdb_editors.txt 24886 alf_movie_imdb_keywords.txt 13691 alf_movie_imdb_locations.txt 11639 alf_movie_imdb_miscellaneous-companies.txt 65584 alf_movie_imdb_miscellaneous.txt 57846 alf_movie_imdb_producers.txt 34519 alf_movie_imdb_production-companies.txt 7574 alf_movie_imdb_production-designers.txt 1333 alf_movie_imdb_special-effects-companies.txt 57309 alf_movie_imdb_writers.txt 769802 total `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ V

^ëëçÅá~íáçå=iáëíë imdb_actors_Bogaert%2C Frans,Bleeke Bet (1923),Bulldog Drummond (1922),Fate's Plaything (1920) imdb_actors_Bogaerts%2C Sam,Ellektra (2004),Sicilia%2C La (1997),Steve + Sky (2004),Witte%2C De (1980) imdb_actors_Bogan%2C Hugh,Men Cry Bullets (1997),Zoo Radio (1990) imdb_actors_Bogard%2C Jerrod,Gatorman (2006),LovecraCked! The Movie (2006) imdb_actors_Bogart%2C Humphrey,Across the Paciﬁc (1942),Action in the North Atlantic (1943),African Queen%2C The (1951),... imdb_costume-designers_Adamcova%2C Ruzena,Frona (1954),Igelfreundschaft%2C Die (1962),Jestrab kontra Hrdlicka (1953) imdb_costume-designers_Adams%2C Annalisa,Billy Schulz (2006),Coffee Date (2006),Confessions of a Late Bloomer (2005)... imdb_costume-designers_Adams%2C Katherine,Safe Side%2C The (2005),Trip%2C The (2003) imdb_costume-designers_Adams-Galligan%2C Kimberly,House Divided%2C A (1998),Minotaur (1994),Minus Man%2C The (1999) imdb_costume-designers_Adams-Santos%2C Stephanie,Last Good Man%2C The (2006),Model Man (2006) imdb_directors_Spielberg%2C Steven (I),,Amistad (1997),Artiﬁcial Intelligence: AI (2001),Close Encounters of the Third Kind ... imdb_directors_Spierig%2C Peter,Big Picture%2C The (2000),Daybreakers (2008),Undead (2003) imdb_directors_Spires%2C Tony,Tears of a Clown (2005),Two Degrees (2001) imdb_directors_Spiridakis%2C Tony,Last Word%2C The (1995),Noise (2004),Tinseltown (1997) imdb_directors_Spiridellis%2C Evan,Ahnuld for Governor (2003),Big Box Mart (2005),Second Term (2005),This Land (2004) mdb_keywords_dog-sled,Alaska: Spirit of the Wild (1997),Alaska's Silver Millions (1936),Arizona Dream (1993)... imdb_keywords_newlywed,10 (1979),Above Suspicion (1943),Another Day%2C Another Man (1966) imdb_keywords_surrealism,2001: A Space Odyssey (1968),300 (2006),8? (1963),8 X 8: A Chess Sonata in 8 Movements (1957)... imdb_keywords_whimsical,10 (1979),10 Things I Hate About You (1999),13 Going on 30 (2004),Addams Family%2C The ,,, `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NM

fíÉã=J=fíÉã= • eÉêÉ=ïÉ=~êÉ=ÇáëÅìëëáÄäÉåÖ=?áíÉãJáíÉã?= êÉÅçããÉåÇ~íáçåë • mÉçéäÉ=ïÜç=äáâÉ=EÄçìÖÜíI=ìëÉI=KKKF=íÜáëI=ïÉ=íÜáåâ= ïáää=äáâÉ=EÄìóI=ìëÉI=KKKF=íÜ~í • iáâÉ=~=ÄáÖ=ã~íêáñ=ïáíÜ=~ää=íÜÉ=áíÉãë=~ë=ÄçíÜ= êçïë=~åÇ=Åçäìãåë • b~ÅÜ=Éåíêó=E~IÄF=áë=~=äáâÉäáÜççÇ=íÜ~í=Ä=áë=~= ÖççÇ=êÉÅçããÉåÇ~íáçå=Ñçê=~ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NN

grep -h -e \"$seed\" $dataDir/alf_*.txt The seed is that from which we are generating the recommendations, in this case “Casablanca (1942)” Hence this ﬁrst step returns all the lists from all the ALF ﬁles that contain “Casablanca (1942)” (imagine huge list of lists, here) `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NO

... | tr , '\\n' | grep -v -e \"$seed\" -e \"^imdb_\" • Pipe all those lists into tr, converting the commas into newlines, putting each movie on its own line, then remove the list headers and the seed itself ... 13th Letter%2C The (1951) 633 Squadron (1964) Mr. Brown Comes Down the Hill (1965) Fox%2C The (1967) imdb_writers_Epstein%2C Julius J. Letter from an Unknown Woman (1948) Happy Deathday (1968) In Our Time (1944) Casablanca (1942) In This Our Life (1942) Intimate Stranger%2C The (1956) Journey Out of Darkness (1967) Letter%2C The (1940) Mission to Moscow (1943) Greengage Summer%2C The (1961) ... `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NP

... | sort | uniq -c • Sort the list and qive to uniq -c to replace duplicate lines with a single line plus the count of occurrences ... 1 Angel's Flight (1965) 1 Angel's Holiday (1937) 1 Angela's Ashes (1999) 1 Angels One Five (1952) 2 Angels Over Broadway (1940) 1 Angels Wash Their Faces%2C The (1939) 1 Angels of Death Island%2C The (2003) 14 Angels with Dirty Faces (1938) 3 Anger Management (2003) 3 Angst essen Seele auf (1974) 1 Angus (1995) 2 Anima (1998) ... `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NQ

... | sort -k1,1nr | head • Now sort again, numerically in reverse order, by the counts. Then take the top of the list, and those are your recommendations! 14 Angels with Dirty Faces (1938) 14 To Have and Have Not (1944) 13 Conspirators%2C The (1944) 13 Now%2C Voyager (1942) 12 Maltese Falcon%2C The (1941) 12 Passage to Marseille (1944) 12 Special Agent K-7 (1936) 11 Big Sleep%2C The (1946) 11 Dark Victory (1939) 11 Key Largo (1948) `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NR

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NS

oÉîáÉï • fíÉã=ñ=áíÉã=êÉÅçããÉåÇ~íáçåë=êÉíìêå=íÜÉ=ãçëí= ~ëëçÅá~íÉÇ=áíÉãë=íç=~=ÖáîÉå=ëÉÉÇ • lêÖ~åáòÉ=óçìê=áíÉãë=áå=~ëëçÅá~íáçåë=äáëíë • cáåÇ=~ää=íÜÉ=äáëíë=ïáíÜ=íÜÉ=ëÉÉÇ • oÉíìêå=íÜÉ=áíÉãë=íÜ~í=~ééÉ~ê=çå=íÜÉ=ãçëí=äáëíë=== `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NT

bñéä~å~íáçåë `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NU

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ NV

`çåëáÇÉê~íáçåë • qÜÉ=a~í~ • tÉáÖÜíáåÖ • mçéìä~êáíó=L=råéçéìä~êáíó • mÉêëçå~äáò~íáçå=C=mêÉëÉåí~íáçå • j~âáåÖ=áí=c~ëí • qÉëíáåÖ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OM

qÜÉ=a~í~ • qÜÉ=a~í~=Ü~ë=íç=ÅçãÉ=Ñêçã=ëçãÉïÜÉêÉI=~åÇ= óçì=éêçÄ~Ääó=åÉÉÇ=~=äçí=çÑ=áí • qÜÉ=ÄÉëí=~äÖçêáíÜã=ïáíÜ=åç=Ç~í~=áë=~ää=ÇêÉëëÉÇ= ìé=ïáíÜ=åç=éä~ÅÉ=íç=Öç=EÄìí=ÅçåîÉêëÉäóKKKF • håçïáåÖ=ïÜÉå=íïç=áíÉãë=~êÉ=?íÜÉ=ë~ãÉ? • j~íÅÜáåÖ=íç=~=Å~í~äçÖ • _É=ÅêÉ~íáîÉ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ ON

tÉáÖÜíáåÖ • pÅçêÉë=ÇçåÛí=Ü~îÉ=íç=ÄÉ=N=éçáåí=éÉê=çÅÅìêêÉåÅÉ • iáëí=ïÉáÖÜíáåÖ • fë=ëÜ~êáåÖ=~=ÇáêÉÅíçê=ãçêÉ=áãéçêí~åí=íÜÉå= ëÜ~êáåÖ=~=ÅçëíìãÉ=ÇÉëáÖåÉê\\ • qÉêã=ïÉáÖÜíáåÖ • fë=NM=éä~óäáëí=ÅçJçÅÅìêêÉåÅÉë=ïáíÜ=`çäÇéä~ó= ïçêíÜ=íÜÉ=ë~ãÉ=~ë=NM=ÅçJçÅÅìêêÉåÅÉë=ïáíÜ= gçå~íÜ~å=oáÅÜã~å\\ • j~óÄÉI=ã~óÄÉ=åçí=JJJ=ãçêÉ=ä~íÉê== `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OO

mçéìä~êáíó • _ÉÜ~îáçê~ä=Ç~í~ • qÜÉ=?`çäÇéä~ó?=mêçÄäÉã • ^ëëçÅá~íáçåë=îëK=`çêêÉä~íáçåë • jçêÉ=~ëëçÅá~íÉÇ=Äìí=äÉëë=ÅçêêÉä~íÉÇ\\ 25 C B A 800 23 100 27 `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OP

mçéìä~êáíó • pçäìíáçåW=qÉêã=tÉáÖÜíáåÖ=Et^icF • list1,A,B:0.2,C:0.3 • qÉêã=cêÉèìÉåÅó=J=fåîÉêëÉ=açÅìãÉåí=cêÉèìÉåÅó • qcJfac • ^äëç=ëçäîÉë=aáäìíÉÇ=iáëí=mêçÄäÉã • ^ëëçÅá~íáçåë=ã~ó=ÄÉ=äÉëë=ãÉ~åáåÖÑìä=áå=îÉêó= ä~êÖÉ=äáëíë • pÉÉ=~äëçW=mÉêëçå~äáò~íáçå `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OQ

råéçéìä~êáíó • oÉÅçããÉåÇ~íáçåë=Ñçê=~å=áíÉã=çå=çåÉ=çê=îÉêó= ÑÉï=äáëíë • ^â~=íÜÉ=?ïÜó=íÜÉ=ÜÉÅâ=ÇáÇ=óçì=êÉÅçããÉåÇ= íÜáë\\?=éêçÄäÉã • jççåäáÖÜí=pçå~í~=JJ[=qçñáÅ=E_êáíåÉó=péÉ~êëF • pçäìíáçåW=qeobpelia • oÉèìáêÉ=ëçãÉ=ãáåáãìã=ëÅçêÉ=L=ÉîáÇÉåÅÉ • qÜêÉëÜçäÇ=J=`çîÉê~ÖÉ=íê~ÇÉçÑÑ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OR

mÉêëçå~äáò~íáçå • êÉÅEìëÉêJNI=ëÉÉÇF=≠=êÉÅEìëÉêJOI=ëÉÉÇF • rëÉ=Üáëíçêó=~åÇ=éêçÑáäÉ=Ñçê=áãéäáÅáí=ÑáäíÉêáåÖ=çê= ïÉáÖÜíáåÖ=E~äêÉ~Çó=ÄçìÖÜíI=îáëáíÉÇ=êÉÅÉåíäóI=KKKF • ^ääçï=ÉñéäáÅáí=ÑáäíÉêáåÖI=Åìëíçãáò~íáçå=EëÉÉW=éçéF • cÉÉÇÄ~Åâ=J=Ñ~îçêáíÉëI=Ä~ååáåÖ • pÉÉÇ=ÖÉåÉê~íáçå=L=~ìÖãÉåí~íáçå • =x=ÑEìëÉêF=Z=êÉÅçããÉåÇ~íáçåë=z `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OS

mêÉëÉåí~íáçå • dççÇ=îëK=rëÉÑìä • lÄîáçìëåÉëë • péêáåÖëíÉÉå=“_çêå=íç=oìåÒ=J[=péêáåÖëíÉÉå= “qÜìåÇÉê=oç~ÇÒ=Eilqo=f=J[=ilqo=ffF • aáîÉêëáíó • kçí=~ää=ëÜçÉë=Äó=íÜÉ=ë~ãÉ=ÇÉëáÖåÉê • cçê=ÑêÉèìÉåí=îáëáíÉêI=ÇçåÛí=ëÜçï=ë~ãÉ= êÉÅçããÉåÇ~íáçåë=ÉîÉêó=íáãÉ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OT

péÉÉÇ • fÑ=åÉÅÉëë~êó> recommender • mêÉJÅçãéìí~íáçå ofﬂine lists association matrix top n correlations • fåÅêÉãÉåí~ä=ìéÇ~íÉë per item `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OU

qÉëíáåÖ • pìÄàÉÅíáîÉ=~å~äóëáë • eçäÇJçìíë • _ÉÑçêÉ=L=^ÑíÉê • ^J_=qÉëíáåÖ • fåÇáêÉÅí=ãÉíêáÅë • jÉ~ëìêáåÖ=ïÜ~íÉîÉê=?ëìÅÅÉëë?=ãÉ~åë=Ñçê= óçìê=ëáíÉ `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OV

píê~åÇë=pçÅá~ä=oÉÅçããÉåÇÉê • qÜÉ=ÅçêÉ=ÅçåÅÉéí=áë=ëíê~áÖÜíÑçêï~êÇ • ^=ÑÉï=êÉ~äáíáÉë=ã~âÉ=áí=ãçêÉ=ÇáÑÑáÅìäí • píê~åÇë=áë=áå=íÜÉ=ÄìëáåÉëë=çÑ=ã~âáåÖ=áí=~ë=É~ëó= ~ë=éçëëáÄäÉ • áÜÉ~êíãçîáÉëKÅçã=~ÇÇÉÇ=êÉÅçããÉåÇ~íáçåë= íç=íÜÉáê=ëáíÉ=áå=çåÉ=Üçìê • q~äâ=íç=gÉëúë=çê=Åçåí~Åí=ìë=Ñçê=ãçêÉ=áåÑç • ÜííéWLLÄáòëçäìíáçåëKëíê~åÇëKÅçã `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ PM

p~~p=oÉÅçããÉåÇÉê dÉåÉêáÅ=ÉåÖáåÉ=Ñçê=ÉJÅçããÉêÅÉI=îáÇÉçI=~åÇ=ëçÅá~ä=ÅçåíÉåí=ëáíÉë `lkcfabkqf^i=C=molmofbq^ov PN `çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

pçÅá~ä=oÉÅçããÉåÇÉê=^mf Item visited Recommended item visited Item reviewed Capturing behavior in a retail site Item added to shopping cart Item added to favorites Item added to wishlist Item purchased Content visited Recommended content visited Content reviewed Capturing behavior in a content site Content searched Content uploaded Content downloaded Get item recommendation Get user recommendation Getting recommendations Explain a recommendation Session start Session start/end Session end Rate an item Rating items Show item rating Set user tags Get user tags Tagging users and items Set item tags Get item tags `lkcfabkqf^i=C=molmofbq^ov PO `çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

bñ~ãéäÉW=ÄÉÜ~îáçê=áå=êÉí~áä ‣ customerId ‣ itemId Item visited Tracks an item that has been visited ‣ userId by a user ‣ sessionId /tracking/event_visited.sbs ‣ needResult ‣ customerId ‣ itemId Recommended item ‣ userId Tracks a recommended item that has ‣ reqId visited been clicked ‣ sessionId ‣ needResult /tracking/event_clickedrecommendation.sbs ‣ widgetId ‣ customerId ‣ itemId Item reviewed Tracks an item reviewed or commented ‣ userId by a user ‣ sessionId /tracking/event_reviewed.sbs ‣ needResult ‣ customerId Item added to ‣ itemId Tracks an item that has been added to ‣ userId shopping cart the shopping cart ‣ sessionId ‣ needResult /tracking/event_addshoppingcart.sbs `lkcfabkqf^i=C=molmofbq^ov PP `çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

aÉîÉäçéÉêë=páíÉ ÜííéWLLÄáòëçäìíáçåëKëíê~åÇëKÅçãLíÉÅÜ oÉÖáëíÉê=C dÉí= `~éíìêÉ `~í~äçÖ=ìéäç~Ç oÉÅçããÉåÇ~íáçåë _ÉÜ~îáçê~ä=bîÉåíë `lkcfabkqf^i=C=molmofbq^ov PQ `çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

bñ~ãéäÉ=rëÉêë `lkcfabkqf^i=C=molmofbq^ov PR `çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

a~ëÜÄç~êÇ=^å~äóíáÅë `lkcfabkqf^i=C=molmofbq^ov PS `çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

^ÅâåçïäÉÇÖÉãÉåíë • qÜ~åâë=íç=hÉå=_~äÇïáå=ïÜç=çêáÖáå~ääó= ÅçåÅÉáîÉÇ=çÑ=íÜÉ=?çåÉJäáåÉ=êÉÅçããÉåÇÉê?=áå= íÜáë=í~äâ • qÜ~åâë=íç=t~âççé~=~åÇ=píê~åÇë=Ñçê= ëéçåëçêáåÖ=ob`hba • qÜ~åâë=íç=~ää=Ñçê=~ííÉåÇáåÖ> `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ PT

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ PU

Recommendations 101

0 comments

Notes on slide 1

8 Favorites