Recommendations 101

RECKED
Jim Shur
Amsterdam, The Netherlands
26 Jan 2009
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
N

^ÖÉåÇ~

• qçé=päáÅÉW=tÜ~í=áë=píê~åÇë
• qÜÉ=jÉ~í=Eçê=îÉÖÖáÉëFW=
====J=_ìáäÇáåÖ=~=oÉÅçããÉåÇÉê

• _çííçã=päáÅÉW=píê~åÇë=pçÅá~ä=oÉÅçããÉåÇÉê

O

tÜ~í=áë=píê~åÇë

P

tÜ~í=áë=píê~åÇë

píê~åÇë=ÇÉîÉäçéë=
êÉÅçããÉåÇ~íáçå=íÉÅÜåçäçÖáÉë=íç=
ÜÉäé=éÉçéäÉ=ÇáëÅçîÉê=åÉï=íÜáåÖë
aÉîÉäçéÉêëW= _ìëáåÉëëÉëW=
cêÉÉ=~åÇ=É~ëó=íç=~ÇÇ= båÇ=ìëÉêëW=
qÜêçìÖÜ=~=ÜçëíÉÇ=ëçäìíáçåI=
êÉÅçããÉåÇ~íáçå= píê~åÇëKÅçã
píê~åÇë=ÜÉäéë=ÄìëáåÉëëÉë=
ïáÇÖÉíë=ïáíÜ=çìê=^mfë= ãçåÉóKpíê~åÇëKÅçã=
ÇÉäáîÉê=íÜÉ=êáÖÜí=ÅçåíÉåí=íç=
íÜÉ=êáÖÜí=éÉçéäÉ=~í=íÜÉ=
cçê=ÇÉîÉäçéÉêë=~åÇ=
êáÖÜí=íáãÉ
ëã~ää=éêçàÉÅíë

Q

mêÉëÉåíáåÖ

qÜÉ=låÉJäáåÉJçÑJ`çÇÉ=
oÉÅçããÉåÇÉê

R

T

^ic
• ^ëëçÅá~íáçå=iáëí=cçêã~í
• iáëíë=çÑ=áíÉãë=Ñçê=ïÜáÅÜ=óçì=Ü~îÉ=ëçãÉ=êÉ~ëçå=
íç=ÄÉäáÉîÉ=íÜ~í=É~ÅÜ=áíÉã=ÅçìäÇ=ëÉêîÉ=~ë=~=
êÉÅçããÉåÇ~íáçå=Ñçê=íÜÉ=çíÜÉê
• _ÉÜ~îáçê~ä
• jìëáÅ=éä~óäáëíëI=éä~ó=ëíêÉ~ãëI=Ñ~îçêáíÉëI=KKK
• mìêÅÜ~ëÉ=Ä~ëâÉíëI=é~ÖÉ=îáëáíëI=KKK
• `çããçå=ãÉí~Ç~í~=EÅçäÇ=ëí~êíF
U

labs:~/cgi-bin/data/movie$ wc -l alf_*.txt
214711 alf_movie_imdb_actors.txt
129865 alf_movie_imdb_actresses.txt
24970 alf_movie_imdb_cinematographers.txt
19018 alf_movie_imdb_composers.txt
7120 alf_movie_imdb_costume-designers.txt
47145 alf_movie_imdb_directors.txt
28048 alf_movie_imdb_distributors.txt
24544 alf_movie_imdb_editors.txt
24886 alf_movie_imdb_keywords.txt
13691 alf_movie_imdb_locations.txt
11639 alf_movie_imdb_miscellaneous-companies.txt
65584 alf_movie_imdb_miscellaneous.txt
57846 alf_movie_imdb_producers.txt
34519 alf_movie_imdb_production-companies.txt
7574 alf_movie_imdb_production-designers.txt
1333 alf_movie_imdb_special-effects-companies.txt
57309 alf_movie_imdb_writers.txt
769802 total

V

^ëëçÅá~íáçå=iáëíë
imdb_actors_Bogaert%2C Frans,Bleeke Bet (1923),Bulldog Drummond (1922),Fate's Plaything (1920)
imdb_actors_Bogaerts%2C Sam,Ellektra (2004),Sicilia%2C La (1997),Steve + Sky (2004),Witte%2C De (1980)
imdb_actors_Bogan%2C Hugh,Men Cry Bullets (1997),Zoo Radio (1990)
imdb_actors_Bogard%2C Jerrod,Gatorman (2006),LovecraCked! The Movie (2006)
imdb_actors_Bogart%2C Humphrey,Across the Paciﬁc (1942),Action in the North Atlantic (1943),African Queen%2C The (1951),...

imdb_costume-designers_Adamcova%2C Ruzena,Frona (1954),Igelfreundschaft%2C Die (1962),Jestrab kontra Hrdlicka (1953)
imdb_costume-designers_Adams%2C Annalisa,Billy Schulz (2006),Coffee Date (2006),Confessions of a Late Bloomer (2005)...
imdb_costume-designers_Adams%2C Katherine,Safe Side%2C The (2005),Trip%2C The (2003)
imdb_costume-designers_Adams-Galligan%2C Kimberly,House Divided%2C A (1998),Minotaur (1994),Minus Man%2C The (1999)
imdb_costume-designers_Adams-Santos%2C Stephanie,Last Good Man%2C The (2006),Model Man (2006)

imdb_directors_Spielberg%2C Steven (I),,Amistad (1997),Artiﬁcial Intelligence: AI (2001),Close Encounters of the Third Kind ...
imdb_directors_Spierig%2C Peter,Big Picture%2C The (2000),Daybreakers (2008),Undead (2003)
imdb_directors_Spires%2C Tony,Tears of a Clown (2005),Two Degrees (2001)
imdb_directors_Spiridakis%2C Tony,Last Word%2C The (1995),Noise (2004),Tinseltown (1997)
imdb_directors_Spiridellis%2C Evan,Ahnuld for Governor (2003),Big Box Mart (2005),Second Term (2005),This Land (2004)

mdb_keywords_dog-sled,Alaska: Spirit of the Wild (1997),Alaska's Silver Millions (1936),Arizona Dream (1993)...
imdb_keywords_newlywed,10 (1979),Above Suspicion (1943),Another Day%2C Another Man (1966)
imdb_keywords_surrealism,2001: A Space Odyssey (1968),300 (2006),8? (1963),8 X 8: A Chess Sonata in 8 Movements (1957)...
imdb_keywords_whimsical,10 (1979),10 Things I Hate About You (1999),13 Going on 30 (2004),Addams Family%2C The ,,,

NM

fíÉã=J=fíÉã=
• eÉêÉ=ïÉ=~êÉ=ÇáëÅìëëáÄäÉåÖ=?áíÉãJáíÉã?=
êÉÅçããÉåÇ~íáçåë
• mÉçéäÉ=ïÜç=äáâÉ=EÄçìÖÜíI=ìëÉI=KKKF=íÜáëI=ïÉ=íÜáåâ=
ïáää=äáâÉ=EÄìóI=ìëÉI=KKKF=íÜ~í
• iáâÉ=~=ÄáÖ=ã~íêáñ=ïáíÜ=~ää=íÜÉ=áíÉãë=~ë=ÄçíÜ=
êçïë=~åÇ=Åçäìãåë
• b~ÅÜ=Éåíêó=E~IÄF=áë=~=äáâÉäáÜççÇ=íÜ~í=Ä=áë=~=
ÖççÇ=êÉÅçããÉåÇ~íáçå=Ñçê=~
NN

grep -h -e quot;$seedquot; $dataDir/alf_*.txt

The seed is that from which we are
generating the recommendations, in this
case “Casablanca (1942)”

Hence this ﬁrst step returns all the lists from all
the ALF ﬁles that contain “Casablanca (1942)”

(imagine huge list of lists, here)

NO

... | tr , 'n' | grep -v -e quot;$seedquot; -e quot;^imdb_quot;
• Pipe all those lists into tr, converting the commas into newlines,
putting each movie on its own line, then remove the list
headers and the seed itself
...
13th Letter%2C The (1951)
633 Squadron (1964)
Mr. Brown Comes Down the Hill (1965)
Fox%2C The (1967)
imdb_writers_Epstein%2C Julius J.
Letter from an Unknown Woman (1948)
Happy Deathday (1968)
In Our Time (1944)
Casablanca (1942)
In This Our Life (1942)
Intimate Stranger%2C The (1956)
Journey Out of Darkness (1967)
Letter%2C The (1940)
Mission to Moscow (1943)
Greengage Summer%2C The (1961)
... `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NP

... | sort | uniq -c
• Sort the list and qive to uniq -c to replace duplicate lines
with a single line plus the count of occurrences
...
1 Angel's Flight (1965)
1 Angel's Holiday (1937)
1 Angela's Ashes (1999)
1 Angels One Five (1952)
2 Angels Over Broadway (1940)
1 Angels Wash Their Faces%2C The (1939)
1 Angels of Death Island%2C The (2003)
14 Angels with Dirty Faces (1938)
3 Anger Management (2003)
3 Angst essen Seele auf (1974)
1 Angus (1995)
2 Anima (1998)
...
NQ

... | sort -k1,1nr | head
• Now sort again, numerically in reverse order, by the
counts. Then take the top of the list, and those are your
recommendations!

14 Angels with Dirty Faces (1938)
14 To Have and Have Not (1944)
13 Conspirators%2C The (1944)
13 Now%2C Voyager (1942)
12 Maltese Falcon%2C The (1941)
12 Passage to Marseille (1944)
12 Special Agent K-7 (1936)
11 Big Sleep%2C The (1946)
11 Dark Victory (1939)
11 Key Largo (1948)

NR

NS

oÉîáÉï

• fíÉã=ñ=áíÉã=êÉÅçããÉåÇ~íáçåë=êÉíìêå=íÜÉ=ãçëí=
~ëëçÅá~íÉÇ=áíÉãë=íç=~=ÖáîÉå=ëÉÉÇ
• lêÖ~åáòÉ=óçìê=áíÉãë=áå=~ëëçÅá~íáçåë=äáëíë
• cáåÇ=~ää=íÜÉ=äáëíë=ïáíÜ=íÜÉ=ëÉÉÇ
• oÉíìêå=íÜÉ=áíÉãë=íÜ~í=~ééÉ~ê=çå=íÜÉ=ãçëí=äáëíë===

NT

bñéä~å~íáçåë

NU

NV

`çåëáÇÉê~íáçåë
• qÜÉ=a~í~
• tÉáÖÜíáåÖ
• mçéìä~êáíó=L=råéçéìä~êáíó
• mÉêëçå~äáò~íáçå=C=mêÉëÉåí~íáçå
• j~âáåÖ=áí=c~ëí
• qÉëíáåÖ

OM

qÜÉ=a~í~
• qÜÉ=a~í~=Ü~ë=íç=ÅçãÉ=Ñêçã=ëçãÉïÜÉêÉI=~åÇ=
óçì=éêçÄ~Ääó=åÉÉÇ=~=äçí=çÑ=áí
• qÜÉ=ÄÉëí=~äÖçêáíÜã=ïáíÜ=åç=Ç~í~=áë=~ää=ÇêÉëëÉÇ=
ìé=ïáíÜ=åç=éä~ÅÉ=íç=Öç=EÄìí=ÅçåîÉêëÉäóKKKF
• håçïáåÖ=ïÜÉå=íïç=áíÉãë=~êÉ=?íÜÉ=ë~ãÉ?
• j~íÅÜáåÖ=íç=~=Å~í~äçÖ
• _É=ÅêÉ~íáîÉ
ON

tÉáÖÜíáåÖ
• pÅçêÉë=ÇçåÛí=Ü~îÉ=íç=ÄÉ=N=éçáåí=éÉê=çÅÅìêêÉåÅÉ
• iáëí=ïÉáÖÜíáåÖ
• fë=ëÜ~êáåÖ=~=ÇáêÉÅíçê=ãçêÉ=áãéçêí~åí=íÜÉå=
ëÜ~êáåÖ=~=ÅçëíìãÉ=ÇÉëáÖåÉê
• qÉêã=ïÉáÖÜíáåÖ
• fë=NM=éä~óäáëí=ÅçJçÅÅìêêÉåÅÉë=ïáíÜ=`çäÇéä~ó=
ïçêíÜ=íÜÉ=ë~ãÉ=~ë=NM=ÅçJçÅÅìêêÉåÅÉë=ïáíÜ=
gçå~íÜ~å=oáÅÜã~å
• j~óÄÉI=ã~óÄÉ=åçí=JJJ=ãçêÉ=ä~íÉê==
OO

mçéìä~êáíó
• _ÉÜ~îáçê~ä=Ç~í~
• qÜÉ=?`çäÇéä~ó?=mêçÄäÉã
• ^ëëçÅá~íáçåë=îëK=`çêêÉä~íáçåë
• jçêÉ=~ëëçÅá~íÉÇ=Äìí=äÉëë=ÅçêêÉä~íÉÇ

25 C
B A
800
23
100
27
OP

mçéìä~êáíó
• pçäìíáçåW=qÉêã=tÉáÖÜíáåÖ=Et^icF
• list1,A,B:0.2,C:0.3
• qÉêã=cêÉèìÉåÅó=J=fåîÉêëÉ=açÅìãÉåí=cêÉèìÉåÅó
• qcJfac
• ^äëç=ëçäîÉë=aáäìíÉÇ=iáëí=mêçÄäÉã
• ^ëëçÅá~íáçåë=ã~ó=ÄÉ=äÉëë=ãÉ~åáåÖÑìä=áå=îÉêó=
ä~êÖÉ=äáëíë
• pÉÉ=~äëçW=mÉêëçå~äáò~íáçå
OQ

råéçéìä~êáíó
• oÉÅçããÉåÇ~íáçåë=Ñçê=~å=áíÉã=çå=çåÉ=çê=îÉêó=
ÑÉï=äáëíë
• ^â~=íÜÉ=?ïÜó=íÜÉ=ÜÉÅâ=ÇáÇ=óçì=êÉÅçããÉåÇ=
íÜáë?=éêçÄäÉã
• jççåäáÖÜí=pçå~í~=JJ[=qçñáÅ=E_êáíåÉó=péÉ~êëF
• pçäìíáçåW=qeobpelia
• oÉèìáêÉ=ëçãÉ=ãáåáãìã=ëÅçêÉ=L=ÉîáÇÉåÅÉ
• qÜêÉëÜçäÇ=J=`çîÉê~ÖÉ=íê~ÇÉçÑÑ
OR

mÉêëçå~äáò~íáçå
• êÉÅEìëÉêJNI=ëÉÉÇF=≠=êÉÅEìëÉêJOI=ëÉÉÇF
• rëÉ=Üáëíçêó=~åÇ=éêçÑáäÉ=Ñçê=áãéäáÅáí=ÑáäíÉêáåÖ=çê=
ïÉáÖÜíáåÖ=E~äêÉ~Çó=ÄçìÖÜíI=îáëáíÉÇ=êÉÅÉåíäóI=KKKF
• ^ääçï=ÉñéäáÅáí=ÑáäíÉêáåÖI=Åìëíçãáò~íáçå=EëÉÉW=éçéF
• cÉÉÇÄ~Åâ=J=Ñ~îçêáíÉëI=Ä~ååáåÖ
• pÉÉÇ=ÖÉåÉê~íáçå=L=~ìÖãÉåí~íáçå
• =x=ÑEìëÉêF=Z=êÉÅçããÉåÇ~íáçåë=z
OS

mêÉëÉåí~íáçå
• dççÇ=îëK=rëÉÑìä
• lÄîáçìëåÉëë
• péêáåÖëíÉÉå=“_çêå=íç=oìåÒ=J[=péêáåÖëíÉÉå=
“qÜìåÇÉê=oç~ÇÒ=Eilqo=f=J[=ilqo=ffF
• aáîÉêëáíó
• kçí=~ää=ëÜçÉë=Äó=íÜÉ=ë~ãÉ=ÇÉëáÖåÉê
• cçê=ÑêÉèìÉåí=îáëáíÉêI=ÇçåÛí=ëÜçï=ë~ãÉ=
êÉÅçããÉåÇ~íáçåë=ÉîÉêó=íáãÉ
OT

péÉÉÇ
• fÑ=åÉÅÉëë~êó> recommender
• mêÉJÅçãéìí~íáçå
ofﬂine

lists
association
matrix top n
correlations
• fåÅêÉãÉåí~ä=ìéÇ~íÉë per item
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OU

qÉëíáåÖ
• pìÄàÉÅíáîÉ=~å~äóëáë
• eçäÇJçìíë
• _ÉÑçêÉ=L=^ÑíÉê
• ^J_=qÉëíáåÖ
• fåÇáêÉÅí=ãÉíêáÅë
• jÉ~ëìêáåÖ=ïÜ~íÉîÉê=?ëìÅÅÉëë?=ãÉ~åë=Ñçê=
óçìê=ëáíÉ
OV

píê~åÇë=pçÅá~ä=oÉÅçããÉåÇÉê
• qÜÉ=ÅçêÉ=ÅçåÅÉéí=áë=ëíê~áÖÜíÑçêï~êÇ
• ^=ÑÉï=êÉ~äáíáÉë=ã~âÉ=áí=ãçêÉ=ÇáÑÑáÅìäí
• píê~åÇë=áë=áå=íÜÉ=ÄìëáåÉëë=çÑ=ã~âáåÖ=áí=~ë=É~ëó=
~ë=éçëëáÄäÉ
• áÜÉ~êíãçîáÉëKÅçã=~ÇÇÉÇ=êÉÅçããÉåÇ~íáçåë=
íç=íÜÉáê=ëáíÉ=áå=çåÉ=Üçìê
• q~äâ=íç=gÉëúë=çê=Åçåí~Åí=ìë=Ñçê=ãçêÉ=áåÑç
• ÜííéWLLÄáòëçäìíáçåëKëíê~åÇëKÅçã
PM

p~~p=oÉÅçããÉåÇÉê
dÉåÉêáÅ=ÉåÖáåÉ=Ñçê=ÉJÅçããÉêÅÉI=îáÇÉçI=~åÇ=ëçÅá~ä=ÅçåíÉåí=ëáíÉë

`lkcfabkqf^i=C=molmofbq^ov
PN
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

pçÅá~ä=oÉÅçããÉåÇÉê=^mf
Item visited
Recommended item visited
Item reviewed
Capturing behavior in a retail site Item added to shopping cart
Item added to favorites
Item added to wishlist
Item purchased
Content visited
Recommended content visited
Content reviewed
Capturing behavior in a content site Content searched
Content uploaded
Content downloaded

Get item recommendation
Get user recommendation
Getting recommendations Explain a recommendation

Session start
Session start/end Session end

Rate an item
Rating items Show item rating

Set user tags
Get user tags
Tagging users and items Set item tags
Get item tags

PO

bñ~ãéäÉW=ÄÉÜ~îáçê=áå=êÉí~áä
‣ customerId
‣ itemId
Item visited Tracks an item that has been visited
‣ userId
by a user
‣ sessionId
/tracking/event_visited.sbs
‣ needResult
‣ customerId
‣ itemId
Recommended item ‣ userId
Tracks a recommended item that has
‣ reqId
visited been clicked
‣ sessionId
‣ needResult
/tracking/event_clickedrecommendation.sbs
‣ widgetId
‣ customerId
‣ itemId
Item reviewed Tracks an item reviewed or commented
‣ userId
by a user
‣ sessionId
/tracking/event_reviewed.sbs
‣ needResult
‣ customerId
Item added to ‣ itemId
Tracks an item that has been added to
‣ userId
shopping cart the shopping cart
‣ sessionId
‣ needResult
/tracking/event_addshoppingcart.sbs

PP

aÉîÉäçéÉêë=páíÉ
ÜííéWLLÄáòëçäìíáçåëKëíê~åÇëKÅçãLíÉÅÜ

oÉÖáëíÉê=C dÉí=
`~éíìêÉ
`~í~äçÖ=ìéäç~Ç oÉÅçããÉåÇ~íáçåë
_ÉÜ~îáçê~ä=bîÉåíë

PQ

bñ~ãéäÉ=rëÉêë

PR

a~ëÜÄç~êÇ=^å~äóíáÅë

PS

^ÅâåçïäÉÇÖÉãÉåíë
• qÜ~åâë=íç=hÉå=_~äÇïáå=ïÜç=çêáÖáå~ääó=
ÅçåÅÉáîÉÇ=çÑ=íÜÉ=?çåÉJäáåÉ=êÉÅçããÉåÇÉê?=áå=
íÜáë=í~äâ
• qÜ~åâë=íç=t~âççé~=~åÇ=píê~åÇë=Ñçê=
ëéçåëçêáåÖ=ob`hba
• qÜ~åâë=íç=~ää=Ñçê=~ííÉåÇáåÖ>

PT

PU

Recommendations 101

Recommended

Recommended

More Related Content

Similar to Recommendations 101

Similar to Recommendations 101 (11)

Recently uploaded

Recently uploaded (19)

Recommendations 101