RECKED
Jim Shur
Amsterdam, The Netherlands
26 Jan 2009
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
N

^ÖÉåÇ~
! qçé=päáÅÉW=tÜ~í=áë=píê~åÇë\\
! qÜÉ=jÉ~í=Eçê=îÉÖÖáÉëFW=
====J=_ìáäÇáåÖ=~=oÉÅçããÉåÇÉê
! _çííçã=päáÅÉW=píê~åÇë=pçÅá~ä=oÉÅçããÉåÇÉê
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
O

tÜ~í=áë=píê~åÇë\\
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
P

tÜ~í=áë=píê~åÇë\\
píê~åÇë=ÇÉîÉäçéë=
êÉÅçããÉåÇ~íáçå=íÉÅÜåçäçÖáÉë=íç=
ÜÉäé=éÉçéäÉ=ÇáëÅçîÉê=åÉï=íÜáåÖë
aÉîÉäçéÉêëW= _ìëáåÉëëÉëW=
cêÉÉ=~åÇ=É~ëó=íç=~ÇÇ= båÇ=ìëÉêëW=
qÜêçìÖÜ=~=ÜçëíÉÇ=ëçäìíáçåI=
êÉÅçããÉåÇ~íáçå= píê~åÇëKÅçã
píê~åÇë=ÜÉäéë=ÄìëáåÉëëÉë=
ïáÇÖÉíë=ïáíÜ=çìê=^mfë= ãçåÉóKpíê~åÇëKÅçã=
ÇÉäáîÉê=íÜÉ=êáÖÜí=ÅçåíÉåí=íç=
íÜÉ=êáÖÜí=éÉçéäÉ=~í=íÜÉ=
cçê=ÇÉîÉäçéÉêë=~åÇ=
êáÖÜí=íáãÉ
ëã~ää=éêçàÉÅíë
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
Q

mêÉëÉåíáåÖ
qÜÉ=låÉJäáåÉJçÑJ`çÇÉ=
oÉÅçããÉåÇÉê
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
R

my $recdata =
`grep -h -e \"$seed\" $dataDir/alf_*.txt
| tr , '\\n'
| grep -v -e \"$seed\" -e \"^imdb_\"
| sort
| uniq -c
| sort -k1,1nr
| head`;
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
S

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
T

^ic
! ^ëëçÅá~íáçå=iáëí=cçêã~í
! iáëíë=çÑ=áíÉãë=Ñçê=ïÜáÅÜ=óçì=Ü~îÉ=ëçãÉ=êÉ~ëçå=
íç=ÄÉäáÉîÉ=íÜ~í=É~ÅÜ=áíÉã=ÅçìäÇ=ëÉêîÉ=~ë=~=
êÉÅçããÉåÇ~íáçå=Ñçê=íÜÉ=çíÜÉê
! _ÉÜ~îáçê~ä
! jìëáÅ=éä~óäáëíëI=éä~ó=ëíêÉ~ãëI=Ñ~îçêáíÉëI=KKK
! mìêÅÜ~ëÉ=Ä~ëâÉíëI=é~ÖÉ=îáëáíëI=KKK
! `çããçå=ãÉí~Ç~í~=EÅçäÇ=ëí~êíF
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
U

labs:~/cgi-bin/data/movie$ wc -l alf_*.txt
214711 alf_movie_imdb_actors.txt
129865 alf_movie_imdb_actresses.txt
24970 alf_movie_imdb_cinematographers.txt
19018 alf_movie_imdb_composers.txt
7120 alf_movie_imdb_costume-designers.txt
47145 alf_movie_imdb_directors.txt
28048 alf_movie_imdb_distributors.txt
24544 alf_movie_imdb_editors.txt
24886 alf_movie_imdb_keywords.txt
13691 alf_movie_imdb_locations.txt
11639 alf_movie_imdb_miscellaneous-companies.txt
65584 alf_movie_imdb_miscellaneous.txt
57846 alf_movie_imdb_producers.txt
34519 alf_movie_imdb_production-companies.txt
7574 alf_movie_imdb_production-designers.txt
1333 alf_movie_imdb_special-effects-companies.txt
57309 alf_movie_imdb_writers.txt
769802 total
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
V

^ëëçÅá~íáçå=iáëíë
imdb_actors_Bogaert%2C Frans,Bleeke Bet (1923),Bulldog Drummond (1922),Fate's Plaything (1920)
imdb_actors_Bogaerts%2C Sam,Ellektra (2004),Sicilia%2C La (1997),Steve + Sky (2004),Witte%2C De (1980)
imdb_actors_Bogan%2C Hugh,Men Cry Bullets (1997),Zoo Radio (1990)
imdb_actors_Bogard%2C Jerrod,Gatorman (2006),LovecraCked! The Movie (2006)
imdb_actors_Bogart%2C Humphrey,Across the Pacific (1942),Action in the North Atlantic (1943),African Queen%2C The (1951),...
imdb_costume-designers_Adamcova%2C Ruzena,Frona (1954),Igelfreundschaft%2C Die (1962),Jestrab kontra Hrdlicka (1953)
imdb_costume-designers_Adams%2C Annalisa,Billy Schulz (2006),Coffee Date (2006),Confessions of a Late Bloomer (2005)...
imdb_costume-designers_Adams%2C Katherine,Safe Side%2C The (2005),Trip%2C The (2003)
imdb_costume-designers_Adams-Galligan%2C Kimberly,House Divided%2C A (1998),Minotaur (1994),Minus Man%2C The (1999)
imdb_costume-designers_Adams-Santos%2C Stephanie,Last Good Man%2C The (2006),Model Man (2006)
imdb_directors_Spielberg%2C Steven (I),,Amistad (1997),Artificial Intelligence: AI (2001),Close Encounters of the Third Kind ...
imdb_directors_Spierig%2C Peter,Big Picture%2C The (2000),Daybreakers (2008),Undead (2003)
imdb_directors_Spires%2C Tony,Tears of a Clown (2005),Two Degrees (2001)
imdb_directors_Spiridakis%2C Tony,Last Word%2C The (1995),Noise (2004),Tinseltown (1997)
imdb_directors_Spiridellis%2C Evan,Ahnuld for Governor (2003),Big Box Mart (2005),Second Term (2005),This Land (2004)
mdb_keywords_dog-sled,Alaska: Spirit of the Wild (1997),Alaska's Silver Millions (1936),Arizona Dream (1993)...
imdb_keywords_newlywed,10 (1979),Above Suspicion (1943),Another Day%2C Another Man (1966)
imdb_keywords_surrealism,2001: A Space Odyssey (1968),300 (2006),8? (1963),8 X 8: A Chess Sonata in 8 Movements (1957)...
imdb_keywords_whimsical,10 (1979),10 Things I Hate About You (1999),13 Going on 30 (2004),Addams Family%2C The ,,,
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NM

fíÉã=J=fíÉã=
! eÉêÉ=ïÉ=~êÉ=ÇáëÅìëëáÄäÉåÖ=?áíÉãJáíÉã?=
êÉÅçããÉåÇ~íáçåë
! mÉçéäÉ=ïÜç=äáâÉ=EÄçìÖÜíI=ìëÉI=KKKF=íÜáëI=ïÉ=íÜáåâ=
ïáää=äáâÉ=EÄìóI=ìëÉI=KKKF=íÜ~í
! iáâÉ=~=ÄáÖ=ã~íêáñ=ïáíÜ=~ää=íÜÉ=áíÉãë=~ë=ÄçíÜ=
êçïë=~åÇ=Åçäìãåë
! b~ÅÜ=Éåíêó=E~IÄF=áë=~=äáâÉäáÜççÇ=íÜ~í=Ä=áë=~=
ÖççÇ=êÉÅçããÉåÇ~íáçå=Ñçê=~
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NN

grep -h -e \"$seed\" $dataDir/alf_*.txt
The seed is that from which we are
generating the recommendations, in this
case “Casablanca (1942)”
Hence this first step returns all the lists from all
the ALF files that contain “Casablanca (1942)”
(imagine huge list of lists, here)
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NO

... | tr , '\\n' | grep -v -e \"$seed\" -e \"^imdb_\"
• Pipe all those lists into tr, converting the commas into newlines,
putting each movie on its own line, then remove the list
headers and the seed itself
...
13th Letter%2C The (1951)
633 Squadron (1964)
Mr. Brown Comes Down the Hill (1965)
Fox%2C The (1967)
imdb_writers_Epstein%2C Julius J.
Letter from an Unknown Woman (1948)
Happy Deathday (1968)
In Our Time (1944)
Casablanca (1942)
In This Our Life (1942)
Intimate Stranger%2C The (1956)
Journey Out of Darkness (1967)
Letter%2C The (1940)
Mission to Moscow (1943)
Greengage Summer%2C The (1961)
... `çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NP

... | sort | uniq -c
• Sort the list and qive to uniq -c to replace duplicate lines
with a single line plus the count of occurrences
...
1 Angel's Flight (1965)
1 Angel's Holiday (1937)
1 Angela's Ashes (1999)
1 Angels One Five (1952)
2 Angels Over Broadway (1940)
1 Angels Wash Their Faces%2C The (1939)
1 Angels of Death Island%2C The (2003)
14 Angels with Dirty Faces (1938)
3 Anger Management (2003)
3 Angst essen Seele auf (1974)
1 Angus (1995)
2 Anima (1998)
...
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NQ

... | sort -k1,1nr | head
• Now sort again, numerically in reverse order, by the
counts. Then take the top of the list, and those are your
recommendations!
14 Angels with Dirty Faces (1938)
14 To Have and Have Not (1944)
13 Conspirators%2C The (1944)
13 Now%2C Voyager (1942)
12 Maltese Falcon%2C The (1941)
12 Passage to Marseille (1944)
12 Special Agent K-7 (1936)
11 Big Sleep%2C The (1946)
11 Dark Victory (1939)
11 Key Largo (1948)
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NR

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NS

oÉîáÉï
! fíÉã=ñ=áíÉã=êÉÅçããÉåÇ~íáçåë=êÉíìêå=íÜÉ=ãçëí=
~ëëçÅá~íÉÇ=áíÉãë=íç=~=ÖáîÉå=ëÉÉÇ
! lêÖ~åáòÉ=óçìê=áíÉãë=áå=~ëëçÅá~íáçåë=äáëíë
! cáåÇ=~ää=íÜÉ=äáëíë=ïáíÜ=íÜÉ=ëÉÉÇ
! oÉíìêå=íÜÉ=áíÉãë=íÜ~í=~ééÉ~ê=çå=íÜÉ=ãçëí=äáëíë===
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NT

bñéä~å~íáçåë
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NU

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
NV

`çåëáÇÉê~íáçåë
! qÜÉ=a~í~
! tÉáÖÜíáåÖ
! mçéìä~êáíó=L=råéçéìä~êáíó
! mÉêëçå~äáò~íáçå=C=mêÉëÉåí~íáçå
! j~âáåÖ=áí=c~ëí
! qÉëíáåÖ
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OM

qÜÉ=a~í~
! qÜÉ=a~í~=Ü~ë=íç=ÅçãÉ=Ñêçã=ëçãÉïÜÉêÉI=~åÇ=
óçì=éêçÄ~Ääó=åÉÉÇ=~=äçí=çÑ=áí
! qÜÉ=ÄÉëí=~äÖçêáíÜã=ïáíÜ=åç=Ç~í~=áë=~ää=ÇêÉëëÉÇ=
ìé=ïáíÜ=åç=éä~ÅÉ=íç=Öç=EÄìí=ÅçåîÉêëÉäóKKKF
! håçïáåÖ=ïÜÉå=íïç=áíÉãë=~êÉ=?íÜÉ=ë~ãÉ?
! j~íÅÜáåÖ=íç=~=Å~í~äçÖ
! _É=ÅêÉ~íáîÉ
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
ON

tÉáÖÜíáåÖ
! pÅçêÉë=ÇçåÛí=Ü~îÉ=íç=ÄÉ=N=éçáåí=éÉê=çÅÅìêêÉåÅÉ
! iáëí=ïÉáÖÜíáåÖ
! fë=ëÜ~êáåÖ=~=ÇáêÉÅíçê=ãçêÉ=áãéçêí~åí=íÜÉå=
ëÜ~êáåÖ=~=ÅçëíìãÉ=ÇÉëáÖåÉê\\
! qÉêã=ïÉáÖÜíáåÖ
! fë=NM=éä~óäáëí=ÅçJçÅÅìêêÉåÅÉë=ïáíÜ=`çäÇéä~ó=
ïçêíÜ=íÜÉ=ë~ãÉ=~ë=NM=ÅçJçÅÅìêêÉåÅÉë=ïáíÜ=
gçå~íÜ~å=oáÅÜã~å\\
! j~óÄÉI=ã~óÄÉ=åçí=JJJ=ãçêÉ=ä~íÉê==
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OO

mçéìä~êáíó
! _ÉÜ~îáçê~ä=Ç~í~
! qÜÉ=?`çäÇéä~ó?=mêçÄäÉã
! ^ëëçÅá~íáçåë=îëK=`çêêÉä~íáçåë
! jçêÉ=~ëëçÅá~íÉÇ=Äìí=äÉëë=ÅçêêÉä~íÉÇ\\
25 C
B A
800
23
100
27
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OP

mçéìä~êáíó
! pçäìíáçåW=qÉêã=tÉáÖÜíáåÖ=Et^icF
! list1,A,B:0.2,C:0.3
! qÉêã=cêÉèìÉåÅó=J=fåîÉêëÉ=açÅìãÉåí=cêÉèìÉåÅó
! qcJfac
! ^äëç=ëçäîÉë=aáäìíÉÇ=iáëí=mêçÄäÉã
! ^ëëçÅá~íáçåë=ã~ó=ÄÉ=äÉëë=ãÉ~åáåÖÑìä=áå=îÉêó=
ä~êÖÉ=äáëíë
! pÉÉ=~äëçW=mÉêëçå~äáò~íáçå
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OQ

råéçéìä~êáíó
! oÉÅçããÉåÇ~íáçåë=Ñçê=~å=áíÉã=çå=çåÉ=çê=îÉêó=
ÑÉï=äáëíë
! ^â~=íÜÉ=?ïÜó=íÜÉ=ÜÉÅâ=ÇáÇ=óçì=êÉÅçããÉåÇ=
íÜáë\\?=éêçÄäÉã
! jççåäáÖÜí=pçå~í~=JJ[=qçñáÅ=E_êáíåÉó=péÉ~êëF
! pçäìíáçåW=qeobpelia
! oÉèìáêÉ=ëçãÉ=ãáåáãìã=ëÅçêÉ=L=ÉîáÇÉåÅÉ
! qÜêÉëÜçäÇ=J=`çîÉê~ÖÉ=íê~ÇÉçÑÑ
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OR

mÉêëçå~äáò~íáçå
! êÉÅEìëÉêJNI=ëÉÉÇF=!=êÉÅEìëÉêJOI=ëÉÉÇF
! rëÉ=Üáëíçêó=~åÇ=éêçÑáäÉ=Ñçê=áãéäáÅáí=ÑáäíÉêáåÖ=çê=
ïÉáÖÜíáåÖ=E~äêÉ~Çó=ÄçìÖÜíI=îáëáíÉÇ=êÉÅÉåíäóI=KKKF
! ^ääçï=ÉñéäáÅáí=ÑáäíÉêáåÖI=Åìëíçãáò~íáçå=EëÉÉW=éçéF
! cÉÉÇÄ~Åâ=J=Ñ~îçêáíÉëI=Ä~ååáåÖ
! pÉÉÇ=ÖÉåÉê~íáçå=L=~ìÖãÉåí~íáçå
! =x=ÑEìëÉêF=Z=êÉÅçããÉåÇ~íáçåë=z
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OS

mêÉëÉåí~íáçå
! dççÇ=îëK=rëÉÑìä
! lÄîáçìëåÉëë
! péêáåÖëíÉÉå=\"_çêå=íç=oìåÒ=J[=péêáåÖëíÉÉå=
\"qÜìåÇÉê=oç~ÇÒ=Eilqo=f=J[=ilqo=ffF
! aáîÉêëáíó
! kçí=~ää=ëÜçÉë=Äó=íÜÉ=ë~ãÉ=ÇÉëáÖåÉê
! cçê=ÑêÉèìÉåí=îáëáíÉêI=ÇçåÛí=ëÜçï=ë~ãÉ=
êÉÅçããÉåÇ~íáçåë=ÉîÉêó=íáãÉ
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OT

péÉÉÇ
! fÑ=åÉÅÉëë~êó> recommender
! mêÉJÅçãéìí~íáçå
offline
lists
association
matrix top n
correlations
! fåÅêÉãÉåí~ä=ìéÇ~íÉë per item
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ OU

qÉëíáåÖ
! pìÄàÉÅíáîÉ=~å~äóëáë
! eçäÇJçìíë
! _ÉÑçêÉ=L=^ÑíÉê
! ^J_=qÉëíáåÖ
! fåÇáêÉÅí=ãÉíêáÅë
! jÉ~ëìêáåÖ=ïÜ~íÉîÉê=?ëìÅÅÉëë?=ãÉ~åë=Ñçê=
óçìê=ëáíÉ
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
OV

píê~åÇë=pçÅá~ä=oÉÅçããÉåÇÉê
! qÜÉ=ÅçêÉ=ÅçåÅÉéí=áë=ëíê~áÖÜíÑçêï~êÇ
! ^=ÑÉï=êÉ~äáíáÉë=ã~âÉ=áí=ãçêÉ=ÇáÑÑáÅìäí
! píê~åÇë=áë=áå=íÜÉ=ÄìëáåÉëë=çÑ=ã~âáåÖ=áí=~ë=É~ëó=
~ë=éçëëáÄäÉ
! áÜÉ~êíãçîáÉëKÅçã=~ÇÇÉÇ=êÉÅçããÉåÇ~íáçåë=
íç=íÜÉáê=ëáíÉ=áå=çåÉ=Üçìê
! q~äâ=íç=gÉë#ë=çê=Åçåí~Åí=ìë=Ñçê=ãçêÉ=áåÑç
! ÜííéWLLÄáòëçäìíáçåëKëíê~åÇëKÅçã
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
PM

p~~p=oÉÅçããÉåÇÉê
dÉåÉêáÅ=ÉåÖáåÉ=Ñçê=ÉJÅçããÉêÅÉI=îáÇÉçI=~åÇ=ëçÅá~ä=ÅçåíÉåí=ëáíÉë
`lkcfabkqf^i=C=molmofbq^ov
PN
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

pçÅá~ä=oÉÅçããÉåÇÉê=^mf
Item visited
Recommended item visited
Item reviewed
Capturing behavior in a retail site Item added to shopping cart
Item added to favorites
Item added to wishlist
Item purchased
Content visited
Recommended content visited
Content reviewed
Capturing behavior in a content site Content searched
Content uploaded
Content downloaded
Get item recommendation
Get user recommendation
Getting recommendations Explain a recommendation
Session start
Session start/end Session end
Rate an item
Rating items Show item rating
Set user tags
Get user tags
Tagging users and items Set item tags
Get item tags
`lkcfabkqf^i=C=molmofbq^ov
PO
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

bñ~ãéäÉW=ÄÉÜ~îáçê=áå=êÉí~áä
! customerId
! itemId
Item visited Tracks an item that has been visited
! userId
by a user
! sessionId
/tracking/event_visited.sbs
! needResult
! customerId
! itemId
Recommended item ! userId
Tracks a recommended item that has
! reqId
visited been clicked
! sessionId
! needResult
/tracking/event_clickedrecommendation.sbs
! widgetId
! customerId
! itemId
Item reviewed Tracks an item reviewed or commented
! userId
by a user
! sessionId
/tracking/event_reviewed.sbs
! needResult
! customerId
Item added to ! itemId
Tracks an item that has been added to
! userId
shopping cart the shopping cart
! sessionId
! needResult
/tracking/event_addshoppingcart.sbs
`lkcfabkqf^i=C=molmofbq^ov
PP
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

aÉîÉäçéÉêë=páíÉ
ÜííéWLLÄáòëçäìíáçåëKëíê~åÇëKÅçãLíÉÅÜ
oÉÖáëíÉê=C dÉí=
`~éíìêÉ
`~í~äçÖ=ìéäç~Ç oÉÅçããÉåÇ~íáçåë
_ÉÜ~îáçê~ä=bîÉåíë
`lkcfabkqf^i=C=molmofbq^ov
PQ
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

bñ~ãéäÉ=rëÉêë
`lkcfabkqf^i=C=molmofbq^ov
PR
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

a~ëÜÄç~êÇ=^å~äóíáÅë
`lkcfabkqf^i=C=molmofbq^ov
PS
`çéóêáÖÜí=OMMPJOMMUI=píê~åÇëI=fåÅ

^ÅâåçïäÉÇÖÉãÉåíë
! qÜ~åâë=íç=hÉå=_~äÇïáå=ïÜç=çêáÖáå~ääó=
ÅçåÅÉáîÉÇ=çÑ=íÜÉ=?çåÉJäáåÉ=êÉÅçããÉåÇÉê?=áå=
íÜáë=í~äâ
! qÜ~åâë=íç=t~âççé~=~åÇ=píê~åÇë=Ñçê=
ëéçåëçêáåÖ=ob`hba
! qÜ~åâë=íç=~ää=Ñçê=~ííÉåÇáåÖ>
`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
PT

`çéóêáÖÜí=OMMPJOMMVI=píê~åÇëI=fåÅ
PU