Using Grids to support
Information Filtering Systems
Running Collaborative Filtering
Recommendations on gLite middleware
Leandro N. Ciuffo
leandro.ciuffo@ct.infn.it
INFN-Catania - Italy

Recommender system settings

http://canalcinefilia.com.br
Movielens-like recommender system
(http://movielens.umn.edu)
Explicit data collection
(users need to rate > 20 movies)

No dedicated servers / standard technologies
Hosted by a comercial Web Hosting service
www
MySQL

Data set
225 out of 390 users
are able to get movie recommendations
921 movies
33,147 ratings

Rating Matrix
225 users X 921 movies
~207,000 „cells‟

Users’ similarity - Pearson r correlation
The complexity of maintaining a similarity
matrix with the Pearson correlation
between every pair of users is O(m2n)

Neighborhood: similarity threshold r > 0.3
Pearson
Correlation
0.6 -1 < r < 1
0.5
0.7
0.4
1
0.3
0.8
0.9

Generating recommendations
Will this user enjoy the movie “Yes Man” ?
Weighted mean
0.9 0.8 0.6 0.4 0.3
Rating
User’s neighborhood Prediction
generated
A movie must be rated by
This repeats for every movie not rated
at least 8 neighbors

www.canalcinefilia.com.br - predictions
65,388 predictions

Grid Computing

Computational intensive research

Computation intensive applications / experiments

EGEE Grid (www.eu-egee.org)
EGEE numbers:
>260 sites
54 countries
~114,000 CPUs
>20 PetaBytes
>16,000 users
>200 VOs
>150,000 jobs/day

The global network coverage

The global Grid coverage
SEE-Grid
DEISA BalticGrid
TeraGrid
EGEE EUChinaGrid
OSG NAREGI
EUMedGrid
EUIndiaGrid
EUAsiaGrid
EELA

Implementation on the
gLite-based Grid

GILDA (https://gilda.ct.infn.it)
 Grid INFN Laboratory for Dissemination Activities
 Grid test-bed for training
 A „standard‟ t-Infrastructure adopted by
many projects
 Users can practice prior to run their codes
on the production e-Infrastructures
 ~ 11 sites - 285 CPUs *
(*) # of Sites may change in time
they are managed on a “best effort” basis

EELA (www.eu-eela.eu)
 E-science grid facility for Europe and Latin America
 Co-funded by EC (FP7)
 ~ 5800 CPUs

EELA (www.eu-eela.eu)

www
MySQL

.JDL
LFC
Grid UI

SE
SE

Input sandbox
mdclient.config
Start.sh
Recommender.class
Grid UI

WMS
Grid UI

SE SE
WN WN WN
CE 1 CE 2 CE n

Output - Version I

.SQL
.SQL .SQL
.SQL .SQL
.SQL
.SQL .SQL .SQL
WN WN WN
CE 1 CE 2 CE n

Grid UI WMS
.SQL
.SQL
.SQL
Output sandbox

www
MySQL

Output - Version II

AMGA
WN WN WN
CE 1 CE 2 CE n

www
MySQL

Implementation on OurGrid

OurGrid (www.ourgrid.org)
 Opportunistic Grid
 Job submissions can be handled by a web portal

What’s next?

Future works
 Run experiments using the Netflix prize database
 Create a new version using the Amazon EC2
 Provide performance comparisons among
EELA (gLite) X OurGrid X Amazon EC2
I‟m looking for partners

The End
leandro.ciuffo@ct.infn.it
http://www.canalcinefilia.com.br
http://canalcinefilia.com.br/en/credits/about.php
http://applications.eu-eela.eu/application_details.php?ID=59