Automatic Selection of Linked Open Data features in Graph-based Recommender Systems

Automatic Selection of
Linked Open Data features in
Graph-based Recommender Systems
Cataldo Musto, Pierpaolo Basile, Marco de Gemmis
Pasquale Lops, Giovanni Semeraro, Simone Rutigliano
(Università degli Studi di Bari ‘Aldo Moro’, Italy - SWAP Research Group)
CBRecSys 2015
Workshop on New Trends in
Content-based Recommender Systems
Vienna (Austria)
September 20, 2015

Outline
• Basics
• Linked Open Data
• Graph-based Recommendations
• PageRank with Priors
• Methodology
• Introducing LOD-based features
• Selecting LOD-based features
• Experiments
• Impact of LOD-based features
• Comparison to baselines
• Trade-off F1/Diversity
• Conclusions
2Cataldo Musto, Pierpaolo Basile, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro, Simone Rutigliano.
Automatic Selection of Linked Open features in Graph-based Recommender Systems. CBRecSys 2015 Workshop, Vienna, 20.09.2015

3
Linked Open Data
Cataldo Musto, Pierpaolo Basile, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro, Simone Rutigliano.

4
Linked Open Data
What are you talking about?
(*) basic italian gesture
(*)

5
Linked Open Data
Methodology to publish, share and link
structured data on the Web
Deﬁnition

6
Linked Open Data (cloud)
What is it?
A (large) set of interconnected semantic datasets

7
What kind of datasets?
Each bubble is a dataset!

8
How many data?
1048 datasets and 58 billions triplessource: http://stats.lod2.eu
(slide from CBRecSys
2014 presentation)

9
How many data?
3426 datasets and 86 billions triplessource: http://stats.lod2.eu
today!

10
DBpedia is the structured mapping of Wikipedia
http://dbpedia.org
It is the core of the LOD cloud.
DBpedia

11
Example: unstructured content from Wikipedia
example (Wikipedia page)

12
How are these data represented?
The Matrix
Don Davis
http://dbpedia.org/resource/Category:Films_shot_in_Australia
Films shot in
Australia
dcterms:subject
dbpedia-owl:m
usicCom
poser
http://dbpedia.org/resource/Don_Davis_(composer)
dcterms:subject
dcterm
s:subject
dcterms:subjectdbpedia-owl:director
dcterm
s:subject
dcterm
s:subject
Dystopian Films1999 Films
American Action
Thriller Films
Cyberpunk Films The Wachowskis
http://dbpedia.org/resource/The_Wachowskis
http://dbpedia.org/resource/Dystopian_FIlmshttp://dbpedia.org/resource/1999_FIlms
http://dbpedia.org/resource/Cyberpunk_Films
http://dbpedia.org/resource/American_Action_Thriller_FIlms
http://dbpedia.org/resource/Films_About_Rebellions
Films about
Rebellions

13
The Matrix
Don Davis
Films shot in
Australia
dcterms:subject
dbpedia-owl:m
usicCom
poser
dcterms:subject
dcterm
s:subject
dcterms:subjectdbpedia-owl:director
dcterm
s:subject
dcterm
s:subject
Dystopian Films1999 Films
American Action
Thriller Films
http://dbpedia.org/resource/Dystopian_FIlmshttp://dbpedia.org/resource/1999_FIlms
Films about
Rebellions

14
Semantic Web cake
Information from the
LOD cloud is
represented in RDF

15
The Matrix
Don Davis
Films shot in
Australia
dcterms:subject
dbpedia-owl:m
usicCom
poser
dcterms:subject
dcterm
s:subject
dbpedia-owl:director
dcterm
s:subject
dcterm
s:subject
Dystopian Films
American Action
Thriller Films
http://dbpedia.org/resource/Dystopian_FIlms
Films about
Rebellions
dbo:runtime
136

16
The Matrix
Don Davis
Films shot in
Australia
dcterms:subject
dbpedia-owl:m
usicCom
poser

dcterms:subject
dcterm
s:subject
dbo:runtimedbpedia-owl:director
dcterm
s:subject
dcterm
s:subject
Dystopian Films136
American Action
Thriller Films
http://dbpedia.org/resource/Dystopian_FIlms
Films about
Rebellions
Several interesting (non-trivial) features come into play!

Research Question
Can we use Linked Open Data for
Recommender Systems?

Contribution
We investigate the impact of the injection of
exogenousknowledge coming from the LOD cloud
in graph-based recommender systems

Graphs
Why graphs?

Graphs
First, because LOD cloud is a graph

Graphs
Graphs are the most straightforward
representation for LOD-based data points

Why graphs?
Second, graphs can easily model
the recommendation task

i4
(Bipartite graph)
Users = nodes
Items = nodes
Preferences = edges
Very intuitive
representation!
u1
i1
u2 i2
u3 i3
u4
i4
Graph-based RecSys

i4
u1
i1
u2 i2
u3 i3
u4
i4
Graph-based RecSys
Recommendations
are obtained by
identifying the
most relevant
(item) nodes for a
target user,
according to the
graph topology.

i4
u1
i1
u2 i2
u3 i3
u4
i4
Graph-based RecSys
Recommendations
are obtained by
identifying the
most relevant
(item) nodes for a
target user,
according to the
graph topology.
How can we
obtain such
information?

Graph-based Recommendations
PageRank calculates the ‘importance’ of a node according
to the quality and the number of its connections
PageRank

i4
u1
i1
u2 i2
u3 i3
u4
i4
Graph-based RecSys
It is likely that i4 is
suggested to u3, since it
is more (and better)
connected in the graph

i4
u1
i1
u2 i2
u3 i3
u4
i4
Graph-based RecSys
Issue:
Classic PageRank
is not personalized!

i4
Graph-based RecSys
Issue:
Classic PageRank
is not personalized!
1/8
1/8
1/8
1/8
1/8
1/8
1/8 1/8
When PageRank is
run, all the nodes
are provided with an
even distributed
probability

i4
Graph-based RecSys
connected in the graph1/8
1/8
1/8
1/8
1/8
1/8
1/8 1/8
When PageRank is
run, all the nodes
even distributed
probability
Page, L., Brin, S., Motwani, R., &
Winograd, T. (1999). The PageRank
citation ranking: bringing order to
the Web.
Reference

i4
Graph-based RecSys
1/8
1/8
1/8
1/8
1/8
1/8 1/8
When PageRank is
run, all the nodes
even distributed
probability
the Web.
Reference
All the users are provided
with the same ranking,
which it is independent
from user preferences

i4
Graph-based RecSys
1/8
1/8
1/8
1/8
1/8
1/8 1/8
When PageRank is
run, all the nodes
even distributed
probability
the Web.
ReferenceSolution:
PageRank with
Priors

Graph-based Recommendations
Rationale:
PageRank with Priors
. T. H. Haveliwala. Topic-Sensitive PageRank: A
Context-Sensitive Ranking Algorithm for Web
Search. IEEE Trans. Knowl. Data Eng., 15(4):
784–796, 2003.  
Reference
Random Walks have to be inﬂuenced by previous users
behaviors (preferences!). Probability cannot be even
distributed.

A large probability
(e.g. 80%) is
assigned a priori to
speciﬁc items
0.33/8
0.33/8
0.33/8
0.33/8
0.33/8
0.33/8
3/8
3/8

A large probability
(e.g. 80%) is
speciﬁc items
0.33/8
0.33/8
0.33/8
0.33/8
0.33/8
0.33/8
3/8
3/8
e.g., the items a
user liked!

A large probability
(e.g. 80%) is
speciﬁc items
0.33/8
0.33/8
0.33/8
0.33/8
0.33/8
0.33/8
3/8
3/8
e.g., the items a
user liked!
PageRank is run,
and calculations
are thus inﬂuenced
by such a different
distribution of a
priori probabilities.

PageRank with Priors is a very good recommendation model.
Musto, C., Basile, P., Lops, P., de
Gemmis, M., & Semeraro, G. (2014).
Linked Open Data-enabled Strategies
for Top-N Recommendations.
CBRecSys 2014
Reference

PageRank with Priors is a very good recommendation model.
Musto, C., Basile, P., Lops, P., de
Gemmis, M., & Semeraro, G. (2014).
Linked Open Data-enabled Strategies
for Top-N Recommendations.
CBRecSys 2014
Reference
We want to exploit Linked Open Data to further improve it.

Methodology
Graphs provide a uniform and solid representation
to model collaborative (user preferences) data points
as well as LOD-based ones

If we are able to map the items in the dataset with the
entities in the LOD cloud, our representation can
be extended with new data points
Methodology

i4
u1
u2
u3
u4
Methodology
example - original representation

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
example - LOD-boosted representation
1999 ﬁlms
http://dbpedia.org/resource/1999_ﬁlms
dcterms:subject
dcterms:subject

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
Many new information
can be injected in the
graph

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
can be run again
against this novel
representation

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
How do the
recommendations
change with such a
new topology?

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
Is there any signiﬁcant
increase in terms of
computational load?

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
Is it necessary to inject
all of the properties
available in
the LOD cloud?

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
X
X

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
X
X
X

i4
u1
u2
u3
u4
dcterms:subject
Films about
Rebellions
dbprop:director
Quentin Tarantino
dbprop:director
Methodology
1999 ﬁlms
dcterms:subject
dcterms:subject
Is it possibile to
automatically select
the most promising
properties?

51
Experiments

Research Questions
Do graph-based
recommender systems
beneﬁt of the introduction
of LOD-based features?
Do graph-based
recommender systems
exploiting LOD beneﬁt of
the adoption of feature
selection techniques?
1/2
1.
2.

Research Questions
3.
4.
2/2
Is there any correlation
between the choice of the
FS technique and the
behavior of the algorithm?
(e.g., better diversity or
better F1) ?
How does our
methodology perform
with respect to state-of-
the-art algorithms?

Experimental Evaluation
Description of the dataset
MovieLens 100k
983 users
1,682 movies
100,000 ratings
55.17% positive ratings
84.43 ratings/user (avg.)
48.48 ratings/item (avg.)
93.7% sparsity

DBpedia mapping
1,600 movies (95%) were successfully mapped to DBpedia
by querying a SPARQL endpoint with the title of the movie.

DBpedia mapping
60 properties were extracted from the LOD cloud

Most Popular LOD properties
Top-10 properties in the Movie domain
dcterms:subject
dbprop:starring
dbprop:producer
dbprop:title
dbprop:writer
dbprop:country
dbprop:distributor
dbprop:music
dbprop:director
dbprop:runtime

Experimental Protocol
Algorithm PageRank with Priors
Data Split 5-fold Cross Validation
Graph Representation G, GLOD, GLOD+FS
Feature Selection Techniques
PageRank, Chi-Square, Information
Gain, Gain Ratio, mRMR, PCA, SVM
#Selected Features 10, 30, 50 (out of 60 overall features)
Evaluation Metrics F1, Intra-List Diversity, Run Time

Graph Representations :: Recap
G
Basic Graph with
collaborative data points

GLOD Graph extended with all the properties
gathered from the LOD cloud

GLOD+FS
Graph encoding only the most relevant properties
selected by a feature selection technique FS

Experiment 1
62
Impact of LOD-based features.

F1@5
F1@10
F1@15
G
G_LOD
G
G_LOD
G
G_LOD
53 55 57 59 61
59,63
60,83
54,24
59,41
60,23
53,89
Experiment 1
63
Impact of LOD-based features :: F1-measure
Signiﬁcant improvement in all the metrics (Wilcoxon test)

Run Time (min.)
G
G_LOD
50 262,5 475 687,5 900
880
72
Experiment 1
Tremendous increase in the run time, as well
Impact of LOD-based features :: Run Time

Experiment 1
Impact of LOD-based features :: Graph Representation
Nodes Edges
G 2,625 100,000
GLOD 53,794 178,020
Tremendous increase in the run time
depending on the amount of information encoded in the graph

Experiment 1
66
Impact of LOD-based features :: LESSONS LEARNED
F1 RunTime
LOD features can have a good impact on the overall F1
Tremendous Run Time increase
1.
2.

Experiment 2
67
Impact of Feature Selection techniques

PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18
Experiment 2
68
Impact of Feature Selection techniques :: F1@5
Typically, the larger the number of features, the better the F1

PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18
Experiment 2
69
Typically, the larger the number of features, the better the F1
#50
#50
#50
#50
#50
#30
#30
(best)

PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18
Experiment 2
70
#50
#50
#50
#50
#50
#30
#30
(best)
VERY IMPORTANT: we noted a dataset-dependant behavior

PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18
Experiment 2
71
How does it perform with respect to the baseline?

Experiment 2
72
GLOD (baseline) = 54,24
PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18

PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18
Experiment 2
73
Only three out of seven techniques
(and only with 50 features) overcome the baseline

PageRank
mRMR
Chi-Square
SVM
Gain Ratio
Inf. Gain
PCA
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
10
30
50
53 53,5 54 54,5 55
54,31
54,12
54,06
54,21
54,2
54,21
54,12
54,13
53,96
53,98
54,13
54,19
54,29
54,29
54,06
53,97
53,72
53,82
54,14
53,97
54,18
Experiment 2
74
Overall, PCA is the best-performing feature selection technique

Run Time (min.)
GLOD
GLOD+PCA
50 262,5 475 687,5 900
581
880
Experiment 2
33.9% decrease
Impact of Feature Selection techniques :: Run Time

Nodes Edges
GLOD 53,794 178,020
GLOD+PCA 49,158 169,405
Experiment 2
Impact of Feature Selection techniques :: Run Time
-8.6% nodes and -4.8% edges

Experiment 2
Recap of the Experiment
G GLOD GLOD+PCA
F1@5 0,5406 0,5424 0,5431
F1@10 0,6068 0,6083 0,6088
F1@15 0,5956 0,5963 0,5970
Run Time 72 880 581
LOD Properties 0 60 50
The adoption of Feature Selection Techniques
improves F1 and decreases run time

Experiment 2
78
Impact of Features Selection techniques :: LESSONS LEARNED
F1 RunTime
Computational load drops down, as expected
Features Selection techniques are useful, but they
do not always improve F11.
2.
3. The optimal number of features is dataset-dependant

Experiment 3
79
Trade-off between F1 and diversity

Experiment 3
80
Trade-off between F1 and diversity
Can the choice of the feature selection technique
endogenously induce an higher diversity (or,
respectively, an higher F1) of the recommendations?

Experiment 3
81
Trade-off between F1 and diversity :: F1@5
Features Selection techniques can be split into four classes

Experiment 3
82
Low Diversity
Low Accuracy

Experiment 3
83
Low Diversity
Low Accuracy
Low Diversity
High Accuracy

Experiment 3
84
Low Diversity
Low Accuracy
Low Diversity
High Accuracy
High Diversity
Low Accuracy

Experiment 3
85
Low Diversity
Low Accuracy
Low Diversity
High Accuracy
High Diversity
Low Accuracy
High Diversity
High Accuracy

Experiment 3
86
Low Diversity
Low Accuracy
Low Diversity
High Accuracy
High Diversity
Low Accuracy
High Diversity
High Accuracy

Experiment 3
87
mRMR, Information Gain and ChiSquare are not useful

Experiment 3
88
PCA maximizes F1, at the expense of a little diversity

Experiment 3
89
Gain Ratio and SVM sacriﬁce F1,
to induce an higher diversity

Experiment 3
90
PageRank obtains a good compromise
between F1 and Diversity

Experiment 3
91
Trade-off between F1 and Diversity :: LESSONS LEARNED
F1
Behavior needs to be generalized by analyzing different
datasets
Features Selection techniques can maximize a speciﬁc
evaluation metric, thus a graph-based recsys can be
tuned according to the requirements of a scenario
1.
2.

Experiment 4
92
Comparison to State of the art
BPRMF (Bayesian Personalized Ranking) [+]
U2U-KNN (User to User CF)
I2I-KNN (Item to Item CF)
POPULAR (Popularity-based baseline)
[+] S. Rendle, C.Freudenthaler, Z. Gantner, L. Schmidt-Thieme: BPR:
Bayesian Personalized Ranking from Implicit Feedback. UAI 2009.

Experiment 4
93
Comparison to State of the Art
50
54
58
62
66
F1@5 F1@10
60,88
54,31
59,16
51,4
59,16
51,78
59,7
52,2
58,35
50,22
I2I-KNN U2U-KNN BPRMF POPULAR PR (G_LOD+PCA)

Experiment 4
94
50
54
58
62
66
F1@5 F1@10
60,88
54,31
59,16
51,4
59,16
51,78
59,7
52,2
58,35
50,22
PageRank with Priors boosted with LOD
is the best-performing approach

Experiment 4
95
50
54
58
62
66
F1@5 F1@10
60,88
54,31
59,16
51,4
59,16
51,78
59,7
52,2
58,35
50,22
Even state-of-the-art approaches based on Matrix
Factorization are overcame by our methodology

Conclusions

Recap
97
Methodolology
1. PageRank with Priors as base algorithm
2. Mapping of the items with nodes in the Linked
Open Data Cloud
3. Expansion of the data points and injection of new
nodes and edges
4. Use of feature selection to automatically select the
most promising properties
INVESTIGATION ABOUT THE EFFECTIVENESS OF LINKED OPEN DATA IN
GRAPH-BASED RECOMMENDER SYSTEMS

Lessons Learned
98
Evaluation
1. PageRank with Priors benefit of the injection of data points
coming from the LOD cloud
2. Feature Selection techniques improve the results but need
to be properly tuned, since its usage is not always useful.
3. A significant connection between the choice of the feature
selection technique and the maximization of a specific
evaluation metric exists.
4. PageRank with Priors boosted with LOD significantly
overcomes state-of-the-art approaches.
INVESTIGATION ABOUT THE EFFECTIVENESS OF LINKED OPEN DATA IN
GRAPH-BASED RECOMMENDER SYSTEMS

Future Research
99
Evaluation against different datasets and
stronger baselines;
Further expansion of the graph, by introducing
more LOD-based data points
Evaluation of Novelty and Serendipity on
LOD-based Recommendations;

questions?
Cataldo Musto, Ph.D
cataldo.musto@uniba.it

Automatic Selection of Linked Open Data features in Graph-based Recommender Systems

Recommended

Recommended

More Related Content

What's hot

What's hot (15)

Viewers also liked

Viewers also liked (20)

Similar to Automatic Selection of Linked Open Data features in Graph-based Recommender Systems

Similar to Automatic Selection of Linked Open Data features in Graph-based Recommender Systems (20)

More from Cataldo Musto

More from Cataldo Musto (20)

Recently uploaded

Recently uploaded (20)

Automatic Selection of Linked Open Data features in Graph-based Recommender Systems