Comparing knowledge graph embeddings such as TransE, TransH, TransR for the task of item recommendation.

1. An Empirical Comparison of Knowledge Graph
Embeddings for Item Recommendation
Enrico Palumbo, Giuseppe Rizzo, Raphael Troncy,
Elena Baralis, Michele Osella, Enrico Ferro

2. MOVIE RECOMMENDATIONS

3. ITEM RECOMMENDATION
USER
ITEMS
ranking function
ρ (u, i)
0.8
0.6
0.5
0.2ρ(u,i)

4. CONTENT-BASED FILTERING
JANE
ρ (u, i)
0.8Kill_Bill_Vol.2
Samuel_Jackson
0.0
Content similarities with items liked in the past

5. COLLABORATIVE FILTERING ρ (u, i)
0.8
0.0
collaborative: users who watch x also like y...
Kill_Bill_Vol.2
Jane
Mark
Taxi_Driver

6. BEYOND CONTENT-BASED AND COLLABORATIVE...
Content-based
● overspecialization
● new users
● requires item metadata
Collaborative
● new users
● new items
● data sparsity
Hybrid recommender systems

7. Knowledge Graph
● K = (E, R, O)
● Entities E
● Users U ⊂
● Items I ⊂
●
●
● ⊂
● feedback ⊂
Kill_Bill_Vol.2
Samuel_Jackson
JaneMark
Quentin_Tarantino
Taxi_Driver
Star_Wars_Ep.1
THE_AVENGERS

8. RECOMMENDATIONS USING A KNOWLEDGE GRAPH
Item recommendation as a
knowledge graph completion
problem!

9. Knowledge Graph Embeddings
Kill_Bill_Vol.2
Samuel_Jackson
Jackie_BrownJaneMark
Quentin_Tarantino
Quentin_Tarantino
Mark
Jane
Jackie_Brown
Samuel_Jackson
Kill_Bill_Vol.2
Taxi_Driver
Taxi_Driver
Star_Wars_Ep.1
Star_Wars_Ep.1
F

10. FEATURES IN THE EMBEDDING SPACE
F
Feature engineering
● Manually define a mapping
function

11. FEATURES IN THE EMBEDDING SPACE
F
Feature engineering
● Manually define a mapping
function
Feature learning
● Define F (W) as a function of
a set of parameters W (e.g. a
neural network)
● Learn W from an
optimization problem

12. FEATURES IN THE EMBEDDING SPACE
F
Feature engineering
● Manually define a mapping
function
Feature learning
● Define F (W) as a function of
a set of parameters W (e.g. a
neural network)
● Learn W from an
optimization problem
Random walks:
● DeepWalk [1]
● Node2vec [2]
● RDF2Vec [3]
● Entity2rec [4]

13. FEATURES IN THE EMBEDDING SPACE
F
Feature engineering
● Manually define a mapping
function
Feature learning
● Define F (W) as a function of
a set of parameters W (e.g. a
neural network)
● Learn W from an
optimization problem
Random walks:
● DeepWalk [1]
● Node2vec [2]
● RDF2Vec [3]
● Entity2rec [4]
Translational models:
● TransE [5]
● TransH [6]
● TransR [7]

14. FEATURES IN THE EMBEDDING SPACE
F
Feature engineering
● Manually define a mapping
function
Feature learning
● Define F (W) as a function of
a set of parameters W (e.g. a
neural network)
● Learn W from an
optimization problem
Random walks:
● DeepWalk [1]
● Node2vec [2]
● RDF2Vec [3]
● Entity2rec [4]
Translational models:
● TransE [5]
● TransH [6]
● TransR [7]

15. TRANSLATIONAL MODELS
TransE [5]
Model: h + r ~ t
Simple, but not suitable
for 1-to-N, N-to-1, N-to-N
relations

16. TRANSLATIONAL MODELS
TransH [6]
Model: h⊥
+ dr
~ t⊥
Multiple representations
for different relations
projecting on hyperplanes
TransE [5]
Model: h + r ~ t
Simple, but not suitable
for 1-to-N, N-to-1, N-to-N
relations

17. TRANSLATIONAL MODELS
TransR [7]
Model: hr
+ r ~ tr
Embed entities and relations
in different vector spaces
through projection matrix Mr
TransH [6]
Model: h⊥
+ dr
~ t⊥
Multiple representations
for different relations
projecting on hyperplanes
TransE [5]
Model: h + r ~ t
Simple, but not suitable
for 1-to-N, N-to-1, N-to-N
relations

18. TRANSLATIONAL MODELS
TransR [7]TransH [6]TransE [5]
Score function:
f(h, r, t) = D(h + r, t)
Score function:
f(h, r, t) = D(h⊥
+ dr
, t⊥
)
Score function:
f(h, r, t) = D(hr
+ r , tr
)

19. RANKING FUNCTION
i1
i3
ρ(u,i) = - D(u + feedback, i)
i2
i3
TransE [5]

20. RANKING FUNCTION
i1
i3
ρ(u,i) = - D(u + feedback, i)
i2
i3
TransE [5]
TransH [6]
ρ(u,i) = - D(u⊥
+ dfeedback
, i⊥
)

21. RANKING FUNCTION
i1
i3
ρ(u,i) = - D(u + feedback, i)
i2
i3
TransE [5]
TransH [6]
ρ(u,i) = - D(u⊥
+ dfeedback
, i⊥
)
TransR [7]
ρ(u,i) = - D(ufeedback
+ feedback, ifeedback
)
D = Euclidean distance

22. DATASET
● Movielens_1M: 1,000,209 ratings, 3900 movies, 6040 users
● Splitting: we split data using 70% train, 10% validation, 20% test of ratings per user
● DBpedia: we leverage DBpedia mappings of Movielens items to build the knowledge
graph created in a related work1
● Property selection: we count the most frequently occurring properties used in
DBpedia for items in Movielens1M, obtaining: ["dbo:director", "dbo:starring",
"dbo:distributor", "dbo:writer","dbo:musicComposer",
"dbo:producer","dbo:cinematography", "dbo:editing"]
● We add: "dct:subject" to this set and we model ratings >= 4 as “feedback” property
1: https://github.com/sisinflab/LODrecsys-datasets

23. EXPERIMENTAL SETUP
● Evaluation Protocol: we apply the same candidate generation strategies to all
systems under evaluation, i.e. “AllUnratedItems” [8]
● Baselines: we use heuristics such as MostPopular and collaborative filtering
algorithms such as Singular Value Decomposition (SVD) and ItemKNN from the
Surprise Library (http://surpriselib.com)
● Scoring: we measure standard information retrieval metrics such as P@5, P@10,
MAP, NDCG. The same scorer is applied to all systems
● Configuration: all systems are used with their default hyper-parameters
● Implementation of translational models is: https://github.com/thunlp/KB2E
● Evaluators, scorers, baselines and python wrappers of translational models can be
found at: https://github.com/D2KLab/entity2rec/tree/dev

24. RESULTS
System P@5 P@10 MAP NDCG
TransH 0.196457 0.170331 0.134170 0.461370
TransR 0.190497 0.165033 0.127401 0.453900
MostPop 0.144603 0.129156 0.092103 0.406294
TransE 0.116656 0.098245 0.071185 0.379548
SVD 0.067815 0.062401 0.042671 0.328776
ItemKNN 0.057483 0.053626 0.040933 0.324887

25. CONCLUSIONS
● Knowledge graphs are a beneficial data structure for hybrid recommender systems
● Item recommendation can be seen as a knowledge graph completion problem
● Translational models conceived to predict missing properties in a knowledge graph
can be used to provide recommendations
● The empirical comparison shows that TransH outperforms TransR and TransE
● Many possible extensions of this work: considering other datasets, other baselines,
other state-of-the-art graph embeddings systems, using knowledge graph
embeddings to model multiple user-item interactions

26. ● Enrico Palumbo, Data Scientist, ISMB, PhD Student, EURECOM - PoliTo
● Mail: palumbo@ismb.it
● Personal page: http://enricopal.github.io
● Slides: http://www.slideshare.net/EnricoPalumbo2
● Github: www.github.com/D2KLAB, www.github.com/enricopal
● Twitter: https://twitter.com/enricopalumbo91
Turin, Italy, www.ismb.it Sophia Antipolis, France, www.eurecom.fr Turin, Italy, www.polito.it
CONTACTS

27. REFERENCES
● [1]: Perozzi, Bryan, Rami Al-Rfou, and Steven Skiena. "Deepwalk: Online learning of social representations." Proceedings of the
20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014.
● [2]: Grover, Aditya, and Jure Leskovec. "node2vec: Scalable feature learning for networks." Proceedings of the 22nd ACM
SIGKDD international conference on Knowledge discovery and data mining. ACM, 2016.
● [3]: Ristoski, Petar, and Heiko Paulheim. "Rdf2vec: Rdf graph embeddings for data mining." International Semantic Web
Conference. Springer, Cham, 2016.
● [4]: Palumbo, Enrico, Giuseppe Rizzo, and Raphaël Troncy. "Entity2rec: Learning user-item relatedness from knowledge graphs
for top-n item recommendation." Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 2017.
● [5]: Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling
multi-relational data. In Advances in neural information processing systems (pp. 2787-2795).
● [6]: Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014, July). Knowledge Graph Embedding by Translating on Hyperplanes. In AAAI
(Vol. 14, pp. 1112-1119).
● [7]: Lin, Yankai, et al. "Learning entity and relation embeddings for knowledge graph completion." AAAI. Vol. 15. 2015.
● [8]: Harald Steck. 2013. Evaluation of recommendations: rating-prediction and ranking. In Proceedings of the 7th ACM
conference on Recommender systems. ACM, 213–220.