Translating Embeddings for Modeling Multi-relational Data

1. Translating Embeddings for
Modeling Multi-relational Data
Tien-Bach-Thanh Do
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
E-mail: osfa19730@catholic.ac.kr
2024/03/11
Antoine Bordes et al.
NIPS 2013

2. 2
Introduction
• Multi-relational data refers to directed graphs whose nodes correspond to entities and edges of the form
(head, label, tail) (denoted (h, l, t)), each of which indicates that there exists a relationship of name label
between the entities head and tail
• Models of multi-relational data play a pivotal role in many areas
○ Friendship/social relationship links
○ Recommender systems where entities are users and products and relationships are buying, rating,
reviewing or searching for a product
○ Knowledge bases like
■ Freebase
■ Google Knowledge Graph
■ GeneOntology

3. 3
TransE
Method

4. 4
TransE
Scoring function

5. 5
TransE
Algorithm

6. 6
TransE
Connectivity Patterns in KG
● Relations in heterogeneous KG have different properties
○ Symmetry: If the edge (h, “Roommate”, t) exists in KG, then the edge (t, “Roommate”, h) should
also exist
○ Inverse relation: If the edge (h, “Advisor”, t) exists in KG, then the edge (t, “Advisee”, h) should
also exist
● Can we categorize these relation patterns?
● Are KG embedding methods expressive enough to model these patterns?

7. 7
TransE
Relation Patterns
● Symmetric (Antisymmetric) Relations:
● Symmetric: Family, Roommate
● Antisymmetric: Hypernym
● Inverse Relations:
● Example: Advisor, Advisee
● Composition (Transitive) Relations:
● Example: My mother’s husband is my father
● 1-to-N relations:
● Example: r is StudentsOf

8. 8
Antisymmetric Relations in TransE
● Antisymmetric Relations:
● TransE can model antisymmetric relations

9. 9
Inverse Relations in TransE
• Inverse Relations:
○ Example: (Advisor, Advisee)
• TransE can model inverse relations

10. 10
Composition in TransE
• Composition (Transitive) Relations
○ Example: My mother’s husband is my father
• TransE can model composition relations

11. 11
Limitation: Symmetric Relations
● Symmetric Relations:
○ Example: Family, Roommate
● TransE cannot model symmetric relations

12. 12
Limitation: 1-to-N relations
● 1-to-N Relations:
○ Example: (h, r, t1) and (h, r, t2) both exist in the knowledge graph, eg r is StudentsOf
● TransE cannot model 1-to-N relations
○ t1 and t2 will map to the same vector, although they are different entities

13. 13
Datasets
Wordnet
• Designed to produce an intuitively usable dictionary and thesaurus, and support automatic text analysis
• Entities correspond to word senses, relationships define lexical relations between them

14. 14
Datasets
Freebase
• A huge and growing KB of general facts
• 1.2 billion triplets and more than 80 million entities
• 2 dataset with Freebase
○ FB15K
○ FB1M has most frequently occurring 1 million entities => 25k relationships and more than 17 millions
training triplets

15. 15
Results
Link prediction

16. 16
Results

17. 17
Results

18. 18
Results

19. 19
Conclusion
• TransE has proven effective in modeling multi-relational data by capturing complex relationships.
• While demonstrating superior performance in certain scenarios, ongoing research is crucial for refining its
capabilities and adapting to diverse datasets and real-world applications.