### Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction

• 1. Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction Paolo Rosso, Dingqi Yang, Philippe Cudré-Mauroux eXascale Infolab University of Fribourg, Switzerland April 2020 – The Web Conference 2020 0
• 2. Knowledge Graph (KG) • Multi-relational graph composed of entities and relations • Each fact is represented as a triplet head entity, relation, tail entity • A fact indicates that two entities are connected by a specific relation 1
• 3. Knowledge Graph embeddings • Represent entities and relations in a Knowledge Graph using a vector space • Learn low-dimensional vector representation of entities and relations from a Knowledge Graph while preserving the graph properties Marie Curie University of Paris educated at Semantic search Question-answering Query expansion Recommendation systems 2 TransE
• 4. Hyper-relational facts • Triplet-based representation of a KG oversimplifies the complex nature of hyper-relational data • A hyper-relational data represents a fact containing multiple relations and entities • A hyper-relational is represented by a triplet and a set of key-value pairs 3
• 5. Hyper-relational facts representations 1. Triplet only: keeping only the base triplet (irreversible information loss) 2. Reification: adding an artificial entity to represent the base triplet and use it as head entity for the k-v pair (creates additional triplets) 3. Relation paths: creating a relation path of relation-key and use it as a relation to connect head and key (creates additional triplets) … we need a method to better learn hyper-relation facts! 4
• 6. 5 Marie Curie educated at University of Paris Hyper-relational facts representations Triplet only
• 7. Hyper-relational facts representations 1. Triplet only: keeping only the base triplet (irreversible information loss) 2. Reification: adding an artificial entity to represent the base triplet and use it as head entity for the k-v pair (creates additional triplets) 3. Relation paths: creating a relation path of relation-key and use it as a relation to connect head and key (creates additional triplets) … we need a method to better learn hyper-relation facts! 6
• 8. 7 q := Marie Curie educated at University of Paris q academic major Physics q academic degree Master of Science Hyper-relational facts representations Reification
• 9. Hyper-relational facts representations 1. Triplet only: keeping only the base triplet (irreversible information loss) 2. Reification: adding an artificial entity to represent the base triplet and use it as head entity for the k-v pair (creates additional triplets) 3. Relation paths: creating a relation path of relation-key and use it as a relation to connect head and key (creates additional triplets) … we need a method to better learn hyper-relation facts! 8
• 10. 9 Marie Curie educated at University of Paris rk1 := educated at AND academic major rk2 := educated at AND academic degree Marie Curie rk1 Physics Marie Curie rk2 Master of Science Relation paths
• 11. Hyper-relational facts representations 1. Triplet only: keeping only the base triplet (irreversible information loss) 2. Reification: adding an artificial entity to represent the base triplet and use it as head entity for the k-v pair (creates additional triplets) 3. Relation paths: creating a relation path of relation-key and use it as a relation to connect head and key (creates additional triplets) 10
• 12. 11 • Hyper-relational fact can be transformed into n-ary representation • E.g., relation education is extracted from the hyper-relational fact • N-ary representation {education_head:Marie Curie, education_tail:University of Paris, education_major:Physics, education_degree:Master of Science} … but triplets serve as the fundamental data structure in the modern KGs because they preserve the essential information for link prediction N-Ary Representation
• 13. 12 • Hyper-relational fact can be transformed into n-ary representation • E.g., relation education is extracted from the hyper-relational fact • N-ary representation {education_head:Marie Curie, education_tail:University of Paris, education_major:Physics, education_degree:Master of Science} … but triplets serve as the fundamental data structure in the modern KGs because they preserve the essential information for link prediction N-Ary Representation
• 14. Limitation of N-Ary Representation 13 • Null model: one triplet is extracted from the n-ary representation of each hyper- relational fact via randomly sampled relation path (h, rhki, vi) • E.g., Marie Curie education_head_education_major Physics • Null hypothesis: the information for link prediction encoded by the original base triplets is not greater than the triplets created by the null model • We test Null hypothesis on link prediction tasks using models on original basic triplets and null model • Rejected Null hypothesis - results: • Performance of the baselines on original base triplets is consistently and significantly better than the performance from the null model • Information encoded in original base triplets is greater than the information encoded in null model
• 15. HINGE: Hyper-relatIonal kNowledge Graph Embedding • KG embedding model to learn hyper-relational facts in a KG • Capturing the primary structural information of the KG encoded in the triplets in order to preserve the essential information for link prediction • Capturing the correlation between each triplet and its associated key- value pairs (if any) 14
• 17. HINGE architecture • The base triplet encodes the primary structural information and capture essential information for link prediction • CNN to learn from the base triplet and capture the triple-wise relatedness between h,r,t embeddings • The triple-wise relatedness vector used to characterize the plausibility of the base triplet of being true 16
• 18. HINGE architecture • CNN to learn from each key-value pair associated with the triplet together and capture the quintuple-wise relatedness between h,r,t,k,v embeddings • The quintuple-wise relatedness vector used to characterize the plausibility of the base triplet associated with the k-v pair being true 17
• 19. HINGE architecture • Concatenate the triple-wise and quintuple- wise relatedness future vectors • Get the minimum value along each dimension • For a valid hyper-relational fact, the relatedness vectors should be high • The minimum score along each dimension is expected to be high • Fully connected layer to output the score for the input hyper-relational fact 18
• 21. Experiments • Learning from triplets facts only: • Translational distance models: TransE, TransH, TransR and TransD • Semantic matching models: Rescal, DistMult, ComplEx, Analogy and ConvE • Learning from hyper-relational facts: • m-TransH, RAE, NaLP, NaLP-Fix 20
• 22. Datasets • Hyper-relational datasets • JF17K: filtered from Freebase to have a significant presence of hyper-relational facts • WikiPeople: extracted from Wikidata and focuses on entities of type human • JF17K and WikiPeople contain triplet facts and hyper-relational facts • Dataset configurations: Triplet Only, Relation Path, Reification, Original Hyper-Relational 21
• 23. Evaluation Tasks and Metrics • Link prediction: given two elements of a triplet in a fact, predict the missing one • E.g., (?,r,t), (h,?,t) or (h,r,?) • Mean Reciprocal Rank (MRR), Hits@10 and Hits@1 22
• 24. Comparison with Baselines Learning from Hyper-Relational Facts 23 • HINGE outperforms all baselines learning from hyper-relational facts • Among the baselines: • NaLP shows the best performance (it learns the relatedness between k-v pairs but it is not aware of the triplet structure) • m-TransH and RAE learn only from entities
• 25. Comparison with Baselines Learning from Triplets Only Dataset Transformation Setting WikiPeople JF17K Head/Tail Prediction (%) Relation Prediction (%) Head/Tail Prediction (%) Relation Prediction (%) Basic 0.81 18.55 41.45 8.41 Relation Path 2.85 24.65 22.96 12.24 Reification 3.28 22.22 29.71 6.53 24
• 26. Comparison with Baselines Learning from Triplets Only Dataset Transformation Setting WikiPeople JF17K Head/Tail Prediction (%) Relation Prediction (%) Head/Tail Prediction (%) Relation Prediction (%) Basic 0.81 18.55 41.45 8.41 Relation Path 2.85 24.65 22.96 12.24 Reification 3.28 22.22 29.71 6.53 • Basic setting discards the key-value pairs (information loss) 25
• 27. Comparison with Baselines Learning from Triplets Only Dataset Transformation Setting WikiPeople JF17K Head/Tail Prediction (%) Relation Prediction (%) Head/Tail Prediction (%) Relation Prediction (%) Basic 0.81 18.55 41.45 8.41 Relation Path 2.85 24.65 22.96 12.24 Reification 3.28 22.22 29.71 6.53 • Basic setting discards the key-value pairs (information loss) • Exception for the head/tail prediction on WikiPeople with Basic setting (dominance of triplet) 26
• 28. Comparison with Baselines Learning from Triplets Only Dataset Transformation Setting WikiPeople JF17K Head/Tail Prediction (%) Relation Prediction (%) Head/Tail Prediction (%) Relation Prediction (%) Basic 0.81 18.55 41.45 8.41 Relation Path 2.85 24.65 22.96 12.24 Reification 3.28 22.22 29.71 6.53 • Basic setting discards the key-value pairs (information loss) • Exception for the head/tail prediction on WikiPeople with Basic setting (dominance of triplet) • Relation Path and Reification create extra entities/relations (not capturing essential information for link prediction) 27
• 29. Conclusions • We investigate the problem of hyper-relational KG embedding showing that triplets serve as the fundamental data structure in modern KGs encoding the essential information for link prediction • We introduce HINGE, a KG embedding model that captures the information encoded in the triplets and the correlation between each triplet and its associated key-value pairs • We show that the triplet structure is the fundamental structure for link prediction and show the limitations of a commonly used representation scheme • We observe an improvement on various link prediction tasks with different data transformation settings 28
Current LanguageEnglish
Español
Portugues
Français
Deutsche