SlideShare a Scribd company logo
FRAMING FEW-SHOT
KNOWLEDGE GRAPH
COMPLETION WITH LARGE
LANGUAGE MODELS
Adrian M.P. Brașoveanu
Lyndon J.B. Nixon
Albert Weichselbraun
Arno Scharl
NLP4KGC@SEMANTICS 2023
LLMs 2020-2023
LARGE LANGUAGE MODELS
Generative AI
ChatGPT 3.5/4.0
Claude 2
Cohere Chat
Falcon
LLaMa2
Flan-T5
Core Innovation:
Ecosystems
Agents
LangChain
KGs
Tools
Problem Solving
Mixture of
Experts (MoE)?
Image Copyright © Language Models are Few-Shot Learners (2020) by Tom
B. Brown et al. NeurIPS 2020.
LLM Reasoning Strategies(1): CoT
LARGE LANGUAGE MODELS
Relation
Extraction with
CoT
Explanation is All
You Need!
Step-by-step
reasoning
Augmented Text
leads to better
results!
Image Copyright © Revisiting Relation Extraction in the era of Large Language
Models by Wadhwa et al. ACL(1) 2023.
LLM Reasoning Strategies (2): ToT
LARGE LANGUAGE MODELS
CoT contains
explanations
ToT extends CoT
Multiple paths
towards an answer
CoT-SC – Majority
voting mechanism
ToT – more similar
to the human
selection process
ToT allows for
parallel exploration
of ideas as
opposed to linear
exploration (CoT).
Image Copyright © Tree of Thoughts: Deliberate Problem Solving with Large
Language Models (2023) by Yao et al.
Knowledge Graphs (KG)
LARGE LANGUAGE MODELS
Sustainability
KG
Built with Wikidata.
Missing relations:
- country-specific
- region-specific
KG Completion
(KGC)
Can we fill the
missing relations
using LLMs?
Evaluating Large Language Models
LARGE LANGUAGE MODELS
Single interface
nat.dev/chat
Includes
ChatGPT3.5/4
(with 32k cw)
Claude1/2
(with 100k cw)
Cohere Chat
MPT30B
Falcon40B
LLaMa2
Functionality
Playground
Compare
Chat
Metrics
Evaluating Large Language Models
LARGE LANGUAGE MODELS
Relations
Only Relations
Explanations
CoT
Completions
Restricted CoT
Self-Scoring
Truthfulness Proxy
Evaluating Large Language Models
LARGE LANGUAGE MODELS
Tools
GPT-3.5
GPT-4.0
Claude2
MPT-30B
Few-Shot
Input: 12-14
annotated texts
Output: 50
annotated texts
We want all the
texts annotated in
a large batch if
possible
Evaluating Large Language Models
LARGE LANGUAGE MODELS
Taxonomy of
Errors
These are only the
most frequent
errors!
And the Winner
Is?
ChatGPT and
Claude2 have
similar
performance
Conclusion?
LARGE LANGUAGE MODELS
Self-Scoring
Consecutive runs
Huge differences
And the Winner
Is?
ChatGPT and
Claude2 have
similar
performance
Acknowledgments
PROJECTS
DWBI Vienna - Vienna Science and Technology Fund (WWTF) [10.47379/ICT20096]
SDG-HUB – FFG (GA No. 892212)
CONTACT
adrian.brasoveanu@modul.ac.at
THANK YOU!

More Related Content

Similar to Framing Few Shot Knowledge Graph Completion with Large Language Models

A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...
A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...
A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...
mlaij
 
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATIONAN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
gerogepatton
 
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATIONAN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
ijaia
 
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATIONAN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
gerogepatton
 
Analysis of the evolution of advanced transformer-based language models: Expe...
Analysis of the evolution of advanced transformer-based language models: Expe...Analysis of the evolution of advanced transformer-based language models: Expe...
Analysis of the evolution of advanced transformer-based language models: Expe...
IAESIJAI
 
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
mathsjournal
 
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
mathsjournal
 
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
mathsjournal
 
ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015
ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015
ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015
RIILP
 
BERT Explained_ State of the art language model for NLP.pdf
BERT Explained_ State of the art language model for NLP.pdfBERT Explained_ State of the art language model for NLP.pdf
BERT Explained_ State of the art language model for NLP.pdf
sudeshnakundu10
 
Topicmodels
TopicmodelsTopicmodels
Topicmodels
Ajay Ohri
 
Fine grained irony classification through transfer learning approach
Fine grained irony classification through transfer learning approachFine grained irony classification through transfer learning approach
Fine grained irony classification through transfer learning approach
CSITiaesprime
 
LLM GPT-3: Language models are few-shot learners
LLM GPT-3: Language models are few-shot learnersLLM GPT-3: Language models are few-shot learners
LLM GPT-3: Language models are few-shot learners
Yan Xu
 
Language Models for Information Retrieval
Language Models for Information RetrievalLanguage Models for Information Retrieval
Language Models for Information Retrieval
Dustin Smith
 
Plug play language_models
Plug play language_modelsPlug play language_models
Plug play language_models
Mohammad Moslem Uddin
 
Deep Learning | Speaker Indentification
Deep Learning | Speaker IndentificationDeep Learning | Speaker Indentification
Deep Learning | Speaker Indentification
Sai Kiran Kadam
 
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTS
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTSAUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTS
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTS
IRJET Journal
 
2010 INTERSPEECH
2010 INTERSPEECH 2010 INTERSPEECH
2010 INTERSPEECH
WarNik Chow
 
The Value and Benefits of Data-to-Text Technologies
The Value and Benefits of Data-to-Text TechnologiesThe Value and Benefits of Data-to-Text Technologies
The Value and Benefits of Data-to-Text Technologies
International Journal of Modern Research in Engineering and Technology
 
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
ijsc
 

Similar to Framing Few Shot Knowledge Graph Completion with Large Language Models (20)

A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...
A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...
A Deep Learning Model to Predict Congressional Roll Call Votes from Legislati...
 
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATIONAN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
 
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATIONAN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
 
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATIONAN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
AN IMPROVED MT5 MODEL FOR CHINESE TEXT SUMMARY GENERATION
 
Analysis of the evolution of advanced transformer-based language models: Expe...
Analysis of the evolution of advanced transformer-based language models: Expe...Analysis of the evolution of advanced transformer-based language models: Expe...
Analysis of the evolution of advanced transformer-based language models: Expe...
 
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
 
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
 
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
OPTIMIZING SIMILARITY THRESHOLD FOR ABSTRACT SIMILARITY METRIC IN SPEECH DIAR...
 
ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015
ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015
ESR11 Hoang Cuong - EXPERT Summer School - Malaga 2015
 
BERT Explained_ State of the art language model for NLP.pdf
BERT Explained_ State of the art language model for NLP.pdfBERT Explained_ State of the art language model for NLP.pdf
BERT Explained_ State of the art language model for NLP.pdf
 
Topicmodels
TopicmodelsTopicmodels
Topicmodels
 
Fine grained irony classification through transfer learning approach
Fine grained irony classification through transfer learning approachFine grained irony classification through transfer learning approach
Fine grained irony classification through transfer learning approach
 
LLM GPT-3: Language models are few-shot learners
LLM GPT-3: Language models are few-shot learnersLLM GPT-3: Language models are few-shot learners
LLM GPT-3: Language models are few-shot learners
 
Language Models for Information Retrieval
Language Models for Information RetrievalLanguage Models for Information Retrieval
Language Models for Information Retrieval
 
Plug play language_models
Plug play language_modelsPlug play language_models
Plug play language_models
 
Deep Learning | Speaker Indentification
Deep Learning | Speaker IndentificationDeep Learning | Speaker Indentification
Deep Learning | Speaker Indentification
 
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTS
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTSAUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTS
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTS
 
2010 INTERSPEECH
2010 INTERSPEECH 2010 INTERSPEECH
2010 INTERSPEECH
 
The Value and Benefits of Data-to-Text Technologies
The Value and Benefits of Data-to-Text TechnologiesThe Value and Benefits of Data-to-Text Technologies
The Value and Benefits of Data-to-Text Technologies
 
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
 

More from MODUL Technology GmbH

How distinct and aligned with UGC is European capitals’ DMO branding on Insta...
How distinct and aligned with UGC is European capitals’ DMO branding on Insta...How distinct and aligned with UGC is European capitals’ DMO branding on Insta...
How distinct and aligned with UGC is European capitals’ DMO branding on Insta...
MODUL Technology GmbH
 
Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...
Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...
Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...
MODUL Technology GmbH
 
Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...
Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...
Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...
MODUL Technology GmbH
 
New Opportunities for Understanding Tourist Photography.pptx
New Opportunities for Understanding Tourist Photography.pptxNew Opportunities for Understanding Tourist Photography.pptx
New Opportunities for Understanding Tourist Photography.pptx
MODUL Technology GmbH
 
How do destinations relate to one another? A study of visual destination bran...
How do destinations relate to one another? A study of visual destination bran...How do destinations relate to one another? A study of visual destination bran...
How do destinations relate to one another? A study of visual destination bran...
MODUL Technology GmbH
 
Do DMOs promote the right aspects of the destination? A study of Instagram ph...
Do DMOs promote the right aspects of the destination? A study of Instagram ph...Do DMOs promote the right aspects of the destination? A study of Instagram ph...
Do DMOs promote the right aspects of the destination? A study of Instagram ph...
MODUL Technology GmbH
 
The Impact of Social Media on perceived Destination Image: case of Mexico Ci...
The Impact of Social Media on perceived Destination Image:  case of Mexico Ci...The Impact of Social Media on perceived Destination Image:  case of Mexico Ci...
The Impact of Social Media on perceived Destination Image: case of Mexico Ci...
MODUL Technology GmbH
 
The Impact of Social Media on perceived Destination Image: the case of Mexico...
The Impact of Social Media on perceived Destination Image:the case of Mexico...The Impact of Social Media on perceived Destination Image:the case of Mexico...
The Impact of Social Media on perceived Destination Image: the case of Mexico...
MODUL Technology GmbH
 
How Instagram influences Visual Destination Image - a case study of Jordan an...
How Instagram influences Visual Destination Image - a case study of Jordan an...How Instagram influences Visual Destination Image - a case study of Jordan an...
How Instagram influences Visual Destination Image - a case study of Jordan an...
MODUL Technology GmbH
 
Media mining for smarter tourism
Media mining for smarter tourismMedia mining for smarter tourism
Media mining for smarter tourism
MODUL Technology GmbH
 
NoTube: Pattern-based Recommendations (part 3)
NoTube: Pattern-based Recommendations (part 3)NoTube: Pattern-based Recommendations (part 3)
NoTube: Pattern-based Recommendations (part 3)
MODUL Technology GmbH
 
NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)
MODUL Technology GmbH
 
NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)
MODUL Technology GmbH
 
NoTube: Recommendations (Collaborative)
NoTube: Recommendations (Collaborative)NoTube: Recommendations (Collaborative)
NoTube: Recommendations (Collaborative)
MODUL Technology GmbH
 
NoTube: User Profiling (Beancounter)
NoTube: User Profiling (Beancounter)NoTube: User Profiling (Beancounter)
NoTube: User Profiling (Beancounter)
MODUL Technology GmbH
 
14 no tube dissemination and showcases [compatibility mode]
14 no tube dissemination and showcases [compatibility mode]14 no tube dissemination and showcases [compatibility mode]
14 no tube dissemination and showcases [compatibility mode]
MODUL Technology GmbH
 
NoTube: BBC show case
NoTube: BBC show caseNoTube: BBC show case
NoTube: BBC show case
MODUL Technology GmbH
 
NoTube: Stoneroos show case
NoTube: Stoneroos show caseNoTube: Stoneroos show case
NoTube: Stoneroos show case
MODUL Technology GmbH
 
NoTube: RAI Show Case
NoTube: RAI Show CaseNoTube: RAI Show Case
NoTube: RAI Show Case
MODUL Technology GmbH
 
NoTube: Architecture
NoTube: ArchitectureNoTube: Architecture
NoTube: Architecture
MODUL Technology GmbH
 

More from MODUL Technology GmbH (20)

How distinct and aligned with UGC is European capitals’ DMO branding on Insta...
How distinct and aligned with UGC is European capitals’ DMO branding on Insta...How distinct and aligned with UGC is European capitals’ DMO branding on Insta...
How distinct and aligned with UGC is European capitals’ DMO branding on Insta...
 
Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...
Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...
Unsupervised Topic Modeling with BERTopic for Coarse and Fine-Grained News Cl...
 
Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...
Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...
Breaking New Ground with EPOCH: AI and Web Intelligence Transform Price Forec...
 
New Opportunities for Understanding Tourist Photography.pptx
New Opportunities for Understanding Tourist Photography.pptxNew Opportunities for Understanding Tourist Photography.pptx
New Opportunities for Understanding Tourist Photography.pptx
 
How do destinations relate to one another? A study of visual destination bran...
How do destinations relate to one another? A study of visual destination bran...How do destinations relate to one another? A study of visual destination bran...
How do destinations relate to one another? A study of visual destination bran...
 
Do DMOs promote the right aspects of the destination? A study of Instagram ph...
Do DMOs promote the right aspects of the destination? A study of Instagram ph...Do DMOs promote the right aspects of the destination? A study of Instagram ph...
Do DMOs promote the right aspects of the destination? A study of Instagram ph...
 
The Impact of Social Media on perceived Destination Image: case of Mexico Ci...
The Impact of Social Media on perceived Destination Image:  case of Mexico Ci...The Impact of Social Media on perceived Destination Image:  case of Mexico Ci...
The Impact of Social Media on perceived Destination Image: case of Mexico Ci...
 
The Impact of Social Media on perceived Destination Image: the case of Mexico...
The Impact of Social Media on perceived Destination Image:the case of Mexico...The Impact of Social Media on perceived Destination Image:the case of Mexico...
The Impact of Social Media on perceived Destination Image: the case of Mexico...
 
How Instagram influences Visual Destination Image - a case study of Jordan an...
How Instagram influences Visual Destination Image - a case study of Jordan an...How Instagram influences Visual Destination Image - a case study of Jordan an...
How Instagram influences Visual Destination Image - a case study of Jordan an...
 
Media mining for smarter tourism
Media mining for smarter tourismMedia mining for smarter tourism
Media mining for smarter tourism
 
NoTube: Pattern-based Recommendations (part 3)
NoTube: Pattern-based Recommendations (part 3)NoTube: Pattern-based Recommendations (part 3)
NoTube: Pattern-based Recommendations (part 3)
 
NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)
 
NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)NoTube: Pattern-based Recommendations (part 1)
NoTube: Pattern-based Recommendations (part 1)
 
NoTube: Recommendations (Collaborative)
NoTube: Recommendations (Collaborative)NoTube: Recommendations (Collaborative)
NoTube: Recommendations (Collaborative)
 
NoTube: User Profiling (Beancounter)
NoTube: User Profiling (Beancounter)NoTube: User Profiling (Beancounter)
NoTube: User Profiling (Beancounter)
 
14 no tube dissemination and showcases [compatibility mode]
14 no tube dissemination and showcases [compatibility mode]14 no tube dissemination and showcases [compatibility mode]
14 no tube dissemination and showcases [compatibility mode]
 
NoTube: BBC show case
NoTube: BBC show caseNoTube: BBC show case
NoTube: BBC show case
 
NoTube: Stoneroos show case
NoTube: Stoneroos show caseNoTube: Stoneroos show case
NoTube: Stoneroos show case
 
NoTube: RAI Show Case
NoTube: RAI Show CaseNoTube: RAI Show Case
NoTube: RAI Show Case
 
NoTube: Architecture
NoTube: ArchitectureNoTube: Architecture
NoTube: Architecture
 

Recently uploaded

Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
UX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business GoalsUX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business Goals
FIDO Alliance
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
bellared2
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
siddu769252
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
AimanAthambawa1
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
OnBoard
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
Enterprise Knowledge
 
Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024
Michael Price
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
DianaGray10
 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
Baishakhi Ray
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
FIDO Alliance
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 

Recently uploaded (20)

Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
UX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business GoalsUX Webinar Series: Aligning Authentication Experiences with Business Goals
UX Webinar Series: Aligning Authentication Experiences with Business Goals
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
COVID-19 and the Level of Cloud Computing Adoption: A Study of Sri Lankan Inf...
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
 
Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 

Framing Few Shot Knowledge Graph Completion with Large Language Models

  • 1. FRAMING FEW-SHOT KNOWLEDGE GRAPH COMPLETION WITH LARGE LANGUAGE MODELS Adrian M.P. Brașoveanu Lyndon J.B. Nixon Albert Weichselbraun Arno Scharl NLP4KGC@SEMANTICS 2023
  • 2. LLMs 2020-2023 LARGE LANGUAGE MODELS Generative AI ChatGPT 3.5/4.0 Claude 2 Cohere Chat Falcon LLaMa2 Flan-T5 Core Innovation: Ecosystems Agents LangChain KGs Tools Problem Solving Mixture of Experts (MoE)? Image Copyright © Language Models are Few-Shot Learners (2020) by Tom B. Brown et al. NeurIPS 2020.
  • 3. LLM Reasoning Strategies(1): CoT LARGE LANGUAGE MODELS Relation Extraction with CoT Explanation is All You Need! Step-by-step reasoning Augmented Text leads to better results! Image Copyright © Revisiting Relation Extraction in the era of Large Language Models by Wadhwa et al. ACL(1) 2023.
  • 4. LLM Reasoning Strategies (2): ToT LARGE LANGUAGE MODELS CoT contains explanations ToT extends CoT Multiple paths towards an answer CoT-SC – Majority voting mechanism ToT – more similar to the human selection process ToT allows for parallel exploration of ideas as opposed to linear exploration (CoT). Image Copyright © Tree of Thoughts: Deliberate Problem Solving with Large Language Models (2023) by Yao et al.
  • 5. Knowledge Graphs (KG) LARGE LANGUAGE MODELS Sustainability KG Built with Wikidata. Missing relations: - country-specific - region-specific KG Completion (KGC) Can we fill the missing relations using LLMs?
  • 6. Evaluating Large Language Models LARGE LANGUAGE MODELS Single interface nat.dev/chat Includes ChatGPT3.5/4 (with 32k cw) Claude1/2 (with 100k cw) Cohere Chat MPT30B Falcon40B LLaMa2 Functionality Playground Compare Chat Metrics
  • 7. Evaluating Large Language Models LARGE LANGUAGE MODELS Relations Only Relations Explanations CoT Completions Restricted CoT Self-Scoring Truthfulness Proxy
  • 8. Evaluating Large Language Models LARGE LANGUAGE MODELS Tools GPT-3.5 GPT-4.0 Claude2 MPT-30B Few-Shot Input: 12-14 annotated texts Output: 50 annotated texts We want all the texts annotated in a large batch if possible
  • 9. Evaluating Large Language Models LARGE LANGUAGE MODELS Taxonomy of Errors These are only the most frequent errors! And the Winner Is? ChatGPT and Claude2 have similar performance
  • 10. Conclusion? LARGE LANGUAGE MODELS Self-Scoring Consecutive runs Huge differences And the Winner Is? ChatGPT and Claude2 have similar performance
  • 11. Acknowledgments PROJECTS DWBI Vienna - Vienna Science and Technology Fund (WWTF) [10.47379/ICT20096] SDG-HUB – FFG (GA No. 892212) CONTACT adrian.brasoveanu@modul.ac.at THANK YOU!