SlideShare a Scribd company logo
A Practical Toolkit for Multilingual Question and
Answer Generation
Computer Science & Informatics, Cardiff University, Cardiff NLP
ACL 2023 System Demonstration
Asahi Ushio, Fernando Alva-Manchego and Jose Camacho-Collados
https://github.com/asahi417/lm-question-generation
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Question & Answer Generation
2
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Three QAG Approaches
3
● Pipeline QAG
○ QG + AE (Separate Model)
○ Prediction per sentence
● Multitask QAG
○ QG + AE (Shared Model)
○ Prediction per sentence
● End2end QAG
○ QAG (Single Model)
○ Prediction per paragraph
*QG: Question Generation
*AE: Answer Extraction
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Three QAG Approaches
4
● Pipeline QAG
○ QG + AE (Separate Model)
○ Prediction per sentence
● Multitask QAG
○ QG + AE (Shared Model)
○ Prediction per sentence
● End2end QAG
○ QAG (Single Model)
○ Prediction per paragraph
*QG: Question Generation
*AE: Answer Extraction
● Slow Inference
● More Storage
● More QA Pairs
● Stable across languages
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Three QAG Approaches
5
● Pipeline QAG
○ QG + AE (Separate Model)
○ Prediction per sentence
● Multitask QAG
○ QG + AE (Shared Model)
○ Prediction per sentence
● End2end QAG
○ QAG (Single Model)
○ Prediction per paragraph
*QG: Question Generation
*AE: Answer Extraction
● Slow Inference
● More Storage
● More QA Pairs
● Stable across languages
● Slow Inference
● Low Storage
● More QA Pairs
● Not the best in some
languages
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Three QAG Approaches
6
● Pipeline QAG
○ QG + AE (Separate Model)
○ Prediction per sentence
● Multitask QAG
○ QG + AE (Shared Model)
○ Prediction per sentence
● End2end QAG
○ QAG (Single Model)
○ Prediction per paragraph
*QG: Question Generation
*AE: Answer Extraction
● Slow Inference
● More Storage
● More QA Pairs
● Stable across languages
● Fast Inference
● Low Storage
● Less QA Pairs
● Failure in some languages
● Slow Inference
● Low Storage
● More QA Pairs
● Not the best in some
languages
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
lmqg: A Python Library for QAG
7
● What I can do with lmqg?🤔
○ QA Generation with fine-tuned models in 8 languages En/Ja/Es/Fr/De/It/Ru/Ko.
■ Check available models at MODEL CARD or https://huggingface.co/lmqg.
○ QAG model training/evaluation.
■ Check available datasets at https://huggingface.co/lmqg.
○ Rest API for QA Generation.
● Link → https://github.com/asahi417/lm-question-generation
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Examples of lmqg
8
(ii) Model Evaluation
(i) Model Training (iii) QA Generation
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Rest API for QA Generation with lmqg
9
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
AutoQG https://autoqg.net/
10
● Multilingual
○ En/Ja/Es/Fr/De/It/Ru/Ko
● Switch QAG Type:
○ Pipeline/End2end/Multitask
● Switch LMs
○ T5/BART/Flan-T5
● QG mode: Specify Answer
Thank you!!

More Related Content

Recently uploaded

Synopsis presentation VDR gene polymorphism and anemia (2).pptx
Synopsis presentation VDR gene polymorphism and anemia (2).pptxSynopsis presentation VDR gene polymorphism and anemia (2).pptx
Synopsis presentation VDR gene polymorphism and anemia (2).pptx
FarhanaHussain18
 
BIOTRANSFORMATION MECHANISM FOR OF STEROID
BIOTRANSFORMATION MECHANISM FOR OF STEROIDBIOTRANSFORMATION MECHANISM FOR OF STEROID
BIOTRANSFORMATION MECHANISM FOR OF STEROID
ShibsekharRoy1
 
Holsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdf
Holsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdfHolsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdf
Holsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdf
frank0071
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
PirithiRaju
 
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE  AND ITS BENIFITS.pptxIMPORTANCE OF ALGAE  AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
OmAle5
 
fermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptxfermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptx
ananya23nair
 
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Sérgio Sacani
 
Reaching the age of Adolescence- Class 8
Reaching the age of Adolescence- Class 8Reaching the age of Adolescence- Class 8
Reaching the age of Adolescence- Class 8
abhinayakamasamudram
 
Embracing Deep Variability For Reproducibility and Replicability
Embracing Deep Variability For Reproducibility and ReplicabilityEmbracing Deep Variability For Reproducibility and Replicability
Embracing Deep Variability For Reproducibility and Replicability
University of Rennes, INSA Rennes, Inria/IRISA, CNRS
 
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
Sérgio Sacani
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
hozt8xgk
 
gastroretentive drug delivery system-PPT.pptx
gastroretentive drug delivery system-PPT.pptxgastroretentive drug delivery system-PPT.pptx
gastroretentive drug delivery system-PPT.pptx
Shekar Boddu
 
Lattice Defects in ionic solid compound.pptx
Lattice Defects in ionic solid compound.pptxLattice Defects in ionic solid compound.pptx
Lattice Defects in ionic solid compound.pptx
DrRajeshDas
 
The cost of acquiring information by natural selection
The cost of acquiring information by natural selectionThe cost of acquiring information by natural selection
The cost of acquiring information by natural selection
Carl Bergstrom
 
Nutaceuticsls herbal drug technology CVS, cancer.pptx
Nutaceuticsls herbal drug technology CVS, cancer.pptxNutaceuticsls herbal drug technology CVS, cancer.pptx
Nutaceuticsls herbal drug technology CVS, cancer.pptx
vimalveerammal
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
PsychoTech Services
 
一比一原版美国佩斯大学毕业证如何办理
一比一原版美国佩斯大学毕业证如何办理一比一原版美国佩斯大学毕业证如何办理
一比一原版美国佩斯大学毕业证如何办理
gyhwyo
 
Post translation modification by Suyash Garg
Post translation modification by Suyash GargPost translation modification by Suyash Garg
Post translation modification by Suyash Garg
suyashempire
 
2001_Book_HumanChromosomes - Genéticapdf
2001_Book_HumanChromosomes - Genéticapdf2001_Book_HumanChromosomes - Genéticapdf
2001_Book_HumanChromosomes - Genéticapdf
lucianamillenium
 
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Creative-Biolabs
 

Recently uploaded (20)

Synopsis presentation VDR gene polymorphism and anemia (2).pptx
Synopsis presentation VDR gene polymorphism and anemia (2).pptxSynopsis presentation VDR gene polymorphism and anemia (2).pptx
Synopsis presentation VDR gene polymorphism and anemia (2).pptx
 
BIOTRANSFORMATION MECHANISM FOR OF STEROID
BIOTRANSFORMATION MECHANISM FOR OF STEROIDBIOTRANSFORMATION MECHANISM FOR OF STEROID
BIOTRANSFORMATION MECHANISM FOR OF STEROID
 
Holsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdf
Holsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdfHolsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdf
Holsinger, Bruce W. - Music, body and desire in medieval culture [2001].pdf
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
 
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE  AND ITS BENIFITS.pptxIMPORTANCE OF ALGAE  AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
 
fermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptxfermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptx
 
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
 
Reaching the age of Adolescence- Class 8
Reaching the age of Adolescence- Class 8Reaching the age of Adolescence- Class 8
Reaching the age of Adolescence- Class 8
 
Embracing Deep Variability For Reproducibility and Replicability
Embracing Deep Variability For Reproducibility and ReplicabilityEmbracing Deep Variability For Reproducibility and Replicability
Embracing Deep Variability For Reproducibility and Replicability
 
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
 
gastroretentive drug delivery system-PPT.pptx
gastroretentive drug delivery system-PPT.pptxgastroretentive drug delivery system-PPT.pptx
gastroretentive drug delivery system-PPT.pptx
 
Lattice Defects in ionic solid compound.pptx
Lattice Defects in ionic solid compound.pptxLattice Defects in ionic solid compound.pptx
Lattice Defects in ionic solid compound.pptx
 
The cost of acquiring information by natural selection
The cost of acquiring information by natural selectionThe cost of acquiring information by natural selection
The cost of acquiring information by natural selection
 
Nutaceuticsls herbal drug technology CVS, cancer.pptx
Nutaceuticsls herbal drug technology CVS, cancer.pptxNutaceuticsls herbal drug technology CVS, cancer.pptx
Nutaceuticsls herbal drug technology CVS, cancer.pptx
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
 
一比一原版美国佩斯大学毕业证如何办理
一比一原版美国佩斯大学毕业证如何办理一比一原版美国佩斯大学毕业证如何办理
一比一原版美国佩斯大学毕业证如何办理
 
Post translation modification by Suyash Garg
Post translation modification by Suyash GargPost translation modification by Suyash Garg
Post translation modification by Suyash Garg
 
2001_Book_HumanChromosomes - Genéticapdf
2001_Book_HumanChromosomes - Genéticapdf2001_Book_HumanChromosomes - Genéticapdf
2001_Book_HumanChromosomes - Genéticapdf
 
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
 

Featured

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
Christy Abraham Joy
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
Vit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
MindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

2023-07, ACL, A Practical Toolkit for Multilingual Question and Answer Generation.pdf

  • 1. A Practical Toolkit for Multilingual Question and Answer Generation Computer Science & Informatics, Cardiff University, Cardiff NLP ACL 2023 System Demonstration Asahi Ushio, Fernando Alva-Manchego and Jose Camacho-Collados https://github.com/asahi417/lm-question-generation
  • 2. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Question & Answer Generation 2
  • 3. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Three QAG Approaches 3 ● Pipeline QAG ○ QG + AE (Separate Model) ○ Prediction per sentence ● Multitask QAG ○ QG + AE (Shared Model) ○ Prediction per sentence ● End2end QAG ○ QAG (Single Model) ○ Prediction per paragraph *QG: Question Generation *AE: Answer Extraction
  • 4. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Three QAG Approaches 4 ● Pipeline QAG ○ QG + AE (Separate Model) ○ Prediction per sentence ● Multitask QAG ○ QG + AE (Shared Model) ○ Prediction per sentence ● End2end QAG ○ QAG (Single Model) ○ Prediction per paragraph *QG: Question Generation *AE: Answer Extraction ● Slow Inference ● More Storage ● More QA Pairs ● Stable across languages
  • 5. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Three QAG Approaches 5 ● Pipeline QAG ○ QG + AE (Separate Model) ○ Prediction per sentence ● Multitask QAG ○ QG + AE (Shared Model) ○ Prediction per sentence ● End2end QAG ○ QAG (Single Model) ○ Prediction per paragraph *QG: Question Generation *AE: Answer Extraction ● Slow Inference ● More Storage ● More QA Pairs ● Stable across languages ● Slow Inference ● Low Storage ● More QA Pairs ● Not the best in some languages
  • 6. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Three QAG Approaches 6 ● Pipeline QAG ○ QG + AE (Separate Model) ○ Prediction per sentence ● Multitask QAG ○ QG + AE (Shared Model) ○ Prediction per sentence ● End2end QAG ○ QAG (Single Model) ○ Prediction per paragraph *QG: Question Generation *AE: Answer Extraction ● Slow Inference ● More Storage ● More QA Pairs ● Stable across languages ● Fast Inference ● Low Storage ● Less QA Pairs ● Failure in some languages ● Slow Inference ● Low Storage ● More QA Pairs ● Not the best in some languages
  • 7. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados lmqg: A Python Library for QAG 7 ● What I can do with lmqg?🤔 ○ QA Generation with fine-tuned models in 8 languages En/Ja/Es/Fr/De/It/Ru/Ko. ■ Check available models at MODEL CARD or https://huggingface.co/lmqg. ○ QAG model training/evaluation. ■ Check available datasets at https://huggingface.co/lmqg. ○ Rest API for QA Generation. ● Link → https://github.com/asahi417/lm-question-generation
  • 8. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Examples of lmqg 8 (ii) Model Evaluation (i) Model Training (iii) QA Generation
  • 9. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados Rest API for QA Generation with lmqg 9
  • 10. A Practical Toolkit for Multilingual Question and Answer Generation Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados AutoQG https://autoqg.net/ 10 ● Multilingual ○ En/Ja/Es/Fr/De/It/Ru/Ko ● Switch QAG Type: ○ Pipeline/End2end/Multitask ● Switch LMs ○ T5/BART/Flan-T5 ● QG mode: Specify Answer