This paper proposes an improved neural question generation model called answer-separated seq2seq. It treats the target answer and passage separately by masking the target answer in the passage. It uses a keyword-net to consistently make the model aware of the target answer and extract key information from it. A retrieval style word generator is also used to generate words based on their meanings. Experimental results show this model generates fewer questions containing the target answer and better predicts the question type given the answer, outperforming previous models on question generation tasks.

1. Yanghoon Kim, Hwanhee Lee, Joongbo Shin and Kyomin Jung
Improving Neural Question Generation
using Answer Separation
김양훈

2. Background
Neural question generation (NQG)
- Generating a question from a given text passage with deep neural networks.
Importance of NQG
- Generating questions for educational materials.
- Generating questions for improving QA systems.
NQG with target Answer
- Generating a question from a given text passage and a given target
answer(question target) with deep neural networks.
2
Original passage: John Francis O’hara was elected president of Notre Dame in 1934.
Generated question 1: Who was elected president of Notre Dame in 1934?
Generated question 2: When was John Francis O’hara elected president of Notre Dame?

3. Problem
Previous NQG systems suffer from a critical problem
- Some models don’t take the target answer into account.
- RNNs often follow a shallow generation process.
- Some models can’t well grasp the target answer(question target) .
- A sophisticated proportion of generated questions include word in the target
answer.
3
Original passage: John Francis O’Hara was elected president of Notre Dame in 1934.
Given target answer: John Francis O’hara
Correctly Generated question: Who was elected president of Notre Dame in 1934?
Incorrectly generated question: Who was elected John Francis?

4. Contribution
We propose answer-separated seq2seq
- Treats the target answer(question target) and the passage separately.
- Prevent the generated question from including words in the target answer.
- Better capture the information from both the target answer and the passage
- We propose keyword-net
- Model is consistently aware of the target answer.
- Extract the key information in the target answer.
- We use retrieval style word generator
- Take the word meaning into account when generating words.
4

5. Task Definition
5

6. Model
Base model
- We use RNN encoder-decoder with attention
Answer-separated seq2seq consist of
- Answer-separated passage encoder
- Target answer encoder
- Answer-separated decoder
- keyword-net
- Retrieval style word generator
6

7. Model
Answer-separated passage encoder
- A simple preprocessing of the input passage
- Original passage: Steve Jobs is the founder of Apple.
- Masked passage: Steve Jobs is the <a> .
- A one-layer bi-LSTM
Answer encoder
- A one-layer bi-LSTM
7

8. Model
Answer-separated decoder
- A one-layer LSTM
- keyword-net
- Let the model consistently be aware of the target answer.
- Extract key information.
- Passage: Steve Jobs is the founder of Apple
- Target answer: founder of Apple
8

9. Model
Answer-separated decoder
- Retrieval style word generator by (Ma et al. 2018)*
- seq2seq has tendency to memorize the sequence pattern rather than
reflecting word meanings
- The word generator produces words by querying the distributed word
representations.
*Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation 9

10. Experiment
Data
- Processed version of SQuAD 1.1
- Data split 1: 70,484/10,570/11,877 (train/dev/test)
- Data split 2: 86,635/8,965/8,964
Evaluation
Our model(Ass2s) outperform the previous state-of-the-art model
10

11. Experiment
Impact of answer separation
- Ability to capture target answer
- We checked if the target answer is included in the generated question
- AP : Answer position Feature (BIO scheme)
- (Song et al. 2018) used the copy mechanism.
11
Our model has better ability to generate the right question given the target answer

12. Experiment
Impact of answer separation
- Interrogative word prediction
- “What” takes up more than half of the whole training set
- “Which” : “Which year” can be represented as “When”
- “why”, “yes/no” : only takes up 1.5% and 1.2% of the training set.
12
Our model has better ability to predict the question type for the given target answer

13. Experiment
Impact of answer separation
- Attention from <answer>
- (a) is the attention matrix from our model
- (b) is the attention matrix from seq2seq + AP
- <a> token gives the highest attention weights to the interrogative word “who” in (a)
13

14. Experiment
Question generation for machine comprehension
- Use named entities as target answers, generate synthetic data for machine
comprehension system(QA net by Google).
- ALL : Evaluation result of SQuAD dev set(10k)
- NER : Evaluation result of partial SQuAD dev set(4k)
- answers of single named entity
14

15. Conclusion
We propose Answer-separated seq2seq for NQG
- Separate utilization of target answer and the passage(without target answer)
- By masking the target answer inside the passage
- By using keyword-net to extract key feature from target answer
- By using retrieval style word generator to capture word meaning information
- Our model can
- Reduce the probability that the target answer is included by the generated question
- Generate fluent and right question for the given passage and the target answer
- Better inference the type of question

16. Thank you for listening!
Code, paper: https://yanghoonkim.github.io
Questions: ad26kr@snu.ac.kr