ACL 2018에 다녀오신 NAVER 개발자 분들께서 그 내용을 공유해 주십니다. 1. Overview - Lucy Park 2. Tutorials - Xiaodong Gu 3. Main conference a. Semantic parsing - Soonmin Bae b. Dialogue - Kyungduk Kim c. Machine translation - Zae Myung Kim d. Summarization - Hye-Jin Min 4. Workshops - Minjoon Seo

1. July 15-20, 2018
@Melbourne, Australia
Recap on
2018-08-13
Naver Techtalks

2. 1. Overview - Lucy Park
2. Tutorials - Xiaodong Gu
3. Main conference
a. Semantic parsing - Soonmin Bae
b. Dialogue - Kyungduk Kim
c. Machine translation - Zae Myung Kim
d. Summarization - Hye-Jin Min
4. Workshops - Minjoon Seo

3. Overview
Lucy Park

4. ACL (Association for Computational Linguistics)
- EMNLP, NAACL, COLING 등과 더불어 NLP 관련 최고 학회
- 이들끼리는 workshop이나 tutorial도 joint-call을 함
- ACL since 1962 (56th annual meeting)
- EACL since 1982
- NAACL since 2000
- AACL (Asia-Pacific) since 2018! (학회는 biannual, 2020년에 시작)

5. 1551 papers submitted
[1]: https://acl2018.org/2018/03/12/reviewing-statistics/
(408 accepted)
4 topics to be
covered today!

6. Schedule
Workshops
Main conference
+ Booth
Tutorials
(메인 세션은 조만간 비디오 공개 예정)

7. Tutorials
Morning:
● 100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
● Neural Approaches to Conversational AI
● Variational Inference and Deep Generative Models
● Connecting Language and Vision to Actions
Afternoon:
● Beyond Multiword Expressions: Processing Idioms and Metaphors
● Neural Semantic Parsing
● Deep Reinforcement Learning for NLP
● Multi-lingual Entity Discovery and Linking
To be covered
by Xiadong

8. Thursday:
● BioNLP 2018 (BioNLP)
● Cognitive Aspects of Computational Language Learning and Processing (CogCL)
● Deep Learning Approaches for Low Resource Natural Language Processing (DeepLo)
● Multilingual Surface Realization: Shared Task and Beyond (MSR)
● The 5th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA)
● Third Workshop on Computational Approaches to Linguistic Code-Switching (CALCS)
● Workshop on Machine Reading for Question Answering (MRQA)
● Workshop on Relevance of Linguistic Structure in Neural Architectures for NLP (RELNLP)
Friday:
● 1st Workshop on Economics and Natural Language Processing (ECONLP)
● 3rd Workshop on Representation Learning for NLP (RepL4NLP)
● First Workshop on Computational Modeling of Human Multimodal Language (MML_Challenge)
● Sixth International Workshop on Natural Language Processing for Social Media (SocialNLP)
● The 2nd Workshop on Neural Machine Translation and Generation (NMT)
● The Seventh Named Entities Workshop (NEWS)
● Workshop for NLP Open Source Software (NLPOSS)
Workshops
7 out of 15
recurring
workshops

9. Thursday:
● BioNLP 2018 (BioNLP): Room 207, MCEC
● Cognitive Aspects of Computational Language Learning and Processing (CogCL)
● Deep Learning Approaches for Low Resource Natural Language Processing (DeepLo)
● Multilingual Surface Realization: Shared Task and Beyond (MSR)
● The 5th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA)
● Third Workshop on Computational Approaches to Linguistic Code-Switching (CALCS)
● Workshop on Machine Reading for Question Answering (MRQA)
● Workshop on Relevance of Linguistic Structure in Neural Architectures for NLP (RELNLP)
Friday:
● 1st Workshop on Economics and Natural Language Processing (ECONLP)
● 3rd Workshop on Representation Learning for NLP (RepL4NLP)
● First Workshop on Computational Modeling of Human Multimodal Language (MML_Challenge)
● Sixth International Workshop on Natural Language Processing for Social Media (SocialNLP)
● The 2nd Workshop on Neural Machine Translation and Generation (NMT)
● The Seventh Named Entities Workshop (NEWS)
● Workshop for NLP Open Source Software (NLPOSS)
Workshops
To be covered
by Minjoon

10. Booth

11. Best papers
Short:
● Know What You Don’t Know: Unanswerable Questions for SQuAD.
Pranav Rajpurkar, Robin Jia and Percy Liang
● ‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions.
Olivia Winn and Smaranda Muresan
Long:
● Finding syntax in human encephalography with beam search.
John Hale, Chris Dyer, Adhiguna Kuncoro and Jonathan Brennan.
● Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information.
Sudha Rao and Hal Daumé III.
● Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers.
Andre Cianflone,* Yulan Feng,* Jad Kabbara* and Jackie Chi Kit Cheung. (* equal contribution)
새로운 태스크 + 새로운 데이터

12. Know what you don’t know:
Unanswerable Questions for SQuAD
배경: MRC (machine reading comprehension)
문제: Unanswerable questions in MRC
제안: SQuAD 2.0
- 50,000 unanswerable questions written
adversarially by crowdworkers
- a model with 86% F1 on SQuAD 1.1
achieves only 66% F1 on SQuAD 2.0

13. Learning to Ask Good Questions:
Ranking Clarification Questions using Neural Expected Value of Perfect Information
문제: Make a machine generate questions
- 기존: Reading comprehension
- 제안: Find missing information
데이터: 77k Unix-related posts from
StackExchange
방법:
1. p와 유사한 10개의 질문에서 q 후보 추출
2. 각 q에 적합한 a 도출
3. p가 a로 개선된 정도(EVPI)로 q 랭킹
p
q
a

15. Best demo: uroman
● Romanization for 290 langs
● Unicode 기반이지만 직접 설계한
1088개의 룰 추가
● https://www.isi.edu/~ulf/uroman.html

16. Lifetime Achievement Award
* https://aclweb.org/aclwiki/ACL_Lifetime_Achievement_Award_Recipients
“Algorithms like LSTM and RNN
may work in practice.
But do they work in theory?”
● Mark Steedman (U. Edinburgh)
● Known for work on CCG
○ Combinatory Categorial Grammar
○ Phrase structure grammar의 일종

17. Tutorials
Xiaodong Gu

18. Variational Inference and Deep Generative Models
Let X and Z be random variables, where X is our observed data and Z is a latent variable. A generative model is any model
that defines their joint distributions: p(x, z)
Our goal: estimate the posterior over latent variables p(z|x) which is often intractable!
Our solution: approximate p(z|x) with q(z;θ) which is computable!
L = log p(x) >= - KL(q(z;θ)||p(z|x)) + log p(x)
ELBO:
VI turns inference into optimization

19. Variational Autoencoder Generative Network Gradient
Inference Network Gradient
Solution - VAE
Again requires approximation by sampling
Generative NetworkInference Network

20. Deep Reinforcement Learning for NLP
- RL introduction
- Fundamentals and Overview
- Deep Reinforcement Learning for Dialogue
- Challenges

21. Fundamentals and Overview
Why use RL in NLP?
1. Learning to search and reason
2. Instead of minimizing the surrogate loss (e.g., XE, hinge loss), optimize the end
metric (e.g., BLEU, ROUGE) directly.
3. Select the right (unlabeled) data.
4. Back-propagate the reward to update the model.

22. Deep Reinforcement Learning for Dialogue
Sequence-to-Sequence Model for Dialogue
Problem1: Repetitive responses
Shut up! -> You shut up! -> You shut up!-> …...
Problem2: Short-sighted conversation decisions
How old are you? -> I’m 16 -> 16? -> I don’t know what you are talking
about -> you don’t know what you’re saying -> I don’t know ……->......
Reinforcement Learning
State: message encoding
Action: response decoding
Rewards:
- Ease of answering
r(response)=-log(dull utt | resp)
- Information Flow
r2
= - log sigmoid(cos(s1
,s2
))
- Meaningfulness
r = log p(resp|message) + log p (message|resp)

23. Challenges
NLP problems that presents new challenges to RL
• An unbounded action space defined by natural language
• Dealing with combinatorial actions and external knowledges
• Learning reward functions for NLG
RL problems that are particularly relevant to NLP
• Sample complexity
• Model-based vs. model free RL
• Acquiring rewards

24. Main conference (1)
- Semantic parsing
Soonmin Bae

25. Intro
Semantic Parsing : mapping natural language utterances to machine interpretable representations
(e.g., executable queries or logical forms)
Semantic Role Labeling : shallow semantic parsing, assigning labels to words
(e.g., predicate-argument structure or “who does what to who, when, and where”)
Semantic Parsing @ ACL 2018
3 Talk Sessions for Semantic Parsing (2A, 6A, 7A) : 12 papers
2 Poster Sessions for Semantics (1B, 3C) : 15 + 14 papers ⇒ half SP/SRL, half embeddings
1 Tutorial Session for Neural Semantic Parsing
2 Talk Sessions for Semantics (5A, 8A) : 8 papers ⇒ mostly embeddings, but a few SRL
2 Talk Sessions for Parsing (3F, 4F) : 8 papers ⇒ grammars/syntax + dependency parsing
1 Poster Session for Morphology, Tagging, Parsing (3F)

26. Improving Text-to-SQL Evaluation Methodology [1/3] - Intro
https://arxiv.org/abs/1806.09029
“How to evaluate Text-to-SQL” must be improved.
- DBs are not realistic
- question-based split vs query-based split
Template-based, Slot-filling Baseline must fail
but works for question-based split.
1B

27. Improving Text-to-SQL Evaluation Methodology [2/3] - DB Issues
Issue 1: Human-written questions require more complex queries than automatically generated data
(e.g., joins, nesting)
Issue 2: Training and test sets should be splitted based on the SQL query instead of question.
1B

28. Improving Text-to-SQL Evaluation Methodology [3/3]
- Template-based, Slot-filling Baseline must fail, but works
Oracle Entities : To provide accurate anonymization, we annotated query variables using a combination of
automatic and manual processing.
DB from NLP communities DB from DB communities
1B

29. Coarse-to-Fine Decoding for Neural Semantic Parsing [1/2]
- Honourable Mention for Best Paper
https://arxiv.org/abs/1805.04793
Structure-aware neural architecture
1. Generate a rough sketch of input utterance’s meaning, where low-level info is glossed over
2. Experimental results on four datasets
2A

30. Coarse-to-Fine Decoding for Neural Semantic Parsing [2/2]
- Honourable Mention for Best Paper
2A

31. Jointly Predicting Predicates and Arguments in Neural
Semantic Role Labeling
https://arxiv.org/abs/1805.04787
End-to-end approach for jointly predicting all predicates, arguments spans, and the relations b/w them
5A

32. Large-Scale QA-SRL Parsing [1/3] - Honourable Mention for Best Paper
https://arxiv.org/abs/1805.05377
Contributions
1. Scalable crowdsourcing QA-SRL
2. New models for QA-SRL parser :
span detection → question generation
Challenges
1. Scale up QA-SRL data annotation
2. Train a QA-SRL Parser
3. Improve Recall
Results
- 133,479 verbs from 64,018 sentences
- across 3 domains (Wikipedia, Wikinews, Science)
- 265,140 question-answer pairs in 9 days
7A

33. Large-Scale QA-SRL Parsing [2/3] - Honourable Mention for Best Paper
Crowdsourcing pipeline
1. Generation
1.1. Write QA-SRL questions for each verb
⇒ Auto-suggest complete questions
reduces # strokes
1.2. Highlight answers in the sentence
2. Validation
2.1. Answer each question or mark it invalid
Model
7A

34. Large-Scale QA-SRL Parsing [3/3]
7A

35. Main conference (2)
- Dialogue
Kyungduk Kim

36. Intro

37. Neural Approaches
- Neural Response Generation
- Dialog Policy Optimization
- Personalized Dialog System
Discourse Analysis
Dialog System @ ACL 2018
Dialog System
Oral presentation: 12 long, 4 short papers
Poster presentation: 16 papers
(Sigdial 2018 is co-located)
Question Answering
Oral presentation: 8 long, 4 short papers
Poster presentation: 11+ papers
Intro

38. Intro
● 결론
○ Task-Oriented 대화 시스템에서 neural approaches를 사용하기 위한 방법론이 활발히 연구 중
○ Amazon Alexa와 같은 산업체에서 발표한, (Clova에도) 실제 적용가능 할만한 연구도 눈에 띔
○ 대화 시스템 평가에 대한 방법론은 아직도 정리 중
■ Perplexity등을 사용하기도 하나, mTurk등을 통해 사람이 직접 성능을 확인
■ ConvAI2 등의 shared task 개최에서도 mTurk을 통한 evaluation이 진행됨

39. Exemplar Encoder-Decoder for Neural Conversation Generation
Neural Conversation Generation using example contexts and their responses

40. Exemplar Encoder-Decoder for Neural Conversation Generation

41. Efficient Large-Scale Neural Domain Classification with Personalized
Attention
● Proposes finding proper domains without explicit invocation name of domain
● Handling 40,000+ 3rd party skills.

42. Efficient Large-Scale Neural Domain Classification with Personalized
Attention

43. Efficient Large-Scale Neural Domain Classification with Personalized
Attention
Evaluation
Week: 100K+ utterances from 600K+ users across 1,000+ Alexa skills
Generated from utterances with explicit invocation patterns.
e.g.) “Ask TAXI to get me a ride” → (get me a ride, TAXI)
MTurk: 20K+ human-paraphrased utterances of randomly sampled commands
from test set of Weak data
e.g.) “get me a ride” → I need a ride, can I get a taxi, order a car for me, …
Evaluate performance with natural language

44. Efficient Large-Scale Neural Domain Classification with Personalized
Attention
Baseline model
1-bit model
(user-enabled domain bitmap)
Attention model
(user-enabled domain attention)

45. Improving Slot Filling in Spoken Language Understanding with
Joint Pointer and Attention

46. Improving Slot Filling in Spoken Language Understanding with
Joint Pointer and Attention

47. Improving Slot Filling in Spoken Language Understanding with
Joint Pointer and Attention

48. Main conference (3)
- Machine translation
Zae Myung Kim

49. Intro
● Machine translation
○ Translating a sequence of tokens in lang. A into that of lang. B
○ All MT papers @ ACL 2018 focused on neural MT (NMT) as opposed to
statistical MT (SMT)
● 11 long, 8 shorts, 1 demo, 1 NMT workshop (16 papers)
○ Main conf. papers (see next page)
○ Papers @ NMT workshop
○ Talks @ NMT workshop
● Trends in MT @ ACL 2018
○ Less exploration on DL architectures than last year, but more NLP-oriented
○ Linguistic structure, document-level translation, data augmentation,
efficient computation, domain adaptation, handling inadequate resources,
and analysis of models

50. List of MT related papers (main conf.)
1. Unsupervised Neural Machine Translation with
Weight Sharing. Yang et al. [link]
2. Subword Regularization: Improving Neural Network
Translation Models with Multiple Subword
Candidates. Kudo. [link]
3. Forest-Based Neural Machine Translation. Ma et al.
[link]
4. Attention Focusing for Neural Machine Translation
by Bridging Source and Target Embeddings. Kuang
et al. [link]
5. Context-Aware Neural Machine Translation Learns
Anaphora Resolution. Voita et al. [link]
6. The Best of Both Worlds: Combining Recent
Advances in Neural Machine Translation. Chen et al.
[link]
7. Towards Robust Neural Machine Translation. Cheng
et al. [link]
8. Document Context Neural Machine Translation with
Memory Networks. Maruf and Haffari. [link]
9. A Stochastic Decoder for Neural Machine
Translation. Schulz et al. [link]
10. Accelerating Neural Transformer via an Average
Attention Network. Zhang et al. [link]
11. Are BLEU and Meaning Representation in
Opposition?. Cífka and Bojar. [link]
12. A Simple and Effective Approach to Coverage-Aware
Neural Machine Translation. Li et al. [link]
13. Compositional Representation of
Morphologically-Rich Input for Neural Machine
Translation. Ataman and Federico. [link]
14. Extreme Adaptation for Personalized Neural Machine
Translation. Michel and Neubig. [link]
15. Learning from Chunk-based Feedback in Neural
Machine Translation. Petrushkov et al. [link]
16. Sparse and Constrained Attention for Neural
Machine Translation. Malaviya et al. [link]
17. Bag-of-Words as Target for Neural Machine
Translation. Ma et al. [link]
18. Improving Beam Search by Removing Monotonic
Constraint for Neural Machine Translation. Shu and
Nakayama. [link]
19. Adaptive Knowledge Sharing in Multi-Task Learning:
Improving Low-Resource Neural Machine
Translation. Zaremoodi et al. [link]
20. Marian: Fast Neural Machine Translation in C++.
Junczys-Dowmunt et al. [link]

51. Document-level translation
Context-Aware Neural Machine Translation Learns Anaphora
Resolution. Voita et al. [link]
● OpenSubtitles 2018; En-Ru; takes the previous
subtitle as “context”
● Modifies Transformer’s encoder part to incorporate
the context
● Model attends to the context when translating
ambiguous pronouns

52. Data augmentation
Towards Robust Neural Machine Translation. Cheng et al. [link]
● Small perturbations in an input can dramatically
deteriorate its translation results
● Perturb input by 1) randomly replacing words, or 2)
adding Gaussian noise to word embedding
● Maintain the consistency of behaviors through the
NMT model for the source sentence x and its
perturbed counterpart x’ through adversarial
learning for the perturbation-invariant encoder

53. Domain adaptation
Extreme Adaptation for Personalized Neural Machine Translation.
Michel and Neubig. [link]

54. Main conference (4)
- Summarization
Hye-Jin Min

55. Overview of Text Summarization
● TASK
○ Extractive summarization / Abstractive Summarization
● DATA
○ CNN/Daily Mail data sets / English giga word corpus
● ISSUES
○ OOV (Out-Of-Vocabulary) / Repetition
● Techniques
○ Seq2seq model + α

56. 3 selected papers
Model TASK DATA Issues Techniques
Unified Model Abstractive
Summarization
CNN/Daily Mail
data sets
advantages from
extractor and
abstractor
Unified model &
Inconsistency
mechanism
SWAP-NET Extractive
summarization
CNN/Daily Mail
data sets
the interaction of
key words and
salient sentences
2-level Pointer
Network
Re
3
Sum
Abstractive
Summarization
English giga
word corpus
Repetition
Short length
IR-approach +
seq2seq

57. Unified Model
● Combining sentence-level and word-level attentions to take advantage of both
extractive and abstractive summarization approaches

58. Unified Model
● Combining sentence-level and word-level attentions to take advantage of both
extractive and abstractive summarization approaches

59. SWAP-NET
● Sentences and Words
from Alternating
Pointer Network
● 2 level pointer network
○ The sentence-level
○ The word-level
● Switch mechanism

60. Retrieve, Rerank and Rewrite (Re3Sum)
● Soft Template Based Neural Summarization
○ to improve the readability and stability of seq2seq summarization systems
● fuse the popular IR-based and seq2seq-based summarization systems

61. Retrieve, Rerank and Rewrite (Re3Sum)
● Jointly Rerank and Rewrite
○ Bi-directional RNN encoder to read input x and template r
○ soft template r resembles the actual summary y∗ as much as possible

62. References
● Unified Model: A Unified Model for Extractive and Abstractive Summarization using Inconsistency
Loss
○ https://arxiv.org/abs/1805.06266
○ https://hsuwanting.github.io/unified_summ/
● SWAP-NET: Extractive Summarization with SWAP-NET: Sentences and Words from Alternating Pointer
Networks
○ http://aclweb.org/anthology/P18-1014
○ https://github.com/aishj10/swap-net/tree/master/swap-net-summer
● Re
3
Sum: Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization
○ http://aclweb.org/anthology/P18-1015
○ http://www4.comp.polyu.edu.hk/~cszqcao/

63. 학생들끼리 AI
워크샵 주최하기
Minjoon Seo

65. MRC vs MRQA?

66. 1. 제안

68. 워크샵 주최? 우린 학생인데?

69. 2. Pivoting

70. MRQA Interpretability?

71. QA에 대해 높아진 관심도

72. WMT 급으로 만들어 보자!

73. MRQA Interpretability → MRQA

74. 3. 사람 모으기

75. Steering & Speakers

77. Program Committee: 35

79. 4. 스폰서 구하기

81. 5. 제안서 제출

82. 10월 22일 제출, 11월 17일 결과

83. 6. 논문제출 받기, 결정하기

84. Encouraging cross-submission

85. ~25 제출, ~50% 합격

86. 6. 갑작스런 상황들

87. 바로 전주에도 스피커가 못온다고...

88. 그리고 ACL의 아주 느린 답변

89. 7. 주최

93. 8. 내년에도?

94. 새로운 사람들? 빠질 사람들?

95. Challenge 추가?

96. Thank you!