SlideShare a Scribd company logo
Human Interface Laboratory
With a Little Help From Speech
- Towards Intention Understanding in Korean
2019. 5. 29
Won Ik Cho
About Me
• 조원익
 B.S. in EE/Mathematics (SNU, ’10~’14)
 Ph.D. student (SNU INMC, ‘14~)
• Academic background
 Interested in mathematics > EE!
 Double major?
• ...
 Early years in Speech processing lab
• Source separation
• Voice activity & endpoint detection
• Automatic music composition
 Currently studying on computational linguistics
1
Contents
• Natural language processing: Overview
• NLProc in Korean – new task?
 Intention understanding: from Definition to Disambiguation
• Done and afterward
2
NLP Overview
• From one-hot encoding to BERT
3
NLP Overview
4
NLP Overview
5
NLP Overview
6
NLP Overview
• Deep NLP 에 대한 다양한 시각
7
(그러나 BERT에 감탄)
NLP Overview
• Tasks
 Morphology
• Word segmentation, morphological analysis ...
 Syntax-semantics
• Consistuency parsing, semantic role labeling, pos tagging ...
 Semantics
• Sentence classification (sentiment, intention etc.)
• Question answering, machine translation, summarization ...
 Pragmatics
• Dialog act tagging, dialog managing ...
 Phonetics
• Speech recognition, multimodal speech understanding
8
NLProc in Korean
• New task?
 Development of free-running speech recognition technologies for
embedded robot system (funded by MOTIE)
 로봇용 free-running 임베디드 자연어 대화음성인식을 위한 원천 기술 개발
• In other words:
 Non wake-up-word based speech understanding system
 ...?
9
오늘 또
떨어졌네
이게 대체
며칠째
파란불이냐
지금 손실이
얼마지
NLProc in Korean
• How?
 Related to many aspects of (speaker-dependent) speech recognition
• Speaker-dependency (in terms of a personal assistant)
• Noisy far-talk recognition and beamforming
• Speech intention understanding
– To which utterances should AI react?
10
오늘 또
떨어졌네
이게 대체
며칠째
파란불이냐
지금 손실이
얼마지
NLProc in Korean
• Speech intention understanding
 Defining what ‘intention’ is
• Discourse components
• Speech act
• Rhetoricalness
 Making up annotation guideline
 Introducing phonetic features
• Intonation-dependency
• Sentence-final intonations
• Text-speech alignment
11
NLProc in Korean
• Intention understanding – how?
 Our approach (for Korean)
12
단일 문장인가?
Intonation 정보로
결정 가능한가?
Question set이 있고
청자의 답을 필요로 하는가?
Effective한 To-do list가
청자에게 부여되는가?
No
Yes
No
Yes
요구 (Commands)
수사명령문 (RC)
Full clause를
포함하는가?
No
No
Compound sentence: 힘이 강한 화행에 중점
(서로 다른 문장도 같은 토픽일 때 한 문장으로 간주)
Fragments (FR)
질문 (Questions)
No
Context-dependent (CD)
Yes
Yes
Yes
Intonation 정보가
필요한가?
Yes
Intonation-dependent (ID)
No Questions /
Embedded form
Requirements /
Prohibitions
수사의문문 (RQ)
Target: single sentence
without context
nor punctuation
Otherwise
서술 (Statements)
NLProc in Korean
• Intonation-dependent utterances
 How to figure out if the utterances is intonation-dependent?
13
천천히 가고 있어! (utterance)
천천 히 가 고 있 어 (transcript)
question
statement
command
?
NLProc in Korean
• Intonation-dependent utterances
 Underspecified sentence enders
• -어, -지, -대, -해, -라고, -다며, etc.
• Sentence type is determined based upon the sentence-final intonations that are
assigned considering the speech act
 Conversation maxim (Levinson, 2000)
• 정보성-원리 Informativeness-principle (단순화 버전)
– 화자: 필요한 것 이상으로 말하지 말라.
» Do not say more than is required (bearing the Q-principle in mind)
– 청자: 화자가 일반적으로 말한 것은 전형적으로 그리고 특칭적으로 해석하라.
» What is generally said is stereotypically and specifically exemplified.
 Wh-intervention
• 뭐 먹고 싶어
– What or something?
14
NLProc in Korean
• Introducing phonetic features: Intonation-dependency
 Annotating proper intention for possible cases of intonation
• 기본적으로 문말 억양을 고려함
• 한 가지 intonation에서 여러 intention이 가능한 경우는 ambiguous한 것으로 봄
• 부사, 수일치 등과 관련하여, 서술이 아닌 것으로 해석하기 어색한 것들은 제외함.
• 너무 많은 정보를 담고 있는 문장을 질문으로 판단하는 것을 피함
• Wh-particle들이 의문사의 기능을 하지 않는 경우들에 주의함
• 많은 한국어 문장이 그렇듯, 주어가 생략되어 1,2,3-인칭 등으로 해석할 수 있을 경
우에는, 각각을 대입해 보고, 어색하지 않은 것들로 판단함
• 호격의 유무에 주의함
15
NLProc in Korean
• System overview: Text-based sieving + Speech-aided analysis
 Compatible to text-speech alignment (submitted to EMNLP 2019)
16
NLProc in Korean
• Prosody-ambiguous statements?
 Problems in: Wh-intervention? (Will be appeared; ICPhS 2019)
• Needs disambiguation
17
몇 개 가져오래
Should I bring some?
How many should I bring?
They told you to bring some?
Done and afterward
• Done
 억양 의존성 및 rhetoricalness를 고려한, 음성인식 output 분석에 적합한 일
반언어학적 speech act 분류 방법론 제시
 한국어를 위한 annotation guideline 정립, corpus 구성 및 모델 학습
 질문/요구 argument 추출을 위한 parallel corpus 제작 (distributed)
 Speech analysis의 disambiguation을 위한 코퍼스 (will be distributed)
• Afterward?
 대화체/비정형 질문/요구의 argument 추출 위한 structured paraphrasing
 Speech disambiguation을 위한 text-speech alignment 개량
 Task-oriented와 non-oriented 간 code switching이 자유로운 dialog
manager 시스템의 개발
18
Reference
• Mikolov, Tomas, et al. "Distributed representations of words and phrases and their
compositionality." Advances in neural information processing systems. 2013.
• Sutskever, Ilya, et al. “Sequence to sequence learning with neural networks.” Advances in neural
information processing systems. 2014.
• Cho, Kyunghyun, et al. “Learning Phrase Representations using RNN Encoder-Decoder for
Statistical Machine Translation.” arXiv preprint arXiv:1406.1078. 2014.
• Kim, Yoon. "Convolutional neural networks for sentence classification." arXiv preprint
arXiv:1408.5882. 2014.
• Transformer, BERT, LISA (Google it for further information!)
• Liu, Bing, and Ian Lane. "Attention-based recurrent neural network models for joint intent
detection and slot filling." arXiv preprint arXiv:1609.01454 (2016).
• Gu, Yue, et al. "Speech intention classification with multimodal deep learning." Canadian
Conference on Artificial Intelligence. Springer, Cham, 2017.
• Sadock, Jerrold M., and Arnold M. Zwicky. "Speech act distinctions in syntax." Language
typology and syntactic description 1 (1985): 155-196.
• Portner, Paul. "The semantics of imperatives within a theory of clause types." Semantics and
linguistic theory. Vol. 14. 2004.
• Searle, John R. "A classification of illocutionary acts." Language in society 5.1 (1976): 1-23.
19
Reference
• Stolcke, Andreas, et al. "Dialogue act modeling for automatic tagging and recognition of
conversational speech." Computational linguistics 26.3 (2000): 339-373.
• Bunt, Harry, et al. "Towards an ISO standard for dialogue act annotation." Seventh conference
on International Language Resources and Evaluation (LREC'10). 2010.
• Vosoughi, Soroush, and Deb Roy. "Tweet Acts: A Speech Act Classifier for Twitter." ICWSM. 2016.
• Levinson, Stephen C. Presumptive meanings: The theory of generalized conversational
implicature. MIT press, 2000.
• 이현정, 서정연. "한국어 대화체 문장의 화행 분석." 한국정보과학회 학술발표논문집 24.2Ⅱ (1997):
259-262.
• 김세종, 이용훈, 이종혁. "이전 문장 자질과 다음 발화의 후보 화행을 이용한 한국어 화행 분석." 정보과
학회논문지: 소프트웨어 및 응용 35.6 (2008): 374-385.
• 이현정, 이재원, 서정연. "자동통역을 위한 한국어 대화 문장의 화행 분석 모델." 정보과학회논문지 (B)
25.10 (1998): 1443-1452.
• 이성욱, 서정연. "결정트리를 이용한 한국어 화행 분석." 한국정보과학회 언어공학연구회 학술발표 논문
집 (1999): 377-381.
• Friedrich, Annemarie, Alexis Palmer, and Manfred Pinkal. "Situation entity types: automatic
classification of clause-level aspect." Proceedings of the 54th Annual Meeting of the Association
for Computational Linguistics (Volume 1: Long Papers). Vol. 1. 2016.
20
Thank you!
EndOfPresentation

More Related Content

What's hot

Natural language-processing
Natural language-processingNatural language-processing
Natural language-processing
Hareem Naz
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
National Institute of Technology Durgapur
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
Abash shah
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
KarenVacca
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
Aanchal Chaurasia
 
Natural Language Processing
Natural Language Processing Natural Language Processing
Natural Language Processing
Adarsh Saxena
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
Mariana Soffer
 
1908 working memory
1908 working memory1908 working memory
1908 working memory
WarNik Chow
 
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Edureka!
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
Yasir Khan
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
Saurabh Kaushik
 
5a use of annotated corpus
5a use of annotated corpus5a use of annotated corpus
5a use of annotated corpus
ThennarasuSakkan
 
Hidden markov model based part of speech tagger for sinhala language
Hidden markov model based part of speech tagger for sinhala languageHidden markov model based part of speech tagger for sinhala language
Hidden markov model based part of speech tagger for sinhala language
ijnlc
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
Rishikese MR
 
Natural language processing (nlp)
Natural language processing (nlp)Natural language processing (nlp)
Natural language processing (nlp)
Kuppusamy P
 
Lesson 40
Lesson 40Lesson 40
Lesson 40
Avijit Kumar
 
Lesson 41
Lesson 41Lesson 41
Lesson 41
Avijit Kumar
 
Amharic WSD using WordNet
Amharic WSD using WordNetAmharic WSD using WordNet
Amharic WSD using WordNet
Seid Hassen
 
Unique Signs for Plural Utterances
Unique Signs for Plural UtterancesUnique Signs for Plural Utterances
Unique Signs for Plural Utterances
Lawrie Hunter
 
Natural language processing with python and amharic syntax parse tree by dani...
Natural language processing with python and amharic syntax parse tree by dani...Natural language processing with python and amharic syntax parse tree by dani...
Natural language processing with python and amharic syntax parse tree by dani...
Daniel Adenew
 

What's hot (20)

Natural language-processing
Natural language-processingNatural language-processing
Natural language-processing
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
 
Natural Language Processing
Natural Language Processing Natural Language Processing
Natural Language Processing
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
1908 working memory
1908 working memory1908 working memory
1908 working memory
 
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
5a use of annotated corpus
5a use of annotated corpus5a use of annotated corpus
5a use of annotated corpus
 
Hidden markov model based part of speech tagger for sinhala language
Hidden markov model based part of speech tagger for sinhala languageHidden markov model based part of speech tagger for sinhala language
Hidden markov model based part of speech tagger for sinhala language
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
Natural language processing (nlp)
Natural language processing (nlp)Natural language processing (nlp)
Natural language processing (nlp)
 
Lesson 40
Lesson 40Lesson 40
Lesson 40
 
Lesson 41
Lesson 41Lesson 41
Lesson 41
 
Amharic WSD using WordNet
Amharic WSD using WordNetAmharic WSD using WordNet
Amharic WSD using WordNet
 
Unique Signs for Plural Utterances
Unique Signs for Plural UtterancesUnique Signs for Plural Utterances
Unique Signs for Plural Utterances
 
Natural language processing with python and amharic syntax parse tree by dani...
Natural language processing with python and amharic syntax parse tree by dani...Natural language processing with python and amharic syntax parse tree by dani...
Natural language processing with python and amharic syntax parse tree by dani...
 

Similar to Warnikchow - SAIT - 0529

L1 nlp intro
L1 nlp introL1 nlp intro
L1 nlp intro
Harshit Yadav
 
Deep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word EmbeddingsDeep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word Embeddings
Roelof Pieters
 
Warnikchow - Naver Tech Talk - 3i4k
Warnikchow - Naver Tech Talk - 3i4kWarnikchow - Naver Tech Talk - 3i4k
Warnikchow - Naver Tech Talk - 3i4k
WarNik Chow
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Saurabh Kaushik
 
Natural Language Processing - Unit 1
Natural Language Processing - Unit 1Natural Language Processing - Unit 1
Natural Language Processing - Unit 1
Mithun B N
 
Natural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptxNatural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptx
SHIBDASDUTTA
 
AI Lesson 40
AI Lesson 40AI Lesson 40
AI Lesson 40
Assistant Professor
 
R3 Setting the Research Agenda for Teaching and Learning Chinese
R3 Setting the Research Agenda for Teaching and Learning Chinese  R3 Setting the Research Agenda for Teaching and Learning Chinese
R3 Setting the Research Agenda for Teaching and Learning Chinese
Center for Global Education at Asia Society
 
5810 oral lang anly transcr wkshp (fall 2014) pdf
5810 oral lang anly transcr wkshp (fall 2014) pdf  5810 oral lang anly transcr wkshp (fall 2014) pdf
5810 oral lang anly transcr wkshp (fall 2014) pdf
SVTaylor123
 
L1
L1L1
2007 CogSci 2020 poster
2007 CogSci 2020 poster2007 CogSci 2020 poster
2007 CogSci 2020 poster
WarNik Chow
 
Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...
Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...
Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...
Ken Urano
 
Definiendo el enfoque lfe
Definiendo el enfoque lfeDefiniendo el enfoque lfe
Definiendo el enfoque lfe
Natalia Rojo
 
5810 day 3 sept 20 2014
5810 day 3 sept 20 2014 5810 day 3 sept 20 2014
5810 day 3 sept 20 2014
SVTaylor123
 
Using Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL PronunicationUsing Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL Pronunication
Rebecca Allen
 
1910 HCLT
1910 HCLT1910 HCLT
1910 HCLT
WarNik Chow
 
A research agenda for leslla_
A research agenda for leslla_A research agenda for leslla_
A research agenda for leslla_
Patsy Vinogradov
 
L2 acquisition
L2 acquisitionL2 acquisition
L2 acquisition
CameliaN
 
REPORT.doc
REPORT.docREPORT.doc
5810 day 3 sept 20 2014
5810 day 3 sept 20 2014 5810 day 3 sept 20 2014
5810 day 3 sept 20 2014
SVTaylor123
 

Similar to Warnikchow - SAIT - 0529 (20)

L1 nlp intro
L1 nlp introL1 nlp intro
L1 nlp intro
 
Deep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word EmbeddingsDeep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word Embeddings
 
Warnikchow - Naver Tech Talk - 3i4k
Warnikchow - Naver Tech Talk - 3i4kWarnikchow - Naver Tech Talk - 3i4k
Warnikchow - Naver Tech Talk - 3i4k
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
 
Natural Language Processing - Unit 1
Natural Language Processing - Unit 1Natural Language Processing - Unit 1
Natural Language Processing - Unit 1
 
Natural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptxNatural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptx
 
AI Lesson 40
AI Lesson 40AI Lesson 40
AI Lesson 40
 
R3 Setting the Research Agenda for Teaching and Learning Chinese
R3 Setting the Research Agenda for Teaching and Learning Chinese  R3 Setting the Research Agenda for Teaching and Learning Chinese
R3 Setting the Research Agenda for Teaching and Learning Chinese
 
5810 oral lang anly transcr wkshp (fall 2014) pdf
5810 oral lang anly transcr wkshp (fall 2014) pdf  5810 oral lang anly transcr wkshp (fall 2014) pdf
5810 oral lang anly transcr wkshp (fall 2014) pdf
 
L1
L1L1
L1
 
2007 CogSci 2020 poster
2007 CogSci 2020 poster2007 CogSci 2020 poster
2007 CogSci 2020 poster
 
Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...
Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...
Types of L2 morphosyntactic knowledge that can and cannot be observed in lea...
 
Definiendo el enfoque lfe
Definiendo el enfoque lfeDefiniendo el enfoque lfe
Definiendo el enfoque lfe
 
5810 day 3 sept 20 2014
5810 day 3 sept 20 2014 5810 day 3 sept 20 2014
5810 day 3 sept 20 2014
 
Using Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL PronunicationUsing Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL Pronunication
 
1910 HCLT
1910 HCLT1910 HCLT
1910 HCLT
 
A research agenda for leslla_
A research agenda for leslla_A research agenda for leslla_
A research agenda for leslla_
 
L2 acquisition
L2 acquisitionL2 acquisition
L2 acquisition
 
REPORT.doc
REPORT.docREPORT.doc
REPORT.doc
 
5810 day 3 sept 20 2014
5810 day 3 sept 20 2014 5810 day 3 sept 20 2014
5810 day 3 sept 20 2014
 

More from WarNik Chow

2312 PACLIC
2312 PACLIC2312 PACLIC
2312 PACLIC
WarNik Chow
 
2311 EAAMO
2311 EAAMO2311 EAAMO
2311 EAAMO
WarNik Chow
 
2211 HCOMP
2211 HCOMP2211 HCOMP
2211 HCOMP
WarNik Chow
 
2211 APSIPA
2211 APSIPA2211 APSIPA
2211 APSIPA
WarNik Chow
 
2211 AACL
2211 AACL2211 AACL
2211 AACL
WarNik Chow
 
2210 CODI
2210 CODI2210 CODI
2210 CODI
WarNik Chow
 
2206 FAccT_inperson
2206 FAccT_inperson2206 FAccT_inperson
2206 FAccT_inperson
WarNik Chow
 
2206 Modupop!
2206 Modupop!2206 Modupop!
2206 Modupop!
WarNik Chow
 
2204 Kakao talk on Hate speech dataset
2204 Kakao talk on Hate speech dataset2204 Kakao talk on Hate speech dataset
2204 Kakao talk on Hate speech dataset
WarNik Chow
 
2108 [LangCon2021] kosp2e
2108 [LangCon2021] kosp2e2108 [LangCon2021] kosp2e
2108 [LangCon2021] kosp2e
WarNik Chow
 
2106 PRSLLS
2106 PRSLLS2106 PRSLLS
2106 PRSLLS
WarNik Chow
 
2106 JWLLP
2106 JWLLP2106 JWLLP
2106 JWLLP
WarNik Chow
 
2106 ACM DIS
2106 ACM DIS2106 ACM DIS
2106 ACM DIS
WarNik Chow
 
2104 Talk @SSU
2104 Talk @SSU2104 Talk @SSU
2104 Talk @SSU
WarNik Chow
 
2103 ACM FAccT
2103 ACM FAccT2103 ACM FAccT
2103 ACM FAccT
WarNik Chow
 
2102 Redone seminar
2102 Redone seminar2102 Redone seminar
2102 Redone seminar
WarNik Chow
 
2011 NLP-OSS
2011 NLP-OSS2011 NLP-OSS
2011 NLP-OSS
WarNik Chow
 
2010 INTERSPEECH
2010 INTERSPEECH 2010 INTERSPEECH
2010 INTERSPEECH
WarNik Chow
 
2010 PACLIC - pay attention to categories
2010 PACLIC - pay attention to categories2010 PACLIC - pay attention to categories
2010 PACLIC - pay attention to categories
WarNik Chow
 
2010 HCLT Hate Speech
2010 HCLT Hate Speech2010 HCLT Hate Speech
2010 HCLT Hate Speech
WarNik Chow
 

More from WarNik Chow (20)

2312 PACLIC
2312 PACLIC2312 PACLIC
2312 PACLIC
 
2311 EAAMO
2311 EAAMO2311 EAAMO
2311 EAAMO
 
2211 HCOMP
2211 HCOMP2211 HCOMP
2211 HCOMP
 
2211 APSIPA
2211 APSIPA2211 APSIPA
2211 APSIPA
 
2211 AACL
2211 AACL2211 AACL
2211 AACL
 
2210 CODI
2210 CODI2210 CODI
2210 CODI
 
2206 FAccT_inperson
2206 FAccT_inperson2206 FAccT_inperson
2206 FAccT_inperson
 
2206 Modupop!
2206 Modupop!2206 Modupop!
2206 Modupop!
 
2204 Kakao talk on Hate speech dataset
2204 Kakao talk on Hate speech dataset2204 Kakao talk on Hate speech dataset
2204 Kakao talk on Hate speech dataset
 
2108 [LangCon2021] kosp2e
2108 [LangCon2021] kosp2e2108 [LangCon2021] kosp2e
2108 [LangCon2021] kosp2e
 
2106 PRSLLS
2106 PRSLLS2106 PRSLLS
2106 PRSLLS
 
2106 JWLLP
2106 JWLLP2106 JWLLP
2106 JWLLP
 
2106 ACM DIS
2106 ACM DIS2106 ACM DIS
2106 ACM DIS
 
2104 Talk @SSU
2104 Talk @SSU2104 Talk @SSU
2104 Talk @SSU
 
2103 ACM FAccT
2103 ACM FAccT2103 ACM FAccT
2103 ACM FAccT
 
2102 Redone seminar
2102 Redone seminar2102 Redone seminar
2102 Redone seminar
 
2011 NLP-OSS
2011 NLP-OSS2011 NLP-OSS
2011 NLP-OSS
 
2010 INTERSPEECH
2010 INTERSPEECH 2010 INTERSPEECH
2010 INTERSPEECH
 
2010 PACLIC - pay attention to categories
2010 PACLIC - pay attention to categories2010 PACLIC - pay attention to categories
2010 PACLIC - pay attention to categories
 
2010 HCLT Hate Speech
2010 HCLT Hate Speech2010 HCLT Hate Speech
2010 HCLT Hate Speech
 

Recently uploaded

Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
rawankhanlove256
 
Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...
Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...
Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...
IJAEMSJORNAL
 
Adv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdfAdv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdf
T.D. Shashikala
 
CGR-Unit-1 Basics of Computer Graphics.pdf
CGR-Unit-1 Basics of Computer Graphics.pdfCGR-Unit-1 Basics of Computer Graphics.pdf
CGR-Unit-1 Basics of Computer Graphics.pdf
Rugved Collection
 
Response & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITHResponse & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITH
IIIT Hyderabad
 
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
sunnuchadda
 
Jet Propulsion and its working principle.pdf
Jet Propulsion and its working principle.pdfJet Propulsion and its working principle.pdf
Jet Propulsion and its working principle.pdf
KIET Group of Institutions
 
Ludo system project report management .pdf
Ludo  system project report management .pdfLudo  system project report management .pdf
Ludo system project report management .pdf
Kamal Acharya
 
EAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagne
EAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagneEAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagne
EAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagne
idelewebmestre
 
Software Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity PlanningSoftware Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity Planning
Prakhyath Rai
 
CONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY ppt
CONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY pptCONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY ppt
CONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY ppt
ASHOK KUMAR SINGH
 
Metrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical EngineeringMetrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical Engineering
leakingvideo
 
Stiffness Method for structure analysis - Truss
Stiffness Method  for structure analysis - TrussStiffness Method  for structure analysis - Truss
Stiffness Method for structure analysis - Truss
adninhaerul
 
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
YanKing2
 
Concepts Basic/ Technical Electronic Material.pdf
Concepts Basic/ Technical Electronic Material.pdfConcepts Basic/ Technical Electronic Material.pdf
Concepts Basic/ Technical Electronic Material.pdf
OBD II
 
libro de modelado de diseño-part-1[160-250].pdf
libro de modelado de diseño-part-1[160-250].pdflibro de modelado de diseño-part-1[160-250].pdf
libro de modelado de diseño-part-1[160-250].pdf
celiosilva66
 
printing of ic circuits.pdf
printing       of        ic     circuits.pdfprinting       of        ic     circuits.pdf
printing of ic circuits.pdf
chidambaramnatarajar
 
How to Formulate A Good Research Question
How to Formulate A  Good Research QuestionHow to Formulate A  Good Research Question
How to Formulate A Good Research Question
rkpv2002
 
SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...
SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...
SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...
Jim Mimlitz, P.E.
 
Thermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subjectThermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subject
JigneshChhatbar1
 

Recently uploaded (20)

Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...
Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...
Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...
 
Adv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdfAdv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdf
 
CGR-Unit-1 Basics of Computer Graphics.pdf
CGR-Unit-1 Basics of Computer Graphics.pdfCGR-Unit-1 Basics of Computer Graphics.pdf
CGR-Unit-1 Basics of Computer Graphics.pdf
 
Response & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITHResponse & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITH
 
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Jet Propulsion and its working principle.pdf
Jet Propulsion and its working principle.pdfJet Propulsion and its working principle.pdf
Jet Propulsion and its working principle.pdf
 
Ludo system project report management .pdf
Ludo  system project report management .pdfLudo  system project report management .pdf
Ludo system project report management .pdf
 
EAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagne
EAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagneEAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagne
EAAP2023 : Durabilité et services écosystémiques de l'élevage ovin de montagne
 
Software Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity PlanningSoftware Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity Planning
 
CONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY ppt
CONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY pptCONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY ppt
CONFINED SPACE ENTRY TRAINING FOR OIL INDUSTRY ppt
 
Metrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical EngineeringMetrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical Engineering
 
Stiffness Method for structure analysis - Truss
Stiffness Method  for structure analysis - TrussStiffness Method  for structure analysis - Truss
Stiffness Method for structure analysis - Truss
 
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
 
Concepts Basic/ Technical Electronic Material.pdf
Concepts Basic/ Technical Electronic Material.pdfConcepts Basic/ Technical Electronic Material.pdf
Concepts Basic/ Technical Electronic Material.pdf
 
libro de modelado de diseño-part-1[160-250].pdf
libro de modelado de diseño-part-1[160-250].pdflibro de modelado de diseño-part-1[160-250].pdf
libro de modelado de diseño-part-1[160-250].pdf
 
printing of ic circuits.pdf
printing       of        ic     circuits.pdfprinting       of        ic     circuits.pdf
printing of ic circuits.pdf
 
How to Formulate A Good Research Question
How to Formulate A  Good Research QuestionHow to Formulate A  Good Research Question
How to Formulate A Good Research Question
 
SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...
SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...
SCADAmetrics Instrumentation for Sensus Water Meters - Core and Main Training...
 
Thermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subjectThermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subject
 

Warnikchow - SAIT - 0529

  • 1. Human Interface Laboratory With a Little Help From Speech - Towards Intention Understanding in Korean 2019. 5. 29 Won Ik Cho
  • 2. About Me • 조원익  B.S. in EE/Mathematics (SNU, ’10~’14)  Ph.D. student (SNU INMC, ‘14~) • Academic background  Interested in mathematics > EE!  Double major? • ...  Early years in Speech processing lab • Source separation • Voice activity & endpoint detection • Automatic music composition  Currently studying on computational linguistics 1
  • 3. Contents • Natural language processing: Overview • NLProc in Korean – new task?  Intention understanding: from Definition to Disambiguation • Done and afterward 2
  • 4. NLP Overview • From one-hot encoding to BERT 3
  • 8. NLP Overview • Deep NLP 에 대한 다양한 시각 7 (그러나 BERT에 감탄)
  • 9. NLP Overview • Tasks  Morphology • Word segmentation, morphological analysis ...  Syntax-semantics • Consistuency parsing, semantic role labeling, pos tagging ...  Semantics • Sentence classification (sentiment, intention etc.) • Question answering, machine translation, summarization ...  Pragmatics • Dialog act tagging, dialog managing ...  Phonetics • Speech recognition, multimodal speech understanding 8
  • 10. NLProc in Korean • New task?  Development of free-running speech recognition technologies for embedded robot system (funded by MOTIE)  로봇용 free-running 임베디드 자연어 대화음성인식을 위한 원천 기술 개발 • In other words:  Non wake-up-word based speech understanding system  ...? 9 오늘 또 떨어졌네 이게 대체 며칠째 파란불이냐 지금 손실이 얼마지
  • 11. NLProc in Korean • How?  Related to many aspects of (speaker-dependent) speech recognition • Speaker-dependency (in terms of a personal assistant) • Noisy far-talk recognition and beamforming • Speech intention understanding – To which utterances should AI react? 10 오늘 또 떨어졌네 이게 대체 며칠째 파란불이냐 지금 손실이 얼마지
  • 12. NLProc in Korean • Speech intention understanding  Defining what ‘intention’ is • Discourse components • Speech act • Rhetoricalness  Making up annotation guideline  Introducing phonetic features • Intonation-dependency • Sentence-final intonations • Text-speech alignment 11
  • 13. NLProc in Korean • Intention understanding – how?  Our approach (for Korean) 12 단일 문장인가? Intonation 정보로 결정 가능한가? Question set이 있고 청자의 답을 필요로 하는가? Effective한 To-do list가 청자에게 부여되는가? No Yes No Yes 요구 (Commands) 수사명령문 (RC) Full clause를 포함하는가? No No Compound sentence: 힘이 강한 화행에 중점 (서로 다른 문장도 같은 토픽일 때 한 문장으로 간주) Fragments (FR) 질문 (Questions) No Context-dependent (CD) Yes Yes Yes Intonation 정보가 필요한가? Yes Intonation-dependent (ID) No Questions / Embedded form Requirements / Prohibitions 수사의문문 (RQ) Target: single sentence without context nor punctuation Otherwise 서술 (Statements)
  • 14. NLProc in Korean • Intonation-dependent utterances  How to figure out if the utterances is intonation-dependent? 13 천천히 가고 있어! (utterance) 천천 히 가 고 있 어 (transcript) question statement command ?
  • 15. NLProc in Korean • Intonation-dependent utterances  Underspecified sentence enders • -어, -지, -대, -해, -라고, -다며, etc. • Sentence type is determined based upon the sentence-final intonations that are assigned considering the speech act  Conversation maxim (Levinson, 2000) • 정보성-원리 Informativeness-principle (단순화 버전) – 화자: 필요한 것 이상으로 말하지 말라. » Do not say more than is required (bearing the Q-principle in mind) – 청자: 화자가 일반적으로 말한 것은 전형적으로 그리고 특칭적으로 해석하라. » What is generally said is stereotypically and specifically exemplified.  Wh-intervention • 뭐 먹고 싶어 – What or something? 14
  • 16. NLProc in Korean • Introducing phonetic features: Intonation-dependency  Annotating proper intention for possible cases of intonation • 기본적으로 문말 억양을 고려함 • 한 가지 intonation에서 여러 intention이 가능한 경우는 ambiguous한 것으로 봄 • 부사, 수일치 등과 관련하여, 서술이 아닌 것으로 해석하기 어색한 것들은 제외함. • 너무 많은 정보를 담고 있는 문장을 질문으로 판단하는 것을 피함 • Wh-particle들이 의문사의 기능을 하지 않는 경우들에 주의함 • 많은 한국어 문장이 그렇듯, 주어가 생략되어 1,2,3-인칭 등으로 해석할 수 있을 경 우에는, 각각을 대입해 보고, 어색하지 않은 것들로 판단함 • 호격의 유무에 주의함 15
  • 17. NLProc in Korean • System overview: Text-based sieving + Speech-aided analysis  Compatible to text-speech alignment (submitted to EMNLP 2019) 16
  • 18. NLProc in Korean • Prosody-ambiguous statements?  Problems in: Wh-intervention? (Will be appeared; ICPhS 2019) • Needs disambiguation 17 몇 개 가져오래 Should I bring some? How many should I bring? They told you to bring some?
  • 19. Done and afterward • Done  억양 의존성 및 rhetoricalness를 고려한, 음성인식 output 분석에 적합한 일 반언어학적 speech act 분류 방법론 제시  한국어를 위한 annotation guideline 정립, corpus 구성 및 모델 학습  질문/요구 argument 추출을 위한 parallel corpus 제작 (distributed)  Speech analysis의 disambiguation을 위한 코퍼스 (will be distributed) • Afterward?  대화체/비정형 질문/요구의 argument 추출 위한 structured paraphrasing  Speech disambiguation을 위한 text-speech alignment 개량  Task-oriented와 non-oriented 간 code switching이 자유로운 dialog manager 시스템의 개발 18
  • 20. Reference • Mikolov, Tomas, et al. "Distributed representations of words and phrases and their compositionality." Advances in neural information processing systems. 2013. • Sutskever, Ilya, et al. “Sequence to sequence learning with neural networks.” Advances in neural information processing systems. 2014. • Cho, Kyunghyun, et al. “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.” arXiv preprint arXiv:1406.1078. 2014. • Kim, Yoon. "Convolutional neural networks for sentence classification." arXiv preprint arXiv:1408.5882. 2014. • Transformer, BERT, LISA (Google it for further information!) • Liu, Bing, and Ian Lane. "Attention-based recurrent neural network models for joint intent detection and slot filling." arXiv preprint arXiv:1609.01454 (2016). • Gu, Yue, et al. "Speech intention classification with multimodal deep learning." Canadian Conference on Artificial Intelligence. Springer, Cham, 2017. • Sadock, Jerrold M., and Arnold M. Zwicky. "Speech act distinctions in syntax." Language typology and syntactic description 1 (1985): 155-196. • Portner, Paul. "The semantics of imperatives within a theory of clause types." Semantics and linguistic theory. Vol. 14. 2004. • Searle, John R. "A classification of illocutionary acts." Language in society 5.1 (1976): 1-23. 19
  • 21. Reference • Stolcke, Andreas, et al. "Dialogue act modeling for automatic tagging and recognition of conversational speech." Computational linguistics 26.3 (2000): 339-373. • Bunt, Harry, et al. "Towards an ISO standard for dialogue act annotation." Seventh conference on International Language Resources and Evaluation (LREC'10). 2010. • Vosoughi, Soroush, and Deb Roy. "Tweet Acts: A Speech Act Classifier for Twitter." ICWSM. 2016. • Levinson, Stephen C. Presumptive meanings: The theory of generalized conversational implicature. MIT press, 2000. • 이현정, 서정연. "한국어 대화체 문장의 화행 분석." 한국정보과학회 학술발표논문집 24.2Ⅱ (1997): 259-262. • 김세종, 이용훈, 이종혁. "이전 문장 자질과 다음 발화의 후보 화행을 이용한 한국어 화행 분석." 정보과 학회논문지: 소프트웨어 및 응용 35.6 (2008): 374-385. • 이현정, 이재원, 서정연. "자동통역을 위한 한국어 대화 문장의 화행 분석 모델." 정보과학회논문지 (B) 25.10 (1998): 1443-1452. • 이성욱, 서정연. "결정트리를 이용한 한국어 화행 분석." 한국정보과학회 언어공학연구회 학술발표 논문 집 (1999): 377-381. • Friedrich, Annemarie, Alexis Palmer, and Manfred Pinkal. "Situation entity types: automatic classification of clause-level aspect." Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vol. 1. 2016. 20

Editor's Notes

  1. .