230305_Characterizing English Variation across Social Media Communities with BERT
•
0 likes•9 views
23년 3월 5일 랩미팅 논문 발표 자료
Recommended
Recommended
More Related Content
Similar to 230305_Characterizing English Variation across Social Media Communities with BERT
Similar to 230305_Characterizing English Variation across Social Media Communities with BERT (20)
More from YongSang Yoo
Recently uploaded
Recently uploaded (20)
230305_Characterizing English Variation across Social Media Communities with BERT
- 1. 2023/3/6 Characterizing English Variation across Social Media Communities with BERT 유용상 TACL 2021
- 2. 1 Introduction • Much previous work characterizing language variation across Internet social groups has focused on the types of words used by these groups. • employing BERT to characterize variation in the senses of words as well, analyzing two months of English comments in 474 Reddit communities.
- 3. 2 Related Work • Online language contains an abundance of “nonstandard” words (Rotabi and Kleinberg, 2016) • Online communities’ linguistic norms and differences are often defined by which words are used (Zhang et al. , 2017) • The strength of BERT to capture word senses presents a new opportunity to measure semantic variation in online communities of practice (Devlin et al., 2019) • different senses tend to be segregated into different regions of BERT’s embedding space (Wiedemann et al. , 2019).
- 4. 1 Data • select the top 500 most popular subreddits based on number of comments and remove subreddits • randomly sample 80,000 comments • exclude too general and not specific • removed 1044 multi-word expressions from analysis 2226 unique glossary words
- 5. 1 Word Sense Disambiguation 1. 배가 불러서 더 이상 못 먹겠다. 2. 올 해에는 배가 풍년이다. 3. 내가 더보다 몇 배는 더 빠르다. 4. 사촌이 땅을 사면 내 배가 아프다.
- 6. 1 Word Sense Induction 1. 배가 불러서~~ 4. 내 배가 아프다~~ 3. 몇 배는 더~~ 2. ~~배가 풍년이다.
- 7. 1 Methods for Identifying Community-Specific Language : Type • focused on lexical choice, examining the word types unique to a community • PMI • TF-IDF • TextRank • Jensen-Shannon divergence (JSD)
- 8. 1 Methods for Identifying Community-Specific Language : Meaning OW Overwatch (r/overwatch) Off-White (r/sneakers) Opening Week (r/BoxOffice) • BERT Embeddings • clusters representatives containing word substitutes predicted by BERT
- 9. 1 Methods for Identifying Community-Specific Language : Meaning(Cont’d)
- 10. 1 Evaluation
- 11. 1 Evaluation
- 12. 1 Conclusion • set a foundation for further investigations on how BERT could help define unknown words or meanings in niche communities • Future work could develop annotated WSI datasets for online language similar to the standard SemEval benchmarks
- 13. 감사합니다