Acl 2020 talk
•
0 likes•165 views
1) The document presents a new model for predicting emotional reactions to events that incorporates label semantics by embedding emotion labels and modeling correlations between labels. 2) The model outperforms baselines by finetuning BERT with labels as input and using label correlations as a regularization signal during semi-supervised training. 3) Results show the new approach achieves state-of-the-art performance on an emotion inference task, with a 4.9 F1 point improvement over the previous best model.
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Acl 2020 talk
- 1. 1 Modeling Label Semantics for Predicting Emotional Reactions ACL 2020 Radhika Gaonkar Heeyoung Kwon Mohaddeseh Bastan Niranjan Balasubramanian Nate Chambers
- 2. 2 Emotion Inference Task Understand how events in a story affect the characters involved (Rashkin et al.) The instructor was furious and threw his chair He cancelled practice and expected us to perform tomorrow Anger Fear Surprise Instructor Students
- 3. 3 Objective Infer what emotional reaction each event evokes given the context of the character ➔ Reactions of characters to events taking place in ROC Stories ➔ Multi-label: Each event can evoke more than one emotion Context The instructor was furious and threw his chair Students Event He cancelled practice and expected us to perform tomorrow Fear Surprise Can be framed as a standard multi-label classification problem!
- 4. 4 The Gap Standard framing treats labels as anonymous classes L1 L2 Joy “Danielle was really short on money” 0.7 -0.8 Fear Sadness- It ignores semantics in labels themselves - Doesn’t directly capture label correlations
- 5. 5 Our Contributions Modeling Label Semantics Embeddings Correlations ➔ Capture meaning of label word ➔ Label embedding input to model ➔ Embeddings updated in training ➔ Capture relations between labels ➔ Correlations learned in training ➔ Correlations used in inference 1. Label Embedding Attention Model 2. Finetuning BERT with labels 1. Correlations learned during training and used for inference 2. Semi-supervision with label correlations as a regularization signal
- 6. 6 Encoding Event and Context ● Encode Event and Context text with BiLSTM encoders (Rashkin et al.) Encoder biLSTMEvent Context Encoder biLSTM PretrainedGloVe Multi-label emotion classification The instructor was furious and threw his chair He cancelled practice and expected us to perform tomorrow
- 7. 7 Label Embedding Attention Model (LEAM) ● Label embeddings guide the encoder to extract emotion related information ● Attention matrix - compatibility between encoder outputs and label embeddings ● Emotion focused input representations used for classification Adapted from (Wang et al.) Encoder biLSTMEvent Context Labels Embeddings Encoder biLSTM Labels Pretrained GloVe Labels Embeddings Label based Attention Event Representation Label based Attention Context Representation Multi-label emotion classification Labels
- 8. 8 Finetuning BERT with Labels ● Exploit self attention of the pretrained BERT model ● Self-attention attend to the labels when constructing representations of the input (Devlin et al.) Fine-tuning BERT Event SEP Context Label Sentences Multi-label emotion classification Pretrained Bertbase He cancelled practice and expected us to perform tomorrow The instructor was furious and threw his chair Students feel fear
- 10. 10 Label Scoring w/. Learned Correlations ● Predictions scores affected by high correlations between labels ● G also used in the loss function for continuous representation of true labels Adapted from (Zhao et al.) Label Correlation matrix G 𝐳 ✕ Label prediction scores Correlation loss Cross Entropy loss Your favourite classification model 😃 𝐳’
- 11. 11 Using Correlations as Semi-supervision ● Label correlation as a regularization signal on the unlabeled data ● Iterative batch-wise training ● Forces the model to have more consistent scores ○ Positively correlated have similar scores ○ Negatively correlated have dissimilar scores Label Correlation matrix G 𝐳 ✕ Label prediction scores Correlation loss Cross Entropy loss Unlabeled Data ~200k Semi-supervision Training Update G 𝐳’ Your favourite classification model 😃
- 12. 12 Results (micro-F1 over all emotions) *REN - Recurrent Entity Network (Henaff et al.); NPN - Neural Process Network (Bosselut et al.) † Implementation without ConceptNet Knowledge (Speer and Havasi) and ELMo embeddings (Peters et al.) Baselines New SOTA on emotion inference
- 13. 13 Take-aways 1. Modeling class labels as semantic embeddings helps to learn better representations and predictions 2. Self-attention over labels fed as input helps more than explicit attention using LEAM model 3. Label correlations help ○ regularize learning ○ allows us to exploit unlabeled data with semi-supervision 4. New state-of-the-art result on emotion inference with a 4.9 F1 points advance over best baseline Project page: https://github.com/StonyBrookNLP/emotion-label-semantics