Successfully reported this slideshow.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

EMNLP 2021 proScript (summary slides)

  1. 1. proScript: Partially Ordered Scripts Generation Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras,
 Niket Tandon, Peter Clark, Yejin Choi
 https://proscript.allenai.org/
  2. 2. What is Script? Why is it important? “a script is a stereotyped sequence of actions that defines a well-known situation and has associated with it”  Roger Schank and Robert Abelson (1977) 2 What is Script? Scenario: Travel to Hawaii
  3. 3. What is Script? Why is it important? Images from: https://images.app.goo.gl/B24EbZbZ8Xzw7STUA https://images.app.goo.gl/iWRLkmm3TTWGD4jJ7 https://images.app.goo.gl/WDjZtLHW3Gw36eS9A “a script is a stereotyped sequence of actions that defines a well-known situation and has associated with it”  Roger Schank and Robert Abelson (1977) 3 ✦Script is an essential part commonsense knowledge. ✦Script helps to represent and understand causal structure of events. ✦Script allows inference about implicit cause and effect relationship. What is Script? Why is it important? Scenario: Travel to Hawaii
  4. 4. Research Problem: trade-off between quality and scale 4 Quality Scalability Induce from texts (e.g., Chambers and Jurafsky, 2008) - + Event sequence alignment (e.g., Regneri et al., 2010) + - proScript + +
  5. 5. Our contributions 5 We collect 6.4k partially ordered scripts, proScript, which is substantially larger than prior datasets. With proScript, we introduced two complementary tasks and models. (edge prediction and script generation) We show the first time that pre-trained neural LM can be adapted to generate partial-order scripts.
  6. 6. 1. proScript: Crowdsourced 6.4k partial-order scripts 6 Suppose a scenario where someone wants to “travel to Hawaii”. Q1: Describe 5 to 7 essential steps and each time duration. (Note: the order does not matter.) decide schedule 1 hour book a flight go to airport Q2. Create a flowchart of the steps (possibly in partial order, where temporal ordering is required only when it is necessary.) 30 minutes 1 hour Collect “scenarios” (e.g., travel to Hawaii, bake a cake) from existing corpora and datasets. 1. DeScript (Wanzare et al., 2016) 2. VirtualHome (Puig et al., 2018) 3. ROCStories (Mostafazadeh et al., 2016)
  7. 7. 2. Two complementary tasks and models 1. proScript Edge Prediction 7 find the cake recipe gather the ingredients turn on the oven mix the ingredients put the cake batter in the oven bake for the right amount of time take the cake out of the oven Scenario: bake a cake Given: Scenario and randomly shuffled events 2. proScript Generation
  8. 8. 2. Two complementary tasks and models 1. proScript Edge Prediction 8 2. proScript Generation find the cake recipe gather the ingredients turn on the oven mix the ingredients put the cake batter in the oven bake for the right amount of time take the cake out of the oven Scenario: bake a cake Given: Scenario and randomly shuffled events Given: Scenario and the number of events (to generate) Scenario: bake a cake Number of events: 7
  9. 9. 2. Two complementary tasks and models 1. proScript Edge Prediction 9 2. proScript Generation find the cake recipe gather the ingredients turn on the oven mix the ingredients put the cake batter in the oven bake for the right amount of time take the cake out of the oven Scenario: bake a cake Given: Scenario and randomly shuffled events Given: Scenario and the number of events (to generate) Scenario: bake a cake Number of events: 7 DAGs are flattened by DOT language
  10. 10. 3. Generate partial-order Scripts with neural LM 10 Scenario: play the organ Scenario: drink a glass of milk walk to the kitchen open the refrigerator remove milk from refrigerator close the refrigerator pour milk into pot to warm a bit pour milk into glass to drink raise glass to lips find sheet music to play sit down at the organ bench set up the sheet warm up on the organ play the music on the organ
  11. 11. Evaluation: Graph Edit Distance (lower GED, the better) 11 random baseline proScript generator Human 0 3 6 9 12 2.78 4.73 11.3 ← smaller the better Human (2.7) < proScript generator (4.7) << Random (11.3) proScript generator
  12. 12. Check out the paper for more details! 12 Pairwise comparison between Human vs. Model, Qualitative analysis of graph edits, Results on edge prediction task, and a lot of other results and analysis…
  13. 13. Summary 13 We collect 6.4k partially ordered scripts, proScript, which is substantially larger than prior datasets. With proScript, we introduced two complementary tasks and models. (edge prediction and script generation) We show the first time that pre-trained neural LM can be adapted to generate partial-order scripts. Data is available: https://proscript.allenai.org/

×