4. Background & Motivation
Parsing Noisy Text
Learners Essay, Web Texts, Speech Transcription
e.g.,
(incorrect) I look in forward hear from you.
(correct) I look forward to hearing from you.
3
5. Background & Motivation
Parsing Noisy Text
Learners Essay, Web Texts, Speech Transcription
e.g.,
(incorrect) I look in forward hear from you.
(correct) I look forward to hearing from you.
Error correction
↓
Parsing
Pipeline
4
6. Background & Motivation
Parsing Noisy Text
Learners Essay, Web Texts, Speech Transcription
e.g.,
(incorrect) I look in forward hear from you.
(correct) I look forward to hearing from you.
Error correction
↓
Parsing
Pipeline
Error-repair
parsing
Joint training
5
7. Error-repair Dependency Parsing
1. Non-directional Easy-first parsing
(Goldberg and Elhadad, 2010)
2. Three new actions to repair errors
6
10. Non-directional Easy-first Parsing
a brown fox jumped with joy
a brown joywith
joy
fox
a brown
9
ATTACHRIGHT(𝑖)
ATTACHLEFT(𝑖)
Iteratively take actions until a complete tree is built.
23. Three new actions to repair errors
SUBSTITUTE (𝑤%) replaces a token to another
(grammatically more probable) token
DELETE (𝑤%) removes an unnecessary token
INSERT (𝑤%) inserts a new token at an index i.
22
24. Three new actions to repair errors
I look in forward xhearx from you
I youyou
23
25. I look in forward xhearx from you
I youyou
Three new actions to repair errors 24
ATTACHRIGHT
ATTACHLEFT
26. I look in forward xhearx from you
I youyou
Three new actions to repair errors 25
SUBSTITUTE / DELETE / INSERT
27. ATTACHRIGHT
I look in forward xhearx from you
I youyou
Three new actions to repair errors 26
28. I look in forward xhearx from you
I youyou
Three new actions to repair errors 27
29. ATTACHLEFT
I look in forward xhearx from you
I youyou
Three new actions to repair errors 28
30. Three new actions to repair errors
I look in forward xhearx from you
I youyou
29
31. Three new actions to repair errors
SUBSTITUTE
I look in forward xhearx from you
I youyou
30
32. Three new actions to repair errors
I look in forward hearing from you
I youyou
31
33. Three new actions to repair errors
DELETE
I look in forward hearing from you
I youyou
32
34. Three new actions to repair errors
I look forward hearing from from you
I youyou
33
35. Three new actions to repair errors
INSERT
I look forward hearing from from you
I youyou
34
36. Three new actions to repair errors
I look forward to hearing from you
I youyou
35
37. Three new actions to repair errors
ATTACHLEFT
I look forward to hearing from you
I youyou
36
38. Three new actions to repair errors
I look look to hearing from you
I youyouI forward
37
39. We are ready to parse noisy texts … ?
Wait!! The new actions may cause infinite loops.
SUB à SUB à SUB à …
INS à DEL à INS à DEL à ...
38
40. We are ready to parse noisy texts … ?
Wait!! The new actions may cause infinite loops.
SUB à SUB à SUB à …
INS à DEL à INS à DEL à ...
Heuristic constraints to avoid infinite loops
1. Limiting the number of new action operations
2. Substituted token cannot be substituted again
39
41. Training the parser 40
Model learns which action to take at each time step.
structured perceptron + learning with exploration
(Goldberg and Nivre, 2013)
features: basic linguistic features
(Goldberg and Elhadad 2010)
42. Training the parser 41
How to know which action is good (i.e., oracle, valid)?
ATTACHLEFT & ATTACHRIGHT (Goldberg and Elhadad, 2010)
1. proposed edge is in the gold parse and
2. the child (to be attached) already has all its children
SUBSTITUTE, DELETE, & INSERT
3. proposed action decreases the (word) edit distance
to the gold (grammatical) sentence.
46. Experiment 2 (real data)
Grammaticality improvement on real ESL corpus
Treebank of Learner English (Berzak et al., 2016)
Grammaticality score (Heilman et al., 2014)
Regression model with linguistic features
1 (incomprehensible) ~ 4 (perfect)
45
48. Summary
Error-repair Dependency Parsing
1. Non-directional Easy-first Parsing
2. Three new actions to repair errors
Experimental results
1. more robust against grammatical errors
2. improves grammaticality
47
I look in forward xhearx from you
I youyou
