MT Study SCFG
•
0 likes•750 views
This document discusses tree-based machine translation using synchronous context-free grammar (SCFG). SCFG is an extension of context-free grammar that can model correspondences between sentences in two languages. The document explains the formal definition of SCFG, provides examples of rewrite rules and derivations, and discusses some key characteristics like the inability to binarize higher-rank rules. It also outlines the contents to be covered, including training SCFG models.
More Related Content
Viewers also liked
Viewers also liked (12)
Similar to MT Study SCFG
Similar to MT Study SCFG (20)
Recently uploaded
Recently uploaded (20)
MT Study SCFG
- 1. Tree-‐Based Machine Transla0on Synchronous Context-‐Free Grammar Introduced by Akiva Miura, AHC-‐Lab 2015/06/18 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 1 MT Study Group
- 2. Contents 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 2 6.2 Synchronous Context-‐Free Grammar 6.2.1 Characteris0cs 6.2.2 Training 6.2.3 Syntac0c Labels 6.2.4 Features 6.2.5 Decoding
- 3. SCFG 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 3 Synchronous Context-‐Free Grammar (SCFG): • bilingual extension of CFG • can be applied for machine transla0on by source language side parsing (transducing)
- 4. Formalism 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 4 SCFG is defined as: where: G = N Σ Δ R A N Σ Δ R A
- 5. Rewrite Rules 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 5 * → α β φ ∈ R , α ( ) 1* 1 1 β ( ) 1* 1 φ : 1 1 1* 1 : α β → α β φ → α β , 1 1 1 : α β
- 6. Rules Example 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 6 Example of rewrite rules: S → <NP1 が VP2, NP1 VP2> VP → <NP1 を V2, V2 NP1> VP → <PP1 V2, V2 PP1> VP → <NP1 V2, V2 NP1> PP → <NP1 の P2, P2 NP1> NP → <NP1 の NP2, NP2 of NP1> V → <開けた, opened> |<座った,sat> P → <上に, on> NP → <犬, the dog> | <ドア, the door> | <本, the book> | <上に, the upper>
- 7. Deriva0on Example 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 7 Example of deriva0on: <S1, S1> ⇒ <NP2 が VP3, NP2 VP3> ⇒ <犬 が VP3, the dog VP3> ⇒ <犬 が NP4 を V5, the dog V5 NP4> ⇒ <犬 が ドア を V5, the dog V5 the door> ⇒ <犬 が ドア を 開けた, the dog opened the door>
- 8. Parse Tree Example 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 8 Example of deriva0on trees: 犬 NP2 が NP3 S1 NP4 を V5 ドア 開けた the dog NP2 VP3 S1 V5 NP4 opened the door
- 9. Contents 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 9 6.2 Synchronous Context-‐Free Grammar 6.2.1 Characteris0cs 6.2.2 Training 6.2.3 Syntac0c Labels 6.2.4 Features 6.2.5 Decoding
- 10. Normal Form 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 10 • SCFG has almost the same characteris0cs with CFG, but does not have normal form Explana0on: rank : # of non-‐terminals in the right part of rule binariza0on : conversion of rules with rank >= 3 to rules with rank <= 2 Any CFG can be converted to Chomsky Normal Form, but SCFG can’t
- 11. Binariza0on of Rank-‐3 Rules 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 11 • Any Rank-‐3 SCFG rule can be binarized: e.g. X → <A1 B2 C3, C3 B2 A1> introducing new non-‐terminal X’ X → <X’ 1 C2, C2 X’ 1> X’ → <A1 B2, B2 A1>
- 12. Binariza0on of Rank-‐4 Rules 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 12 • Not all rank-‐4 SCFG rules can be binarized: e.g. X → <A1 B2 C3 D4, C3 A1 D4 B2> X → <A1 B2 C3 D4, B2 D4 A1 C3> A1 X B2 C3 D4 C3 A1 D4 B2 X A1 X B2 C3 D4 B2 D4 A1 C3 X these are called “inside-out”
- 13. Rela0on of Grammar Ranks 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 13 • r-‐CFG is set of languages produced by rank-‐r rules • Any r-‐CFG can be converted to equivalent 2-‐CFG Ø 1-‐CFG ⊊ 2-‐CFG = 3-‐CFG = 4-‐CFG = … = r-‐CFG • r-‐SCFG is set of language pairs produced by rank-‐r rules • 3-‐SCFG can be converted to equivalent 2-‐SCFG • r-‐SCFG (r ≧ 4) can not be banarized Ø 1-‐SCFG ⊊ 2-‐SCFG = 3-‐SCFG ⊊ 4-‐SCFG ⊊ … ⊊ r-‐SCFG
- 14. Contents 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 14 6.2 Synchronous Context-‐Free Grammar 6.2.1 Characteris0cs 6.2.2 Training 6.2.3 Syntac0c Labels 6.2.4 Features 6.2.5 Decoding
- 15. Training 15/06/18 Automa0c training of synchronous rules: 彼1 は2 近 い3 う ち4 に5 国 会6 を7 解 散8 す る9 he1 ■ will2 disolve3 ■ ■ the4 ■ diet5 ■ in6 ■ the7 near8 ■ ■ future9 ■ ■ Word Alignment 近 い3 う ち4 に5 国 会6 を7 解 散8 す る9 disolve3 ■ ■ the4 ■ diet5 ■ in6 ■ the7 near8 ■ ■ future9 ■ ■ X1 に5 X2 解 散8 す る9 dissolve3 ■ ■ X2 ■ in6 ■ the7 X1 ■ Phrase Extraction ↑ Synchronous Rule Extraction →
- 16. Rule Extrac0on 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 16 These rules are extracted hierarchically, then called “Hierarchical Phrases/Rules” (Hiero) .21 2 ,. 1 1 1 .,1 1 .1. R ← ∅ 2 . ∈ Φ R ← R ∪ →{ } 2 1: → α β . . ∈ Φ α = α α β = β β R ← R ∪ → α α β β{ } 2 R = R ∈ ∪
- 17. Rule Restric0on 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 17 • Hierarchical rule extraction method is exhaustive, then the trained grammar will be oversized and very ambiguous! Ø need to limit the rules: • minimal phrase pairs for the same alignment • span length limitation (e.g. 2 ≦ length ≦ 10) • rule length limitation (e.g. length ≦ 5) • rank of rules (rank ≦ 2) • prohibition of contiguous non-‐‑‒terminals (X1 X2) • including at least 1 word alignment
- 18. Glue Rules 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 18 • Because of the span length limitation, the grammars might be impossible to cover long sentences. Ø introducing heuristically initial synchronous rules called “gleu rules”: S → <S1 X2, S1 X2> S → <X1, X1> • for long distance reordering (such as En↔Ja), we can introduce also: S → <S1 X2, X2 S1>
- 19. Contents 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 19 6.2 Synchronous Context-‐Free Grammar 6.2.1 Characteris0cs 6.2.2 Training 6.2.3 Syntac0c Labels 6.2.4 Features 6.2.5 Decoding
- 20. Syntac0c Labels 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 20 • In standard Hiero rules, using only 2 non-‐terminals: S, X • s0ll very ambiguous (might be slow and inaccurate) Ø introducing syntac0c labels from parse tree 近 い3 う ち4 に5 国 会6 を7 解 散8 す る9 disolve3 ■ ■ the4 ■ diet5 ■ in6 ■ the7 near8 ■ ■ future9 ■ ■ NP PP NP VP IN+DT VP/PP VPVB
- 21. Contents 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 21 6.2 Synchronous Context-‐Free Grammar 6.2.1 Characteris0cs 6.2.2 Training 6.2.3 Syntac0c Labels 6.2.4 Features 6.2.5 Decoding
- 22. Features 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 22 • Decoding with SCFG also uses log linear model, and the features are almost the same with PBMT • If phrase pairs include non-‐terminals, count of phrases is not 1 per occurrence, but normalized by number of matched rules • Addi0onal penal0es: • rule count penalty: • glue rule count penalty: = − = − ∈ ∧ ∈ R{ }
- 23. Contents 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 23 6.2 Synchronous Context-‐Free Grammar 6.2.1 Characteris0cs 6.2.2 Training 6.2.3 Syntac0c Labels 6.2.4 Features 6.2.5 Decoding
- 24. Decoding 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 24 • SCFG decoding maximizes the viterbi deriva0on with linear combina0on of the features: = () , = () , ',* ω ( )( )∑ ',* ω ( )( )∑ ≈ () , ∈D G = = ω ( )
- 25. Transla0on Forest 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 25 • Example of decoding: ⽝犬0,1 が1,2 本2,3 の3,4 座った5,6 上に4,5 NP0,1 VP2,6 S0,6 PP2,5 NP2,5 NP2,3 P4,5 NP4,5 V5,6 the dog sat the upper on the book NP0,1 V5,6 NP4,5 P4,5 NP2,3 PP2,5 of NP2,5 S0,6 ↑ Source language side syntax parsing Target language side transla0on forest ↑
- 26. End Slide 15/06/18 2015©Akiva Miura AHC-‐Lab, IS, NAIST 26