CS571: Natural Language Processing https://github.com/emory-courses/cs571

1. Natural Language Processing
Emory University
Jinho D. Choi
Tree-Adjoining
Grammar

2. Mildly Context-Sensitive Grammar
• Why another grammar?
- Context-free grammar cannot capture all linguistic phenomena.
- Context-sensitive grammar takes exponential time for parsing.
• Mildly context-sensitive grammar
- Can capture all linguistic phenomena in CFG and more.
- Can be parsed in polynomial time.
- Grows linearly as the string length grows.
• Formalisms
- Tree-adjoining grammars (TAG).
- Combinatory categorial grammars (CCG).
2

3. Tree-Adjoining Grammars
• Tree generating system
- What are the elementary objects of CFG?
- What are the elementary objects of TAG?
3
S, NP, VP, N, V, D,
I, bought, a, car
buy a car
ND
V
N
NP
VP
NP VP
will
V
S
buy
V NP1↓
VPNP0↓
S
car
ND↓
NP
a
D
VP*
VP
will
V
N
NP
← Strings ↑ Trees

4. Tree-Adjoining Grammars
• G = (N, Σ, S, I, A)
- N : a finite set of non-terminals.
- Σ : a finite set of terminals.
- S : a start symbol representing the whole sentence, where S ∈ N.
- I : a finite set of initial trees.
- A : a finite set of auxiliary trees.
4
Same as CFG
No production rules?
Substitution & Adjunction
Elementary trees

5. Elementary Tree
• Elementary trees (initial trees ∪ auxiliary trees)
- Interior nodes are labeled by non-terminals.
- Nodes on the frontier are labeled by terminals or non-terminals.
- Non-terminal nodes on the frontier are marked for substitution (↓).
• The foot node in an auxiliary tree is marked for adjunction (*).
• The foot node must have the same label as the root node.
5
VP*
VP
will
V
buy
V NP1↓
VPNP0↓
S
car
ND↓
NP
a
D
N
NP
Initial trees Auxiliary tree

6. Derivation
• Substitution
- Substitute X↓ with a tree whose root node is labeled by X.
- Only trees derived from initial trees can be substituted.
6
buy
V NP1↓
VPNP0↓
S
car
ND↓
NP
a
DN
NP
VP*
VP
will
V
car
ND
NP
a
Which NP?
car
ND
NP
a
V
VP
S
buy
N
NP

7. Derivation
• Adjunction
- Adjoin X* with a tree containing a non-substitution node labeled by X.
- Any adjunction on X↓ is not allowed.
7
car
ND
NP
a
V
VP
S
buy
N
NP VP*
VP
will
VVP
S
N
NP
N VP*
NP VP
will
V
S
car
ND
NP
a
V
VP
buy buy a car
ND
V
N
NP
VP
NP VP
will
V
S

8. Derivation Tree
• Derivation Tree
- Derived tree: a tree composed by two other trees.
- A tree showing how a derived tree was constructed.
8
buy
V NP1↓
VPNP0↓
S
αbuy
car
ND↓
NP
αcar
N
NP
αJohn
a
D
αa
VP*
VP
will
V
αwill
12
2.21
What kind of tree is this?

9. Lexicalization
• Lexicalized grammar
- A finite set of structures each associated with a lexical item (anchor).
- Operations composing the structures.
9
Design elementary trees using only substitutions
bought a car
ND
V
N
NP
VP
NP VP
almost
AP
S
A
by taking
1.The leftmost lexical item
2.The rightmost lexical item
3.The head lexical item
as the anchor of each phrase.

10. Lexicalization with Substitution
10
VP↓NP↓
S
N
VP↓
VP
almost
AP
A
bought
V NP↓
VP
a
N↓D
NP
car
N
Leftmost
lexical item
N
NP
almost
AP
A
bought
V
a
D
car
ND↓
V↓ NP
VP
NP↓ VP
AP↓
S
Rightmost
lexical item
bought
V NP↓
VP
NP↓ VP
AP↓
S
N
NP
almost
AP
A
car
ND↓
NP
a
D
Head
lexical item
Which gives the most meaningful trees?
Is this ideal?

11. Lexicalization with Adjunction
11
John bought a car John almost bought a car John bought a car too
bought
V NP↓
VP
S
NP↓
bought
V NP↓
VP
NP↓ VP
AP↓
S
bought
V NP↓
VP
NP↓
S
VP
AP↓
What should be included in the domain of locality for “bought”?
VP*
VP
A
AP
almost
AP
VP
A
VP*
toobought
V NP↓
VP
S
NP↓

12. Domain of Locality
• CFG vs TAG
- CFG, the domain of locality is constrained to direct children.
- TAG, the domain of locality can be extended much more.
• Reading presentation
- “Domain of Locality”,Aravind Joshi, 2004.
- Presented by Kate Silverstein.
• Next week’s readings
- “Synchronous Tree Adjoining Grammars”.
- Shieber & Schabes, 1990.
12

13. Exercises
• Wh-question
- What did John buy for Mary
• Relative clause
- John bought a car which Mary wanted
• Light-verb construction
- John made a bid for the car
• Verb-particle construction
- Mary threw the car away
13

14. Wh-Question
14
V
NP VP
S
S
V
buy
NP
!
PP
P NP
for
N
Mary
N
Johndid
NP
S
N
What
“What did John buy for Mary”
V
NP↓ VP
S
S
NP↓
buy
NP
!
PP↓
NP
N
John
PP
P NP↓
for
S
V S*
did
NP
N
Wh
NP
N
Mary

15. Relative Clause
15
“John bought a car which Mary wanted”
a
D
bought
V NP↓
VP
S
NP↓
NP
N
John car
ND↓
NP NP
N
Mary NP↓ S
S
NP
NP*
NP↓ VP
V NP
!wanted
NP
N
which
NP S
S
NP
NP VP
V NP
!wanted
N
Mary
N
whicha
ND
NP
car
VP
V
bought
NP
John
N
S
Could it be adjoined earlier?

16. Light-Verb Construction
16
“John made a bid for the car”
made
V NP
VP
NP↓
S
VP
PP↓
D↓ N
bid
NP
N
John
PP
P NP↓
for car
ND↓
NP
a
D
the
D
made
V NP
VP
NP
S
VP
PP
D N
bid
P NP
for car
ND
theaJohn
N

17. Verb-Particle Construction
17
“Mary threw the car away”
threw
V NP↓
VP
S
NP↓
P
away
NP
N
Mary
the
D
car
ND↓
NP
threw
V NP
VP
S
NP
P
awayMary the car
N
ND