Upcoming SlideShare
×

Language Modeling in Turner&Charniak (2007)

402
-1

Published on

Class presentation for a seminar on computational approaches to functional elements.

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• this application can be great for arabic language.not only sentence but alphabet as event.
cordialement

Are you sure you want to  Yes  No
• Be the first to like this

Views
Total Views
402
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
13
1
Likes
0
Embeds 0
No embeds

No notes for slide

Language Modeling in Turner&Charniak (2007)

1. 1. Language Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Language Modeling in Charniak’s LM Determiner Turner&Charniak (2007) Selection Method Results Reasons for Success References Kilian Evang 2009-11-30
2. 2. Language Recap: Language Models Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Charniak’s LM ◮ LMs assign probabilities to sentences Determiner Selection ◮ a sentence is a complex event Method Results ◮ LMs break it up into a sequence of “atomic” events Reasons for Success References ◮ each “atomic” event conditioned on certain previous events ◮ conditional probabilities approximated by counting and smoothing
3. 3. Language N-gram LMs Modeling in Turner&Charniak (2007) Kilian Evang Language Models n-gram LMs Charniak’s LM N-gram LMs Charniak’s LM sequence represents sentence Determiner Selection p(sent) = p(seq) Method events are words, Results Reasons for Success end symbols References conditioned on the n − 1 previous events
4. 4. Language A Sentence – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Sentence Charniak’s LM Determiner Selection Method Results Reasons for Success put the ball in the box References Event sequence put, the, ball, in, the, box, ∆
5. 5. Language A Sentence – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Sentence Charniak’s LM Determiner Selection Method Results Reasons for Success put the ball in the box References Conditional probability p(wi = the|wi −2 = ball, wi −1 = in)
6. 6. Language N-gram LMs vs. Charniak’s Parsing LM Modeling in Turner&Charniak (2007) Kilian Evang n-gram LMs Charniak’s LM Language Models N-gram LMs sequence represents sentence parse tree Charniak’s LM p(sent) = p(seq) p(seq) Determiner Selection seq Method events are words, pre-terminals, Results Reasons for Success end symbols terminals, References constituents, end symbols conditioned on the n − 1 certain previous previous events, depending events on type
7. 7. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree vp Language Models N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box
8. 8. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree vp Language Models N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Event sequence verb, put, M, ∆, M, np, pp, ∆, noun, ball, M, det, ∆, M, ∆, the, prep, in, M, ∆, M, np, ∆, noun, box, M, det, ∆, M, ∆, the
9. 9. Language Digression: Non-head Constituents Modeling in Turner&Charniak (2007) Kilian Evang Language Models Tree fragment N-gram LMs Charniak’s LM l Determiner Selection Method Results Lm ... L1 t R1 ... Rn Reasons for Success References h Event sequence fragment M, L1 , . . ., Lm , ∆, M, R1 , . . ., Rn , ∆
10. 10. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree Language Models vp N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Conditional probability for a head pre-terminal p(t = noun|l = np, m = vp, u = verb, i = put)
11. 11. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree Language Models vp N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Conditional probability for a head terminal p(h = ball|t = noun, l = np, m = vp, u = verb, i = put)
12. 12. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Parse tree Kilian Evang vp Language Models N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Conditional probability for a non-head constituent p(Li = det|Li −1 = M, h = ball, t = noun, l = np, m = vp, u = verb)