Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
×

# Language Modeling in Turner&Charniak (2007)

598 views

Published on

Class presentation for a seminar on computational approaches to functional elements.

Published in: Technology
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• this application can be great for arabic language.not only sentence but alphabet as event.
cordialement

Are you sure you want to  Yes  No
Your message goes here
• Be the first to like this

### Language Modeling in Turner&Charniak (2007)

1. 1. Language Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Language Modeling in Charniak’s LM Determiner Turner&Charniak (2007) Selection Method Results Reasons for Success References Kilian Evang 2009-11-30
2. 2. Language Recap: Language Models Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Charniak’s LM ◮ LMs assign probabilities to sentences Determiner Selection ◮ a sentence is a complex event Method Results ◮ LMs break it up into a sequence of “atomic” events Reasons for Success References ◮ each “atomic” event conditioned on certain previous events ◮ conditional probabilities approximated by counting and smoothing
3. 3. Language N-gram LMs Modeling in Turner&Charniak (2007) Kilian Evang Language Models n-gram LMs Charniak’s LM N-gram LMs Charniak’s LM sequence represents sentence Determiner Selection p(sent) = p(seq) Method events are words, Results Reasons for Success end symbols References conditioned on the n − 1 previous events
4. 4. Language A Sentence – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Sentence Charniak’s LM Determiner Selection Method Results Reasons for Success put the ball in the box References Event sequence put, the, ball, in, the, box, ∆
5. 5. Language A Sentence – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Language Models N-gram LMs Sentence Charniak’s LM Determiner Selection Method Results Reasons for Success put the ball in the box References Conditional probability p(wi = the|wi −2 = ball, wi −1 = in)
6. 6. Language N-gram LMs vs. Charniak’s Parsing LM Modeling in Turner&Charniak (2007) Kilian Evang n-gram LMs Charniak’s LM Language Models N-gram LMs sequence represents sentence parse tree Charniak’s LM p(sent) = p(seq) p(seq) Determiner Selection seq Method events are words, pre-terminals, Results Reasons for Success end symbols terminals, References constituents, end symbols conditioned on the n − 1 certain previous previous events, depending events on type
7. 7. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree vp Language Models N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box
8. 8. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree vp Language Models N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Event sequence verb, put, M, ∆, M, np, pp, ∆, noun, ball, M, det, ∆, M, ∆, the, prep, in, M, ∆, M, np, ∆, noun, box, M, det, ∆, M, ∆, the
9. 9. Language Digression: Non-head Constituents Modeling in Turner&Charniak (2007) Kilian Evang Language Models Tree fragment N-gram LMs Charniak’s LM l Determiner Selection Method Results Lm ... L1 t R1 ... Rn Reasons for Success References h Event sequence fragment M, L1 , . . ., Lm , ∆, M, R1 , . . ., Rn , ∆
10. 10. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree Language Models vp N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Conditional probability for a head pre-terminal p(t = noun|l = np, m = vp, u = verb, i = put)
11. 11. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Kilian Evang Parse tree Language Models vp N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Conditional probability for a head terminal p(h = ball|t = noun, l = np, m = vp, u = verb, i = put)
12. 12. Language A Parse Tree – a Sequence of Events Modeling in Turner&Charniak (2007) Parse tree Kilian Evang vp Language Models N-gram LMs Charniak’s LM Determiner np pp Selection Method Results Reasons for Success np References verb det noun prep det noun put the ball in the box Conditional probability for a non-head constituent p(Li = det|Li −1 = M, h = ball, t = noun, l = np, m = vp, u = verb)