This document discusses statistical machine translation (SMT). It provides an overview of key concepts in SMT including word and sentence alignment, probability, language models, and translation models. The fundamental equation of SMT is presented as using a translation model to find the best target language words for the input and a language model to build up a sentence from those words. Basic SMT architecture and the Bayes' theorem and chain rule for calculating probabilities are also covered.

1. STATISTICAL MACHINE
TRANSLATION
Presented by Asim Ullah Jan

2. Agenda
• What is Statistical Machine Translation?
• How to built an STM System?
• Sentence alignment
• Word alignment
What is Probibility?
History of Probability
Conditional Probability

3. Agenda (Cont..)
Language Model
Translation model
SMT Basic Architecture
Bayes Theorm
Chain Rule

4. SMT Model
• Fundimental Equation of Machine Translation
• P(E|F) = argmax P(F|E).P(E)
• Translation model: Find best target-language word or phrase for each input unit.
Result: a bag of words/phases.
• P(F|E) :-Probability that the input French gives this particular sentence
of English
• Language Model: Pick one word/phrase at a time from the bag of words/phrases
and build up a sentence.
• P(E) :-Probability governing how the English words are properly composed

5. Basic SMT Architecture

6. Baye’s Rule
𝑃 𝐴, 𝐵 = 𝑃(𝐵) ∙ 𝑃 𝐴 𝐵
𝑃 𝐴, 𝐵 = 𝑃(𝐴) ∙ 𝑃 𝐵 𝐴
𝑃(𝐵) ∙ 𝑃 𝐴 𝐵 = 𝑃 𝐴 ∙ 𝑃 𝐵 𝐴
𝑃 𝐴 𝐵 =
𝑃 𝐴 ∙ 𝑃 𝐵 𝐴
𝑃(𝐵)
probability of A given B is equal to the probability of A, multiplied by the probability of B given
A, divided by the probability of B.
Thomas Bayes
1701 – 1761

7. Probabilistic Language Modeling
Goal: compute the probability of a sentence or sequence of words:
P(W) = P(w1,w2,w3,w4,w5…wn)
• Related task: probability of an upcoming word:
P(w5|w1,w2,w3,w4)
• A model that computes either of these:
P(W) or P(wn|w1,w2…wn-1) is called a language model.

8. How to compute P(W)
• How to compute this joint probability:
• P(its, water, is, so, transparent, that)
• Intuition: let’s rely on the Chain Rule of Probability

9. The Chain Rule
• Recall the definition of conditional probabilities
More variables:
P(A,B,C,D) = P(A)P(B|A)P(C|A,B)P(D|A,B,C)
• The Chain Rule in General
P(x1,x2,x3,…,xn) = P(x1)P(x2|x1)P(x3|x1,x2)…P(xn|x1,…,xn-1)

10. The Chain Rule applied to compute joint probability
of words in sentence
P(“its water is so transparent”) =
P(its) × P(water|its) × P(is|its water)
× P(so|its water is) × P(transparent|its water is so)
 
i
iin wwwwPwwwP )|()( 12121 