Maximum Likelihood Estimation, Hidden and Latent Variables, Expectation-Maximization, EM for Naive Bayes

1. Machine
Learning
for
Language
Technology
Lecture
6:
Hidden
Variables
and
Expecta6on
Maximiza6on
(EM)
Marina
San6ni
Department
of
Linguis6cs
and
Philology
Uppsala
University,
Uppsala,
Sweden
Autumn
2014
Acknowledgement:
Thanks
to
Prof.
Joakim
Nivre
for
course
design
and
materials

2. Repe66on:
Baysian
Approach
to
Classifica6on

3. MLE

4. Hidden
and
Latent
Variables

5. Expecta6on-­‐Maximaza6on
(1)

6. Expecta6on-­‐Maximaza6on
(2)

7. Expecta6on-­‐Maximiza6on
(3)

8. Expecta6on-­‐Maximiza6on
(4)

9. Naive
Bayes
Revised
• Equa6ons
are
ok

10. EM
for
Naive
Bayes

11. A
Simple
Example

12. Supervised
Learning

13. Unsupervised
Learning

14. First
Guess

15. First
E-­‐Step

16. First
M-­‐Step

17. Second
E-­‐Step

18. Second
M-­‐Step

19. Third
E-­‐Step

20. Third
M-­‐Step

21. Fourth
E-­‐Step

22. Fourth
M-­‐Step
-­‐
Convergence

23. Uses
of
EM

24. The
end