EM Algorithm (March 15, 2004).

I received an email question about the EM Algorithm. This is a computational approach that works well for missing data problems and data models with latent (unobserved) variabels. The basic approach is to estimate the missing or latent data (E-step), compute maximum likelihood estimates that incorporates the missing/latent estimates (M-step), then update the missing or latent data (E-step) and so forth. There's a book by McLachlan and Krishnan, The EM Algorithm and Extensions, that I have not seen, but which sounds pretty good. There are also a few good web sites about this algorithm.

The Expectation Maximization Algorithm [pdf]. Dellaert F, Georgia Institute of Technology. Accessed on 2004-03-15. www.cc.gatech.edu/~dellaert/em-paper.pdf

The EM Algorithm and its Extensions. Bell Laboratories. Accessed on 2004-03-15. cm.bell-labs.com/cm/ms/departments/sia/project/em/

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models [pdf]. Bilmes JA, U.C. Berkeley. Accessed on 2004-03-15. www.vision.ethz.ch/ml/slides/em_tutorial.pdf

If I get some time, I will show a simple example on my web pages.

07/08/2008. Category: Statistical computing