Recent Changes - Search:


edit SideBar


Minimum Chi-Square, Not Maximum Likelihood!

J. Berkson, Annals of Statistics, Vol 8, 3, 457-487, 1980.

Although many believe Likelihood constitutes a fundamental basis for analysis, Berkson makes the case that it is the Chi-square variable which seems to be absolutely related to a meaningful goodness of fit for a model. While we can argue that Chi-square is only valid for Gaussian distributions, applying the conventional definition of Likelihood to Gaussian data does not generate a Chi-square variable (where are the log normalisation terms?).

The observation that the Likelihood function in common use cannot be directly related to a meaningful expression of probability of the data (due to a missing interval scale which should be related to data accuracy) may seem like nit-picking, until we begin to try to analyse data for which this interval would be expected to change. We have encountered this situation many times in our research.

My comment would be that Likelihood is generally introduced with the intention of using density functions, this hides any direct link to the probability of obtaining the data. If we try to maintain this link and use Likelihood so that the implied intervals are specified according to measurement accuracy of the data (in effect a meaningful quantization of the data) then the outcome for Gaussian data is in fact a Chi-square. Further, this issue seems to be related to why Expectation Maximisation works, and why we cannot obtain an unbiased estimate of the variance for a data sample with simple Likelihood. In both cases, changes in the estimated distribution width should have changed the implied interval terms which have been assumed constant. See for example Tina memo 2004-005.

Of course this all comes down to how you believe Likelihood should be defined to begin with. You may feel that there is no issue at all here based upon your understanding of Likelihood methodology. We can only apply our general observation and say we side with Berkson's opinion of what generally is done. Apparently the paper went through some rigerous reviewing.

NAT 1/2/2013

Edit - History - Print - Recent Changes - Search
Page last modified on February 07, 2013, at 11:54 AM