TITLE:
Fluctuation-Model-Based Discrete Probability Estimation for Small Samples
AUTHORS:
Takashi Isozaki
KEYWORDS:
Probability Estimation, Fluctuation Models, Helmholtz Free Energy, Canonical Distributions
JOURNAL NAME:
Open Journal of Statistics,
Vol.5 No.5,
August
20,
2015
ABSTRACT: A robust method is proposed for estimating discrete probability functions for small samples. The proposed approach introduces and minimizes a parameterized objective function that is analogous to free energy functions in statistical physics. A key feature of the method is a model of the parameter that controls the trade-off between likelihood and robustness in response to the degree of fluctuation. The method thus does not require the value of the parameter to be manually selected. It is proved that the estimator approaches the maximum likelihood estimator at the asymptotic limit. The effectiveness of the method in terms of robustness is demonstrated by experimental studies on point estimation for probability distributions with various entropies.