Consider a situation in which some quantity of interest, μ, is to be estimated from a sample of observations that can be regarded as realizations from some unknown probability distribution, and that in order to do so, it is necessary to specify a model for the distribution. There are usually many alternative plausible models available and, in general, they each lead to different estimates of μ. Consider a sample of data, x, and a set of K models containing the true model M_t. Each M_k consists of a family of distributions, where represents a parameter (or vector of parameters). The prior probability that M_k is the true model is denoted by and the prior distribution of the parameters of M_k (given that M_k is true) by. Conditioning on the data x and integrating out the parameter, one obtains the following posterior model probabilities:

where

is the integrated likelihood under M_k. If is a discrete distribution, the integral in (3) is replaced by a sum.

Bayesian model selection involves selecting the “best” model with some selection criterion; more often the Bayesian information criterion (BIC), also known as the Schwarz criterion [10] is used; it is an asymptotic approximation of the log posterior odds when the prior odds are all equal. More information on Bayesian model selection and applications can be found in Nguefack-Tsague and Ingo [11] , Guan and Stephens [12] , Nguefack- Tsague [13] , Carvalho and Scott [14] , Fridley [15] , Robert [16] , Liang et al. [17] , and Bernado and Smith [18] . Other variants of model selection include Nguefack-Tsague and Ingo [11] who used BMA machinery to derive a focused Bayesian information criterion (FoBMA) which selects different models for different purposes, i.e. their method depends on the parameter singled out for inferences.

Let μ be a quantity of interest depending on x, for example a future observation from the same process that generated x. The idea is to use a weighted average of the estimates of μ obtained using each of the alternative models, rather than the estimate obtained using any single model. More precisely, the posterior distribution of μ is given by

Note that is a weighted average of the posterior distributions, , where the k-th weight, , is the posterior probability that M_k is the true model.

The posterior distribution of μ, conditioned on M_k being true, is given by

The posterior mean and posterior variance are given by

A classical reference is Hoeting et al. [19] with an extensive framework of BMA methodology and applications for different statistical models. Various real data and simulation studies have investigated the predictive performance of BMA (Clyde [20] ; Clyde and George [21] ). A discussion on the issue of using BMA for dealing with model uncertainty is given in Clyde and George [21] . Nguefack-Tsague [13] uses BMA in the context of estimating a multivariate mean. Others references on BMA include Marty et al. [22] for reliable ensemble forecasting, Simmons et al. [23] for benchmark dose estimation, Fan and Wang [24] (Autoregressive regression models), Corani and Mignatti [25] (presence-absence data), Tsiotas [26] (quantile regression), Baran [27] (truncated normal components), Lenkoski et al. [28] (endogenous variables), Fan et al. [29] (regression models), Madadgar [30] (integration of copulas), Koop et al. [31] (instrumental variables), and Clyde et al. [32] (variable selection).

Clyde and Iversen [33] developed a variant of BMA in which it is not assumed that the true model belongs to competing ones (M-open framework). They developed an optimal weighted scheme and showed that their method provides accurate predictions than any of the proxy models.

An R [34] package for BMA is now available for computational purposes; this package provides ways for carrying out BMA for linear regression, generalized linear models, and survival analysis using Cox proportional hazard models. For computations, Monte Carlo methods, or approximating methods, are used; thus many BMA applications are based on the BIC. As one can realize in deriving BMA, there are no unique statistical model and unique prior distribution associated with BMA, taught these are available for each competing model. This renders frequentist properties of BMA hard to obtain from pure Bayesian decision theory. This was the main motivation of this paper in proposing alternative Bayesian model in which the long run properties of resulted estimators could be automatically obtained from Bayesian decision theory. The present paper is organized as follows. Section 2 introduces the new BMA method; Section 3 provides practical examples while Section 4 provides discussions. The paper ends with concluding remarks.

The purpose of this section is to define a new BMA method. The prior of the quantity of interest can be defined as

where is the prior distribution of M_k.

The parametric statistical model can also be defined as

with being the parametric statistical model for model M_k (i.e. the sampling distribution of M_k). The use of Bayes rule leads to the posterior of the quantity of interest as

Defining a loss function, Bayesian estimates are then obtained with its long and short run properties known. All the frequentist properties of Bayes rules now apply, in particular one can find conditions under which there are consistent and admissible. This approach is referred to as Mixed based Bayesian model averaging (MBMA).

Proposition 1. Under (8) and (9), assuming that for all k and j, ,

, the posterior of the quantity of interest in (10) is

Proof.

Since, by Bayes rule,

Dividing (a) by (b) yields the result.

Corollary 2. Suppose that all the models have identical sampling distribution, that is for all k and j, then MBMA reduces to BMA.

Proof.

In the numerator of (11),

The numerator of (11) is therefore

.

Therefore, the denominator of (11),

a mixture of marginal distributions.

Therefore

Thus in this special case, the posteriors mean and variance using the MBMA are those of BMA given in Equations (6) and (7).

Evaluating the long run properties of MBMA involves studying frequentist issues, including: asymptotic methods, consistency, efficiency, unbiasedness, and admissibility. Details about derivations for more general Bayes estimates can be found e.g. in Gelman [35] (p. 83). The following are proven in Gelman [35] for any Bayes estimate, in particular for MBMA. Let be the Fisher information, the observed information, Mod the posterior mode; and μ₀ the value of the parameter that makes the model distribution closest (e.g. in the sense of Kullback-Leiber information) to the true distribution.

1) If the sample size is large and the posterior distribution is unimodal and roughly symmetric, one can approximate it by a normal distribution centered at Mod with variance.

2) If the likelihood () is a continuous function of μ and the true parameter value μ₀ is not on the boundary of the parameter space, as the sample size n tends to ∞, the posterior distribution of μ approaches normality with mean μ₀ and variance and Mod is consistent for μ₀.

3) Suppose the normal approximation for the posterior distribution (), Mod ® μ₀ and the true data distribution is included in the class of models, then, where I is the identity matrix.

4) When the truth is included in the family of models () being fitted, the posteriors mode, mean and median are consistent and asymptotically unbiased and efficient under mild regularity conditions.

5) If a prior distribution () is strictly positive with finite Bayes risk and the risk function is continuous, MBMA is admissible.

One measure of predictive performance is the Good’s logarithm score rule [36] . From the nonnegativity of Kullback-Leiber information divergence, it follows that if f and g two probabilities distribution functions,

Applying this to MBMA leads to

MBMA provides thus better predictive performance than any single model.