In the field of data analysis, measurements are primarily evaluated using two distinct schools of thought: frequentist or classical statistics and Bayesian statistics. This section provides a comparative overview of these two fundamental approaches.

According to [12], classical statistics is characterized by the following principles:

At a given point x i , frequent measurements y i j are carried out where j denotes the measurement series. Thereby, a probability distribution (e.g., a Gaussian distribution) for y i can be estimated for each measurement point i . In classical statistics, a distinction is made between the true value of the model parameter and its estimated value obtained from sample data. While the true model parameter is treated as an a-priori unknown fixed constant, the estimator (the specific value derived from regression analysis) is a random variable with a distribution induced by the sampling process. Consequently, a confidence interval is determined based on the sampling distribution of this estimator, providing a plausible range for the true parameter.

A central method in classical regression analysis is the determination of a curve y = f ( x ) that approximates n measurement points ( x i , y i ) . Measurement uncertainties are considered by including weighting factors in the sum of squared residuals Z lsq :

where w i is the weighting factor of the i -th measurement point and ξ → 1 are the parameters of the model function f ( x i , ξ → 1 ) . The fitting parameters are obtained by minimizing Z lsq , which represents the weighted classical least squares method. In contrast, Bayesian statistics interprets probabilities as measures of belief or uncertainty. In general, the probability P ( B | A ) is the conditional probability of event B given that event A is true. Prior probabilities P ( A ) for an event A express beliefs before observing data. This prior probability is updated by the likelihood P ( B | A ) via Bayes’ theorem (see, e.g., [12] and [13]) and yields the posterior probability P ( A | B ) , which represents updated beliefs after observing event B . The posterior probability is normalized by the evidence P ( B ) (or marginal likelihood):

In this work, Bayes’ theorem is used to infer the model parameters ξ → by determining the conditional probability P ( ξ → | ( x i , y i ) , M ) , where ( x i , y i ) are the observed data and M denotes the underlying model hypothesis. In classical statistics, model parameters are fixed, but unknown constants. In Bayesian statistics, in contrast, model parameters are described by probability density functions rather than being treated as fixed constants. Thus, the model curve is known within a credible interval. Whereas in classical statistics measurements are frequently repeated to determine the uncertainty of each measurement point, the measurement is assumed to be certain in conventional Bayesian statistics.

This paper will investigate the regression problem through a novel Bayesian approach employing a global minimization strategy. We find a measure for unknown systematic errors, regardless of whether they originate from the experiment or the model, by using the residuals in a Bayesian framework. This analysis will reveal that this method leads to a fundamental extension of the classical least-squares sum by introducing an additional significant term.

The determination of weighting factors is a central topic in this study, as introduced above. In a common standard approach, often employed when repeated j measurements are available for each x i , the weighting factor w i ( x i ) for the data point i is determined as follows: Repeated measurements j produce a number of j values y i at x i , the sample variance s i 2 ( x i ) of these repeated measurements at x i is calculated; the reciprocal of this variance is then used as

the weighting factor: w i ( x i ) = 1 s i 2 ( x i ) , see [10].

In the frequentist statistical approach it is implicitly assumed that the model is structurally perfect. The residuals, representing the deviations between the model and the measurements are solely dependent on the fluctuations in the data points, but do not account for systematic errors. In practice, it is often the case that different models describe the measurement data with varying degrees of accuracy, and it is usually not known a priori which model will be the most accurate. In addition to possibly occurring fluctuations in data points, unknown systematic errors may be caused by a too simple model, see [1]. These unknown systematic errors lead to significantly larger residuals than the variability of the data points alone would suggest. These considerations motivate us to assign lower weights to regions with higher residuals.

However, a more advanced regression analysis with observation weights not known a priori is known as iteratively reweighted least squares (IRLS) method [9]. The weights are iteratively optimized in each step. The regression is successful when the resulting parameter estimates do not differ within a certain threshold from those obtained in the previous iteration. This involves using the residuals from the weighted fit to re-estimate the variance (or standard deviation) function and update the weights accordingly. In practice, one or two iterations are usually sufficient to achieve stable estimates of the parameters. The IRLS method is a robust regression technique, but it has some limitations:

The method can be prone to convergency problems and may end up at a local minimum instead of the global minimum. Since IRLS is primarily an optimization method, it also lacks a comprehensive statistical framework for determining the probability distributions of uncertainty parameters.

In order to resolve these limitations, we decided to develop a Bayesian informed extended least squares method. The weighting factors are modeled as functions dependent on a set of weight parameters ξ → 2 , which are optimized simultaneously with the primary model parameters ξ → 1 . This comes at the cost that the dimensionality of the optimization space is increased by the length of the vector ξ → 2 of the weight parameters. In contrast to the iteratively reweighted least squares (IRLS) method, our proposed procedure is straightforward and non-iterative. More details are provided in the following section.

The extended least squares method is derived by applying Bayes’ theorem [14] to jointly infer the model and the weight parameters. The parameter set ξ → is composed of the primary model parameters ξ → 1 and the weight parameters ξ → 2 . The prior probability distribution P ( ξ → | M ) contains prior knowledge of the parameter set ξ → , where M denotes the hypothesis space. Upon availability of a new measurement series, denoted as D = { ( x i , y i ) | i = 1 , ⋯ , n } , the prior distribution is updated via the likelihood function P ( D | ξ → , M ) . This likelihood quantifies how well the parameter vector ξ → explains the observed data, i.e., it reflects the goodness of fit between model and measurements. The joint probability of the data and parameters is given by the product of the likelihood function and the prior distribution, i.e. P ( D | ξ → , M ) ⋅ P ( ξ → | M ) . The evidence P ( D | M ) serves as a normalizing constant for this joint probability. The posterior probability distribution P ( ξ → | D , M ) is then obtained as:

which can be rewritten in logarithmic form:

Since the hypothesis space M of the model is assumed to be constant, the distribution P ( D | M ) , named evidence, also remains constant, see also [12]. In the case of a uniform prior distribution, a so-called “flat” prior, compare [12], where all admissible parameter values are considered equally probable, the prior simplifies to: P ( ξ → | M ) = 1 . Under this assumption, Equation (3) reduces to Equation (5), implying that the posterior distribution P ( ξ → | D , M ) is proportional to the likelihood function P ( D | ξ → , M ) [15],

Assuming a flat prior distribution, where all parameter values have equal prior probabilities, we can directly compare the likelihood of different parameter sets with the observed data. This allows us to focus on evaluating the goodness-of-fit and determining the most plausible parameter values based on the given data,

In the following step, a model function y c is defined which describes the data set D . Then, we define the likelihood as a Gaussian function G i , where y i , y c , i and σ i are the measured y -value, the calculated regression value and the standard deviation at measurement point i , respectively:

The standard deviations σ i are a priori unknowns and must be replaced by the estimated values s i = s i ( y c , i , ξ → 2 ) during regression. The calculated value at at each measurement point D i ( x i , y i ) is denoted as y c , i = y c ( x i , ξ → 1 ) . In particular, we express the likelihood function G i ( y c ( x i , ξ → 1 ) , s i ) at each measurement point i :

The regression values y c , i = y c ( x i , ξ → 1 ) are computed from the model function at the positions x i and depend on the primary model parameters ξ → 1 .

In this approximation, we assume that all individual observations are independent. This means that the likelihood function P ( D | ξ → , M ) for the full dataset D = { ( x i , y i ) | i = 1 , ⋯ , n } can be written as a product of the individual likelihoods P i at each point:

In classical least squares regression, the s i values usually reflect only the measurement uncertainty at each x i , determined from repeated measurements in a frequentist framework. However, in this work, we take a different approach: we let s i account not only for the measurement uncertainty of data point i , but also for discrepancies caused by an imperfect model. These additional contributions are derived from the residuals ( y i − y c , i ) .

The estimated standard deviations s i are directly related to the classical weighting factors w i through:

The weighting factor w i , as employed in our framework, aligns with its definition in the classical least squares method, as discussed by Wolberg (2006) [16]. However, in our approach the standard deviations of the measurement points i are a priori uncertain.

Instead of assigning an individual standard deviation s i to each measurement point ( x i , y i ) , we introduce a single function s c ( y c , ξ → 2 ) that depends on a reduced set of fitting parameters ξ → 2 . The individual variances s i 2 are thus approximated by model-based variances:

where y c , i is the model prediction at x i .

Alternatively, the calculated value y c , i ( ξ → 1 ) can be replaced by the measured value y i , yielding:

The parameters ξ → = { ξ → 1 , ξ → 2 } are treated as random variables within the hypothesis space M . Here, ξ → 1 denotes the parameters of the model function, while ξ → 2 parameterizes the function s c , which describes the (unknown) standard deviation associated with each data point. Both sets of parameters are estimated simultaneously within the Bayesian framework. It is worth noting that the fitted s c term absorbs both measurement noise and model misspecification. These two components are in principle not separately identifiable from a single measurement series unless external information or replicated measurements are available.

The likelihood function

is modified to account for the parameter-dependent variances. Assuming independent normally distributed residuals, it takes the form:

A direct numerical evaluation of Equation (12) is often numerically unstable. This is because the product of exponential terms can rapidly tend towards zero for large negative exponents, potentially leading to severe numerical underflow. To circumvent this, the logarithm of the likelihood function provides a more stable and computationally feasible formulation:

Inserting Equation (13) into Bayes’ theorem, Equation (4) yields the following expression for the posterior distribution:

The posterior distribution for a flat prior is obtained by setting P ( { ξ → 1 , ξ → 2 } | M ) = 1 in Equation (14):

In order to apply our method to applications the following workflow can be used:

The model function y c , i is described by the parameters ξ → 1 within the hypothesis space M . The weights w i are approximated by the function for the inverse of the model based variances s c 2 with the parameters ξ → 2 , which are defined in the hypothesis space M . The prior knowledge is considered by a prior probability distribution. The probability distributions of ξ → 1 and ξ → 2 are determined by using Equation (14).

In the subsequent section, an objective function is derived from the posterior distribution given by Equation (15). This objective function serves as the basis for parameter estimation in the proposed method. Subsequently, it is compared to the classical objective function used in conventional least squares regression.

Rearranging Equation (14) yields:

The objective function Z is defined by multiplying both sides of Equation (16) by (−2), so that this objective function becomes closer to the classical least squares objective function. The Bayesian based problem of maximizing the log-posterior distribution is converted into a problem where the objective function Z is minimized.

The final, simplified expression (the third line) is the formula that is numerically minimized to obtain the Maximum A Posteriori (MAP) estimates [17] for the parameters { ξ → 1 , ξ → 2 } .

For a non-informative or (flat) prior distribution, where the prior probability density P ( ξ → | M ) = 1 , its logarithm ln P ( ξ → | M ) becomes zero. Therefore, the last term in the last line of Equation (17) vanishes. The simplified objective function Z 1 thus reduces to (see also Equation (5)):

This objective function is equivalent to the negative log-likelihood of the model. Therefore, minimizing Z 1 is equivalent to finding the maximum likelihood estimate, which in this case also corresponds to the maximum a posteriori (MAP) solution [18].

The model parameters ξ → 1 and the weight parameters ξ → 2 of the function s c , i are optimized by minimizing the objective function Z 1 .

Let us recall that the objective function Z l s q from the least squares method is given by

Therefore, compared with Z lsq (Equation (19)), Z 1 (Equation (18)) contains the additional term

This term penalizes high standard deviations s c , i by increasing the value of the objective function. It is important to note that when each s c , i is independent of the model parameters ξ → 1 and ξ → 2 (as typically assumed in classical weighted least squares, where uncertainties are fixed based on observed data only), this sum becomes a constant additive term in Z 1 . In such a case, minimizing Z lsq and Z 1 yields the identical optimized parameters ξ → 1 . Thus, our Bayesian informed extended least squares algorithm simplifies to the classical least squares solution in the case of weights that do not depend on model ξ → 1 or the weight parameters ξ → 2 .

This is precisely where the advantage of our method lies: the weights may depend on the parameters ξ → 2 of the weight function and on the parameters ξ → 1 of the model function and can be determined from the simultaneous optimization of ξ → 1 and ξ → 2 .

To determine the uncertainties of the fit parameters [19], the objective function Z can be locally approximated by a second-order Taylor series expansion around its minimum. The minimum of the objective function Z , Equation (17) is denoted as ξ → 0 , in which all first partial derivatives are zero. The parameter vector ξ → includes both the model parameters ξ → 1 and the parameters ξ → 2 of the function s c , i . The second-order Taylor series expansion can thus be written as:

where H ↔ is the Hessian matrix [11] with elements given by:

Consequently, the objective function Z , when approximated by a second-order Taylor expansion around its minimum (cf. Equation (20)), forms an N -dimensional paraboloid in the parameter space.

The covariance matrix Σ ↔ between the element j of the vector ξ → and the element k of the vector ξ → is given by:

The off-diagonal elements j ≠ k are frequently defined as σ j k = Σ j k (see e.g., [16]).

The diagonal elements Σ j j of the covariance matrix Σ ↔ are denoted as the variances σ j 2 :

The correlation between parameters is estimated from the 2^nd order Taylor term according to [16]:

The correlation coefficient ρ j k describes the degree of correlation between two parameters. The values of ρ j k are between −1 to +1. If the value is about zero, the parameters are uncorrelated. For ρ j k close to 1 the parameters are completely correlated, in case of ρ j k close to −1, the parameters are completely anti-correlated. Values of exactly ±1 imply a perfect linear relationship.

The relationship between the posterior probability distribution from Bayes’ theorem and the objective function of the extended least squares method follows from the first line of Equation (17):

The posterior probability distribution P can be approximated by an n -dimensional Gaussian function. This is achieved by approximating the objective function Z ( ξ → ) by a second-order Taylor series expansion.

and

The corresponding probability density function P in Equation (27) is represented by an n -dimensional Gaussian function. It is clear that the Gaussian function can only approximate the solution, since the probability distributions of variances σ k 2 follow a scaled χ 2 -distribution (a special case of the Γ-distribution) and thus deviate from the Gaussian shape.

In this first example, we test the algorithm on a synthetic dataset of eight elements, indexed i = 1 , 2 , ⋯ , 8 . The dataset and detailed calculations are provided in the accompanying GitHub repository [20]. The noise-free values, y i 0 , are set to a constant mean of μ = 1.0 . A normally distributed noise component, y ′ i , is then superimposed using a random number generator, yielding the “experimental” data y i :

The true standard deviation of this noise is set to σ = 0.2 for all data points. Consequently, the observed data y i are drawn from a normal distribution y i ~ N ( μ = 1 , σ 2 = 0.04 ) . The generated data are presented in Figure 1.

Figure 1. Synthetic data y i with superimposed artificial noise ( σ = 0.2 ). The green crosses represent the noise-free (true) values, y i 0 . The arrows illustrate the magnitude and sign of the residuals ( y i − y i 0 ) .

Given the dataset depicted in Figure 1, it is assumed that the true (noise-free) value of each signal is y i 0 = 1 . The model function y c ( x i ) = 1 is chosen. Since the arguments of this function always equal the constant 1, the model parameter vector ξ → 1 contains no adjustable parameters for this specific example (i.e., ξ → 1 = {   } ).

The objective is to estimate the standard deviation s from the “true”, but a-priori unknown standard deviation σ of the noise from the data. Within our Bayesian framework, this is achieved by determining the posterior probability distribution P ( s | D , M ) for the estimated standard deviation s (cf. Equation (15)).

It is assumed that the prior distribution is the same for all s > 0 , and therefore the logarithm of the prior distribution vanishes in this function ln P ( s | D , M ) . The probability density P ( s | D , M ) is calculated for the unknown estimated standard deviation s c , i in the hypothesis space M = ℝ + , with ξ → 2 = { s } and s c , i = s ∈ M :

with the evidence K = P ( { ( x 1 , y 1 ) , ( x 2 , y 2 ) , ⋯ , ( x 8 , y 8 ) } | M ) as a normalization factor.

The posterior probability density P is plotted versus the parameter s as can be seen in Figure 2. The normalization constant K is computed numerically, so that the integral of the posterior probability density over all possible values of s equals 1. In this example, K = 0.2074 .

Figure 2. Directly calculated probability density P as function of the parameter s .

The maximum of the probability density P can be found at s max = 0.235 . This value s max is the estimated standard deviation based on the Bayesian approach.

The directly calculated probability density P is now compared to the results obtained from classical statistics. The standard deviation s classic from classical statistics ([16]) follows from

This result s classic = 0.235 is identical to the value s max = 0.235 obtained from the Bayesian analysis. Since in this case the average μ is known and equals unity, all n = 8 degrees of freedom are maintained, and no degree of freedom is used to calculate the average. Consequently, the denominator of Equation (29) uses n = 8 instead of the typical n − 1 = 7 degrees of freedom. The inflection point on the left side of the maximum s max of the probability density P curve is located at s left = 0.179 , the inflection point on the right side of the maximum lies at s right = 0.291 as depicted in Figure 2.

The “left error” of the estimated value s max is determined to: Δ s left = 0.235 − 0.179 = 0.056

and its “right error” is calculated to: Δ s right = 0.291 − 0.235 = 0.056 . As the result of the Bayesian approach, the probability Φ ip , B between the two inflection points s = 0.179 and s = 0.291 (i.e. the integral over the probability density function P , blue area in Figure 2) is calculated to: Φ ip , B = 0.577 .

We approximate ln P ( s ) by a truncated Taylor series at s = s max as expansion point by considering the 2^nd derivative, since the 1^st derivative is zero at s max . The resulting approximation of P ( s ) is the Gaussian function depicted in Figure 3 (red curve). For comparison, the directly calculated probability density function is also shown in the figure as black curve. The probability Φ ip , B = 0.577 following from the directly calculated distribution (Figure 2) is compared to the probability Φ ip , G = 0.682 between the inflection points, in the case that the standard deviation is approximated by a Gaussian distribution (red curve in Figure 3).

Figure 3. Approximated probability density of the truncated Taylor series (red curve) in comparison with the originally calculated probability P as a function of s (black curve).

The error estimation via Bayesian statistics described above is now compared to the approximation in classical error estimation using the extended objective function, see also Equation (13):

The estimated error Δ s of the standard deviation s max is deduced from the inverse of the Hessian matrix H ↔ . In this example H 1 , 1 is the only non-zero second derivative of the objective function and the estimated error follows as Δ s = ± 0.042 . Comparing Gaussian Δ s with the error Δ s = ± 0.056 obtained from the Bayes’ treatment—for both inflection points, the left and the right one—shows that the values are rather different, however, in the same order of magnitude.

As a third error estimation, the error Δ s of the estimated standard deviation s max follows from the approximation suggested by Lane [21]:

For our example, this yields:

This estimated error Δ s = 0.059 is compared to the values determined from the inflection points ( Δ s = 0.056 ) and from the second derivative ( Δ s = 0.042 ), highlighting the differences between these different methods of uncertainty estimation. The calculation using Maple version 18 [22] can be found in GitHub.

The classical Gaussian distribution is inherently a symmetric distribution with non-zero values in the negative s -range. However, the propability density must be zero for values of s < 0 since a negative standard deviation is not possible. This is the case for the probability density function used in our approach, which can be seen as an advantage compared to the classical usage of a Gaussian distribution.

The same y i values from example 1 are used as measurement points in example 2, see Figure 1. In contrast to example 1, where the model curve was known to be y c , i = 1 , the model for this example is described by a constant but a priori unknown parameter, y c , i = y average . Both this value y average and the standard deviation s c , i = s are estimated simultaneously by analyzing the measurement points y i .

The hypothesis space M is defined as M = { ( y average , s ) ; y average ∈ ℝ , s > 0 } = ℝ × ℝ + .

The posterior probability density P of the a priori unknown value ξ → 1 = { y average } and its unknown standard deviation ξ → 2 = { s } are determined from Equation (15):

In this example, n = 8 . The evidence K = P ( { y i } i = 1 n | M ) serves as a normalization factor, and its value is found to be K = 0.0632 . The resulting probability density, which depends on both y average and s , is presented in Figure 4. The calculation is done using Maple version 18 [22]. The Maple code can be found in the accompanying GitHub repository [20].

The maximum of the posterior probability density is reached at y average = 0.949 and at a standard deviation s = 0.231 . These values are the maximum a posteriori (MAP) estimates for the parameters, as depicted in Figure 4.

The value y average , classic obtained using classical statistics is the sample mean:

Figure 4. Probability density P as a function of the average y average and standard deviation s .

In this example, the standard deviation s classic from classical statistics is given by the sample standard deviation formula:

The use of ( n − 1 ) degrees of freedom in the denominator is necessary because the value of y average was estimated from the data itself, using one degree of freedom.

The objective function Z 1 from Equation (18) applied to example 2 is

The Hessian matrix H ↔ of the objective function Z 1 (Equation (17)) is computed with respect to the parameters y average and s :

The covariance matrix Σ ↔ , which is the inverse of the Hessian matrix H ↔ , is given by:

The error of the average is Δ y average = s 1 2 = s 1 = 0.058 ; and the error of the standard deviation is Δ s = s 2 2 = s 2 = 0.041 . The correlation ρ y average , s between the estimated average y average and the estimated standard deviation s is then calculated as:

This value of ρ y average , s = 0.045 , being close to zero, indicates that no significant linear correlation exists between the two parameters.

In summary, the estimated average is y average = 0.949 ± 0.058 , and the standard deviation is s = 0.231 ± 0.048 .

The regression of datasets with unknown uncertainties of the individual measurement points is a challenging problem. In our example 3, polynomial functions are used to fit data over a wide range. It is inherit in this problem that the values of the a-priori unknown weights of the data points can vary over several orders of magnitude. It illustrates how our extended least squares technique provides a robust uncertainty estimation even under conditions of strongly varying weighting factors. It becomes clear that constant weighting factors would not properly capture the regression problem.

The data set used in this third example are synthetically generated. We use the following linear model:

Here, x i spans the range [ − 10 , 85 ] with a step size of 5, resulting in n = 20 generated values. The measurement error ε i assumed follows a centered Gaussian distribution with a standard deviation σ i that deliberately depends on the expected measurement value, given by:

This choice for σ i intentionally introduces a heteroskedastic error structure (i.e., a variance of measurement errors that is not constant across all measurements), where uncertainties increase with the magnitude of the measured value. This is a characteristic often observed in real-world experimental data and is a crucial aspect because standard least squares methods often assume homoskedasticity (i.e., constant error variance across all measurements), which can lead to inefficient parameter estimates and unreliable uncertainty quantification when applied to such heteroskedastic data.

This generated set of n data pairs ( x i , y i ) is fitted using a linear function y c , i = a 0 + a 1 x , where y c , i denotes the regression value at x = x i :

As already mentioned a key aspect of our approach is that the parameters of the linear fit ( a 0 , a 1 ) and the parameters of the uncertainty model ( s 0 , s 1 ) are estimated simultaneously within a single optimization process. The standard deviations s c , i of the measured data points y i are modeled as a function of the fitted values y c , i according to:

Here, s 0 is the intercept and represents the estimated standard deviation at y c , i = 0 . The parameter s 1 is the slope and characterizes the linear increase of the estimated standard deviation as the absolute value of y c , i increases. Both s 0 and s 1 are positive parameters, i.e., s 0 , s 1 ∈ ℝ + , which are obtained during the optimization process as described in Equation (18). The overall hypothesis space M for this fit is thus defined by: a 0 , a 1 ∈ ℝ and s 0 , s 1 ∈ ℝ + . The formulations for y c , i , Equation (34), and s c , i , Equation (35), are denoted as model 1.

It is worth noting that the standard deviation parameter s 0 from Equation (35) must be positive and does not go to zero. This ensures that both the weighting factors and the natural logarithm ln ( s ) of the standard deviation remain finite.

The parameters a 0 , a 1 , s 0 , s 1 (which define the models in Equations (34) and (35) are determined by minimizing the objective function Z 1 (Equation (18)), as summarized in Table 1. This function represents the negative logarithm of the joint posterior probability density of the parameters, making the minimization equivalent to finding MAP estimation. It is important to note that even though a linear regression model is used, the objective function Z 1 is nonlinear due to the simultaneous estimation of the uncertainty parameters. The estimated parameter values a 0 , a 1 , s 0 , and s 1 result from this optimization procedure, and their uncertainties are derived from the diagonal entries of the inverse Hessian matrix, Equation (23).

Table 1. Estimated parameter values and one- σ uncertainties for model 1.

The fitted linear model (red dashed line) and the generated test data set (black dots) show good agreement, as illustrated in Figure 5.

Figure 5. The generated test dataset (black points) is compared to the fitted linear model (red dashed line) from model 1, which represents the most probable course of the data based on our analysis.

The correlations ρ i j between the model parameters are calculated by means of the covariance matrix Σ ↔ according to Equation (24) and are listed in Table 2.

Table 2. Correlation matrix ρ i j between model parameters a 0 , a 1 and the Gaussian uncertainties s 0 , s 1 of model 1.

The regression parameters for the intercept a 0 and the slope a 1 are significantly negatively correlated with each other ( ρ a 0 , a 1 = − 0.522 ), as are the uncertainty parameters s 0 and s 1 ( ρ s 0 , s 1 = − 0.647 ).

The uncertainty of the standard deviation curve, Δ s c is obtained by propagating the parameter uncertainties, which are derived from the inverse Hessian matrix [23]. In the Bayesian context of our analysis, this propagation provides the credible intervals for the model curve. This approach, while mathematically similar to error propagation in classical statistics, allows for a direct interpretation of the parameter uncertainties as probability distributions,

It is shown in Figure 6 that the determined residuals are consistent with the modeled standard deviation curve ± s c , i.e. with the highest probability density for the errors (see red curves in Figure 6). The uncertainty of the highest probability density of the errors ± s c is depicted by a red shaded regions.

Figure 6. The residuals of model 1 are plotted versus the independent variable x . The red line represents the most probable course of the modeled standard deviation s c , which is the MAP estimation. The shaded red regions represent the credible interval (or ± 1 σ ) for the standard deviation curve, illustrating its uncertainty.

The large relative uncertainty of the intercept parameter, a 0 , combined with the strong anti-correlations between several model parameters, suggests that the four-parameter model is over-parameterized. This indicates a potential for overfitting, where the model is too complex for the given data. Consequently, while the model has a high explanatory power for the training data, this comes at the cost of a significant loss of predictive power for new, unseen data.

A simpler model with fewer optimization parameters may therefore be more appropriate. Since the uncertainty parameter s 0 must be retained to prevent a singularity in the weighting coefficient w i (Equation (10)) for y c = 0 , the model’s intercept parameter, a 0 , is the logical choice to be excluded. In example 4, we investigate the implications of this model deficiency by applying a fitting function without the constant intercept term to the same test data. This new fitting approach intentionally deviates from the created data set, introducing a known systematic error due to the model’s simplified form.

The effect of an unknown systematic error is examined in example 4 (model 2). For this purpose the test data from example 3 are used (black points in Equation (5)). The systematic error is caused by skipping the parameter a 0 (see Table 3) of example 3. Thus, the following model function, Equation (37) is applied to the test data:

The Gaussian uncertainty s c , i of the data points is estimated with the same equation as for example 3, i.e. Equation (38):

The hypothesis space M is defined as: a 1 ∈ ℝ , s 0 and s 1 ∈ ℝ + . The formulations for y c , i , Equation (37), and s c , i , Equation (38), are denoted as model 2.

The fit procedure described here results in the following values for the model parameters and their uncertainties (Table 3):

Table 3. Estimated parameter values and one- σ uncertainties for model 2.

The residuals are calculated as the difference of the values from model 2 with the fit parameters from Table 3 and the test data. The results are depicted in Figure 7. It is worth mentioning that the residuals are higher than those obtained in example 3 (compare Figure 6) in particular for small x -values. However, only one primary model parameter is engaged in this model (example 4), i.e. the slope a 1 .

Figure 7. The residuals of model 1 are plotted versus x with an unknown systematic error due to model simplification and the course of the modeled standard deviation is represented by red pluses connected with a red line (the red regions represents the uncertainties Δ s i ).

The residuals in Figure 7 are influenced by two components: the inherent noise of the measurement and a systematic error introduced by not considering a possible non-zero axis intercept within the model. As expected, the sum of the unweighted residual squares for model 2:

is higher than for model 1, where a value of 57.9 is obtained. Consequently, the curve of the calculated standard deviation s c deviates more strongly from zero over the entire range for model 2 than for model 1, as shown in Figure 6 and Figure 7.

The comparison of model 1 and model 2 highlights a key challenge in model selection: choosing the most appropriate model. When choosing between models, a good strategy is to apply Occam’s razor, using the simplest possible model that fits the data adequately. This principle suggests that the three-parameter model 2 might be a better choice than the more complex four-parameter model 1, especially if the advantages of its simplicity, such as being less prone to overfitting, compensate for the introduced systematic error.

The correlations ρ i j between the model parameters are calculated by means of the covariance matrix Σ ↔ according to Equation (24) and are provided in Table 4.

Table 4. Correlation matrix ρ i j between the model parameter a 1 and the Gaussian uncertainties s 0 and s 1 of model 2.

The regression parameter a 1 shows no significant correlation with the uncertainty parameters s 0 and s 1 ( ρ a 1 , s 0 = − 0.086 , ρ a 1 , s 1 = 0.036 ). This indicates that the values of a 1 are largely independent of the uncertainty parameters. A strong negative correlation between s 0 and s 1 is still present, however, similar to model 1.

The results of example 4 demonstrate that the uncertainty of the parameter s 1 is in the same order of magnitude as the parameter itself. In addition a strong anti-correlation between s 0 and s 1 occurs, ρ s 0 , s 1 = − 0.681 . Thus, the number of fit parameters has to be further reduced. The same model function, Equation (39) as used above is again applied to the test data:

However, the Gaussian uncertainty s of the data points is now the same constant for all measuring points i.e.

The hypothesis space M is defined as: a 1 ∈ ℝ and s 0 ∈ ℝ + .

The optimized fitting parameters are summarized in Table 5. The standard deviations ± s c are constant for the whole measurement region, see Figure 8. Thus, the uncertainty of the standard deviation is also constant (red shaded regions in Figure 8).

Table 5. Estimated parameter values and one- σ uncertainties for example 5.

The correlation coefficient ρ a 1 , s 0 is now close to zero, ρ a 1 , s 0 = 4.61 ⋅ 10 − 10 , i.e. there is no correlation between a 1 and s 0 (see Table 6).

Table 6. Correlation matrix ρ i j between the model parameter a 1 and the gaussian uncertainty s 0 of example 5.

The residuals and the course of the standard deviation for example 5 are plotted in Figure 8. Since both, the model function and the model of the Gaussian uncertainty are strongly simplified (the remaining parameters are s 0 and a 1 only), systematic errors occur. The strong simplification of the model to only two parameters ( a 1 and s 0 ) introduces systematic errors, leading to higher residuals. However, these drawbacks are acceptable as the primary objective, the reduction of parameter correlation, is successfully achieved. The result is a robust model in which the standard deviation is sufficiently described and the correlation between the parameters is negligibly small.

Figure 8. The residuals of example 5 are plotted against the independent variable x . The course of the modeled standard deviation is represented by red pluses connected with a dashed line, and the red regions represent the uncertainties Δ s i .

In classical statistics the weighting factors are calculated from a multitude of measurements. In our approach the weighting factors are estimated from the residuals. Thereby, measurement uncertainties and errors due to the arbitrary choice of the model are both taken into account.

As a final example, we demonstrate the application of our extended least squares method to a materials science problem. We examine the temperature-dependent change in dislocation density ρ in martensite during heating for the Böhler steel grade, N404 [24]. The dislocation density ρ in martensite during heat treatment is evaluated from in-situ X-ray diffraction data by means of a Rietveld method by using the Double-Voigt model [25] considering the peak broadening caused by the instrument itself and by lattice strain. The data for each heat treatment are monitored in a single in-situ experiment and each point of the dislocation density ρ characterized by time and temperature corresponds to a diffractogram. Details with respect to the evaluation of ρ can be found in [26].

Its chemical composition is provided in Table 7. To ensure full reproducibility, the complete Python implementation for model fitting is available in the accompanying GitHub repository [20].

The mechanical properties of this steel, particularly tensile strength and ductility, are strongly dependent on the dislocation density ρ , which is influenced by the final heat treatment. In-situ XRD measurements were conducted to investigate different heat treatments [26], from which the dislocation density ρ in martensite was determined, as shown in Figure 9.

Table 7. Chemical composition of the chromium stainless steel, showing mass fractions (wt%) of the main alloying elements. The remainder is iron (Fe).

Figure 9. Experimentally determined and modeled dislocation density during the heating in a tempering process. Uncertainties ( ± 1 s c ) are estimated by the linear function yielding only positive values for the standard deviation.

In crystalline materials, the reduction of dislocation density during heat treatment is controlled by thermally activated mechanisms such as annihilation, which are described by Arrhenius-type behavior. As temperature increases, the mobility of dislocations rises exponentially, leading to a rapid decrease in dislocation density once a characteristic activation temperature is reached.

Such processes typically exhibit three regimes:

a low-temperature regime with negligible changes. a transition region with a strong temperature dependence. a high-temperature regime where the system approaches a saturated state.

The sigmoidal Boltzmann function was selected based on physical considerations of thermally activated processes governing dislocation annihilation.

The sigmoidal Boltzmann function is given by:

with ρ 1 and ρ 2 representing the initial and final dislocation density, respectively. The parameters ϑ 0 and Δ ϑ are the center temperature and temperature change. The slope ( d ρ / d ϑ ) | ϑ 0 at the center temperature ϑ 0 is given by:

However, it should be noted that the underlying microstructural evolution may involve multiple mechanisms (e.g., recovery, rearrangement, and annihilation of dislocations), each characterized by different activation energies and kinetics. As a result, the actual process cannot be fully described by a single activation energy or a single rate-controlling mechanism.

The adopted sigmoidal function should therefore be interpreted as an effective, phenomenological description that captures the overall transition behavior. In particular, the parameter Δ ϑ can be understood as reflecting a distribution of activation energies or overlapping processes occurring over a finite temperature range.

The Boltzmann function seems to provide a physically motivated and sufficiently flexible approximation of the experimentally observed behavior.

The choice of the uncertainty function is primarily guided by statistical considerations. In particular, the residuals normalized by the standard error, i.e., ( ρ i − ρ ( ϑ i ) ) / s c ( ϑ i ) , should be approximately Gaussian distributed with zero mean and unit variance. This requirement ensures the consistency of the underlying Gaussian likelihood assumption and avoids systematic over- or underestimation of the uncertainty.

In addition, the dislocation density must be a non-negative quantity. Therefore, the model for its uncertainty must ensure that the corresponding credible intervals do not extend into physically meaningless, negative values.

We evaluated several functions for the standard error s c . A simple approach is equal weighting: