The newly developed “2^nd-BERRU-PM” methodology [1] [2] can incorporate arbitrarily high sensitivities, thereby vastly generalizing extant data adjustment [3] [4] and data assimilation [5] [6] [7] methodologies. The essential contributions of the second- and higher-order sensitivities for reducing predicted uncertainties in the model response has been illustrated in [8] [9] [10] by applying the 2^nd-BERRU-PM methodology [1] [2] to the neutron leakage response of the polyethylene-reflected plutonium (PERP) OECD/NEA reactor physics benchmark [11] which was computed [8] [9] [10] using the neutron transport Boltzmann equation, involving 21,976 imprecisely known parameters, the solution of which is representative of “large-scale computations.”

Although the 2^nd-BERRU-PM methodology can incorporate arbitrarily high-order sensitivities of the system response of interest to the imprecisely known parameters underlying the computational model, this methodology is limited to considering just second-order moments (hence the designation “2^nd-”) of the experimentally measured responses. Also, the “output” produced by the 2^nd-BERRU-PM methodology is limited to yielding optimal best-estimate values for the means and covariances (i.e., the first- and second-moments) of the best-estimate predicted distribution of responses and parameters. Since skewness and kurtosis play an essential role in determining the asymmetries of distributions, it is important to generalize the 2^nd-BERRU-PM methodology to enable the incorporation of third- and fourth-order moments of measured and computed responses, as well as to enable the computation of skewness and kurtosis of the best-estimate predicted posterior distribution of calibrated model parameters and responses. Such a generalization has now been enabled by the development of the closed-form expression of the fourth-order moment-constrained MaxEnt presented in the accompanying Part 1 [12] , which underlies the development of the 4^th-BERRU-PM (“4^th-Order Best-Estimate Results with Reduced Uncertainties Predictive Modeling”) methodology to be presented in this work. The 4^th-BERRU-PM methodology encompasses the following components:

1) a mathematical model of a physical system, comprising linear and/or nonlinear equations that relate the system’s independent variables and parameters to the system’s state (i.e., dependent) variables;

2) arbitrarily high-order sensitivities and moments of the distribution of model parameters, which will however be explicitly used only up to fourth-order in the derivation of the practically applicable end-results produced by the 4^th-BERRU-PM methodology;

3) one or several computational results, customarily referred to as system responses (or objective functions, or indices of performance), which are computed using the mathematical model;

4) the first four moments (mean values, variances/covariances, skewness, and kurtosis) of the distribution of experimentally measured responses.

This work is structured as follows: Section 2 presents the fourth-order moment-constrained MaxEnt probabilistic representation of the joint distribution of model parameters and computed responses. This joint distribution is constructed by incorporating the mean values, variances/covariances, skewness, and kurtosis, of model parameters and computed model responses by following the same procedure as in the accompanying Part 1 [12] . Section 3 presents the mathematical framework of the 4th-BERRU-PM methodology. Section 4 presents an inter-comparison of the 4th-BERRU-PM, 2nd-BERRU-PM, and data assimilation methodologies. Concluding remarks are presented in Section 5.

The mathematical/computational model of a physical system relates independent variables and model parameters to the computed “results of interest”, which are customarily referred to as “model/system responses.” The generic form of a mathematical/model has been used in previous works ( [13] [14] , and references therein) when developing the arbitrarily-high order adjoint sensitivity analysis, and will therefore be summarily presented in Appendix A, for convenient reference. Consider that there are a total number of TR (“Total Responses”) responses of interest, which will be represented mathematically by the “vector or responses” denoted as r ≜ ( r 1 , ⋯ , r T R ) † , where r i denotes the “i^th-response”, i = 1 , ⋯ , T R . Matrices will be denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “ ≜ ” will be used to denote “is defined as” or “is by definition equal to.” Transposition will be indicated by a dagger ( † ) superscript.

As shown in Appendix A, the unknown joint probability distribution of the model parameters and the computed model responses, denoted as p c ( α , r ) , is formally, defined on a domain D ≜ D α ∪ D r , where D α denotes the domain of definition of the parameters and D r denotes the domain of definition of the model responses. Although p c ( α , r ) is unknown, its moments can be computed as shown in Appendix A. Consistent with the fourth-order moment-constrained MaxEnt distribution constructed in Part 1 [12] for the measured responses, the first four moments of the unknown joint probability distribution p c ( α , r ) will also be used for constructing the corresponding fourth-order moment-constrained MaxEnt joint distribution of model parameters and computed responses. The expressions of the first four moments of p c ( α , r ) are presented in Appendix A and are enumerated below, as follows:

(i) The expected/nominal values of the model parameters, denoted as α i 0 ;

(ii) The parameter covariance matrix, denoted as C α α ≜ [ c i j α ] T P × T P , where the

quantity c i j α denotes the covariance of two model parameters, α i and α j , where i , j = 1 , ⋯ , T P ;

(iii) The triple-correlations of three model parameters α i , α j , and α k , denoted as t i j k α , where i , j , k = 1 , ⋯ , T P ;

(iv) The quadruple-correlations of four model parameters α i , α j , α k , and α l , denoted as q i j k l α , where i , j , k , l = 1 , ⋯ , T R ;

(v) The vector of expected values of the computed responses, defined as

E c ( r ) ≜ [ E c ( r 1 ) , ⋯ , E c ( r k ) , ⋯ , E c ( r T R ) ] † , (1)

where E c ( r k ) denotes the expectation value of a computed response r k ( α ) , for k = 1 , ⋯ , T P , the expression of which is presented in Appendix A.

(vi) The parameter-response computed correlation matrix, denoted as C α r c and defined as follows:

C α r c ≜ ( cor ( α 1 , r 1 ) ⋯ cor ( α 1 , r T R ) ⋮ ⋱ ⋮ cor ( α T P , r 1 ) ⋯ cor ( α T P , r T R ) ) , (2)

where cor ( α i , r k ) denotes the correlation between a parameter α i and a computed response r k , for i = 1 , ⋯ , T P and k = 1 , ⋯ , T R , the expression of which is presented in Appendix A.

(vii) The covariance matrix of computed-responses, denoted as C r r c and defined as follows:

C r r c ≜ ( cor ( r 1 , r 1 ) ⋯ cor ( r 1 , r T R ) ⋮ ⋱ ⋮ cor ( r T P , r 1 ) ⋯ cor ( r T P , r T R ) ) (3)

where cov ( r k ,   r k ) denotes the covariance between two computed responses r k and r l , for k , l = 1 , ⋯ , T R , the expression of which is presented in Appendix A.

(viii) The triple correlations, denoted as μ 3 ( r k , r l , r m ) , among three responses, denoted as r k , r l and r m , for k , l , m = 1 , ⋯ , T R , the expression of which is presented in Appendix A.

(ix) The quadruple-correlations of the distribution of responses, denoted as μ 4 ( r k , r l , r m , r n ) , among four responses, among four responses, denoted as r k , r l , r m and r n , for k , l , m , n = 1 , ⋯ , T R , the expression of which is presented in Appendix A.

The maximum entropy (MaxEnt) principle, originally formulated by Jaynes [15] , will be applied to reconstruct from the above-mentioned known moments the “MaxEnt probability density of the joint model parameters and computed responses,” which will be denoted as p c M E ( α , r ) , where the subscript “c” indicates “computational model”, while the superscript “ME” indicates “MaxEnt” approximation. To ensure optimal compatibility with the available information while simultaneously ensuring minimal spurious information content according to the MaxEnt principle, the probability density p c M E ( α , r ) would need to satisfy the following conditions:

1) p c M E ( α , r ) maximizes the Shannon [16] information entropy, S, as defined below:

S = − ∫ D p c M E ( α , r ) ln [ p c M E ( α , r ) ] d α   d r ; (4)

2) p c M E ( α , r ) satisfies the “moments constraints” enumerated in items (i) through (ix) above and defined in Appendix A;

3) p c M E ( α , r ) satisfies the normalization condition:

∫ D p c M E ( α , r ) d α   d r = 1 . (5)

It was shown in Part 1 [12] that attempting to include the triple and quadruple cross-correlations among experimentally-measured responses leads to insurmountable computational difficulties for little, if any, gain in accuracy. For this reason, only the self-correlations of order three and four were included in constructing the fourth-order MaxEnt distribution of the measured responses [12] . The same approximation will be used in constructing the MaxEnt distribution p c M E ( α , r ) of the joint distribution of model parameters and computed responses, namely only the self-correlations of third- and fourth-order will be included. Thus, the MaxEnt distribution p c M E ( α , r ) will be obtained as the solution of the variational problem ∂ H ( p c M E ) / ∂ p c M E = 0 , where the entropy (Lagrangian functional) H ( p c M E ) is defined as follows:

H ( p c M E ) = − ∫ D p c M E ( α , r ) ln [ p c M E ( α , r ) ] d α   d r − λ 0 [ ∫ D p c M E ( α , r ) d α   d r − 1 ]     − ∑ k = 1 T R a k ( 1 ) [ ∫ D ( δ r k ) p c M E ( α , r ) d α   d r ] − ∑ i = 1 T P a i ( 2 ) [ ∫ D ( δ α i ) p c M E ( α , r )   d α   d r ]     − 1 2 ∑ k = 1 T R ∑ l = 1 T R b k l ( 11 ) [ ∫ D ( δ r k ) ( δ r l ) p c M E ( α , r ) d α   d r − cov ( r k ,   r l ) ]     − ∑ k = 1 T R ∑ i = 1 T P b k i ( 12 ) [ ∫ D ( δ α i ) ( δ r k ) p c M E ( α , r ) d α   d r − cor ( α i , r k ) ]

− 1 2 ∑ i = 1 T P ∑ j = 1 T P b i j ( 22 ) [ ∫ D ( δ α i ) ( δ α j ) p c M E ( α , r ) d α   d r − c i j α ] − ∑ k = 1 T R ψ k ( 1 ) [ ∫ D ( δ r k ) 3 p c M E ( α , r ) d α   d r − μ 3 ( r k ) ] − ∑ i = 1 T P ψ i ( 2 ) [ ∫ D ( δ α i ) 3 p c M E ( α , r )   d α   d r − t i α ] − ∑ k = 1 T R χ k ( 1 ) [ ∫ D ( δ r k ) 4 p c M E ( α , r ) d α   d r − μ 4 ( r k ) ] − ∑ i = 1 T P χ i ( 2 ) [ ∫ D ( δ α i ) 4 p c M E ( α , r )   d α   d r − q i α ] . (6)

In Equation (6), the quantities a k ( 1 ) , a i ( 2 ) , b k l ( 11 ) , b k i ( 12 ) , b i j ( 22 ) , ψ k ( 1 ) , ψ i ( 2 ) , χ k ( 1 ) , and χ i ( 2 ) denote the respective Lagrange multipliers, and the factors 1/2 have been introduced for subsequent computational convenience.

Solving the equation ∂ H ( p c M E ) / ∂ p c M E = 0 yields the following expression for the resulting MaxEnt distribution p c M E ( α , r ) :

p c M E ( α , r ) = Z c − 1 φ ( α , r ) ;           Z c ≜ ∫ D φ ( α , r ) d α   d r   ; (7)

where:

φ ( α , r ) ≜ exp { − ∑ k = 1 T R a k ( 1 ) ( δ r k ) − ∑ i = 1 T P a i ( 2 ) ( δ α i ) − 1 2 ∑ k = 1 T R ∑ l = 1 T R b k l ( 11 ) ( δ r k ) ( δ r l )     − ∑ k = 1 T R ∑ i = 1 T P b k i ( 12 ) ( δ α i ) ( δ r k ) − 1 2 ∑ i = 1 T P ∑ j = 1 T P b i j ( 22 ) ( δ α i ) ( δ α j ) − ∑ k = 1 T R ψ k ( 1 ) ( δ r k ) 3

    − ∑ i = 1 T P ψ i ( 2 ) ( δ α i ) 3 − ∑ k = 1 T R χ k ( 1 ) ( δ r k ) 4 − ∑ i = 1 T P χ i ( 2 ) ( δ α i ) 4 } . (8)

The Lagrange multipliers which appear in Equation (8) will be determined by following the same conceptual procedure as was used to determine the Lagrange multipliers in Part 1 [12] by expanding the third- and fourth-order terms in parameter and response variations in a Taylor series, to obtain the following relation:

φ ( α , r ) ≅ φ 2 ( u ) exp [ − φ 1 ( u ) ] , (9)

where the following definitions have been used:

φ 1 ( u ) ≜ a † u + 1 2 u † B u , (10)

φ 2 ( u ) ≜ 1 − ∑ i = 1 T R ψ k ( 1 ) ( δ r k ) 3 − ∑ i = 1 T P ψ i ( 2 ) ( δ α i ) 3 − ∑ k = 1 T R χ k ( 1 ) ( δ r k ) 4 − ∑ i = 1 T P χ i ( 2 ) ( δ α i ) 4 , (11)

and where the various vectors and matrices are defined as follows:

u = ( r − E c ( r ) α − α 0 ) ;       a ≜ ( a ( 1 ) a ( 2 ) ) ;         a ( 1 ) ≜ ( a 1 ( 1 ) · a T R ( 1 ) ) ;         a ( 2 ) ≜ ( a 1 ( 2 ) · a T P ( 1 ) ) ; (12)

B ≜ ( B ( 11 ) B ( 12 ) [ B ( 12 ) ] † B ( 22 ) ) ;         B ( 11 ) ≜ ( b 11 ( 11 ) · b 1 , T R ( 11 ) · b k l ( 11 ) · b T R , 1 ( 11 ) · b T R , T R ( 11 ) ) ; B ( 12 ) ≜ ( b 11 ( 12 ) · b 1 , T P ( 12 ) · b k l ( 12 ) · b T R , 1 ( 12 ) · b T R , T P ( 12 ) ) ;         B ( 22 ) ≜ ( b 11 ( 22 ) · b 1 , T P ( 22 ) · b k l ( 22 ) · b T P , 1 ( 22 ) · b T P , T P ( 22 ) ) . (13)

Using Equations (9)-(13) in Equation (7) yields the following approximate expression for the normalization integral Z c ( a ) :

Z c ( a ) ≅ ∫ D φ 2 ( u ) exp [ − φ 1 ( u ) ] d u = Φ ( a ) − ∑ k = 1 T R ψ k ( 1 ) Ψ k ( 1 ) ( a ) − ∑ i = 1 T P ψ i ( 2 ) Ψ i ( 2 ) ( a )     − ∑ k = 1 T R χ k ( 1 ) Χ k ( 1 ) ( a ) − ∑ i = 1 T P χ i ( 2 ) Χ i ( 2 ) ( a )   , (14)

where:

Φ ( a ) ≜ ∫ D exp ( − a † u − 1 2 u † B u ) d u = K c exp ( 1 2 a † B − 1 a ) ; (15)

K c ≜ ( 2 π ) ( T R + T P ) / 2 D e t ( B ) ; (16)

Ψ k ( 1 ) ( a ) ≜ ∫ D ( δ r k ) 3 exp ( − a † u − 1 2 u † B u ) d u   ;     k = 1 , ⋯ , T R ; (17)

Ψ i ( 2 ) ( a ) ≜ ∫ D ( δ α i ) 3 exp ( − a † u − 1 2 u † B u ) d u   ;     i = 1 , ⋯ , T P ​   ; (18)

Χ k ( 1 ) ( a ) ≜ ∫ D ( δ r k ) 4 exp ( − a † u − 1 2 u † B u ) d u   ;     k = 1 , ⋯ , T R ; (19)

Χ i ( 2 ) ( a ) ≜ ∫ D ( δ α i ) 4 exp ( − a † u − 1 2 u † B u ) d u   ;       i = 1 , ⋯ , T P ​   . (20)

The Lagrange multipliers will be determined by differentiating the “free energy” [ − ln Z c ( a ) ] with respect to the components of the vector of Lagrange multipliers a ≜ ( a ( 1 ) ,   a ( 2 ) ) † . For the sake of consistency, the same approximations which have been made in Part 1 [12] will also be employed for determining the Lagrange multipliers which appear in Equation (8). Thus, using Equation (15) and neglecting, for the present case, the corresponding third-and fourth-order correlations in Equation (14) yields the following relations:

∂ [ − ln Z c ( a ) ] ∂ a k ( 1 ) = 1 Z c ∫ D ( δ r k ) exp ( − a † u − 1 2 u † B u ) d u = 0 = ( B − 1 a ) k ;     k = 1 , ⋯ , T R ; (21)

∂ [ − ln Z c ( a ) ] ∂ a i ( 2 ) = 1 Z c ∫ D ( δ α i ) exp ( − a † u − 1 2 u † B u ) d u = 0 = ( B − 1 a ) i ;     i = 1 , ⋯ , T P . (22)

It follows from Equations (21) and (22) that the following result holds when the triple-correlations are neglected:

a = 0 . (23)

Differentiating the relations provided in Equations (21) and (22) yields the following relations:

∂ 2 [ − ln Z c ( a ) ] ∂ a j ( 1 ) ∂ a k ( 1 ) = − cov ( r j , r k ) ;         j , k = 1 , ⋯ , T R ; (24)

∂ 2 [ − ln Z c ( a ) ] ∂ a i ( 2 ) ∂ a k ( 1 ) = − cor ( α i , r k ) ;         i = 1 , ⋯ , T P ; ​ ​ ​         k = 1 , ⋯ , T R ; (25)

∂ 2 [ − ln Z c ( a ) ] ∂ a i ( 2 ) ∂ a j ( 2 ) = − cov ( α i , α j ) ;         i , j = 1 , ⋯ , T P . (26)

The results obtained in Equations (24)-(26) can be collectively written in vector-matrix form as follows:

∂ 2 [ − ln Z c ( a ) ] ∂ a ∂ a = − C c ;       C c ≜ ( C r r c C r α c C α r c C α α ) ;         C r r c ≜ [ cov ( r j , r k ) ] T R × T R ; C r α c ≜ [ cor ( r k , α i ) ] T R × T P = ( C α r c ) † ;         C α α ≜ [ cov ( α i , α j ) ] T P × T P . (27)

On the other hand, it follows from Equation (14) that the following result holds when the quadruple-correlations are neglected:

∂ 2 [ − ln Z c ( a ) ] ∂ a ∂ a = − B − 1 . (28)

The relations obtained in Equations (27) and (28) imply the following relation:

B − 1 = C c . (29)

Using the result obtained in Equation (29) simplifies the derivations of the expressions of the functions Ψ k ( 1 ) ( a ) , Χ k ( 1 ) ( a ) , Ψ i ( 2 ) ( a ) and Χ i ( 2 ) ( a ) , which are obtained in Appendix B, and are as follows:

Ψ k ( 1 ) ( a ) = − K c Φ k , 1 ( 3 ) exp [ η ( a ) ] ;         k = 1 , ⋯ , T R ; (30)

Χ k ( 1 ) ( a ) = K c Φ k , 1 ( 4 ) exp [ η ( a ) ] ;         k = 1 , ⋯ , T R ; (31)

Ψ i ( 2 ) ( a ) = − K c Φ i , 2 ( 3 ) exp [ η ( a ) ] ;       i = 1 , ⋯ , T P ; (32)

Χ i ( 2 ) ( a ) = K c Φ i , 2 ( 4 ) exp [ η ( a ) ] ;         i = 1 , ⋯ , T P ; (33)

where the following definitions were used:

η ( a ) ≜ 1 2 a † C c a = 1 2 a † B − 1 a ; (34)

Φ k , 1 ( 3 ) ≜ 3 Φ k , 1 ( 1 ) ( a ) var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 3 ; (35)

Φ k , 1 ( 4 ) ≜ 3 [ var ( r k , r k ) ] 2 + 6 [ Φ k , 1 ( 1 ) ( a ) ] 2 var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 4 ; (36)

Φ i , 2 ( 3 ) ≜ 3 Φ i , 2 ( 1 ) ( a ) var ( α i , α i ) + [ Φ i , 2 ( 1 ) ( a ) ] 3 ; (37)

Φ i , 2 ( 4 ) ≜ 3 [ var ( α i , α i ) ] 2 + 6 [ Φ i , 2 ( 1 ) ( a ) ] 2 var ( α i , α i ) + [ Φ i , 2 ( 1 ) ( a ) ] 4 ; (38)

Φ k , 1 ( 1 ) ( a ) ≜   ∑ j = 1 T R cov ( r j , r k ) a j ( 1 ) + ∑ j = 1 T P cor ( α j , r k ) a j ( 2 ) ; (39)

Φ i , 2 ( 1 ) ≜ ∑ j = 1 T P a j ( 1 ) cor ( α i , r j ) + ∑ j = 1 T P a j ( 2 ) cov ( α i , α j ) . (40)

Using the results obtained in Equations (30)-(33) into Equation (14) yields the following expression for the normalization integral Z c ( a ) :

Z c ( a ) = K c { 1 + ∑ k = 1 T R ψ k ( 1 ) Φ k , 1 ( 3 ) ( a ) + ∑ i = 1 T P ψ i ( 2 ) Φ i , 2 ( 3 ) ( a )                               − ∑ k = 1 T R χ k ( 1 ) Φ k , 1 ( 4 ) ( a ) − ∑ i = 1 T P χ i ( 2 ) Φ i , 2 ( 4 ) ( a ) } exp [ η ( a ) ] . (41)

It follows from the expression provided in Equation (41) that:

ln Z c ( a ) = ln K c + η ( a ) + ln { 1 + ∑ k = 1 T R ψ k ( 1 ) Φ k , 1 ( 3 ) ( a ) + ∑ i = 1 T P ψ i ( 2 ) Φ i , 2 ( 3 ) ( a )                                     − ∑ k = 1 T R χ k ( 1 ) Φ k , 1 ( 4 ) ( a ) − ∑ i = 1 T P χ i ( 2 ) Φ i , 2 ( 4 ) ( a ) } . (42)

The Lagrange multipliers ψ k ( 1 ) , χ k ( 1 ) , for ( k = 1 , ⋯ , T R ), are obtained by using Equation (42) to determine the third-order derivatives of [ − ln Z c ( a ) ] with respect to components of the vector of Lagrange multipliers a ≜ ( a ( 1 ) ,   a ( 2 ) ) † . The procedure for determining these third-order derivatives is the same as was used to obtain the corresponding expression presented in Part 1 [12] . In particular, the third-order derivatives of [ − ln Z c ( a ) ] with respect to components of the vector of Lagrange multipliers a ( 1 ) are obtained [see also Appendix B] in the following form:

− { ∂ 3 ln Z c ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) } a = 0 = − { 1 Z c ( a ) ∂ 3 Z c ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) } a = 0 = μ 3 ( r k ) ≅ − ∑ j = 1 T R ψ j ( 1 ) ∂ 3 Φ j , 1 ( 3 ) ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) = − 6 ψ k ( 1 ) [ var ( r k , r k ) ] 3 ;         k = 1 , ⋯ , T R   . (43)

It follows from Equation (43) that the Lagrange multipliers ψ k ( 1 ) , k = 1 , ⋯ , T R , have the following expressions:

ψ k ( 1 ) = − μ 3 ( r k ) 6 [ var ( r k , r k ) ] 3 ;         k = 1 , ⋯ , T R   . (44)

The third-order derivatives of [ − ln Z c ( a ) ] with respect to components of the vector of Lagrange multipliers a ( 2 ) are obtained by following the same procedure as was used for obtaining the expression shown in Equation (44), which yields the following results:

− { ∂ 3 ln Z c ( a ) ∂ a i ( 2 ) ∂ a i ( 2 ) ∂ a i ( 2 ) } a = 0 = t i α ≅ − 6 ψ i ( 2 ) ( c i i α ) 3 ;         i = 1 , ⋯ , T P   ; (45)

which implies that:

ψ i ( 2 ) = − t i α 6 ( c i i α ) 3 ;         i = 1 , ⋯ , T P . (46)

As expected, the results obtained in Equations (44) and (46) correspond to those previously obtained in Part 1 [12] , thus confirming that the approximations incurred in the course of determining the Lagrange multipliers for the third-order self-correlations among responses and/or parameters are consistent with each other.

The Lagrange multipliers ψ i ( 2 ) and χ i ( 2 ) ( i = 1 , ⋯ , T P ) are obtained by using Equation (41) to determine the fourth-order derivatives of [ − ln Z c ( a ) ] with respect to components of the vector of Lagrange multipliers a ≜ ( a ( 1 ) ,   a ( 2 ) ) † . The procedure for determining these fourth-order derivatives is the same as was used in Part 1 [12] . In particular, the fourth-order derivatives of [ − ln Z c ( a ) ] with respect to components of the vector of Lagrange multipliers a ( 1 ) are obtained [see also Appendix B] in the following form:

− { ∂ 4 ln Z c ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) } a = 0 = − { 1 Z c ( a ) ∂ 4 Z c ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) } a = 0 + { 3 [ Z c ( a ) ] 2 [ ∂ 2 Z c ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ] 2 } β = 0

= − μ 4 ( r k ) + 3 [ var ( r k , r k ) ] 2 ≅ ∑ j = 1 T R χ j ( 1 ) ∂ 4 Φ j , 1 ( 4 ) ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) = 24 χ k ( 1 ) [ var ( r k , r k ) ] 4 ;         k = 1 , ⋯ , T R   . (47)

It follows from Equation (47) that the Lagrange multipliers χ k ( 1 ) , k = 1 , ⋯ , T R , have the following expressions:

χ k ( 1 ) = 3 [ var ( r k , r k ) ] 2 − μ 4 ( r k ) 24 [ var ( r k , r k ) ] 4 ;         k = 1 , ⋯ , T R   . (48)

The fourth-order derivatives of [ − ln Z c ( a ) ] with respect to components of the vector of Lagrange multipliers a ( 2 ) are obtained by following the same procedure as was used for obtaining the expression shown in Equation (48), which ultimately yields the following result:

χ i ( 2 ) = 3 ( c i i α ) 2 − q i α 24 ( c i i α ) 4 ;         i = 1 , ⋯ , T P   . (49)

As expected, the results obtained in Equations (48) and (49) correspond to those previously obtained in Part 1 [12] , thus confirming that the approximations incurred in the course of determining the Lagrange multipliers for the fourth-order self-correlations among responses and/or parameters are consistent with each other.

Collecting the results obtained in Equations (7), (8), (12), (23), (29), (44) (46), (48) and (49) yields the following expression for the fourth-order MaxEnt joint distribution, p c M E ( α , r ) , of the computed model responses and parameters:

p c M E ( α , r ) = Z c − 1 φ ( α , r ) ;         Z c ≜ ∫ D φ ( α , r ) d α   d r   ; (50)

where:

φ ( α , r ) = exp { − 1 2 ( r − E c ( r ) α − α 0 ) † ( C r r c C r α c C α r c C α α ) − 1 ( r − E c ( r ) α − α 0 )   − ∑ i = 1 T P ψ i ( 2 ) ( α i − α i 0 ) 3 − ∑ k = 1 T R ψ k ( 1 ) [ r k ( α ) − E c ( r k ) ] 3   − ∑ i = 1 T P χ i ( 2 ) ( α i − α i 0 ) 4 − ∑ k = 1 T R χ k ( 1 ) [ r k ( α ) − E c ( r k ) ] 4 } . (51)

Notably, if the triple- and the quadruple correlations are negligeable (or unavailable) then the MaxEnt distribution p c M E ( r ) , presented in Equation (51), of the computed model responses and parameters reduces to a multivariate Gaussian with mean [ E c ( r ) , α 0 ] and covariance matrix C c .

This Section presents the mathematical and physical considerations leading to the development of the “4^th-BERRU-PM” methodology; this acronym stands for the “Fourth-Order Best-Estimate Results with Reduced Uncertainties−Predictive Modeling.” This methodology probabilistically incorporates, using Bayes’ Theorem, the fourth-order moments-constrained MaxEnt distribution representing the computational model, which was obtained in Section 2, above, with the fourth-order moments-constrained MaxEnt distribution of measured responses which was obtained in Part 1 [12] .

The known moments of the experimentally-measured responses were defined in Part 1 [12] and are provided below for convenient reference, as follows:

(i) Known mean/expectation values, denoted as r i e , for the system responses r i , where i = 1 , ⋯ , T R :

r i e ≜ ∫ D e r i p e M E ( z ) d r   ;     r e ≜ ( r 1 e , ⋯ , r i e , ⋯ , r T R e ) † ;       i = 1 , ⋯ , T R . (52)

(ii) Known covariances, denoted as c i j e , for two system responses r i and r j , where i , j = 1 , ⋯ , T R :

c i j e ≜ cov ( r i , r j ) e ≜ ∫ D e ( r i − r i e ) ( r j − r j e ) p e M E ( z ) d r   ;         i , j = 1 , ⋯ , T R   ; (53)

The covariances cov ( r i , r j ) e , i , j = 1 , ⋯ , T R , of the system responses are considered to be components of the T R × T R -dimensional covariance matrix of

system responses, which will be denoted as C r r e ≜ [ cov ( r i , r j ) e ] T R × T R ≜ [ c i j e ] T R × T R .

(iii) Known triple correlations, denoted as t i j k e , for three system responses denoted as r i , r j and r k , where i , j , k = 1 , ⋯ , T R :

t i j k e ≜ ∫ D e ( r i − r i e ) ( r j − r j e ) ( r k − r k e ) p e M E ( z ) d r   ;         t k e ≜ t k k k e ;           i , j , k = 1 , ⋯ , T R   ; (54)

(iv) Known quadruple correlations, denoted as q i j k l e , for four system responses denoted as r i , r j , r k , r l , where i , j , k , l = 1 , ⋯ , T R :

q i j k l e ≜ ∫ D e ( r i − r i e ) ( r j − r j e ) ( r k − r k e ) ( r l − r l e ) p e M E ( z ) d r   ; q k e ≜ q k k k k e ;       i , j , k , l = 1 , ⋯ , T R   . (55)

Recall that the following expression was obtained in Part 1 [12] for the fourth-order moments-constrained MaxEnt distribution, denoted as p e M E ( z ) , of experimentally-measured responses r e ≜ ( r 1 e , ⋯ , r T R e ) † :

p e M E ( z ) = exp { − 1 2 ( r − r e ) † Λ ( r − r e ) − ∑ i = 1 T R θ i ( r i − r i e ) 3 − ∑ i = 1 T R ω i ( r i − r i e ) 4 } ∫ D r exp { − 1 2 ( r − r e ) † Λ ( r − r e ) − ∑ i = 1 T R θ i ( r i − r i e ) 3 − ∑ i = 1 T R ω i ( r i − r i e ) 4 } d r , (56)

where the Lagrange multipliers Λ , θ i and ω i , i = 1 , ⋯ , T R , have the following expressions, respectively:

(a) the matrix of Lagrange multipliers Λ ≜ ( λ i j ) ;   i , j = 1 , ⋯ , T R is obtained in terms of the covariances of the measured responses as follows:

( Λ − 1 ) m n = c m n e ;         Λ − 1 = C e ;       C e ≜ ( c m n e ) T R × T R ;       m , n = 1 , ⋯ , T R   ; (57)

where C e ≜ ( c m n e ) T R × T R denotes the known T R × T R -dimensional covariance matrix of the system responses;

(b) the vector θ ≜ ( θ 1 , ⋯ , θ T R ) † is obtained in terms of the triple self-correlations t e ≜ ( t 1 e , ⋯ , t T R e ) † and the covariances ( c m n e ) of the measured responses as follows:

θ = − T − 1 t e / 6 ;         T ≜ ( τ m n ) T R × T R ;       τ m n ≜ ( c m n e ) 3 = ( c n m e ) 3 ;         m , n = 1 , ⋯ , T R ; (58)

(c) the vector ω ≜ ( ω 1 , ⋯ , ω T R ) † is obtained in terms of the quadruple self-correlations q m e , m = 1 , ⋯ , T R , and the covariances ( c m n e ) of the measured responses as follows:

ω = P − 1 s / 24 ;         P ≜ ( p m n ) T R × T R ;         p m n ≜ ( c m n e ) 4 = ( c m n e ) 4 ;         m , n = 1 , ⋯ , T R s ≜ ( s 1 , ⋯ , s T R ) † ;     s m ≜ 3 ( c m m e ) 2 − q m e ;         m = 1 , ⋯ , T R . (59)

In particular, if only the system response variances are available but the second-order correlations among the system responses are negligible or unavailable, then the result obtained in Equation (58) reduces to the following simple expression for determining the Lagrange multipliers θ k , k = 1 , ⋯ , T R :

θ k = − t k e 6 ( c k k e ) 3 ; ​ ​ ​ ​ ​ ​ ​ ​         k = 1 , ⋯ , T R . (60)

Similarly, if the (second-order) correlations among the measured system responses are negligible or unavailable, then the result obtained in Equation (59) reduces to the following simple expression for determining the Lagrange multipliers ω k , k = 1 , ⋯ , T R :

ω k = 3 ( c k k e ) 2 − q k e 24 ( c k k e ) 4 ; ​ ​ ​ ​ ​ ​ ​ ​       k = 1 , ⋯ , T R . (61)

Using Bayes’ Theorem to combine the pdf’s obtained in Equations (56) and (50) yields the following expression for the “best-estimate” posterior distribution of all available computational and experimental information, which will be denoted as p 4 b e ( r , α ) , where the superscript “be” denotes “best-estimate” while the subscript “4” denotes “fourth-order”:

p 4 b e ( r , α ) ≜ p e M E ( r ) p c M E ( r , α ) = exp [ − Q ( r , α ) ] Z p ,       Z p ≜ ∫ D exp [ − Q ( r , α ) ] d α   d r , (62)

where:

Q ( r , α ) ≜ 1 2 ( r − r e ) † ( C e ) − 1 ( r − r e ) + ∑ i = 1 T R θ i ( r i − r i e ) 3 + ∑ i = 1 T R ω i ( r i − r i e ) 4   + 1 2 ( r − E c ( r ) α − α 0 ) † ( C r r c C r α c C α r c C α α ) − 1 ( r − E c ( r ) α − α 0 )   + ∑ i = 1 T P ψ i ( 2 ) ( α i − α i 0 ) 3 + ∑ k = 1 T R ψ k ( 1 ) [ r k ( α ) − E c ( r k ) ] 3   + ∑ i = 1 T P χ i ( 2 ) ( α i − α i 0 ) 4 + ∑ k = 1 T R χ k ( 1 ) [ r k ( α ) − E c ( r k ) ] 4 . (63)

The moments of the best-estimate distribution p 4 b e ( r , α ) will be expressed in terms of ratios of integrals of the following form:

I 1 ≜ ∫ D exp [ − Q ( r , α ) ] d α   d r ;     I 2 ≜ ∫ D A ( r , α ) exp [ − Q ( r , α ) ] d α   d r . (64)

Integrals such as those appearing in Equation (64), where A ( r , α ) is a specified function, can be evaluated accurately, within a user-defined degree of accuracy, by using the saddle-point (Laplace) method [17] [18] . The integrals I 1 and I 2 have identical saddle points, which will be denoted as ( r s , α s ) , and which are defined as the points where the gradient of the functional Q ( r , α ) vanishes in the phase-space ( r , α ) , i.e.:

∂ Q ( r , α ) ∂ r = 0 ,           ∂ Q ( r , α ) ∂ α = 0 ,           at     ( r , α ) = ( r s , α s ) . (65)

In order to solve the systems of equations represented by Equation (65), it is convenient to commence with the component-wise representation of the quadratic form Q ( r , α ) defined in Equation (63), which is as follows:

Q ( r , α ) ≜ 1 2 ∑ i , j = 1 T R λ i j ( r i − r i e ) ( r i − r j e ) + ∑ i = 1 T R θ i ( r i − r i e ) 3 + ∑ i = 1 T R ω i ( r i − r i e ) 4   + 1 2 ∑ k = 1 T R ∑ l = 1 T R b k l ( 11 ) [ r k − E c ( r k ) ] [ r l − E c ( r l ) ]   + ∑ k = 1 T R ∑ i = 1 T P b k i ( 12 ) ( α i − α i 0 ) [ r k − E c ( r k ) ]   + 1 2 ∑ i = 1 T P ∑ j = 1 T P b i j ( 22 ) ( α i − α i 0 ) ( α j − α j 0 ) + ∑ i = 1 T P ψ i ( 2 ) ( α i − α i 0 ) 3   + ∑ k = 1 T R ψ k ( 1 ) [ r k − E c ( r k ) ] 3 + ∑ i = 1 T P χ i ( 2 ) ( α i − α i 0 ) 4 + ∑ k = 1 T R χ k ( 1 ) [ r k − E c ( r k ) ] 4 . (66)

Using the component-form expression of Q ( r , α ) presented in Equation (67) in Equation (65) yields the following relations which hold at the saddle-point ( r , α ) = ( r s , α s ) :

∂ Q ( r , α ) ∂ r μ = 0 = ∑ j = 1 T R λ μ j ( r j − r j e ) + 3 θ μ ( r μ − r μ e ) 2 + 4 ω μ ( r μ − r μ e ) 3                                           + ∑ l = 1 T R b μ l ( 11 ) [ r l − E c ( r l ) ] + ∑ i = 1 T P b μ i ( 12 ) ( α i − α i 0 )                                           + 3 ψ μ ( 1 ) [ r μ − E c ( r μ ) ] 2 + 4 χ μ ( 1 ) [ r μ − E c ( r μ ) ] 3 ;       μ = 1 , ⋯ , T R . (67)

∂ Q ( r , α ) ∂ α ν = 0 = ∑ j = 1 T P b ν j ( 22 ) ( α j − α j 0 ) + ∑ k = 1 T R b ν k ( 12 ) [ r k − E c ( r k ) ]                                           + 3 ψ ν ( 2 ) ( α ν − α ν 0 ) 2 + 4 χ ν ( 2 ) ( α ν − α ν 0 ) 3 ;           ν = 1 , ⋯ , T P . (68)

Equations (67) and (68) can be arranged in the following matrix-vector form which holds at the saddle-point ( r , α ) = ( r s , α s ) :

  ( B ( 11 ) B ( 12 ) [ B ( 12 ) ] † B ( 22 ) ) ( r − E c ( r ) α − α 0 ) + ( Λ ( r − r e ) 0 ) + ( v ( r ) w ( α ) ) = ( 0 0 ) , (69)

where:

v ( r ) ≜ [ v 1 ( r ) , ⋯ , v T R ( r ) ] † ;         w ( α ) ≜ [ w 1 ( α ) , ⋯ , w T P ( α ) ] † ; (70)

with

v μ ( r ) ≜ 3 ψ μ ( 1 ) [ r μ − E c ( r μ ) ] 2 + 4 χ μ ( 1 ) [ r μ − E c ( r μ ) ] 3                                   + 3 θ μ ( r μ − r μ e ) 2 + 4 ω μ ( r μ − r μ e ) 3 ;         μ = 1 , ⋯ , T R ; (71)

w ν ( α ) ≜ 3 ψ ν ( 2 ) ( α ν − α ν 0 ) 2 + 4 χ ν ( 2 ) ( α ν − α ν 0 ) 3 ;         ν = 1 , ⋯ , T P . (72)

Multiplying Equation (69) on the left by the matrix B − 1 = C c . yields the following equation satisfied at the saddle-point ( r , α ) = ( r s , α s ) :

( r − E c ( r ) α − α 0 ) + ( C r r c C r α c C α r c C α α ) ( Λ ( r − r e ) 0 ) + ( C r r c C r α c C α r c C α α ) ( v ( r ) w ( α ) ) = ( 0 0 ) . (73)

Carrying out the matrix-vector multiplications in Equation (73) yields the following (vector-valued) equations:

r − E c ( r ) + C r r c Λ ( r − r e ) + C r r c v ( r ) + C r α c w ( α ) = 0 , (74)

α − α 0 + C α r c Λ ( r − r e ) + C α r c v ( r ) + C α α w ( α ) = 0 . (75)

It is convenient to rearrange Equation (74) into the following form:

( I r + C r r c Λ ) ( r − r e ) + r e − E c ( r ) + C r r c v ( r ) + C r α c w ( α ) = 0 . (76)

where I r denotes the TR-dimensional identity matrix. Multiplying Equation (76) on the left by the matrix ( I r + C r r c Λ ) − 1 and recalling that Λ − 1 = C e transforms this equation into the following form at the saddle-point ( r , α ) = ( r s , α s ) :

r − r e − C e ( C e + C r r c ) − 1 [ E c ( r ) − r e ]   + C e ( C e + C r r c ) − 1 [ C r r c v ( r ) + C r α c w ( α ) ] = 0 . (77)

Using Equation (77) to replace the quantity ( r − r e ) in Equation (75) transforms the latter equation into the following form, which holds at the saddle-point ( r , α ) = ( r s , α s ) :

α − α 0 + C α r c ( C e + C r r c ) − 1 [ E c ( r ) − r e ] + C α r c [ I r − ( C e + C r r c ) − 1 C r r c ] v ( r ) + [ C α α − C α r c ( C e + C r r c ) − 1 C r α c ] w ( α ) = 0 . (78)

It is important to note that if the triple- and quadruple computed model response and parameter correlations are neglected in Equations (77) and (78), then v ( r ) = 0 and w ( α ) = 0 , so these equations can be solved exactly to obtain the following results for the components of the saddle-point ( r s , α s ) :

r s ( 0 ) = r e + C e ( C e + C r r c ) − 1 [ E c ( r ) − r e ] , (79)

α s ( 0 ) = α 0 − C α r c ( C e + C r r c ) − 1 [ E c ( r ) − r e ] . (80)

The superscript “(0)” has been used to indicate that the quantities r s ( 0 ) and α s ( 0 ) do not account for the triple and quadruple correlations among computed model responses, but do take into account the triple and quadruple correlations among model parameters which may have been used to compute the vector of mean values of the computed responses, E c ( r ) , and the matrices C r r c and C α r c . The quantities r s ( 0 ) and α s ( 0 ) fully account for all first- and second-order correlations among computed and measured responses and model parameters.

If the triple and quadruple correlations among computed model responses, and among model parameters are not neglected, then v ( r ) ≠ 0 and w ( α ) ≠ 0 , in which case the coupled system consisting of Equations (77) and (78) can be solved by using Newton’s iteration method to determine the saddle-point ( r , α ) = ( r s , α s ) , as follows:

( r s ( n + 1 ) α s ( n + 1 ) ) = ( r s ( n ) α s ( n ) ) − [ D ξ ( r s ( n ) , α s ( n ) ) ] − 1 ξ ( r s ( n ) , α s ( n ) ) ,         n = 0 , 1 , ⋯ (81)

where:

ξ ( r , α ) ≜ ( ξ 1 ( r , α ) ξ 2 ( r , α ) ) ;         D ξ ( r , α ) ≜ ( ∂ ξ 1 ( r , α ) / ∂ r ∂ ξ 1 ( r , α ) / ∂ α ∂ ξ 2 ( r , α ) / ∂ r ∂ ξ 2 ( r , α ) / ∂ α ) ; (82)

with:

ξ 1 ( r , α ) ≜ r − r s ( 0 ) + C e ( C e + C r r c ) − 1 [ C r r c v ( r ) + C r α c w ( α ) ] ; (83)

ξ 2 ( r , α ) ≜ α − α s ( 0 ) + C α r c [ I r − ( C e + C r r c ) − 1 C r r c ] v ( r )                                             + [ C α α − C α r c ( C e + C r r c ) − 1 C r α c ] w ( α ) ; (84)

∂ ξ 1 ( r , α ) ∂ r = I r + C e ( C e + C r r c ) − 1 C r r c ∂ v ( r ) ∂ r ; (85)

∂ ξ 1 ( r , α ) ∂ α = C e ( C e + C r r c ) − 1 C r α c ∂ w ( α ) ∂ α ; (86)

∂ ξ 2 ( r , α ) ∂ r = C α r c [ I r − ( C e + C r r c ) − 1 C r r c ] ∂ v ( r ) ∂ r (87)

∂ ξ 2 ( r , α ) ∂ α = I α + [ C α α − C α r c ( C e + C r r c ) − 1 C r α c ] ∂ w ( α ) ∂ α . (88)

In Equation (88), the quantity I α denotes the TP-dimensional identity matrix.

Using the definitions provided in Equations (70) and (71) enables the computation of the T R × T R -dimensional diagonal matrix V ( r ) , having zero-valued non-diagonal elements and non-zero diagonal elements as defined below:

V ( r ) ≜ ∂ v ( r ) ∂ r = D i a g { 6 ψ μ ( 1 ) [ r μ − E c ( r μ ) ] + 12 χ μ ( 1 ) [ r μ − E c ( r μ ) ] 2                                                                 + 6 θ μ ( r μ − r μ e ) + 12 ω μ ( r μ − r μ e ) 2 } ;         μ = 1 , ⋯ , T R   . (89)

Notably, the components of the matrix V ( r ) contain first-order terms in the triple and quadruple self-correlations of the computed model responses.

Similarly, using the definitions provided in Equations (70) and (72) enables the computation of the T P × T P -dimensional diagonal matrix W ( α ) , having zero-valued non-diagonal elements and non-zero diagonal elements as defined below:

W ( α ) ≜ ∂ w ( α ) ∂ α = D i a g [ 6 ψ ν ( 2 ) ( α ν − α ν 0 ) + 12 χ ν ( 2 ) ( α ν − α ν 0 ) 2 ] ;         ν = 1 , ⋯ , T P . (90)

Notably, the components of the matrix W ( α ) contain first-order terms in the triple and quadruple self-correlations of the model parameters.

It follows from the expressions obtained in Equations (85)-(90) that:

[ D ξ ( r , α ) ] − 1 = [ ( I r 0 0 I α ) + M ( r , α ) ] − 1 ≅ ( I r 0 0 I α ) − M ( r , α ) + H . O . T . , (91)

where the acronym “HOT” denotes “higher-order terms” that comprise second- (and higher-) order powers of the triple and quadruple computed model responses and parameters, and where the matrix M ( r , α ) is defined below:

M ( r , α ) ≜ ( M 11 ( r , α ) M 12 ( r , α ) M 21 ( r , α ) M 22 ( r , α ) ) ; M 11 ( r , α ) ≜ C e ( C e + C r r c ) − 1 C r r c V ( r ) ; M 12 ( r , α ) ≜ C e ( C e + C r r c ) − 1 C r α c W ( α ) ; M 21 ( r , α ) ≜ C α r c [ I r − ( C e + C r r c ) − 1 C r r c ] V ( r ) ; M 22 ( r , α ) ≜ [ C α α − C α r c ( C e + C r r c ) − 1 C r α c ] W ( α ) . (92)

Inserting the expression obtained in Equation (91) into Equation (81), and using the expressions provided in Equations (83), (84) and (92), yields the following form for the Newton iteration for determining the saddle-point ( r , α ) = ( r s , α s ) :

( r s ( n + 1 ) α s ( n + 1 ) ) ≅ ( r s ( n ) α s ( n ) ) − ( ξ 1 ( r s ( n ) , α s ( n ) ) ξ 2 ( r s ( n ) , α s ( n ) ) ) + M ( r s ( n ) , α s ( n ) ) ( ξ 1 ( r s ( n ) , α s ( n ) ) ξ 2 ( r s ( n ) , α s ( n ) ) ) ,       n = 0 , 1 , ⋯ . (93)

Inserting the expressions obtained in Equations (83), (84), and (92) into Equation (93) yields the following expression for the Newton iteration aimed at determining the saddle-point ( r s , α s ) , for n = 0 , 1 , ⋯

( r s ( n + 1 ) α s ( n + 1 ) ) ≅ ( r s ( 0 ) + p 1 ( r s ( n ) , α s ( n ) ) α s ( 0 ) + p 2 ( r s ( n ) , α s ( n ) ) ) + ( M 11 ( r s ( n ) , α s ( n ) ) M 12 ( r s ( n ) , α s ( n ) ) M 21 ( r s ( n ) , α s ( n ) ) M 22 ( r s ( n ) , α s ( n ) ) ) ( r s ( n ) − r s ( 0 ) α s ( n ) − α s ( 0 ) ) , (94)

where:

p 1 ( r s ( n ) , α s ( n ) ) ≜ C e ( C e + C r r c ) − 1 [ C r r c v ( r s ( n ) ) + C r α c w ( α s ( n ) ) ] ; (95)

p 2 ( r s ( n ) , α s ( n ) ) ≜ C α r c [ I r − ( C e + C r r c ) − 1 C r r c ] v ( r s ( n ) ) + [ C α α − C α r c ( C e + C r r c ) − 1 C r α c ] w ( α s ( n ) ) . (96)

The starting point, ( r s ( 0 ) , α s ( 0 ) ) , for the Newton iteration is provided by Equations (79) and (80), which are exact if the triple and quadruple response correlations are neglected. Setting n = 0 in Equation (94) provides the following form for the first Newton iteration:

( r s ( 1 ) α s ( 1 ) ) = ( r s ( 0 ) + p 1 ( r s ( 0 ) , α s ( 0 ) ) α s ( 0 ) + p 2 ( r s ( 0 ) , α s ( 0 ) ) ) . (97)

Notably, the contributions stemming from the matrix M ( r s ( n ) , α s ( n ) ) are nullified for the first-iterate ( r s ( 1 ) , α s ( 1 ) ) of the saddle-point ( r s , α s ) but the third- and fourth-order response correlations do contribute through the terms p 1 ( r s ( 0 ) , α s ( 0 ) ) and p 2 ( r s ( 0 ) , α s ( 0 ) ) , respectively, which have the following specific expressions:

p 1 ( r s ( 0 ) , α s ( 0 ) ) = C e ( C e + C r r c ) − 1 [ C r r c v ( r s ( 0 ) ) + C r α c w ( α s ( 0 ) ) ] ; (98)

p 2 ( r s ( 0 ) , α s ( 0 ) ) = C α r c [ I r − ( C e + C r r c ) − 1 C r r c ] v ( r s ( 0 ) )   + [ C α α − C α r c ( C e + C r r c ) − 1 C r α c ] w ( α s ( 0 ) ) . (99)

v ( r s ( 0 ) ) ≜ [ v 1 ( r s ( 0 ) ) , ⋯ , v T R ( r s ( 0 ) ) ] † ;         w ( α s ( 0 ) ) ≜ [ w 1 ( α s ( 0 ) ) , ⋯ , w T P ( α s ( 0 ) ) ] † ; (100)

v k ( r s ( 0 ) ) ≜ − μ 3 ( r k ) 2 [ var ( r k , r k ) ] 3 [ r k ( 0 ) − E c ( r k ) ] 2 − t k e 2 ( c k k e ) 3 ( r k ( 0 ) − r k e ) 2   + 3 [ var ( r k , r k ) ] 2 − μ 4 ( r k ) 6 [ var ( r k , r k ) ] 4 [ r k ( 0 ) − E c ( r k ) ] 3 + 3 ( c k k e ) 2 − q k e 6 ( c k k e ) 4 ( r k ( 0 ) − r k e ) 3 ; (101)

w i ( α s ( 0 ) ) ≜ − t i α 2 ( c i i α ) 3 ( α i ( 0 ) − α i 0 ) 2 + 3 ( c i i α ) 2 − q i α 6 ( c i i α ) 4 ( α i ( 0 ) − α i 0 ) 3 ;       i = 1 , ⋯ , T P   . (102)

The moments of the best-estimate fourth-order distribution p 4 b e ( r , α ) can now be determined by using the saddle-point expression obtained in Equation (97), if the first-order contributions from the triple- and quadruple-correlations of the measured and computed responses suffice, of from subsequent Newton-iterates of Equation (94), if higher-order contributions from these triple and quadruple response correlations are deemed to be important. Considering the generic representation of the integrals defined in Equation (64), it follows that the moments of the best-estimate fourth-order distribution p 4 b e ( r , α ) have the following expressions:

1) The best-estimate fourth-order expression of the predicted mean values of the responses is denoted as r b e ≜ ( r 1 b e , ⋯ , r T R b e ) † and is defined as follows:

r b e ≜ ∫ D r p 4 b e ( r , α ) d α d r ≅ r s ( n + 1 ) . (103)

The lowest-order expression for the best-estimate predicted mean values of the responses which contains the contributions from the triple and quadruple response and parameter correlations, will be denoted as r 1 b e and is provided by Equation (97), namely:

r 1 b e = r s ( 0 ) + p 1 ( r s ( 0 ) , α s ( 0 ) ) = r e + C e ( C e + C r r c ) − 1 [ E c ( r ) − r e ] + p 1 ( r s ( 0 ) , α s ( 0 ) ) . (104)

2) The best-estimate fourth-order expression of the mean values of the predicted calibrated model parameters, denoted as α b e ≜ ( α 1 b e , ⋯ , α T P b e ) † , is defined as follows:

α b e ≜ ∫ D α p 4 b e ( r , α ) d α   d r ≅ α s ( n + 1 ) . (105)

The lowest-order expression for the best-estimate predicted calibrated model parameters, which contains the contributions from the triple and quadruple response and parameter correlations, will be denoted as α 1 b e and is provided by Equation (97), namely:

α 1 b e = α s ( 0 ) + p 2 ( r s ( 0 ) , α s ( 0 ) ) = α 0 − C α r c ( C e + C r r c ) − 1 [ E c ( r ) − r e ] + p 2 ( r s ( 0 ) , α s ( 0 ) ) . (106)

3) The predicted best-estimate covariance matrix of the predicted best-estimate responses is denoted as C r r b e and is defined as follows:

C r r b e ≜ ∫ D ( r − r b e ) ( r − r b e ) † p 4 b e ( r , α ) d α   d r . (107)

The lowest-order expression for C r r b e will be denoted as ( C r r b e ) 1 and is determined as follows:

( C r r b e ) 1 ≜ ∫ D ( r − r 1 b e ) ( r − r 1 b e ) † p 4 b e ( r , α ) d α   d r = ∫ D [ r − r s ( 0 ) − p 1 ( r s ( 0 ) , α s ( 0 ) ) ] [ r − r s ( 0 ) − p 1 ( r s ( 0 ) , α s ( 0 ) ) ] † p 4 b e ( r , α ) d α   d r = ( C r r b e ) 0 + O [ μ 3 2 ( r k ) ] + O [ ( t k e ) 2 ] + O [ μ 4 2 ( r k ) ] + O [ ( q k e ) 2 ] , (108)

where:

( C r r b e ) 0 ≜ ∫ D ( r − r s ( 0 ) ) ( r − r s ( 0 ) ) † p 4 b e ( r , α ) d α d r = C r r e − C r r e ( C r r e + C r r c ) − 1 C r r e . (109)

The second-order terms involving the “squared triple-correlations” and “squared quadruple-correlations” shown in Equation (108) arise from terms involving the quantity p 1 ( r s ( 0 ) , α s ( 0 ) ) and are expected to be negligible by comparison to the leading term ( C r r b e ) 0 . Not explicitly shown in Equation (108) are terms involving products of triple correlations among parameters and responses, which also stem from the quantity p 1 ( r s ( 0 ) , α s ( 0 ) ) and which are also negligible by comparison to the leading term ( C r r b e ) 0 .

The expression provided in Equation (109) has been obtained by using Equation (79) and the result below:

∫ D [ E c ( r ) − r e ] [ E c ( r ) − r e ] † p 4 b e ( r , α ) d α   d r = ∫ D [ r − r e − r + E c ( r ) ] [ r − r e − r + E c ( r ) ] † p 4 b e ( r , α ) d α   d r = C r r e + C r r c . (110)

4) The predicted best-estimate covariance matrix of the predicted best-estimate calibrated model parameters is denoted as C α α b e and is defined as follows:

C α α b e ≜ ∫ D ( α − α b e ) ( α − α b e ) † p 4 b e ( r , α ) d α   d r   . (111)

The lowest-order expression for C α α b e will be denoted as ( C α α b e ) 1 and is determined as follows:

( C α α b e ) 1 ≜ ∫ D ( α − α 1 b e ) ( α − α 1 b e ) † p 4 b e ( r , α ) d α   d r = ∫ D [ α − α s ( 0 ) − p 2 ( r s ( 0 ) , α s ( 0 ) ) ] [ α − α s ( 0 ) − p 2 ( r s ( 0 ) , α s ( 0 ) ) ] † p 4 b e ( r , α ) d α   d r = ( C α α b e ) 0 + O [ ( t i α ) 2 ] + O [ ( q i α ) 2 ] , (112)

where:

( C α α b e ) 0 ≜ ∫ D ( α − α s ( 0 ) ) ( α − α s ( 0 ) ) † p 4 b e ( r , α ) d α   d r = C α α − C α r c ( C r r e + C r r c ) − 1 C r α c . (113)

The second-order terms involving the “squared triple-correlations” and “squared quadruple-correlations” shown in Equation (112) arise from terms involving the quantity p 2 ( r s ( 0 ) , α s ( 0 ) ) and are expected to be negligible by comparison to the leading term ( C α α b e ) 0 . Not explicitly shown in Equation (112) are terms involving products of triple correlations among parameters and responses, which also stem from the quantity p 2 ( r s ( 0 ) , α s ( 0 ) ) and which are also negligible by comparison to the leading term ( C α α b e ) 0 . The expression provided in Equation (113) has been obtained by using Equations (80) and (110).

5) The predicted best-estimate correlation matrix of the predicted best-estimate responses and calibrated model parameters is denoted as C α r b e and is defined as follows:

C α r b e ≜ ∫ D ( α − α b e ) ( r − r b e ) † p 4 b e ( r , α ) d α   d r . (114)

The lowest-order expression for C α r b e will be denoted as ( C α r b e ) 1 and is determined as follows:

( C α r b e ) 1 ≜ ∫ D ( α − α 1 b e ) ( r − r 1 b e ) † p 4 b e ( r , α ) d α   d r = ∫ D [ α − α s ( 0 ) − p 2 ( r s ( 0 ) , α s ( 0 ) ) ] [ r − r s ( 0 ) − p 1 ( r s ( 0 ) , α s ( 0 ) ) ] † p 4 b e ( r , α ) d α   d r = ( C α r b e ) 0 + S O T , (115)

where:

( C α r b e ) 0 ≜ ∫ D ( α − α s ( 0 ) ) ( r − r s ( 0 ) ) † p 4 b e ( r , α ) d α   d r = C α r c − C α r c ( C r r e + C r r c ) − 1 C r r e . (116)

and where “SOT” denotes “second-order terms” involving products of triple and quadruple correlations which arise from terms involving the quantities p 1 ( r s ( 0 ) , α s ( 0 ) ) and p 2 ( r s ( 0 ) , α s ( 0 ) ) , and which are expected to be negligible by comparison to the leading term ( C r r b e ) 0 . The expression provided in Equation (113) has been obtained by using Equations (80) and (110).

The transposed matrix C r α b e = ( C α r b e ) † is defined below:

C r α b e ≜ ∫ D ( r − r b e ) ( α − α b e ) † p 4 b e ( r , α ) d α   d r . (117)

Following the same steps as those leading to the result obtained in Equation (115) yields the following expression for the lowest-order approximation ( C r α b e ) 1 :

( C r α b e ) 1 ≜ ∫ D ( r − r 1 b e ) ( α − α 1 b e ) † p 4 b e ( r , α ) d α   d r = ∫ D [ r − r s ( 0 ) − p 1 ( r s ( 0 ) , α s ( 0 ) ) ] [ α − α s ( 0 ) − p 2 ( r s ( 0 ) , α s ( 0 ) ) ] † p 4 b e ( r , α ) d α   d r = ( C r α b e ) 0 + S O T , (118)

where:

( C r α b e ) 0 ≜ ∫ D ( r − r s ( 0 ) ) ( α − α s ( 0 ) ) † p 4 b e ( r , α ) d α   d r = C r α c − C r r e ( C r r e + C r r c ) − 1 C r α c . (119)

6) Best-estimate triple correlations among best-estimate predicted responses and best-estimate calibrated model parameters, as follows:

a) the best-estimate predicted triple correlations among three best-estimate predicted responses, r k b e , r l b e , r m b e , which will be denoted as μ 3 b e ( r k b e , r l b e , r m b e ) for k , l , m = 1 , ⋯ , T R , and is defined as follows:

μ 3 b e ( r k b e , r l b e , r m b e ) ≜ ∫ D ( r k − r k b e ) ( r l − r l b e ) ( r m − r m b e ) p 4 b e ( r , α ) d α   d r   . (120)

The lowest-order expression for μ 3 b e ( r k b e , r l b e , r m b e ) will be denoted as [ μ 3 b e ( r k b e , r l b e , r m b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses obtained in Equation (104) in Equation (120), which consequently takes on the following approximate form:

[ μ 3 b e ( r k b e , r l b e , r m b e ) ] 1 = ∫ D ( δ r k − ρ k ) ( δ r l − ρ l ) ( δ r m − ρ m ) p 4 b e ( r , α ) d α   d r , (121)

where:

δ r k ≜ r k − r k e ;         ρ k ≜ { C e ( C e + C r r c ) − 1 [ E c ( r ) − r e ] + p 1 ( r s ( 0 ) , α s ( 0 ) ) } k . (122)

Recalling the definition provided in Equation (62), recalling the definitions for the moments of the experimentally-measured responses provided in Equations (52)-(55), and performing the integration in Equation (121) yields the following result for the first-order approximation, [ μ 3 b e ( r k b e , r l b e , r m b e ) ] 1 , of the best-estimate triple correlations for the best-estimate responses:

[ μ 3 b e ( r k b e , r l b e , r m b e ) ] 1 = t i j k e − ρ l c k m e − ρ m c k l e − ρ k c l m e − ρ k ρ l ρ m ;       k , l , m = 1 , ⋯ , T R . (123)

b) the best-estimate predicted triple correlations among a best-estimate calibrated model parameter, α k b e , and two best-estimate predicted responses, r l b e , r m b e , which will be denoted as μ 3 b e ( α k b e , r l b e , r m b e ) for k = 1 , ⋯ , T P and l , m = 1 , ⋯ , T R , is defined as follows:

μ 3 b e ( α k b e , r l b e , r m b e ) ≜ ∫ D ( α k − α k b e ) ( r l − r l b e ) ( r m − r m b e ) p 4 b e ( r , α ) d α   d r . (124)

The lowest-order expression for μ 3 b e ( α k b e , r l b e , r m b e ) will be denoted as [ μ 3 b e ( α k b e , r l b e , r m b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses and parameters, which consequently takes on the following approximate form:

μ 3 b e ( α k b e , r l b e , r m b e ) = ∫ D ( δ α k + β k ) ( δ r l − ρ l ) ( δ r m − ρ m ) p 4 b e ( r , α ) d α   d r , (125)

where:

δ α k ≜ α k − α k 0 ;         β k ≜ { C α r c ( C e + C r r c ) − 1 [ E c ( r ) − r e ] + p 2 ( r s ( 0 ) , α s ( 0 ) ) } k . (126)

Performing the integration in Equation (125) and noting that the model parameters are uncorrelated with the experimentally-measured responses yields the following result:

μ 3 b e ( α k b e , r l b e , r m b e ) = β k ( c l m e + ρ l ρ m ) . (127)

c) the best-estimate predicted triple correlations among two best-estimate calibrated model parameters, α k b e , α l b e , and a best-estimate predicted responses, r m b e , which will be denoted as μ 3 b e ( α k b e , α l b e , r m b e ) for k , l = 1 , ⋯ , T P and m = 1 , ⋯ , T R , is defined as follows:

μ 3 b e ( α k b e , α l b e , r m b e ) ≜ ∫ D ( α k − α k b e ) ( α l − α l b e ) ( r m − r m b e ) p 4 b e ( r , α ) d α   d r . (128)

The lowest-order expression for μ 3 b e ( α k b e , α l b e , r m b e ) will be denoted as [ μ 3 b e ( α k b e , α l b e , r m b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses and parameters, which consequently takes on the following approximate form:

[ μ 3 b e ( α k b e , α l b e , r m b e ) ] 1 = ∫ D ( δ α k + β k ) ( δ α l + β l ) ( δ r m − ρ m ) p 4 b e ( r , α ) d α   d r = − ρ m ( c k l α + β k β l ) . (129)

d) the best-estimate predicted triple correlations among three best-estimate calibrated model parameters, α k b e , α l b e , α m b e , which will be denoted as μ 3 b e ( α k b e , α l b e , α m b e ) for k , l , m = 1 , ⋯ , T P , is defined as follows:

μ 3 b e ( α k b e , α l b e , α m b e ) ≜ ∫ D ( α k − α k b e ) ( α l − α l b e ) ( α m − α m b e ) p 4 b e ( r , α ) d α   d r . (130)

The lowest-order expression for μ 3 b e ( α k b e , α l b e , α m b e ) will be denoted as [ μ 3 b e ( α k b e , α l b e , α m b e ) ] 1 and is determined by using the first-order expression of the best-estimate calibrated parameters, which consequently takes on the following approximate form:

μ 3 b e ( α k b e , α l b e , α m b e ) = ∫ D ( δ α k + β k ) ( δ α l + β l ) ( δ α m + β m ) p 4 b e ( r , α ) d α   d r = t k l m α + β l c k m α + β m c k l α + β k c l m α + β k β l β m . (131)

7) Best-estimate quadruple correlations among best-estimate predicted responses and best-estimate calibrated model parameters, as follows:

a) the best-estimate predicted quadruple correlations among four best-estimate predicted responses, r k b e , r l b e , r m b e , r n b e , which will be denoted as μ 4 b e ( r k b e , r l b e , r m b e , r n b e ) for k , l , m , n = 1 , ⋯ , T R , and is defined as follows:

μ 4 b e ( r k b e , r l b e , r m b e , r n b e ) ≜ ∫ D ( r k − r k b e ) ( r l − r l b e ) ( r m − r m b e ) ( r n − r n b e ) p 4 b e ( r , α ) d α   d r . (132)

The lowest-order expression for μ 4 b e ( r k b e , r l b e , r m b e , r n b e ) will be denoted as [ μ 4 b e ( r k b e , r l b e , r m b e , r n b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses, which consequently takes on the following approximate form:

[ μ 4 b e ( r k b e , r l b e , r m b e , r n b e ) ] 1 = ∫ D ( δ r k − ρ k ) ( δ r l − ρ l ) ( δ r m − ρ m ) ( δ r n − ρ n ) p 4 b e ( r , α ) d α   d r = q k l m n e − ρ l t k m n e − ρ m t k l n e − ρ n t k l m e − ρ k t l m n e + ρ l ρ m c k n e + ρ k ρ l c m n e       + ρ k ρ m c l n e + ρ n ρ l c k m e + ρ n ρ m c k l e + ρ k ρ n c l m e + ρ k ρ l ρ m ρ n . (133)

b) the best-estimate predicted quadruple correlations among a best-estimate calibrated model parameter, α k b e , and three best-estimate predicted responses, r l b e , r m b e , r n b e , which will be denoted as μ 4 b e ( α k b e , r l b e , r m b e , r n b e ) for k = 1 , ⋯ , T P and l , m , n = 1 , ⋯ , T R , is defined as follows:

μ 4 b e ( α k b e , r l b e , r m b e , r n b e ) ≜ ∫ D ( α k − α k b e ) ( r l − r l b e ) ( r m − r m b e ) ( r n − r n b e ) p 4 b e ( r , α ) d α   d r . (134)

The lowest-order expression for μ 4 b e ( α k b e , r l b e , r m b e , r n b e ) will be denoted as [ μ 4 b e ( α k b e , r l b e , r m b e , r n b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses and parameters, which consequently takes on the following approximate form:

[ μ 4 b e ( α k b e , r l b e , r m b e , r n b e ) ] 1 = ∫ D ( δ α k + β k ) ( δ r n − ρ n ) ( δ r l − ρ l ) ( δ r m − ρ m ) p 4 b e ( r , α ) d α   d r = − ρ l t k m n e − ρ m t k l n e + β k t l m n e + c k n e ρ l ρ m + β k ( ρ n c l m e − ρ l c m n e − ρ m c l n e − ρ n ρ l ρ m ) . (135)

c) the best-estimate predicted quadruple correlations among two best-estimate calibrated model parameters, α k b e , α l b e , and two best-estimate predicted responses, r m b e , r n b e , which will be denoted as μ 4 b e ( α k b e , α n b e , r l b e , r m b e ) for k , l = 1 , ⋯ , T P and m , n = 1 , ⋯ , T R , is defined as follows:

μ 4 b e ( α k b e , α n b e , r l b e , r m b e ) ≜ ∫ D ( α k − α k b e ) ( α l − α l b e ) ( r m − r m b e ) ( r n − r n b e ) p 4 b e ( r , α ) d α   d r   . (136)

The lowest-order expression for μ 4 b e ( α k b e , α l b e , r m b e , r n b e ) will be denoted as [ μ 4 b e ( α k b e , α l b e , r m b e , r n b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses and parameters, which consequently takes on the following approximate form:

[ μ 4 b e ( α k b e , α l b e , r m b e , r n b e ) ] 1 = ∫ D ( δ α k + β k ) ( δ α n + β n ) ( δ r l − ρ l ) ( δ r m − ρ m ) p 4 b e ( r , α ) d α   d r = c k n e ρ l ρ m + β k β n ( c l m e + ρ l ρ m ) . (137)

d) the best-estimate predicted quadruple correlations among three best-estimate calibrated model parameters, α k b e , α l b e , α m b e , and one best-estimate predicted response, r n b e , which will be denoted as μ 4 b e ( α k b e , α l b e , α m b e , r n b e ) for k , l , m = 1 , ⋯ , T P and n = 1 , ⋯ , T R , is defined as follows:

μ 4 b e ( α k b e , α l b e , α m b e , r n b e ) ≜ ∫ D ( α k − α k b e ) ( α l − α l b e ) ( α m − α m b e ) ( r n − r n b e ) p 4 b e ( r , α ) d α   d r   . (138)

The lowest-order expression for μ 4 b e ( α k b e , α l b e , α m b e , r n b e ) will be denoted as [ μ 4 b e ( α k b e , α l b e , α m b e , r n b e ) ] 1 and is determined by using the first-order expression of the best-estimate responses and parameters, which consequently takes on the following approximate form:

μ 4 b e ( α k b e , α l b e , α m b e , r n b e ) = ∫ D ( δ α k + β k ) ( δ α l + β l ) ( δ α m + β m ) ( δ r n − ρ n ) p 4 b e ( r , α ) d α   d r = ρ n ( c k m α β l − β k c l m α − β m c k l α + β k β m β l ) . (139)

e) the best-estimate predicted quadruple correlations among four best-estimate calibrated model parameters, α k b e , α l b e , α m b e , α n b e , which will be denoted as μ 4 b e ( α k b e , α l b e , α m b e , α n b e ) for k , l , m , n = 1 , ⋯ , T P , is defined as follows:

μ 4 b e ( α k b e , α l b e , α m b e , α n b e ) ≜ ∫ D ( α k − α k b e ) ( α l − α l b e ) ( α m − α m b e ) ( α n − α n b e ) p 4 b e ( r , α ) d α   d r   . (140)

The lowest-order expression for μ 4 b e ( α k b e , α l b e , α m b e , α n b e ) will be denoted as [ μ 4 b e ( α k b e , α l b e , α m b e , α n b e ) ] 1 and is determined by using the first-order expression of the best-estimate parameters, which consequently takes on the following approximate form:

[ μ 4 b e ( α k b e , α l b e , α m b e , α n b e ) ] 1 = ∫ D ( δ α k + β k ) ( δ α l + β l ) ( δ α m + β m ) ( δ α n + β n ) p 4 b e ( r , α ) d α   d r = q k l m n α + β l t k m n α + β m t k l n α + β k t l m n α + β n t k l m α + β k β l c m n α + β k β m c l n α       + β k β n c l m α + β l β m c k n α + β l β n c k m α + β m β n c k l α + β k β l β m β n . (141)

It is evident from the expressions obtained in this Section, cf. Equations (103)-(141), that the triple and quadruple correlations among the best estimate predicted responses and calibrated model parameters do not vanish in general. Therefore, the fourth-order posterior distribution p 4 b e ( r , α ) of the predicted responses and calibrated model parameters is not a multivariate normal distribution (i.e., it is not a multivariate Gaussian). However, if all of the a priori triple and quadruple correlations among responses and model parameters are neglected, then all of the posterior results obtained in this Section reduce to the results predicted by the 2^nd-BERRU-PM (“second-order best estimate results with reduced uncertainties—predictive modeling) methodology [1] [2] . In this case, the quantity Q ( α , r ) , which was defined in Equation (63), becomes a quadratic form, which was shown in [1] [2] to take on the following minimum value, denoted as Q min ( 0 ) , at the second-order saddle point ( r s ( 0 ) , α s ( 0 ) ) :

Q min ( 0 ) = ( r ( 0 ) − E c ( r ) α ( 0 ) − α 0 ) † ( C r r c C r α c C α r c C α α ) − 1 ( r ( 0 ) − E c ( r ) α ( 0 ) − α 0 )   + ( r ( 0 ) − r e ) † ( C r r e ) − 1 ( r ( 0 ) − r e ) = [ E c ( r ) − r e ] † ( C r r e + C r r c ) − 1 [ E c ( r ) − r e ] . (142)

As the expression obtained in Equation (142) indicates, the quantity Q min ( 0 ) represents the square of the length of the vector [ E c ( r ) − r e ] , measuring (in the corresponding metric) the deviations between the experimental and nominally computed responses. The quantity Q min ( 0 ) is independent of calibrating (or adjusting) the original data, so it can be evaluated directly from the given data (i.e., model parameters and computed and measured responses, together with their original uncertainties) after having computed the matrix ( C r r e + C r r c ) − 1 . As the dimension of the vector [ E c ( r ) − r e ] indicates, the number of degrees of freedom characteristic of the calibration under consideration is equal to the number TR of experimental responses. The quantity Q min ( 0 ) plays the role of a “ χ 2 -like consistency indicator,” which can be used in the course of quantitative model validation since it quantifies the degree of agreement between the computed and the experimentally-measured responses, before actually combining the computational and experimental information. Agreement between experimental and computational results is indicated when Q min ( 0 ) ≃ 1 (per degree of freedom); values of Q min ( 0 ) that differ greatly from unity indicates “inconsistency” (i.e., lack of validation) or perhaps even gross errors. In such cases, the individual contributions to Q min ( 0 ) must be examined, and the outliers (i.e., very large or very small individual contributions) need to be investigated as possible sources of inconsistencies that could invalidate either parts of the model or parts of the data (or both).

The results provided in Equations (103)-(141) highlight the fact that they can be utilized both for “forward/direct predictive modeling” and for “inverse predictive modeling.” The “forward” or “direct problem” solves the “parameter-to-output” mapping that describes the “cause-to-effect” relationship in the physical process being modeled. The “inverse problem” attempts to solve the “output-to-parameters” mapping. Since the framework of 4^th-BERRU-PM methodology comprises the combined phase-space of parameters and responses, it can be used for solving both forward/direct and inverse problems. The solution of the “forward/direct problem” is provided by the expression for the predicted best-estimate responses together with the corresponding reduced predicted uncertainties. Conversely, the solution of the “inverse problem” is provided by the expressions for the predicted best-estimate calibrated parameters and their reduced predicted uncertainties.

This work has presented the fourth-order predictive modeling methodology designated by the acronym 4^th-BERRU-PM methodology, which uses the MaxEnt principle to incorporate fourth-order moments (means, covariances, skewness, kurtosis) of model parameters, computed and measured model responses, as well as fourth (and higher) order sensitivities of computed model responses to model parameters. The acronym 4^th-BERRU-PM, which designates this new methodology, stands for “fourth-order best-estimate results with reduced uncertainties.” The results predicted by the 4^th-BERRU-PM incorporates, as particular cases, the results previously predicted by the first-order BERRU-PM methodology [19] , the second-order predictive modeling methodology 2^nd-BERRU-PM [1] [2] and vastly generalizes the results produced by extant inverse methods [20] , data adjustment [3] [4] , and data assimilation [5] [6] [7] procedures, all of which rely on the minimization of user-defined least-squares-type functionals for estimating the perceived discrepancies between measured and computed results/responses. The key novel features of the 4^th-BRERRU-PM methodology include the following:

1) The mean values, the second-, third-, and fourth-order correlations among the computed responses, which are incorporated within the 4^th-BERRU-PM methodology, include high-order (as high as needed) response sensitivities to model parameters, thus generalizing all of the previous formulas of this type found in data adjustment/assimilation procedures published to date. These arbitrarily high order sensitivities can be computed most efficiently by applying the high-order adjoint methods developed by Cacuci [13] [14] .

2) The 4^th-BERRU-PM methodology predicted posterior parameter-response correlations are not obtainable by any extant data assimilation methods.

3) The 4th-BERRU-PM methodology enjoys unrivalled computational efficiency, since most intensive computation, required to compute the inverse matrix ( C e r r + C c r r ) − 1 , entails the inversion of a matrix of size T R × T R , where TR denotes the number of distinct computed (or measured) responses. This is computationally very advantageous, since in most practical situations, the number of responses is much smaller than the number of model parameters.

4) The 4^th-BERRU-PM methodology is the first and thus far only predictive modeling methodology which incorporates triple and quadruple correlations for model parameters and experimentally-measured response and is also the only methodology thus far to yield closed-form expressions for computing the predicted best-estimates triple and quadruple correlations among the best-estimate predicted responses and calibrated model parameters.

The author declares no conflict of interest regarding the publication of this paper.

Cacuci, D.G. (2023) Fourth-Order Predictive Modelling: II. 4^th-BERRU-PM Methodology for Combining Measurements with Computations to Obtain Best-Estimate Results with Reduced Uncertainties. American Journal of Computational Mathematics, 13, 439-475. https://doi.org/10.4236/ajcm.2023.134025

The generic mathematical/computational model used in this work is the same as has been used in previous works [13] [14] for computing high-order sensitivities, comprising equations that relate the system’s state variables to the system’s independent variables and parameters, which are considered to be afflicted by uncertainties. The parameters will be denoted as α 1 , ⋯ , α T P , where the subscript TP indicates “total number of imprecisely known parameters.” Without loss of generality, the imprecisely known model parameters can be considered to be real-valued scalars which are considered to be the components of a “vector of parameters” denoted as α ≜ ( α 1 , ⋯ , α T P ) † ∈ ℝ T P , where ℝ T P denotes the TP-dimensional subset of the set of real scalars. The components of α ∈ ℝ T P are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase-space of the model’s independent variables. The nominal parameter values will be denoted as α 0 ≜ [ α 1 0 , ⋯ , α i 0 , ⋯ , α T P 0 ] † ; the superscript “0” will be used to denote “nominal values.”

The generic nonlinear model is considered to comprise TI independent variables which will be denoted as x i ,   i = 1 , ⋯ , T I , where the sub/superscript “TI” denotes the “total number of independent variables.” The independent variables are considered to be components of a TI-dimensional column vector denoted as x ≜ ( x 1 , ⋯ , x T I ) † ∈ ℝ T I . The vector x ∈ ℝ T I is considered to be defined on a phase-space domain denoted as Ω ( α ) and defined as follows: Ω ( α ) ≜ { − ∞ ≤ λ i ( α ) ≤ x i ≤ ω i ( α ) ≤ ∞ ;     i = 1 , ⋯ , T I } . The lower boundary-point of an independent variable is denoted as λ i ( α ) and the corresponding upper boundary-point is denoted as ω i ( α ) , either or both of which could be unbounded. The boundary of Ω ( α ) , which will be denoted as ∂ Ω ( α ) , comprises the set of all of the endpoints λ i ( α ) ,   ω i ( α ) ,   i = 1 , ⋯ , T I of the respective intervals on which the components of x are defined, i.e., ∂ Ω ( α ) ≜ { λ i ( α ) ∪ ω i ( α ) ,     i = 1 , ⋯ , T I } . The boundary ∂ Ω ( α ) is also considered to be imprecisely known since it may depend on both geometrical parameters and material properties. For example, the “extrapolated boundary” in models based on diffusion theory depends both on the imprecisely known physical dimensions of the problem’s domain and also on the medium’s properties (atomic number densities, microscopic transport cross sections, etc.).

The model of a nonlinear physical system comprises coupled equations which can be represented in operator form as follows:

N [ u ( x ) , α ] = Q ( x , α )   ,       x ∈ Ω x ( α ) . (143)

The quantities which appear in Equation (143) are defined as follows:

(i) u ( x ) ≜ [ u 1 ( x ) , ⋯ , u T D ( x ) ] † is a TD-dimensional column vector of dependent variables (also called “state functions”), where “TD” denotes “total number of dependent variables;”

(ii) N [ u ( x ) ; α ] ≜ [ N 1 ( u ; α ) , ⋯ , N T D ( u ; α ) ] † denotes a TD-dimensional column vector, having components N i ( u ; α ) ,   i = 1 , ⋯ , T D , which are operators that act on the dependent variables u ( x ) , the independent variables x and the model parameters α ;

(iii) Q ( x , α ) ≜ [ q 1 ( x ; α ) , ⋯ , q T D ( x ; α ) ] † is a TD-dimensional column vector which represents inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters α ;

Since the right-side of Equation (143) may contain “generalized functions/ functionals” (e.g., Dirac-distributions and derivatives thereof), the equalities in this work are considered to hold in the distributional (“weak”) sense. When differential operators appear in Equation (143), their domains of definition must be specified by providing boundary and/or initial conditions. Mathematically, these boundaries and/or initial conditions can be represented in operator form as follows:

B [ u ( x ) ; α ;   x ] − C ( x , α ) = 0 ,       x ∈ ∂ Ω x ( α ) . (144)

where the column vector 0 has TD components, all of which are zero. The components B i ( u ; α ) ,   i = 1 , ⋯ , T D of B ( u ; α ) ≜ [ B 1 ( u ; α ) , ⋯ , B T D ( u ; α ) ] † are nonlinear operators in u ( x ) and α , which are defined on the boundary ∂ Ω x ( α ) of the model’s domain Ω x ( α ) . The components C i ( x ; α ) ,   i = 1 , ⋯ , T D of C ( x , α ) ≜ [ C 1 ( x ; α ) , ⋯ , C T D ( x ; α ) ] † comprise inhomogeneous boundary sources which are nonlinear functions of α , in general.

The model’s nominal solution, denoted as u 0 ( x ) , is obtained by solving Equations (143) and (144) at the nominal parameter values, namely :

N [ u 0 ( x ) ; α 0 ] = Q ( x , α 0 ) ,       x ∈ Ω x , (145)

B [ u 0 ( x ) ; α 0 ; x ] − C ( x , α 0 ) = 0 ,       x ∈ ∂ Ω x ( α 0 ) . (146)

The model response considered in this work is a nonlinear functional of the model’s state functions and parameters which can be generically represented as follows:

R [ u ( x ) ; α ] ≜ ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) S [ u ( x ) ; α ; x ] d x 1 ⋯ d x T I , (147)

where S [ u ( x ) ; α ] is suitably differentiable nonlinear function of u ( x ) and of α . Noteworthy, the components of α also include parameters that may occur just in the definition of the response under consideration, in addition to the parameters that appear in Equations (143) and (144). Since the system domain’s boundary, ∂ Ω ( α ) , is considered to be subject to uncertainties (e.g., stemming from manufacturing uncertainties), the model response R [ u ( x ) ; α ] will also be affected by the uncertainties that affect the endpoints λ i ( α ) ,   ω i ( α ) ,   i = 1 , ⋯ , T I , of ∂ Ω ( α ) . Since the response R [ u ( x ) ; α ] is a scalar quantity, it can be used, in particular, to represent mathematically any measured quantity that depends on the model’s state functions at a point in phase-space.

The model parameters usually stem from processes that are external to the physical system. Without loss of generality, the model parameters will be considered in this work to be real-valued scalars, and will be denoted as α 1 , ⋯ , α T P , where the quantity “TP” denotes the “total number of model parameters.” Mathematically, these parameters are considered as components of a TP-dimensional vector denoted as α ≜ ( α 1 , ⋯ , α T P ) † ∈ D α ∈ ℝ T P , defined over a domain D α , which is included in a TP-dimensional subset of the ℝ T P . The components of the TP-dimensional column vector α ∈ ℝ T P are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase-space of the model’s independent variables. The model parameters can be considered to be quasi-random scalar-valued quantities which follow an unknown multivariate distribution denoted as p α ( α ) . The moments of p α ( α ) are defined as follows:

(i) The nominal (or expected, or mean) values of the model parameters, assumed to be known, are denoted as α i 0 and are formally defined as follows:

α i 0 ≜ ∫ D α α i p α ( α )   d α   ,       i = 1 , ⋯ , T P . (148)

(ii) The covariance, cov ( α i , α j ) ≜ c i j α , of two model parameters, α i and α j , is also assumed to be known and is defined as follows:

cov ( α i , α j ) ≜ c i j α ≜ ∫ D α ( δ α i ) ( δ α j ) p α ( α )   d α ;       δ α i ≜ α i − α i 0 ;       i , j = 1 , ⋯ , T P . (149)

The covariances cov ( α i , α j ) ≜ c i j α are considered to be the components of the parameter covariance matrix, denoted as C α α ≜ [ cov ( α i , α j ) ] T P × T P ≜ [ c i j α ] T P × T P . The standard deviation of a parameter α i is defined as follows: σ i α ≜ c i i α , i = 1 , ⋯ , T P .

(iii) The triple correlations, assumed to be known, of three model parameters α i , α j , and α k , are denoted as t i j k α , i , j , k = 1 , ⋯ , T P , and are defined as follows:

t i j k α ≜ ∫ D α ( δ α i ) ( δ α j ) ( δ α k ) p α ( α )   d α   ;         i , j , k = 1 , ⋯ , T P   ; (150)

In particular, the triple self-correlation for a parameter, α i , is defined as follows:

t i α ≜ ∫ D α ( δ α i ) 3 p α ( α )   d α   ;         i = 1 , ⋯ , T P   ; (151)

(iv) The quadruple correlations, assumed to be known, of four model parameters α i , α j , α k , and α l , are denoted as q i j k l α , and are defined as follows for i , j , k , l = 1 , ⋯ , T R :

q i j k l α ≜ ∫ D α ( δ α i ) ( δ α j ) ( δ α k ) ( δ α l ) p α ( α )   d α   ;       i , j , k , l = 1 , ⋯ , T P   . (152)

In particular, the quadruple self-correlation for a parameter, α i , is defined as follows:

q i α ≜ ∫ D α ( δ α i ) 4 p α ( α )   d α   ;         i = 1 , ⋯ , T P   ; (153)

The results computed using a mathematical model are customarily called “model responses” (or “system responses” or “objective functions” or “indices of performance”). Each of these model responses is formally a function (implicit and/or explicit) of the model parameters α . Consider that there are a total number of TR such model responses, each response being denoted as r k ( α ) , k = 1 , ⋯ , T R . The uncertainties affecting the model parameters α will “propagate” both directly and indirectly, through the model’s dependent variables, to induce uncertainties in the computed responses. Each computed response, r k ( α ) , can be formally expanded in a multivariate Taylor-series around the parameters’ mean values. In particular, the fourth-order Taylor-series of a model response around the expected (or nominal) parameter values α 0 has the following formal expression:

r k ( α ) = r k ( α 0 ) + ∑ j 1 = 1 T P { ∂ r k ( α ) ∂ α j 1 } α 0 δ α j 1 + 1 2 ∑ j 1 = 1 T P ∑ j 2 = 1 T P { ∂ 2 r k ( α ) ∂ α j 1 ∂ α j 2 } α 0 δ α j 1 δ α j 2   + 1 3 ! ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P { ∂ 3 r k ( α ) ∂ α j 1 ∂ α j 2 ∂ α j 3 } α 0 δ α j 1 δ α j 2 δ α j 3   + 1 4 ! ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P ∑ j 4 = 1 T P { ∂ 4 r k ( α ) ∂ α j 1 ∂ α j 2 ∂ α j 3 ∂ α j 4 } α 0 δ α j 1 δ α j 2 δ α j 3 δ α j 4 + ε k . (154)

In Equation (154), the quantity r k ( α 0 ) indicates the computed value of the response using the expected/nominal parameter values α 0 ≜ ( α 1 0 , ⋯ , α T P 0 ) † . The notation {   } α 0 indicates that the quantities within the braces are also computed using the expected/nominal parameter values. The quantity ε k in Equation (154) comprises all quantifiable errors in the representation of the computed response as a function of the model parameters, including the truncation errors O ( δ α j ) 5 of the Taylor-series expansion, possible bias-errors due to incompletely modeled physical phenomena, and possible random errors due to numerical approximations. The radius/domain of convergence of the series in Equation (154) determines the largest values of the parameter variations δ α j which are admissible before the respective series becomes divergent. In turn, these maximum admissible parameter variations limit, through Equation (154), the largest parameter covariances/standard deviations which can be considered for using the Taylor-expansion for the subsequent purposes of computing moments of the distribution of computed responses.

As is well known, and as indicated by Equation (154), the Taylor-series of a function of TP-variables [e.g., r k ( α ) ] comprises TP 1^st-order derivatives, T P ( T P + 1 ) / 2 distinct 2^nd-order derivatives, and so on. The computation by conventional methods of the n^th-order functional derivatives (called “sensitivities” in the field of sensitivity analysis) of a response with respect to the TP-parameters (on which it depends) would require at least O ( T P n ) large-scale computations. The exponential increase—with the order of response sensitivities—of the number of large-scale computations needed to determine higher-order sensitivities is the manifestation of the “curse of dimensionality in sensitivity analysis,” by analogy to the expression coined by Bellman [21] to express the difficulty of using “brute-force” grid search when optimizing a function with many input variables. The “n^th-order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (n^th-CASAM-L) conceived by Cacuci [13] and the “n^th-order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (n^th-CASAM-N) conceived by Cacuci [14] are currently the only methodologies that enable the exact and efficient computation of arbitrarily high-order sensitivities of model responses to model parameters while overcoming the curse of dimensionality.

Uncertainties in the model’s parameters will evidently give rise to uncertainties in the computed model responses r k ( α ) . The computed model responses are considered to be distributed according to an unknown distribution denoted as p c ( r ) . The unknown joint probability distribution of model parameters and responses will be denoted as p c ( α , r ) ≜ p α ( α ) p c ( r ) ; this joint probability distribution is formally, defined on a domain D ≜ D α ∪ D r , where D α denotes the domain of definition of the parameters and D r denotes the domain of definition of the model responses. The approximate moments of the unknown distribution of r k ( α ) are obtained by using the so-called “propagation of errors” methodology, which entails the formal integration over p c ( α , r ) ≜ p α ( α ) p c ( r ) of various expressions involving the truncated Taylor-series expansion of the response provided in Equation (154). This procedure was first used by Tukey [22] ; Tukey’s results were explicitly generalized to 6^th-order by Cacuci [13] [14] .

The expectation value, E c ( r k ) , of a computed response r k ( α ) is obtained by integrating formally Equation (154) over p c ( α , r ) , which yields the following expression:

E c ( r k ) ≜ ∫ D r k ( α ) p c ( α , r ) d α d r = r k ( α 0 ) + 1 2 ∑ j 1 = 1 T P ∑ j 2 = 1 T P { ∂ 2 r k ( α ) ∂ α j 1 ∂ α j 2 } α 0 c j 1 j 2 α + 1 6 ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P { ∂ 3 r k ( α ) ∂ α j 1 ∂ α j 2 ∂ α j 3 } α 0 t j 1 j 2 j 3 α       + 1 24 ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P ∑ j 4 = 1 T P { ∂ 4 r k ( α ) ∂ α j 1 ∂ α j 2 ∂ α j 3 ∂ α j 4 } α 0 q j 1 j 2 j 3 j 4 α + ⋯ (155)

The expectation values E c ( r k ) , k = 1 , ⋯ , T R , are considered to be the components of a vector defined as follows: E c ( r ) ≜ [ E c ( r 1 ) , ⋯ , E c ( r k ) , ⋯ , E c ( r T R ) ] † . Notably, the expected value, E c ( r k ) , of the expected computed response, differs from the value, r k ( α 0 ) , of the model response computed at the nominal/mean value of the model parameters.

The expression of the correlation between a computed responses and a parameter variance, which will be denoted as cor ( α i , r k ) , is presented below:

cor ( α i , r k ) ≜ ∫ D ( δ α i ) ( δ r k ) p c ( α , r ) d α d r

= ∑ j = 1 T P { ∂ r k ( α ) ∂ α j } α 0 c i j α + 1 2 ∑ j 1 = 1 T P ∑ j 2 = 1 T P { ∂ 2 r k ( α ) ∂ α j 1 ∂ α j 2 } α 0 t i , j 1 j 2 α   + 1 6 ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P { ∂ 3 r k ( α ) ∂ α j 1 ∂ α j 2 ∂ α j 3 } α 0 q i , j 1 j 2 j 3 α + ⋯ (156)

where

δ r k ≜ [ r k ( α ) − E c ( r k ) ] ,       k = 1 , ⋯ , T R . (157)

The parameter-response correlations, cor ( α i , r k ) = cor ( r k , α i ) , i = 1 , ⋯ , T P , k = 1 , ⋯ , T R , are considered to be the components of a “parameter-response computed correlation matrix” denoted as C α r c and defined as follows:

C α r c ≜ ( cor ( α 1 , r 1 ) ⋯ cor ( α 1 , r T R ) ⋮ ⋱ ⋮ cor ( α T P , r 1 ) ⋯ cor ( α T P , r T R ) ) = ( C r α c ) † . (158)

The expression of the covariance between two responses r k and r l , denoted as cov ( r k ,   r l ) , is presented below:

The covariances cov ( r k ,   r l ) , k , l = 1 , ⋯ , T R , are considered to be the components of a “computed-responses covariance matrix” denoted as C r r c and defined as follows:

C r r c ≜ ( cov ( r 1 , r 1 ) ⋯ cov ( r 1 , r T R ) ⋮ ⋱ ⋮ cov ( r T P , r 1 ) ⋯ cov ( r T P , r T R ) ) . (160)

The triple correlations, denoted as μ 3 ( r k , r l , r m ) , among three responses r k , r l and r m , are defined as follows:

μ 3 ( r k , r l , r m ) ≜ ∫ D ( δ r k ) ( δ r l ) ( δ r m ) p c ( α , r ) d α d r = ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P { ∂ r k ( α ) ∂ α j 1 ∂ r l ( α ) ∂ α j 2 ∂ r m ( α ) ∂ α j 3 } α 0 t j 1 j 2 j 3 α   + 1 2 ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P ∑ j 4 = 1 T P { ∂ r k ( α ) ∂ α j 1 ∂ r l ( α ) ∂ α j 2 ∂ 2 r m ( α ) ∂ α j 3 ∂ α j 4 ( q j 1 j 2 j 3 j 4 α − c j 1 j 2 α c j 3 j 4 α )

  + ∂ r k ( α ) ∂ α j 1 ∂ 2 r l ( α ) ∂ α j 2 ∂ α j 3 ∂ r m ( α ) ∂ α j 4 ( q j 1 j 2 j 3 j 4 α − c j 1 j 4 α c j 2 j 3 α ) } α 0 + ⋯ (161)

In particular, setting k = l = m in Equation (161) yields the expression of the (third-order) triple self-correlation, μ 3 ( r k ) , for a response r k , which is defined as follows:

μ 3 ( r k ) ≜ ∫ D ( δ r k ) 3 p c ( α , r ) d α   d r ,         k = 1 , ⋯ , T R . (162)

The expressions of the triple-correlations μ 3 ( α i , r k , r l ) and μ 3 ( α i , α j , r k ) among parameters and responses will not be considered in this work but are provided in [13] [14] .

The expression of the quadruple-correlations of the distribution of responses, denoted as μ 4 ( r k , r l , r m , r n ) , among four responses, r k , r l , r m and r n , is defined as follows:

μ 4 ( r k , r l , r m , r n ) ≜ ∫ D ( δ r k ) ( δ r l ) ( δ r m ) ( δ r n ) p c ( α , r ) d α   d r = ∑ j 1 = 1 T P ∑ j 2 = 1 T P ∑ j 3 = 1 T P ∑ j 4 = 1 T P { ∂ r k ( α ) ∂ α j 1 ∂ r l ( α ) ∂ α j 2 ∂ r m ( α ) ∂ α j 3 ∂ r n ( α ) ∂ α j 4 } α 0 q j 1 j 2 j 3 j 4 α + ⋯ (163)

In particular, setting k = l = m = n in Equation (163) yields the fourth-order self-correlation, μ 4 ( r k ) , for a response r k , which is defined as follows:

μ 4 ( r k ) ≜ ∫ D ( δ r k ) 4 p c ( α , r ) d α   d r ,         k = 1 , ⋯ , T R . (164)

The expressions of the quadruple-correlations among parameters and responses will not be considered in this work but are provided in [13] [14] .

It follows from Equations (15)-(29) that the functions Ψ k ( 1 ) ( a ) , Χ k ( 1 ) ( a ) , for k = 1 , ⋯ , T R , and Ψ i ( 2 ) ( a ) , Χ i ( 2 ) ( a ) for i = 1 , ⋯ , T P , can be obtained by differentiating the function Φ ( a ) with respect to the components of the Lagrange multiplier a ≜ ( a ( 1 ) , a ( 2 ) ) † , the definition of which is reproduced below:

Φ ( a ) ≜ ∫ D exp ( − a † u − 1 2 u † B u ) d u = K c exp ( 1 2 a † B − 1 a ) = K c exp [ η ( a ) ] , (165)

where:

K c ≜ ( 2 π ) ( T R + T P ) / 2 D e t ( B ) = ( 2 π ) ( T R + T P ) / 2 [ D e t ( C c ) ] 1 / 2 ; (166)

η ( a ) ≜ 1 2 a † C c a   ;   C c ≜ ( C r r c C r α c C α r c C α α ) . (167)

The quantity η ( a ) , which appears in the exponent of the quantity Φ ( a ) on the rightmost side of Equation (165), has the following expression:

η ( a ) ≜ 1 2 a † C c a = 1 2 ( a ( 1 ) , a ( 2 ) ) † ( C r r c C r α c C α r c C α α ) ( a ( 1 ) a ( 2 ) ) = 1 2 ( a ( 1 ) ) † C r r c a ( 1 ) + 1 2 ( a ( 1 ) ) † C r α c a ( 2 ) + 1 2 ( a ( 2 ) ) † C α r c a ( 1 ) + 1 2 ( a ( 2 ) ) † C α α a ( 2 ) = 1 2 ∑ i = 1 T R ∑ j = 1 T R a i ( 1 ) a j ( 1 ) cov ( r i , r j ) + ∑ i = 1 T R ∑ j = 1 T P a i ( 1 ) a j ( 2 ) cor ( α i , r j ) + 1 2 ∑ i = 1 T P ∑ j = 1 T P a i ( 2 ) a j ( 2 ) cov ( α i , α j )   . (168)

Differentiating Equation (165) with respect to a Lagrange multiplier a k ( 1 ) yields the following relation, for k = 1 , ⋯ , T R :

∂ Φ ( a ) ∂ a k ( 1 ) = − ∫ D ( δ r k ) exp ( − a † u − 1 2 u † B u ) d u = K c Φ k , 1 ( 1 ) ( a ) exp [ η ( a ) ] , (169)

where, for each k = 1 , ⋯ , T R , the quantity Φ k , 1 ( 1 ) is defined as follows:

Φ k , 1 ( 1 ) ( a ) ≜ ∂ η ( a ) ∂ a k ( 1 ) = ∑ j = 1 T R cov ( r j , r k ) a j ( 1 ) + ∑ j = 1 T P cor ( α j , r k ) a j ( 2 ) ;       k = 1 , ⋯ , T R . . (170)

Differentiating the expression in Equation (169) with respect to the same Lagrange multiplier a k ( 1 ) yields the following relation for the second-order unmixed derivatives of Φ ( a ) with respect to a k ( 1 ) , for k = 1 , ⋯ , T R :

∂ 2 Φ ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) = ∫ D ( δ r k ) 2 exp ( − a † u − 1 2 u † B u ) d u = K c Φ k , 1 ( 2 ) exp [ η ( a ) ] , (171)

where, for each k = 1 , ⋯ , T R , the quantity Φ k , 1 ( 2 ) is defined as follows:

Φ k , 1 ( 2 ) ≜ ∂ Φ k , 1 ( 1 ) ( a ) ∂ a k ( 1 ) + Φ k , 1 ( 1 ) ( a ) ∂ η ( a ) ∂ a k ( 1 ) = var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 2 ;       k = 1 , ⋯ , T R . (172)

Differentiating the expressions in Equation (171) with respect to the same Lagrange multiplier a k ( 1 ) yields the following relation for the third-order unmixed derivatives of Φ ( a ) with respect to a k ( 1 ) , for k = 1 , ⋯ , T R :

∂ 3 Φ ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) = − ∫ D ( δ r k ) 3 exp ( − a † u − 1 2 u † B u ) d u = Φ k , 1 ( 3 ) K c exp [ η ( a ) ] = − Ψ k ( 1 ) ( a ) ;         k = 1 , ⋯ , T R ; (173)

where:

Φ k , 1 ( 3 ) ≜ ∂ Φ k , 1 ( 2 ) ( a ) ∂ a k ( 1 ) + Φ k , 1 ( 2 ) ( a ) ∂ η ( a ) ∂ a k ( 1 ) = 2 Φ k , 1 ( 1 ) ( a ) ∂ Φ k , 1 ( 1 ) ( a ) ∂ a k ( 1 ) + Φ k , 1 ( 2 ) ( a ) Φ k , 1 ( 1 ) ( a ) = 3 Φ k , 1 ( 1 ) ( a ) var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 3 ;       k = 1 , ⋯ , T R . (174)

Differentiating the expressions in Equation (173) with respect to the same Lagrange multiplier a k ( 1 ) yields the following relation for the fourth-order unmixed derivatives of Φ ( a ) with respect to a k ( 1 ) , for k = 1 , ⋯ , T R :

∂ 4 Φ ( a ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) ∂ a k ( 1 ) = ∫ D ( δ r k ) 4 exp ( − a † u − 1 2 u † B u ) d u = Φ k , 1 ( 4 ) K c exp [ η ( a ) ] = Χ k ( 1 ) ( a ) ;         k = 1 , ⋯ , T R ; (175)

where:

Φ k , 1 ( 4 ) ≜ ∂ Φ k , 1 ( 3 ) ( a ) ∂ a k ( 1 ) + Φ k , 1 ( 3 ) ( a ) ∂ η ( a ) ∂ a k ( 1 ) = 3 var ( r k , r k ) ∂ Φ k , 1 ( 1 ) ( a ) ∂ a k ( 1 ) + 3 [ Φ k , 1 ( 1 ) ( a ) ] 2 ∂ Φ k , 1 ( 1 ) ( a ) ∂ a k ( 1 ) + Φ k , 1 ( 3 ) ( a ) Φ k , 1 ( 1 ) ( a ) = 3 { var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 2 } var ( r k , r k )     + { 3 Φ k , 1 ( 1 ) ( a ) var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 3 } Φ k , 1 ( 1 ) ( a ) = 3 [ var ( r k , r k ) ] 2 + 6 [ Φ k , 1 ( 1 ) ( a ) ] 2 var ( r k , r k ) + [ Φ k , 1 ( 1 ) ( a ) ] 4 ;     k = 1 , ⋯ , T R . (176)

Differentiating Equation (165) with respect to a Lagrange multiplier a i ( 2 ) yields the following relation for the first-order derivatives of Φ ( a ) with respect to a i ( 2 ) , for i = 1 , ⋯ , T P :

∂ Φ ( a ) ∂ a i ( 2 ) = − ∫ D ( δ α i ) exp ( − a † u − 1 2 u † B u ) d u = K c Φ i , 2 ( 1 ) exp [ η ( a ) ] , (177)

where:

Φ i , 2 ( 1 ) ≜ ∂ η ( a ) ∂ a i ( 2 ) = ∑ j = 1 T P a j ( 1 ) cor ( α i , r j ) + ∑ j = 1 T P a j ( 2 ) cov ( α i , α j )   ;         i = 1 , ⋯ , T P . (178)

Differentiating the expression in Equation (177) with respect to the same Lagrange multiplier a i ( 2 ) yields the following relation for the second-order unmixed derivatives of Φ ( a ) with respect to a i ( 2 ) , for i = 1 , ⋯ , T P :

∂ 2 Φ ( a ) ∂ a i ( 2 ) ∂ a i ( 2 ) = ∫ D ( δ α i ) 2 exp ( − a † u − 1 2 u † B u ) d u = K c Φ i , 2 ( 2 ) exp [ η ( a ) ] , (179)

where, for each i = 1 , ⋯ , T P , the quantity Φ i , 2 ( 2 ) is defined as follows:

Φ i , 2 ( 2 ) ≜ ∂ Φ i , 2 ( 1 ) ( a ) ∂ a i ( 2 ) + Φ i , 2 ( 1 ) ( a ) ∂ η ( a ) ∂ a i ( 2 ) = var ( α i , α i ) + [ Φ i , 2 ( 1 ) ( a ) ] 2 ;       i = 1 , ⋯ , T P . (180)

Differentiating the expression in Equation (179) with respect to the same Lagrange multiplier a i ( 2 ) yields the following relation for the third-order unmixed derivatives of Φ ( a ) with respect to a i ( 2 ) , for i = 1 , ⋯ , T P :

∂ 3 Φ ( a ) ∂ a i ( 2 ) ∂ a i ( 2 ) ∂ a i ( 2 ) = − ∫ D ( δ α i ) 3 exp ( − a † u − 1 2 u † B u ) d u = Φ i , 2 ( 3 ) K c exp [ η ( a ) ] = − Ψ i ( 2 ) ( a ) ;         i = 1 , ⋯ , T P . (181)

where:

Φ i , 2 ( 3 ) ≜ ∂ Φ i , 2 ( 2 ) ( a ) ∂ a 2 ( 2 ) + Φ i , 2 ( 2 ) ( a ) ∂ η ( a ) ∂ a i ( 2 ) = 2 Φ i , 2 ( 1 ) ( a ) ∂ Φ i , 2 ( 1 ) ( a ) ∂ a i ( 2 ) + Φ i , 2 ( 2 ) ( a ) Φ i , 2 ( 1 ) ( a ) = 3 Φ i , 2 ( 1 ) ( a ) var ( α i , α i ) + [ Φ i , 2 ( 1 ) ( a ) ] 3 ;         i = 1 , ⋯ , T P . (182)

Differentiating the expressions in Equation (181) with respect to the same Lagrange multiplier a i ( 2 ) yields the following relation for the fourth-order unmixed derivatives of Φ ( a ) with respect to a i ( 2 ) , for i = 1 , ⋯ , T P :

∂ 4 Φ ( a ) ∂ a i ( 2 ) ∂ a i ( 2 ) ∂ a i ( 2 ) ∂ a i ( 2 ) = ∫ D ( δ α i ) 4 exp ( − a † u − 1 2 u † B u ) d u = Φ i , 2 ( 4 ) K c exp [ η ( a ) ] = Χ i ( 2 ) ( a ) ;         i = 1 , ⋯ , T P . (183)

where:

Φ i , 2 ( 4 ) ≜ ∂ Φ i , 2 ( 3 ) ∂ a i ( 2 ) + Φ i , 2 ( 3 ) ( a ) ∂ η ( a ) ∂ a i ( 2 ) = 3 [ var ( α i , α i ) ] 2 + 6 [ Φ i , 2 ( 1 ) ( a ) ] 2 var ( α i , α i ) + [ Φ i , 2 ( 1 ) ( a ) ] 4 ;     i = 1 , ⋯ , T P . (184)