AJCMAmerican Journal of Computational Mathematics2161-1203Scientific Research Publishing10.4236/ajcm.2022.121005AJCM-115812ArticlesPhysics&Mathematics Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N): I. Mathematical Framework DanGabriel Cacuci1*Center for Nuclear Science and Energy, University of South Carolina, Columbia, SC, USA18022022120144782, February 20228, March 2022 11, March 2022© Copyright 2014 by authors and Scientific Research Publishing Inc. 2014This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/

This work presents the mathematical framework of the “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5 th-CASAM-N),” which generalizes and extends all of the previous works performed to date on this subject. The 5 th-CASAM-N enables the exact and efficient computation of all sensitivities , up to and including fifth-order , of model responses to uncertain model parameters and uncertain boundaries of the system’s domain of definition, thus enabling, inter alia, the quantification of uncertainties stemming from manufacturing tolerances. The 5th-CASAM-N provides a fundamental step towards overcoming the curse of dimensionality in sensitivity and uncertainty analysis.

Nonlinear Models of Physical Systems High-Order Sensitivity Analysis 1st-Order Sensitivities 2nd-Order Sensitivities 3rd-Order Sensitivities 4th-Order Sensitivities 5th-Order Sensitivities1. Introduction

This work presents the mathematical framework of the “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” abbreviated as “5th-CASAM-N.” The 5th-CASAM-N generalizes the previous mathematical works on this topic, which stems from the framework of the first-order adjoint sensitivity analysis methodology for generic nonlinear systems established in [1]. Numerous specific applications using adjoint functions for the deterministic computation of first-order sensitivities of scalar-valued model responses to model parameters have been published over the years, as discussed and referenced in the book by Cacuci [2]. The history regarding the deterministic computation of second-order sensitivities of model responses to model parameters reveals that although many particular applications which used specifically-computed 2nd-order response sensitivities have been published over the years, the generic mathematical framework of the 2nd-order adjoint sensitivity analysis methodology was presented in [3] for linear systems and in [4] nonlinear systems. The mathematical frameworks and notable applications of the 2nd-order adjoint sensitivity analysis methodology for both linear and nonlinear systems are presented and referenced in the book by Cacuci [5].

The largest application to date of the 2nd-order adjoint sensitivity analysis methodology for linear systems to date has been presented in [6] - [11] for the polyethylene-reflected plutonium (acronym: PERP) OECD/NEA reactor physics benchmark [12]. The numerical model [6] - [11] of the PERP benchmark comprises 21,976 uncertain parameters; 7477 of these uncertain model parameters have nonzero nominal values, as follows: 180 microscopic total cross sections; 7101 microscopic scattering sections; 60 microscopic fission cross sections; 60 parameters that characterize the average number of neutron per fission; 60 parameters that characterize the fission spectrum; 10 parameters that characterize the fission source; and 6 parameters that characterize the isotope number densities. Applying the second-order adjoint sensitivity analysis methodology made it possible to compute exactly the 7477 non-zero first-order sensitivities and the (7477)2 second-order sensitivities of the PERP benchmark’s leakage response with respect to the benchmark’s imprecisely known parameters. The results of these computations have indicated that 13 first-order sensitivities attain values between 1.0 and 10.0, while 126 second-order relative sensitivities have values greater than 10.0, and 1853 second-order relative sensitivities have values between 1.0 and 10.0. These results were contrary to the previously held belief that 2nd-order relative sensitivities are smaller than 1st-order relative sensitivities for neutron transport models such as the PERP benchmark’s model.

The finding that many 2nd-order sensitivities were significantly larger than the 1st-order ones has motivated the subsequent computation of the 3rd-order sensitivities of the leakage response with respect to the PERP benchmark’s total cross sections. The largest 3rd-order sensitivities were computed in [13] [14] [15] by applying the 3rd-order adjoint sensitivity analysis methodology. It has been found that the number of 3rd-order mixed relative sensitivities that have large values (and are therefore important) is significantly greater than the number of important 2nd- and 1st-order sensitivities. For example, the first-order relative sensitivity of the benchmark’s leakage response with respect to the total microscopic cross section of hydrogen in the lowest energy group, denoted as S ( 1 ) ( σ t , 6 30 ) = − 9.366 , was the largest of the 7477 first-order sensitivities. But the unmixed second-order and third-order relative sensitivities of the benchmark’s leakage response with respect to the same parameter had the following values: S ( 2 ) ( σ t , 6 30 , σ t , 6 30 ) = 429.6 and S ( 3 ) ( σ t , 1 g = 30 , σ t , 6 g ′ = 30 , σ t , 6 g ″ = 30 ) = − 1.88 × 10 5 , respectively. The results obtained in [13] [14] [15] have motivated the development in [16] of the 4th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Linear Systems (4th-CASAM-L), which was applied in [17] [18] [19] [20] to the PERP benchmark for computing exactly and efficiently the most important 4th-order sensitivities of the benchmark’s total leakage response with respect to the benchmark’s 180 microscopic total cross sections, which include 180 4th-order unmixed sensitivities and 360 4th-order mixed sensitivities corresponding to the largest 3rd-order ones. The numerical results obtained in [17] [18] [19] [20] indicated, in particular, that the largest overall 4th-order relative sensitivity was the 4th-order relative sensitivity of the benchmark’s leakage response with respect to the total microscopic cross section of hydrogen in the lowest energy group, namely S ( 4 ) ( σ t , 6 g = 30 , σ t , 6 g = 30 , σ t , 6 g = 30 , σ t , 6 g = 30 ) = 2.720 × 10 6 . This value is around 291,000 times larger than the 1st-order relative sensitivity S ( 1 ) ( σ t , 6 30 ) = − 9.366 , 6350 times larger than the 2nd-order relative sensitivity S ( 2 ) ( σ t , 6 30 , σ t , 6 30 ) = 429.6 , and 90 times larger than the 3rd-order relative sensitivity S ( 3 ) ( σ t , 1 g = 30 , σ t , 6 g ′ = 30 , σ t , 6 g ″ = 30 ) = − 1.88 × 10 5 . The results obtained in [6] - [19] have indicated that higher-order sensitivities cannot be simply ignored out of hand but must be computed and their impact (e.g., on uncertainty analysis) must be evaluated in the context of the application under consideration.

The results obtained in [6] - [20] have also motivated the development of the general mathematical framework for computing exactly and efficiently arbitrarily-high-order sensitivities of model responses to model parameters. Since only linear systems admit bona-fide adjoint operators (in contradistinction to nonlinear operators, which do not admit adjoint operators), Cacuci has developed [21] “The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (nth-CASAM-L), which enables the exact and efficient computation of sensitivities, of any order, of model responses to model parameters, including imprecisely known domain boundaries, thus enabling the quantification of uncertainties stemming, among other factors, from manufacturing tolerances. The nth-CASAM-L overcomes the “curse of dimensionality” [22] in sensitivity and uncertainty analysis, as detailed in [23].

For nonlinear systems, the first general methodology which also enabled the exact and efficient computation of model response sensitivities to uncertain domains of definition of the model’s independent variables (in addition to response sensitivities to model parameters) were the works [24] [25] [26] [27] [28] on the “1st-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems.” The “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N)” to be presented in this work generalizes and extends the mathematical framework presented in [24] [25] [26] in order to enable the computation of 5th-order sensitivities. This work is structured as follows: Section 2 presents the mathematical framework of the 5th-CASAM-N, which builds on the lower-order adjoints sensitivity analysis methodologies [24] [25] [26] [27] [28]. The significance of the potential applications of the innovative 5th-CASAM-N are discussed in Section 3. A paradigm illustrative application to a Bernoulli model with uncertain parameters and boundaries will be presented in an accompanying work [29].

2. The Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5<sup>th</sup>-CASAM-N) Methodology

The mathematical framework of the “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N)” builds upon the framework of the 4th-CASAM-N, which is presented in [27]. It is therefore necessary to recall the mathematical framework underlying the 4th-CASAM-N, which will be summarized in this Section. Matrices will be denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “ ≜ ” will be used to denote “is defined as” or “is by definition equal to.” Transposition will be indicated by a dagger ( † ) superscript.

The computational model of a physical system comprises equations that relate the system’s state variables to system’s independent variables and parameters, which are considered to be afflicted by uncertainties. The information customarily available about the model parameters comprises their nominal (expected/mean) values and, possibly, higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are usually determined from evaluation processes external to the physical system under consideration. Occasionally, only lower and/or upper bounds may be known for some model parameters. The parameters will be denoted as α 1 , ⋯ , α T P , where the subscript TP indicates “total number of imprecisely known parameters.” Without loss of generality, the imprecisely known model parameters can be considered to be real-valued scalars which are considered to be the components of a “vector of parameters” denoted as α ≜ ( α 1 , ⋯ , α T P ) † ∈ ℝ T P , where ℝ T P denotes the TP-dimensional subset of the set of real scalars. The components of α ∈ ℝ T P are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase-space of the model’s independent variables. The nominal parameter values will be denoted as α 0 ≜ [ α 1 0 , ⋯ , α i 0 , ⋯ , α T P 0 ] † ; the superscript “0” will be used throughout this work to denote “nominal values.”

The generic nonlinear model is considered to comprise TI independent variables which will be denoted as x i , i = 1 , ⋯ , T I , where the sub/superscript “TI” denotes the “total number of independent variables.” The independent variables are considered to be components of a TI-dimensional column vector denoted as x ≜ ( x 1 , ⋯ , x T I ) † ∈ ℝ T I . The vector x ∈ ℝ T I is considered to be defined on a phase-space domain, denoted as Ω ( α ) and defined as follows: Ω ( α ) ≜ { − ∞ ≤ λ i ( α ) ≤ x i ≤ ω i ( α ) ≤ ∞ ; i = 1 , ⋯ , T I } . The lower boundary-point of an independent variable is denoted as λ i ( α ) and the corresponding upper boundary-point is denoted as ω i ( α ) . The boundary of Ω ( α ) , which will be denoted as ∂ Ω ( α ) , comprises the set of all of the endpoints λ i ( α ) , ω i ( α ) , i = 1 , ⋯ , T I of the respective intervals on which the components of x are defined, i.e., ∂ Ω ( α ) ≜ { λ i ( α ) ∪ ω i ( α ) , i = 1 , ⋯ , T I } . The boundary Ω ( α ) is also considered to be imprecisely known since it may depend on both geometrical parameters and material properties. For example, the “extrapolated boundary” in models based on diffusion theory depends both on the imprecisely known physical dimensions of the problem’s domain and also on the medium’s properties (atomic number densities, microscopic transport cross sections, etc.).

The model of a nonlinear physical system comprises coupled equations which can be represented in operator form as follows:

N [ u ( x ) , α ] = Q ( x , α ) ,       x ∈ Ω x ( α ) . (1)

The quantities which appear in Equation (1) are defined as follows: 1) u ( x ) ≜ [ u 1 ( x ) , ⋯ , u T D ( x ) ] † is a TD-dimensional column vector of dependent variables (also called “state functions”), where “TD” denotes “total number of dependent variables;” 2) N [ u ( x ) ; α ] ≜ [ N 1 ( u ; α ) , ⋯ , N T D ( u ; α ) ] † denotes a TD-dimensional column vector, having components N i ( u ; α ) , i = 1 , ⋯ , T D , which are operators that act on the dependent variables u ( x ) , the independent variables x and the model parameters α ; 3) Q ( x , α ) ≜ [ q 1 ( x ; α ) , ⋯ , q T D ( x ; α ) ] † is a TD-dimensional column vector which represents inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters α ; 4) since the right-side of Equation (1) may contain “generalized functions/functionals” (e.g., Dirac-distributions and derivatives thereof), the equalities in this work are considered to hold in the distributional (“weak”) sense.

When differential operators appear in Equation (1), their domains of definition must be specified by providing boundary and/or initial conditions. Mathematically, these boundaries and/or initial conditions can be represented in operator form as follows:

B [ u ( x ) ; α ; x ] − C ( x , α ) = 0 ,       x ∈ ∂ Ω x ( α ) . (2)

where the column vector 0 has TD components, all of which are zero. The components B i ( u ; α ) , i = 1 , ⋯ , T D of B ( u ; α ) ≜ [ B 1 ( u ; α ) , ⋯ , B T D ( u ; α ) ] † are nonlinear operators in u ( x ) and α , which are defined on the boundary ∂ Ω x ( α ) of the model’s domain Ω x ( α ) . The components C i ( x ; α ) , i = 1 , ⋯ , T D of C ( x ; α ) ≜ [ C 1 ( x ; α ) , ⋯ , C T D ( x ; α ) ] † comprise inhomogeneous boundary sources which are nonlinear functions of α .

The model’s nominal solution, denoted as u 0 ( x ) , is obtained by solving Equations (1) and (2) at the nominal parameter values, namely:

N [ u 0 ( x ) ; α 0 ] = Q ( x , α 0 ) ,       x ∈ Ω x , (3)

B [ u 0 ( x ) ; α 0 ; x ] − C ( x , α 0 ) = 0 ,       x ∈ ∂ Ω x ( α 0 ) . (4)

The model response considered in this work is a nonlinear functional of the model’s state functions and parameters which can be generically represented as follows:

R [ u ( x ) ; α ] ≜ ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) S [ u ( x ) ; α ; x ] d x 1 ⋯ d x T I , (5)

where S [ u ( x ) ; α ] is suitably differentiable nonlinear function of u ( x ) and of α . Noteworthy, the components of α also include parameters that may occur just in the definition of the response under consideration, in addition to the parameters that appear in Equations (1) and (2). Since the system domain’s boundary, ∂ Ω ( α ) , is considered to be subject to uncertainties (e.g., stemming from manufacturing uncertainties), the model response R [ u ( x ) ; α ] will also be affected by the uncertainties that affect the endpoints λ i ( α ) , ω i ( α ) , i = 1 , ⋯ , T I , of ∂ Ω ( α ) .

2.1. The First-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (1<sup>st</sup>-CASAM-N)

The model and boundary parameters α are considered to be uncertain quantities, having unknown true values. The nominal (or mean) parameter vales α 0 are considered to be known, and these will differ from the true values by quantities denoted as δ α ≜ ( δ α 1 , ⋯ , δ α T P ) , where δ α i ≜ α i − α i 0 . Since the forward state functions u ( x ) are related to the model and boundary parameters α through Equations (1) and (2), it follows that the variations δ α in the model and boundary parameters will cause corresponding variations v ( 1 ) ( x ) ≜ [ δ u 1 ( x ) , ⋯ , δ u T D ( x ) ] † around the nominal solution u 0 ( x ) in the forward state functions. In turn, the variations δ α and v ( 1 ) ( x ) will induce variations in the system’s response.

As shown in [1], the 1st-order sensitivities of a model response R ( e ) , where e ≜ ( α , u ) ∈ E , with respect to variations h ≜ ( δ α , v ( 1 ) ) in the model parameters and state functions in a neighborhood around the nominal functions and parameter values e 0 ≜ ( α 0 , u 0 ) ∈ E , are obtained by determining the 1st-order Gateaux- (G-) variation of the response. The 1st-order Gateaux- (G-) variation, denoted as δ R ( e 0 ; h ) , of the response will exist and will be linear in h ≜ ( δ α , v ( 1 ) ) if and only if the following two conditions are satisfied by R ( e ) :

1) R ( e ) satisfies a weak Lipschitz condition at e 0 , which has the following form:

‖ R ( e 0 + ε h ) − R ( e 0 ) ‖ ≤ k ‖ ε ( e 0 ) ‖ ,       k < ∞ , (6)

2) R ( e ) satisfies the following condition:

R ( e 0 + ε h 1 + ε h 2 ) − R ( e 0 + ε h 1 ) − R ( e 0 + ε h 2 ) + R ( e 0 ) = o ( ε ) ;       h 1 , h 2 ∈ E ;       ε ∈ F   . (7)

In Equation (7), the symbol F denotes the underlying field of scalars. Numerical methods (e.g., Newton’s method and variants thereof) for solving Equations (1) and (2) also require the existence of the first-order G-derivatives of original model equations. Therefore, the conditions provided in Equations (6) and (7) are henceforth considered to be satisfied by the model responses and also by the operators underlying the physical system modeled by Equations (1) and (2). When the response R ( e ) satisfies the conditions provided in Equations (6) and (7), the 1st-order G-differential δ R ( e 0 ; h ) can be written as follows:

δ R ( e 0 ; h ) ≡ { δ R [ u ( x ) ; α ; v ( 1 ) ( x ) ; δ α ] } α 0 ≜ { δ R [ u ( x ) ; α ; δ α ] } d i r + { δ R [ u ( x ) ; α ; v ( 1 ) ( x ) ] } i n d . (8)

In Equation (8), the “direct-effect” term { δ R [ u ( x ) ; α ; δ α ] } d i r comprises only dependencies on δ α and is defined as follows:

{ δ R [ u ( x ) ; α ; δ α ] } d i r ≜ { ∂ R ( u ; α ) ∂ α } α 0 δ α ≜ ∑ j 1 = 1 T P { R ( 1 ) [ j 1 ; u ( x ) ; α ] } d i r δ α j 1 , (9)

where ∂ R ( u ; α ) / ∂ α denotes the partial G-derivatives of R ( e ) with respect to α , evaluated at the nominal parameter values, and where the following definitions were used:

∂ [ ] ∂ α δ α ≜ ∑ i = 1 T P ∂ [ ] ∂ α i δ α i . (10)

{ R ( 1 ) [ j 1 ; u ( x ) ; α ] } d i r ≜ { ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) ∂ S ( u ; α ; α ) ∂ α j 1 d x 1 ⋯ d x T I } α 0   + ∑ j = 1 T I { ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ j − 1 ( α ) ω j − 1 ( α ) ∫ λ j + 1 ( α ) ω j + 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) S [ u ( x 1 , . , ω j ( α ) , . , x N x ) ; α ] ∂ ω j ( α ) ∂ α j 1 d x 1 ⋯ d x T I } α 0   − ∑ j = 1 T I { ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ j − 1 ( α ) ω j − 1 ( α ) ∫ λ j + 1 ( α ) ω j + 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) S [ u ( x 1 , . , λ j ( α ) , . , x N x ) ; α ] ∂ λ j ( α ) ∂ α j 1 d x 1 ⋯ d x T I } α 0 . (11)

The direct-effect term can be computed once the nominal values e 0 = ( u 0 , α 0 ) are available. The notation { } α 0 will be used in this work to indicate that the quantity enclosed within the bracket is to be evaluated at the respective nominal parameter and state functions values.

On the other hand, the quantity { δ R [ u ( x ) ; α ; v ( 1 ) ( x ) ] } i n d in Equation (8) comprises only variations in the state functions and is therefore called the “indirect-effect term,” having the following expression:

{ δ R [ u ( x ) ; α ; v ( 1 ) ( x ) ] } i n d ≜ ∫ λ 1 ( α 0 ) ω 1 ( α 0 ) ... ∫ λ T I ( α 0 ) ω T I ( α 0 ) {   ∂ S ( u ; α ; x ) ∂ u } α 0 v ( 1 ) ( x ) d x 1 ⋯ d x T I , (12)

where

∂ [ ] ∂ u v ( 1 ) ( x ) ≜ ∑ i = 1 T D ∂ [ ] ∂ u i ( x ) δ u i ( x ) . (13)

The “indirect-effect” term induces variations in the response through the variations in the state functions, which are, in turn, caused by the parameter variations through the equations underlying the model. Evidently, the indirect-effect term can be quantified only after having determined the variations v ( 1 ) ( x ) in terms of the variations δ α . The first-order relationship between the vectors v ( 1 ) ( x ) and δ α is determined by solving the equations obtained by applying the definition of the G-differential to Equations (1) and (2), which yields the following equations:

{ V ( 1 ) ( u ; α ) v ( 1 ) ( x ) } α 0 = { q V ( 1 ) ( u ; α ; δ α ) } α 0 ,   x ∈ Ω x , (14)

{ b V ( 1 ) ( u ; α ; v ( 1 ) ; δ α ) } α 0 = 0 ,   x ∈ ∂ Ω x ( α 0 ) . (15)

In Equations (14) and (15), the superscript “(1)” indicates “1st-Level” and the various quantities which appear in these equations are defined as follows:

V ( 1 ) ( u ; α ) ≜ { ∂ N ( u ; α ) ∂ u } ≜ ( ∂ N 1 ∂ u 1 … ∂ N 1 ∂ u T D ⋮ ⋱ ⋮ ∂ N T D ∂ u 1 ⋯ ∂ N T D ∂ u T D ) ; (16)

q V ( 1 ) ( u ; α ; δ α ) ≜ ∂ [ Q ( α ) − N ( u ; α ) ] ∂ α δ α ≜ ∑ j 1 = 1 T P s V ( 1 ) ( j 1 ; u ; α ) δ α j 1 ; (17)

s V ( 1 ) ( j 1 ; u ; α ) ≜ ∂ [ Q ( α ) − N ( u ; α ) ] ∂ α j 1 ; (18)

{ b V ( 1 ) ( u ; α ; v ( 1 ) ; δ α ) } α 0 ≜ { ∂ B ( u ; α ) ∂ u } α 0 v ( 1 ) + { ∂ [ B ( u ; α ) − C ( α ) ] ∂ α } α 0 δ α . (19)

The system comprising Equations (14) and (15) is called the “1st-Level Variational Sensitivity System” (1st-LVSS). In order to determine the solutions of the 1st-LVSS that would correspond to every parameter variation δ α j 1 , j 1 = 1 , ⋯ , T P , the 1st-LVSS would need to be solved TP times, with distinct right-sides for each δ α j 1 , thus requiring TP large-scale computations. In other words, the actual form of the 1st-LVSS that would need to be solved in practice is as follows:

{ V ( 1 ) ( u ; α ) v ( 1 ) ( j 1 ; x ) } α 0 = { s V ( 1 ) ( j 1 ; u ; α ) } α 0 ,       j 1 = 1 , ⋯ , T P ;     x ∈ Ω x , (20)

{ b V ( 1 ) [ u ; α ; v ( 1 ) ( j 1 ; x ) ] } α 0 = 0 ;       j 1 = 1 , ⋯ , T P ;       x ∈ ∂ Ω x ( α 0 ) . (21)

Evidently, Equations (14), (15), (20) and (21) indicate that v ( 1 ) ( x ) = ∑ j 1 = 1 T P v ( 1 ) ( j 1 ; x ) δ α j 1 . In most practical situations, the number of model parameters significantly exceeds the number of functional responses of interest, i.e., T R ≪ T P , so it would be advantageous to perform just TR (rather than TP) computations. The goal of the “1st-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (1st-CASAM-N)” is to compute exactly and efficiently the “indirect effect term” defined in Equation (12) without needing to compute explicitly the vectors v ( 1 ) ( j 1 ; x ) , j 1 = 1 , ⋯ , T P .

As has been originally shown by Cacuci [1], the need for computing the vectors v ( 1 ) ( j 1 ; x ) , j 1 = 1 , ⋯ , T P is eliminated by expressing the indirect-effect term defined in Equation (12) in terms of the solutions of the “1st-Level Adjoint Sensitivity System” (1st-LASS), the construction of which requires the introduction of adjoint operators. This is accomplished by introducing a (real) Hilbert space denoted as H 1 ( Ω x ) , endowed with an inner product of two vectors u ( a ) ( x ) ∈ H 1 and u ( b ) ( x ) ∈ H 1 denoted as 〈 u ( a ) , u ( b ) 〉 1 and defined as follows:

〈 u ( a ) , u ( b ) 〉 1 ≜ { ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) [ u ( a ) ( x ) ⋅ u ( b ) ( x ) ] d x 1 ⋯ d x T I } α 0 , (22)

where the dot indicates the scalar product u ( a ) ( x ) ⋅ u ( b ) ( x ) ≜ ∑ i = 1 T D u i ( a ) ( x ) u i ( b ) ( x ) .

Using the inner product defined in Equation (22), construct the inner product of Equation (14) with a vector a ( 1 ) ( x ) to obtain the following relation:

{ { 〈 a ( 1 ) , V ( 1 ) ( u ; α ) v ( 1 ) 〉 1 } α 0 } α 0 = { 〈 a ( 1 ) , q V ( 1 ) ( u ; α ; δ α ) 〉 1 } α 0 ,       x ∈ Ω x . (23)

Using the definition of the adjoint operator in H 1 ( Ω x ) , the left-side of Equation (23) is transformed as follows:

{ 〈 a ( 1 ) , V ( 1 ) ( u ; α ) v ( 1 ) 〉 1 } α 0 = { 〈 A ( 1 ) ( u ; α ) a ( 1 ) , v ( 1 ) 〉 1 } α 0 + { [ P ( 1 ) ( u ; α ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x } α 0 , (24)

where A ( 1 ) ( u ; α ) is the operator adjoint to V ( 1 ) ( u ; α ) , i.e., A ( 1 ) ( u ; α ) ≜ [ V ( 1 ) ( u ; α ) ] * , and where [ P ( 1 ) ( u ; α ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x denotes the associated bilinear concomitant evaluated on the space/time domain’s boundary ∂ Ω x ( α 0 ) . The symbol [ ] ∗ is used in this work to indicate “adjoint” operator. In certain situations, it might be computationally advantageous to include certain boundary components of [ P ( 1 ) ( u ; α ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x into the components of A ( 1 ) ( u ; α ) .

The first term on the right-side of Equation (24) is required to represent the indirect-effect term defined in Equation (12) by imposing the following relationship:

{ A ( 1 ) ( u ; α ) a ( 1 ) ( x ) } α 0 = { ∂ S ( u ; α ) / ∂ u } α 0 ≜ q A ( 1 ) [ u ( x ) ; α ] ,     x ∈ Ω x , (25)

The domain of A ( 1 ) ( u ; α ) is determined by selecting appropriate adjoint boundary and/or initial conditions, which will be denoted in operator form as:

{ b A ( 1 ) ( u ; a ( 1 ) ; α ) } α 0 = 0 ,       x ∈ ∂ Ω x ( α 0 ) . (26)

The above boundary conditions for A ( 1 ) ( u ; α ) are usually inhomogeneous, i.e., b A ( 1 ) ( 0 ; 0 ; α ) ≠ 0 , and are obtained by imposing the following requirements: 1) they must be independent of unknown values of v ( 1 ) ( x ) and δ α ; 2) the substitution of the boundary and/or initial conditions represented by Equations (15) and (26) into the expression of { [ P ( 1 ) ( u ; α ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x } α 0 must cause all terms containing unknown values of v ( 1 ) ( x ) to vanish. Constructing the adjoint initial and/or boundary conditions for A ( 1 ) ( u ; α ) as described above and implementing them together with the variational boundary and initial conditions represented by Equations (15) into Equation (24) reduces the bilinear concomitant

{ [ P ( 1 ) ( u ; α ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x } α 0 to a quantity denoted as { [ P ^ ( 1 ) ( u ; α ; a ( 1 ) ; δ α ) ] ∂ Ω x } α 0 , which will contain boundary terms involving only known values of δ α , α 0 , u 0 , and ψ ( 1 ) Since { [ P ^ ( 1 ) ( u ; α ; a ( 1 ) ; δ α ) ] ∂ Ω x } α 0 is linear in δ α , it can be expressed in the following form: [ P ^ ( 1 ) ( u ; α ; a ( 1 ) ; δ α ) ] ∂ Ω x = ∑ j 1 = 1 T P [ ∂ P ^ ( 1 ) ( u ; α ; a ( 1 ) ) / ∂ α j 1 ] δ α j 1 .

The results obtained in Equations (24) and (25) are now replaced in Equation (12) to obtain the following expression of the indirect-effect term as a function of a ( 1 ) ( x ) :

{ δ R [ u ( x ) ; α ; v ( 1 ) ( x ) ] } i n d = { 〈 a ( 1 ) ,   q V ( 1 ) ( u ; α ; δ α ) 〉 1 } α 0 − { [ P ^ ( 1 ) ( u ; α ; a ( 1 ) ; δ α ) ] ∂ Ω x } α 0 , (27)

Replacing in Equation (8) the result obtained in Equation (27) together with the expression for the direct-effect term provided in Equation (9) yields the following expression for the first G-differential of the response R [ u ( x ) ; α ] :

{ δ R [ u ( x ) ; α ; v ( 1 ) ( x ) ; δ α ] } α 0 = { δ R [ u ( x ) ; α ; δ α ] } d i r + { 〈 a ( 1 ) , q V ( 1 ) ( u ; α ; δ α ) 〉 1 } α 0     − { [ P ^ ( 1 ) ( u ; α ; a ( 1 ) ; δ α ) ] ∂ Ω x } α 0 ≜ ∑ j 1 = 1 T P { R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] } α 0 δ α j 1 , (28)

where, for each j 1 = 1 , ⋯ , T P , the quantity R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] denotes the 1st-order sensitivities of the response R [ u ( x ) ; α ] with respect to the model parameters α j 1 and has the following expression:

R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] = ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) a ( 1 ) ( x ) ⋅ ∂ [ Q ( α ) − N ( u ; α ) ] ∂ α j 1 d x 1 ⋯ d x T I   − ∂ P ^ ( 1 ) ( u ; α ; a ( 1 ) ) ∂ α j 1 + ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) ∂ S ( u ; α ; α ) ∂ α j 1 d x 1 ⋯ d x T I

  + ∑ j = 1 T I { ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ j − 1 ( α ) ω j − 1 ( α ) ∫ λ j + 1 ( α ) ω j + 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) S [ u ( x 1 , . , ω j ( α ) , . , x N x ) ; α ] ∂ ω j ( α ) ∂ α j 1 d x 1 ⋯ d x T I } α 0   − ∑ j = 1 T I { ∫ λ 1 ( α ) ω 1 ( α ) ⋯ ∫ λ j − 1 ( α ) ω j − 1 ( α ) ∫ λ j + 1 ( α ) ω j + 1 ( α ) ⋯ ∫ λ T I ( α ) ω T I ( α ) S [ u ( x 1 , . , λ j ( α ) , . , x N x ) ; α ] ∂ λ j ( α ) ∂ α j 1 d x 1 ⋯ d x T I } α 0 . (29)

As indicated by Equation (29), each of the 1st-order sensitivities R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] of the response R [ u ( x ) ; α ] with respect to the model parameters α j 1 (including boundary and initial conditions) can be computed inexpensively after having obtained the function a ( 1 ) ( x ) ∈ H 1 , using quadrature formulas to evaluate the various inner products involving a ( 1 ) ( x ) ∈ H 1 . The function a ( 1 ) ( x ) ∈ H 1 is obtained by solving numerically Equations (25) and (26), which is the only large-scale computation needed for obtaining all of the first-order sensitivities. Equations (26) and (25) are called the 1st-Level Adjoint Sensitivity System (1st-LASS), and its solution, a ( 1 ) ( x ) ∈ H 1 ( Ω x ) , is called the 1st-level adjoint function. It is very important to note that the 1st-LASS is independent of parameter variation δ α j 1 , j 1 = 1 , ⋯ , T P , and therefore needs to be solved only once, regardless of the number of model parameters under consideration. Furthermore, since Equation (25) is linear in a ( 1 ) ( x ) ψ 1 , i 1 ( 2 ) ( x ) , solving it requires less computational effort than solving the original Equation (1), which is nonlinear in u ( x ) .

2.2. The Second-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (2<sup>nd</sup>-CASAM-N)

The 2nd-CASAM-N relies on the same fundamental concepts as introduced in [4], but in addition to the capabilities described in [4], the 2nd-CASAM-N also enables the computation of response sensitivities with respect to imprecisely known domain boundaries, thus including all possible types of uncertain parameters. Fundamentally, the 2nd-order sensitivities are defined as the “1st-order sensitivities of the 1st-order sensitivities.” This definition stems from the inductive definition of the 2nd-order total G-differential of correspondingly differentiable function, which is also defined inductively as “the total 1st-order differential of the 1st-order total differential” of a function. The 1st-order sensitivities R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] are assumed to satisfy the conditions stated in Equations (6) and (7), for each j 1 = 1 , ⋯ , T P , which ensures the existence of the 2nd-order sensitivities. The G-variation { δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ; δ a ( 1 ) ( x ) ; δ α ] } α 0 of a 1st-order sensitivity R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] has the following expression:

{ δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ; δ a ( 1 ) ( x ) ; δ α ] } α 0 = { δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; δ α ] } d i r         + { δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ; δ a ( 1 ) ( x ) ] } i n d . (30)

In Equation (30), the quantity { δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; δ α ] } d i r denotes the direct-effect term, which comprises all dependencies on the vector δ α of parameter variations, and is defined as follows:

{ δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; δ α ] } d i r ≜ { ∂ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] ∂ α δ α } α 0 . (31)

Also in Equation (30), the indirect-effect term { δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ; δ a ( 1 ) ( x ) ] } i n d comprises all dependencies on the vectors v ( 1 ) ( x ) and δ a ( 1 ) ( x ) of variations in the state functions u ( x ) and a ( 1 ) ( x ) , respectively, and is defined as follows:

{ δ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ; δ a ( 1 ) ( x ) ] } i n d ≜ { ∂ R ( 1 ) [ j 1 ; ⋯ ; α ] / ∂ u } α 0 v ( 1 ) ( x ) + { ∂ R ( 1 ) [ j 1 ; ⋯ ; α ] / ∂ a ( 1 ) } α 0 δ a ( 1 ) ( x ) . (32)

The functions v ( 1 ) ( x ) and δ a ( 1 ) ( x ) are obtained by solving the following 2nd-Level Variational Sensitivity System (2nd-LVSS):

{ V M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] V ( 2 ) ( 2 ; x ) } α 0 = { Q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ] } α 0 ,       x ∈ Ω x , (33)

{ B V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] } α 0 = 0 [ 2 ] ,     0 [ 2 ] ≜ [ 0 , 0 ] † ,       x ∈ ∂ Ω x ( α 0 ) .(34)

The argument “2” which appears in the list of arguments of the vector U ( 2 ) ( 2 ; x ) and the “variational vector” V ( 2 ) ( 2 ; x ) in Equation (33) indicates that each of these vectors is a 2-block column vector (each block comprising a column-vector of dimension TD), defined as follows:

U ( 2 ) ( 2 ; x ) ≜ ( u ( 1 ) ( x ) a ( 1 ) ( x ) ) ;       V ( 2 ) ( 2 ; x ) ≜ δ U ( 2 ) ( 2 ; x ) ≜ ( v ( 2 ) ( 1 ; x ) v ( 2 ) ( 2 ; x ) ) ≜ ( v ( 1 ) ( x ) δ a ( 1 ) ( x ) ) . (35)

To distinguish block-vectors from block matrices, two capital bold letter have been used (and will henceforth be used) to denote block matrices, as in the case of “the second-level variational matrix” V M ( 2 ) [ 2 × 2 ; u ( 2 ) ( x ) ; α ] . The “2nd-level” is indicated by the superscript “(2)”. Subsequently in this work, levels higher than second will also be indicated by a corresponding superscript attached to the appropriate block-vectors and/or block-matrices. The argument “ 2 × 2 ”, which appears in the list of arguments of V M ( 2 ) [ 2 × 2 ; u ( 2 ) ( x ) ; α ] , indicates that this matrix is a 2 × 2 -dimensional block-matrix comprising four matrices, each of dimensions T D × T D , having the following structure:

V M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] ≜ ( V ( 1 ) 0 V 21 ( 2 ) V 22 ( 2 ) ) . (36)

The other quantities which appear in Equations (33) and (34) are 2-block vectors having the same structure as V ( 2 ) ( 2 ; x ) , and are defined as follows:

Q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ] ≜ ( q V ( 2 ) ( 1 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ) q V ( 2 ) ( 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ) ) ≜ ( q V ( 1 ) ( u ; α ; δ α ) q 2 ( 2 ) ( u ; a ( 1 ) ; α ; δ α ) ) ; (37)

q 2 ( 2 ) ( u ; α ; a ( 1 ) ; δ α ) ≜ ∂ q A ( 1 ) [ u ( x ) ; α ] ∂ α δ α − ∂ [ A ( 1 ) ( u ; α ) a ( 1 ) ( x ) ] ∂ α δ α ; (38)

B V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] ≜ ( b V ( 2 ) [ 1 ; U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] b V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] ) ≜ ( b V ( 1 ) ( u ( 1 ) ; α ; δ u ( 1 ) ; δ α ) δ b A ( 1 ) [ U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] ) . (39)

V 21 ( 2 ) ( u ; a ( 1 ) ; α ) ≜ ∂ [ A ( 1 ) ( u ; α ) a ( 1 ) ] ∂ u − q A ( 1 ) [ u ( x ) ; α ] ∂ u ; (40)

V 22 ( 2 ) ( u ; α ) ≜ A ( 1 ) ( u ; α ) ; (41)

δ b A ( 1 ) ( u ; a ( 1 ) ; α ) ≜ ∂ b A ( 1 ) ( u ; a ( 1 ) ; α ) ∂ u v ( 1 ) ( x ) + ∂ b A ( 1 ) ( u ; a ( 1 ) ; α ) ∂ a ( 1 ) δ a ( 1 ) ( x )     + ∂ b A ( 1 ) ( u ; a ( 1 ) ; α ) ∂ α δ α . (42)

The structure of the second component of the source-term Q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ] defined in Equation (37) is as follows:

q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ] ≜ ∑ j 2 = 1 T P s V ( 2 ) [ 2 ; j 2 ; U ( 2 ) ( 2 ; x ) ; α ] δ α j 2 , (43)

where

s V ( 2 ) [ 2 ; j 2 ; U ( 2 ) ( 2 ; x ) ; α ] ≜ ∂ q A ( 1 ) [ u ( x ) ; α ] ∂ α j 2 − ∂ [ A ( 1 ) ( u ; α ) a ( 1 ) ( x ) ] ∂ α j 2 . (44)

Taking into account the expressions in Equations (43) and (44) while recalling the expressions in Equations (17) and (18) indicates the actual form of 2nd-LVSS to be solved (if one would wish to solve it) would be as follows:

{ V M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] V ( 2 ) ( 2 ; j 2 ; j 1 ; x ) } α 0 = { Q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; j 2 ; j 1 ; x ) ; α ; δ α ] } α 0 ,     j 1 = 1 , ⋯ , T P ;     j 2 = 1 , ⋯ , T P ;     x ∈ Ω x , (45)

{ B V ( 2 ) [ 2 ; j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] } α 0 = 0 [ 2 ] ,       x ∈ ∂ Ω x ( α 0 ) . (46)

Thus, there would be T P 2 “variational vectors” V ( 2 ) ( 2 ; j 2 ; j 1 ; x ) to be computed. The need to avoid such impractical, extremely expensive, computations provides the fundamental motivation underlying the development of the adjoint sensitivity analysis methodologies for computing sensitivities (of all orders) of model responses with respect to the model’s parameters. Thus, since “variational sensitivity systems” will never need to be actually solved if the 5th-CASAM-N methodology developed and presented in this work is utilized, the dependence on the indices j 1 , j 2 of the variation sensitivity systems will be suppressed in this work, in order to simplify the mathematical notation. On the other hand, since the solutions of the adjoint sensitivity systems of various levels will actually be computed in practice, the dependence on the indices j 1 , j 1 = 1 , ⋯ , T P will be displayed explicitly.

The need for solving the 2nd-LVSS is circumvented by deriving an alternative expression for the indirect-effect term defined in Equation (32), in which the function V ( 2 ) ( 2 ; x ) is replaced by a 2nd-level adjoint function which is independent of variations in the model parameter and state functions, and is the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS) which is constructed by using the 2nd-LVSS as starting point and following the same principles as outlined in Section 2.1. The 2nd-LASS is constructed in a Hilbert space, denoted as H 2 ( Ω x ) , which comprises as elements block-vectors of the same form as V ( 2 ) ( 2 ; x ) . The inner product of two vectors Ψ ( 2 ) ( 2 ; x ) ≜ [ ψ ( 2 ) ( 1 ; x ) , ψ ( 2 ) ( 2 ; x ) ] † ∈ H 2 ( Ω x ) and Φ ( 2 ) ( 2 ; x ) ≜ [ φ ( 2 ) ( 1 ; x ) , φ ( 2 ) ( 2 ; x ) ] † ∈ H 2 ( Ω x ) in the Hilbert space H 2 ( Ω x ) will be denoted as 〈 Ψ ( 2 ) ( 2 ; x ) , Φ ( 2 ) ( 2 ; x ) 〉 2 and defined as follows:

〈 Ψ ( 2 ) ( 2 ; x ) , Φ ( 2 ) ( 2 ; x ) 〉 2 ≜ ∑ i = 1 2 〈 ψ ( 2 ) ( i ; x ) , φ ( 2 ) ( i ; x ) 〉 1 . (47)

Following the same principles as outlined in Section 2.1, the inner product defined in Equation (47) is used to construct the following 2nd-Level Adjoint Sensitivity System (2nd-LASS) for the 2nd-level adjoint function A ( 2 ) ( 2 ; j 1 ; x ) ≜ [ a ( 2 ) ( 1 ; j 1 ; x ) , a ( 2 ) ( 2 ; j 1 ; x ) ] † ∈ H 2 ( Ω x ) , for each j 1 = 1 , ⋯ , T P :

{ A M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] A ( 2 ) ( 2 ; j 1 ; x ) } α 0 = { Q A ( 2 ) [ 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; α ] } α 0 ,     j 1 = 1 , ⋯ , T P ;       x ∈ Ω x , (48)

subject to boundary conditions represented as follows:

{ B A ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] } α 0 = 0 [ 2 ] ;       j 1 = 1 , ⋯ , T P ;       x ∈ ∂ Ω x ( α 0 ) .(49)

where:

Q A ( 2 ) [ 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; α ] ≜ ( q A ( 2 ) ( 1 ; j 1 ; U ( 2 ) ; α ) q A ( 2 ) ( 2 ; j 1 ; U ( 2 ) ; α ) ) ≜ ( ∂ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ] / ∂ u ∂ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ; v ( 1 ) ( x ) ] / ∂ a ( 1 ) ) ,     j 1 = 1 , ⋯ , T P . (50)

A M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] ≜ [ V M ( 2 ) ( 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ) ] * = ( [ V ( 1 ) * ] † [ V 21 ( 2 ) * ] † 0 [ V 22 ( 2 ) * ] † ) . (51)

The matrix A M ( 2 ) [ 2 × 2 ; u ( 2 ) ( x ) ; α ] comprises ( 2 × 2 ) block-matrices, each of dimensions T D 2 , thus comprising a total of ( 2 × 2 ) T D 2 components (or elements) and is obtained from the following relation:

{ 〈 A ( 2 ) ( 2 ; x ) , V M ( 2 ) V ( 2 ) ( 2 ; x ) 〉 2 } α 0 = { [ P ( 2 ) ( U ( 2 ) ; A ( 2 ) ; V ( 2 ) ; α ) ] ∂ Ω x } α 0     + { 〈 V ( 2 ) ( 2 ; x ) , A M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] A ( 2 ) ( 2 ; x ) 〉 2 } α 0 , (52)

where the quantity { [ P ( 2 ) ( U ( 2 ) ; A ( 2 ) ; V ( 2 ) ; α ) ] ∂ Ω x } α 0 denotes the corresponding bilinear concomitant on the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions. The 2nd-level adjoint boundary/initial conditions represented by Equation (49) are determined by requiring that: 1) they must be independent of unknown values of V ( 2 ) ( 2 ; x ) ; 2) the substitution of the boundary and/or initial conditions represented by Equations (49)and (34) into the expression of { [ P ( 2 ) ( U ( 2 ) ; A ( 2 ) ; V ( 2 ) ; α ) ] ∂ Ω x } α 0 must cause all terms containing unknown values of V ( 2 ) ( 2 ; x ) to vanish.

Using the 2nd-LASS to obtain the alternative expression for the indirect-effect term in terms of A ( 2 ) ( 2 ; j 1 ; x ) ≜ [ a ( 2 ) ( 1 ; j 1 ; x ) , a ( 2 ) ( 2 ; j 1 ; x ) ] † and the expression for the direct-effect term provided in Equation (31) yields the following expression for the total differential defined by Equation (30):

{ δ R ( 1 ) [ j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ; δ α ] } α 0 = { ∂ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] ∂ α δ α } α 0 + { 〈 A ( 2 ) ( 2 ; j 1 ; x ) , Q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ] 〉 2 } α 0 − { [ P ^ ( 2 ) ( U ( 2 ) ; A ( 2 ) ; α ; δ α ) ] ∂ Ω x } α 0 = ∑ j 2 = 1 T P { R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] } α 0 δ α j 2 ,     j 1 = 1 , ⋯ , T P   . (53)

where { [ P ^ ( 2 ) ( U ( 2 ) ; A ( 2 ) ; α ; δ α ) ] ∂ Ω x } α 0 denotes residual boundary terms which may not have vanished after having used the boundary and/or initial conditions represented by Equations (34) and (49). The detailed operations leading to the expression given in Equation (53) are provided in [27]. The quantity R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] denotes the second-order sensitivity of the generic scalar-valued response R [ u ( x ) ; α ] with respect to the parameters α j 1 and α j 2 computed at the nominal values of the parameters and respective state functions, and has the following expression:

For     j 1 , j 2 = 1 , ⋯ , T P :       R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] ≜ ∂ R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; α ] ∂ α j 2 − { ∂ P ^ ( 2 ) [ U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] } ∂ Ω x ∂ α j 2   + ∑ i = 1 2 〈 a ( 2 ) ( i ; j 1 ; x ) , s V ( 2 ) [ i ; j 2 ; U ( 2 ) ( 2 ; x ) ; α ] 〉 1 ≜ ∂ 2 R [ u ( x ) ; α ] ∂ α j 2 ∂ α j 1 . (54)

If the 2nd-LASS is solved TP-times, the 2nd-order mixed sensitivities R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] ≡ ∂ 2 R / ∂ α j 2 ∂ α j 1 will be computed twice, in two different ways, in terms of two distinct 2nd-level adjoint functions. Consequently, the symmetry property ∂ 2 R [ u ( x ) ; α ] / ∂ α j 2 ∂ α j 1 = ∂ 2 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 2 enjoyed by the second-order sensitivities provides an intrinsic (numerical) verification that the components of the 2nd-level adjoint function A ( 2 ) ( 2 ; j 1 ; x ) and the 1st-level adjoint function a ( 1 ) ( x ) are computed accurately.

2.3. The Third-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (3<sup>rd</sup>-CASAM-N)

The 3rd-order sensitivities will be computed by considering them to be the “sensitivities of a 2nd-order sensitivity.” Thus, each of the 2nd-order sensitivities R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] ≡ ∂ 2 R / ∂ α j 2 ∂ α j 1 will be considered to be a “model response” which is assumed to satisfy the conditions stated in Equations (6) and (7) for each j 1 , j 2 = 1 , ⋯ , T P , so that the 1st-order total G-differential of R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] will exist and will be linear in the variations V ( 2 ) ( 2 ; x ) and δ A ( 2 ) ( 2 ; j 1 ; x ) in a neighborhood around the nominal values of the parameters and the respective state functions. By definition, the 1st-order total G-differential of R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] , which will be denoted as { δ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ; V ( 2 ) ( 2 ; x ) ; δ A ( 2 ) ( 2 ; j 1 ; x ) ; δ α ] } α 0 , is given by the following expression:

{ δ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ; V ( 2 ) ( 2 ; x ) ; δ A ( 2 ) ( 2 ; j 1 ; x ) ; δ α ] } α 0 ≜ { ∂ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ] ∂ α δ α } α 0 + { δ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ; V ( 2 ) ( 2 ; x ) ; δ A ( 2 ) ( 2 ; j 1 ; x ) ] } i n d , (55)

where:

{ δ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ; V ( 2 ) ( 2 ; x ) ; δ A ( 2 ) ( 2 ; j 1 ; x ) ] } i n d ≜ { ∂ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ; A ( 2 ) ; α ] ∂ U ( 2 ) ( 2 ; x ) } α 0 V ( 2 ) ( 2 ; x ) ​ ​ ​ + { ∂ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ; A ( 2 ) ; α ] ∂ A ( 2 ) ( 2 ; j 1 ; x ) } α 0 δ A ( 2 ) ( 2 ; j 1 ; x ) . (56)

The indirect-effect term { δ R ( 2 ) [ j 2 ; j 1 ; U ( 2 ) ( 2 ; x ) ; A ( 2 ) ( 2 ; j 1 ; x ) ; α ; δ α ] } d i r can be computed after having determined the vectors V ( 2 ) ( 2 ; x ) and δ A ( 2 ) ( 2 ; j 1 ; x ) , which are the solutions of the following 3rd-Level Variational Sensitivity System (3rd-LVSS):

{ V M ( 3 ) [ 4 × 4 ; U ( 3 ) ( 4 ; x ) ; α ] V ( 3 ) ( 4 ; x ) } α 0 = { Q V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] } α 0 ,     x ∈ Ω x , (57)

{ B V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; V ( 3 ) ( 4 ; x ) ; α ; δ α ] } α 0 = 0 [ 4 ] ;     x ∈ ∂ Ω x ( α 0 ) (58)

where 0 [ 4 ] ≜ [ 0 , 0 , 0 , 0 ] † and where:

V M ( 3 ) ( 4 × 4 ; U ( 3 ) ; α ) ≜ ( V M ( 2 ) ( 2 × 2 ) 0 [ 2 × 2 ] V M 21 ( 3 ) ( 2 × 2 ) V M 22 ( 3 ) ( 2 × 2 ) ) ; (59)

U ( 3 ) ( 4 ; x ) ≜ ( U ( 2 ) ( 2 ; x ) A ( 2 ) ( 2 ; j 1 ; x ) ) ; ​ ​ ​ ​ ​       V ( 3 ) ( 4 ; x ) ≜ δ U ( 3 ) ( 4 ; x ) = ( V ( 2 ) ( 2 ; x ) δ A ( 2 ) ( 2 ; j 1 ; x ) ) ; (60)

V M 21 ( 3 ) ( 2 × 2 ; x ) ≜ ∂ { A M ( 2 ) [ 2 × 2 ; U ( 2 ) ; α ] A ( 2 ) ( 2 ; x ) } ∂ U ( 2 ) ( 2 ; x ) − ∂ Q A ( 2 ) [ 2 ; u ( 2 ) ( x ) ; α ] ∂ U ( 2 ) ( 2 ; x ) ; (61)

V M 22 ( 3 ) ( 2 × 2 ; x ) ≜ A M ( 2 ) [ 2 × 2 ; U ( 2 ) ; α ] ;         0 [ 2 × 2 ] ≜ ( 0 0 0 0 ) ; (62)

Q V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] ≜ ( Q V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; α ; δ α ] Q 2 ( 3 ) [ 2 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] ) ≜ { q V ( 3 ) [ 1 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] , ⋯ , q V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] } † ; (63)

q V ( 3 ) [ i ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] ≡ ∑ j 3 = 1 T P s V ( 3 ) [ i ; j 3 ; U ( 3 ) ( 4 ; x ) ; α ] δ α j 3 ;       i = 1 , 2 , 3 , 4 ; (64)

Q 2 ( 3 ) [ 2 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] ≜ ∂ Q A ( 2 ) [ 2 ; u ( 2 ) ( x ) ; α ] ∂ α ∂ α − ∂ { A M ( 2 ) [ 2 × 2 ; U ( 2 ) ( 2 ; x ) ; α ] A ( 2 ) ( 2 ; j 1 ; x ) } ∂ α ∂ α ; (65)

B V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; V ( 3 ) ( 4 ; x ) ; α ; δ α ] ≜ ( B V ( 2 ) [ 2 ; U ( 2 ) ( 2 ; x ) ; V ( 2 ) ( 2 ; x ) ; α ; δ α ] δ B A ( 2 ) [ 2 ; U ( 3 ) ( 4 ; x ) ; V ( 3 ) ( 4 ; x ) ; α ; δ α ] ) . (66)

The right-side of the 3rd-LVSS actually depends on the indices j 1 , j 2 , j 3 = 1 , ⋯ , T P , so the 3rd-LVSS would need to be solved T P 3 times to obtain each of the variational functions V ( 3 ) ( 4 ; j 1 , j 2 , j 3 ; x ) . Thus, solving the 3rd-LVSS would require T P 3 large-scale computations, which is unrealistic for large-scale systems comprising many parameters. Since the 3rd-LVSS is never actually solved but is only used to construct the corresponding adjoint sensitivity system, the actual dependence of the 3rd-LVSS on the indices j 1 , j 2 , j 3 = 1 , ⋯ , T P has been suppressed.

The 3rd-CASAM-N circumvents the need for solving the 3rd-LVSS by deriving an alternative expression for the indirect-effect term defined in Equation (56), in which the function V ( 3 ) ( 4 ; x ) is replaced by a 3rd-level adjoint function which is independent of parameter variations. This 3rd-level adjoint function is the solution of a 3rd-Level Adjoint Sensitivity System (3rd-LASS) which is constructed by applying the same principles as those used for constructing the 1st-LASS and the 2nd-LASS. The Hilbert space appropriate for constructing the 3rd-LASS, denoted as H 3 ( Ω x ) , comprises as elements block-vectors of the same form as V ( 3 ) ( 4 ; x ) . Thus, a generic block-vector in H 3 ( Ω x ) , denoted as Ψ ( 3 ) ( 4 ; x ) ≜ [ ψ ( 3 ) ( 1 ; x ) , ψ ( 3 ) ( 2 ; x ) , ψ ( 3 ) ( 3 ; x ) , ψ ( 3 ) ( 4 ; x ) ] † ∈ H 3 ( Ω x ) , comprises four TD-dimensional vector-components of the form ψ ( 3 ) ( i ; x ) ≜ [ ψ 1 ( 3 ) ( i ; x ) , ⋯ , ψ T D ( 3 ) ( i ; x ) ] † ∈ H 1 ( Ω x ) , i = 1 , 2 , 3 , 4 , where each of these four components is a TD-dimensional column vector. The inner product of two vectors Ψ ( 3 ) ( 4 ; x ) ∈ H 3 ( Ω x ) and Φ ( 3 ) ( 4 ; x ) ∈ H 3 ( Ω x ) in the Hilbert space H 3 ( Ω x ) will be denoted as 〈 Ψ ( 3 ) ( 4 ; x ) , Φ ( 3 ) ( 4 ; x ) 〉 3 and defined as follows:

〈 Ψ ( 3 ) ( 4 ; x ) , Φ ( 3 ) ( 4 ; x ) 〉 3 ≜ ∑ i = 1 4 〈 ψ ( 3 ) ( i ; x ) , φ ( 3 ) ( i ; x ) 〉 1 . (67)

The steps for constructing the 3rd-LASS are conceptually similar to those described in Sections 2.1 and 2.2 and are detailed in [27]. The final expressions for the 3rd-order sensitivities are as follows:

{ δ R ( 2 ) [ j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; δ α ] } α 0 = { ∂ R ( 2 ) [ j 2 ; j 1 ; U ( 3 ) ; α ] ∂ α δ α } α 0 − { [ P ^ ( 3 ) ( U ( 3 ) ; A ( 3 ) ; δ α ) ] ∂ Ω x } α 0 + { 〈 A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) , Q V ( 3 ) [ 4 ; U ( 3 ) ; α ; δ α ] 〉 3 } α 0 , (68)

where { [ P ^ ( 3 ) ( U ( 3 ) ; A ( 3 ) ; δ α ) ] ∂ Ω x } α 0 denotes residual boundary terms which may have not vanished automatically, and where the 3rd-level adjoint function A ( 3 ) ( 4 ; x ) ≜ [ a ( 3 ) ( 1 ; x ) , a ( 3 ) ( 2 ; x ) , a ( 3 ) ( 3 ; x ) , a ( 3 ) ( 4 ; x ) ] † ∈ H 3 ( Ω x ) is the solution of the following 3rd-LASS:

{ A M ( 3 ) [ 4 × 4 ; U ( 3 ) ( 4 ; x ) ; α ] A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) } α 0 = { Q A ( 3 ) [ 4 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; α ] } α 0 ,     j 1 = 1 , ⋯ , T P ;     j 2 = 1 , ⋯ , j 1 ; (69)

where:

Q A ( 3 ) [ 4 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; α ] ≜ [ q A ( 3 ) ( 1 ; j 2 ; j 1 ; U ( 3 ) ; α ) , ⋯ , q A ( 3 ) ( 4 ; j 2 ; j 1 ; U ( 3 ) ; α ) ] ; (70)

q A ( 3 ) ( 1 ; j 2 ; j 1 ; U ( 3 ) ; α ) ≜ ∂ R ( 2 ) [ j 2 ; j 1 ; u ( 2 ) ; a ( 2 ) ; α ] / ∂ u ( 1 ) ; (71)

q A ( 3 ) ( 2 ; j 2 ; j 1 ; U ( 3 ) ; α ) ≜ ∂ R ( 2 ) [ j 2 ; j 1 ; u ( 2 ) ; a ( 2 ) ; α ] / ∂ a ( 1 ) ; (72)

q A ( 3 ) ( 3 ; j 2 ; j 1 ; U ( 3 ) ; α ) ≜ ∂ R ( 2 ) [ j 2 ; j 1 ; u ( 2 ) ; a ( 2 ) ; α ] / ∂ a ( 2 ) ( 1 ; j 1 ; x ) ; (73)

q A ( 3 ) ( 3 ; j 2 ; j 1 ; U ( 3 ) ; α ) ≜ ∂ R ( 2 ) [ j 2 ; j 1 ; u ( 2 ) ; a ( 2 ) ; α ] / ∂ a ( 2 ) ( 2 ; j 1 ; x ) . (74)

A M ( 3 ) [ 4 × 4 ; U ( 3 ) ( 4 ; x ) ; α ] ≜ [ V M ( 3 ) ( 4 × 4 ; U ( 3 ) ; α ) ] * = ( { [ V M ( 2 ) ( 2 × 2 ) ] * } † { [ V M 21 ( 3 ) ( 2 × 2 ) ] * } † 0 [ 2 × 2 ] { [ V M 22 ( 3 ) ( 2 × 2 ) ] * } † ) , (75)

The boundary conditions to be satisfied by each of the 3rd-level adjoint functions A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ≜ [ a ( 3 ) ( 1 ; j 2 ; j 1 ; x ) , a ( 3 ) ( 2 ; j 2 ; j 1 ; x ) , a ( 3 ) ( 3 ; j 2 ; j 1 ; x ) , a ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ] † can be represented in operator form as follows:

{ B A ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] } α 0 = 0 [ 4 ] ; for     j 1 = 1 , ⋯ , T P ;     j 2 = 1 , ⋯ , j 1 ;     x ∈ ∂ Ω x ( α 0 ) . (76)

In component form, the total differential expressed by Equation (68) has the following expression:

{ δ R ( 2 ) [ j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; δ α ] } α 0 = ∑ j 3 = 1 T P { R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] } α 0 δ α j 3 ,     j 1 ; j 2 = 1 , ⋯ , T P , (77)

where the quantity R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] denotes the third-order sensitivity of the generic scalar-valued response R [ u ( x ) ; α ] with respect to any three model parameters α j 1 , α j 2 , α j 3 , and has the following expression:

R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] ≜ ∂ R ( 2 ) [ j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; α ] ∂ α j 3 − [ ∂ P ^ ( 3 ) ( U ( 3 ) ; A ( 3 ) ; δ α ) ] ∂ Ω x ∂ α j 3   + ∑ i = 1 4 〈 a ( 3 ) ( i ; j 2 ; j 1 ; x ) , s V ( 3 ) [ i ; j 3 ; j 1 ; U ( 3 ) ( 4 ; x ) ; α ] 〉 1 ≜ ∂ 3 R [ u ( x ) ; α ] ∂ α j 3 ∂ α j 2 ∂ α j 1 ,         for     j 1 , j 2 , j 3 = 1 , ⋯ , T P . (78)

2.4. The Fourth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (4<sup>th</sup>-CASAM-N)

Assuming that the 3rd-order sensitivities R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] ≡ ∂ 3 R [ u ( x ) ; α ] / ∂ α j 3 ∂ α j 2 ∂ α j 1 satisfy the conditions stated in Equations (6) and (7) for each j 1 , j 2 , j 3 = 1 , ⋯ , T P , the 1st-order total G-differential of R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] will exist and will be linear in the variations V ( 3 ) ( 4 ; x ) ≜ δ U ( 3 ) ( 4 ; x ) and δ A ( 3 ) ( 4 ; x ) in a neighborhood around the nominal values of the parameters and the respective state functions. By definition, the 1st-order total G-differential of R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] , which will be denoted as { δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; V ( 3 ) ( 4 ; j 1 ; x ) ; δ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; δ α ] } α 0 ,

is given by the following expression:

{ δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; V ( 3 ) ( 4 ; j 1 ; x ) ; δ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; δ α ] } α 0 ≜ { δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; δ α ] } d i r + { δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; V ( 3 ) ( 4 ; j 1 ; x ) ; δ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ] } i n d . (79)

where:

{ δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; δ α ] } d i r ≜ { ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] ∂ α δ α } α 0 , (80)

{ δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; V ( 3 ) ( 4 ; j 1 ; x ) ; δ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ] } i n d ≜ { ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] ∂ U ( 3 ) ( 4 ; j 1 ; x ) } α 0 V ( 3 ) ( 4 ; j 1 ; x )     + { ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] ∂ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) } α 0 δ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) , (81)

The vectors V ( 3 ) ( 4 ; x ) and δ A ( 3 ) ( 4 ; x ) are the solutions of the following “4th-order variational sensitivity system” (4th-LVSS), which is derived in detail in [27]:

{ V M ( 4 ) [ 8 × 8 ; U ( 4 ) ; α ] V ( 4 ) ( 8 ; x ) } α 0 = { Q V ( 4 ) [ 8 ; U ( 4 ) ( 8 ; x ) ; α ; δ α ] } α 0 ,     x ∈ Ω x , (82)

{ B V ( 4 ) [ 8 ; U ( 4 ) ( 8 ; x ) ; V ( 4 ) ( 8 ; x ) ; α ; δ α ] } α 0 = 0 [ 8 ] ≜ [ 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 ] † ;     x ∈ ∂ Ω x ( α 0 ) , (83)

where

V M ( 4 ) ( 8 × 8 ) ≜ ( V M ( 3 ) ( 4 × 4 ) 0 [ 4 × 4 ] V M 21 ( 4 ) ( 4 × 4 ) V M 22 ( 4 ) ( 4 × 4 ) ) ; (84)

U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ≜ ( U ( 3 ) ( 4 ; j 1 ; x ) A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ) ; (85)

V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ≜ δ U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) = ( V ( 3 ) ( 4 ; j 1 ; x ) δ A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) )

= [ v ( 1 ) ( x ) , δ a ( 1 ) ( x ) , δ a ( 2 ) ( 1 ; j 1 ; x ) , δ a ( 2 ) ( 2 ; j 1 ; x ) , δ a ( 3 ) ( 1 ; j 2 ; j 1 ; x ) ,             ⋯ , δ a ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ] † ; (86)

V M 21 ( 4 ) ( 4 × 4 ; x ) ≜ ∂ { A M ( 3 ) ( 4 × 4 ; U ( 3 ) ; α ) A ( 3 ) ( 4 ; x ) } ∂ U ( 3 ) ( 4 ; j 1 ; x ) − ∂ Q A ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; α ] ∂ U ( 3 ) ( 4 ; j 1 ; x ) ; (87)

V M 22 ( 4 ) ( 4 × 4 ; j 1 ; x ) ≜ A M ( 3 ) [ 4 × 4 ; U ( 3 ) ( 4 ; j 1 ; x ) ; α ] ; (88)

Q V ( 4 ) [ 8 ; U ( 4 ) ( 4 ; x ) ; α ; δ α ] ≜ ( Q V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; x ) ; α ; δ α ] Q 2 ( 4 ) [ 4 ; U ( 4 ) ( 4 ; x ) ; α ; δ α ] ) ≜ { q V ( 4 ) [ 1 ; U ( 4 ) ( 4 ; x ) ; α ; δ α ] , ⋯ , q V ( 4 ) [ 8 ; U ( 4 ) ( 4 ; x ) ; α ; δ α ] } † ; (89)

q V ( 4 ) [ i ; U ( 4 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; δ α ] ≡ ∑ j 4 = 1 T P s V ( 4 ) [ i ; j 4 ; U ( 4 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] δ α j 4 ;       i = 1 , ⋯ , 8 ; (90)

Q 2 ( 4 ) [ 4 ; U ( 4 ) ( 4 ; j 2 ; j 1 ; x ) ; α ; δ α ] ≜ ∂ Q A ( 3 ) [ 4 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; α ] ∂ α ∂ α − ∂ { A M ( 3 ) [ 4 × 4 ; U ( 3 ) ( 4 ; j 1 ; x ) ; α ] A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) } ∂ α ∂ α ; (91)

B V ( 4 ) [ 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ; δ α ] ≜ ( B V ( 3 ) [ 4 ; U ( 3 ) ( 4 ; j 1 ; x ) ; V ( 3 ) ( 4 ; j 1 ; x ) ; α ; δ α ] δ B A ( 3 ) [ 4 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ; δ α ] ) . (92)

The right-side of the 4th-LVSS actually depends on the indices j 1 , j 2 , j 3 , j 4 = 1 , ⋯ , T P , so the 4th-LVSS would need to be solved T P 4 times in order to obtain each of the variational functions V ( 4 ) ( 4 ; j 1 , j 2 , j 3 , j 4 ; x ) , which is unrealistic for large-scale systems comprising many parameters. Since the 4th-LVSS is never actually solved but is only used to construct the corresponding adjoint sensitivity system, the actual dependence of the 4th-LVSS on the indices j 1 , j 2 , j 3 = 1 , ⋯ , T P has been suppressed.

The 4th-CASAM-N circumvents the need for solving the 4th-LVSS by deriving an alternative expression for the indirect-effect term defined in Equation (81), in which the function V ( 4 ) ( 8 ; x ) is replaced by a 4th-level adjoint function which is independent of parameter variations. This 4th-level adjoint function will be the solution of a 4th-Level Adjoint Sensitivity System (4th-LASS) which will be constructed by applying the same principles as those used for constructing the 1st-LASS, the 2nd-LASS and the 3rd-LASS. The Hilbert space appropriate for constructing the 4th-LASS will be denoted as H 4 ( Ω x ) and comprises as elements block-vectors of the same form as V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) . Thus, a generic block-vector in H 4 ( Ω x ) will have the structure Ψ ( 4 ) ( 8 ; x ) ≜ [ ψ ( 4 ) ( 1 ; x ) , ⋯ , ψ ( 4 ) ( 8 ; x ) ] † ∈ H 4 ( Ω x ) , comprising 8 TD-dimensional vectors of the form ψ ( 4 ) ( i ; x ) ≜ [ ψ 1 ( 4 ) ( i ; x ) , ⋯ , ψ T D ( 4 ) ( i ; x ) ] † ∈ H 1 ( Ω x ) , i = 1 , ⋯ , 8 . The inner product of two vectors Ψ ( 4 ) ( 8 ; x ) ∈ H 4 ( Ω x ) and Φ ( 4 ) ( 8 ; x ) ∈ H 4 ( Ω x ) in the Hilbert space H 4 ( Ω x ) will be denoted as 〈 Ψ ( 4 ) ( 8 ; x ) , Φ ( 4 ) ( 8 ; x ) 〉 4 and defined as follows:

〈 Ψ ( 4 ) ( 8 ; x ) , Φ ( 4 ) ( 8 ; x ) 〉 4 ≜ ∑ i = 1 8 〈 ψ ( 4 ) ( i ; x ) , φ ( 4 ) ( i ; x ) 〉 1 . (93)

The steps for constructing the 4th-LASS are conceptually similar to those described in Sections 2.1 and 2.2 and are detailed in [27]. The final expressions for the 4th-order sensitivities are as follows:

{ δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; δ α ] } α 0 = { ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] ∂ α δ α } α 0 − { [ P ^ ( 4 ) ( U ( 4 ) ; A ( 4 ) ; δ α ) ] ∂ Ω x } α 0 + { 〈 A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) , Q V ( 4 ) [ 8 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ; δ α ] 〉 3 } α 0 . (94)

In component form, the total differential expressed by Equation (94) can be written in the following form, for each j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 :

{ δ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; δ α ] } α 0 = ∑ j 4 = 1 T P { R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] } α 0 δ α j 4 , (95)

where the quantity R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] denotes the fourth-order sensitivity of the generic scalar-valued response R [ u ( x ) ; α ] with respect to any four model parameters α j 1 , α j 2 , α j 3 , α j 4 , and has the following expression:

R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] ≜ ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ( 4 ; j 1 ; x ) ; A ( 3 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] ∂ α j 4 − [ ∂ P ^ ( 4 ) ( U ( 4 ) ; A ( 4 ) ; δ α ) ] ∂ Ω x ∂ α j 4 + ∑ i = 1 8 〈 a ( 4 ) ( i ; j 3 ; j 2 ; j 1 ; x ) , s V ( 4 ) [ i ; j 4 ; j 1 ; U ( 4 ) ( 4 ; j 2 ; j 1 ; x ) ; α ] 〉 1 ≡ ∂ 4 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 ∂ α j 4 ;         j 1 , j 2 , j 3 , j 4 = 1 , ⋯ , j 3 = 1 , ⋯ , T P . (96)

In Equation (96), the quantity { [ P ^ ( 4 ) ( U ( 4 ) ; A ( 4 ) ; δ α ) ] ∂ Ω x } α 0 denotes residual boundary terms which may have not vanish automatically, and the 4th-level adjoint functions A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ≜ [ a ( 4 ) ( 1 ; j 3 ; j 2 ; j 1 ; x ) , ⋯ , a ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ] † ∈ H 4 ( Ω x ) , satisfy the following 4th-LASS, for each value of the indices j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 :

A M ( 4 ) [ 8 × 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) = Q A ( 4 ) [ 8 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] , (97)

subject to boundary conditions represented in operator form as follows:

{ B A ( 4 ) [ 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] } α 0 = 0 [ 8 ] ,       x ∈ ∂ Ω x ( α 0 ) ;       j 1 = 1 , ⋯ , T P ;     j 2 = 1 , ⋯ , j 1 ;     j 3 = 1 , ⋯ , j 2 . (98)

The quantities which appear in Equations (97) and (98) are defined as follows:

Q A ( 4 ) [ 8 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] ≜ { q A ( 4 ) [ 1 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] , ⋯ , q A ( 4 ) [ 8 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] } † ,     j 1 = 1 , ⋯ , T P ;     j 2 = 1 , ⋯ , j 1 ;   j 3 = 1 , ⋯ , j 2 ; (99)

q A ( 4 ) [ 1 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] ≜ ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] / ∂ u ( x ) ; (100)

q A ( 4 ) [ 2 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] ≜ ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] / ∂ a ( 1 ) ( x ) ; (101)

q A ( 4 ) [ 2 + i ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] ≜ ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] ∂ a ( 2 ) ( i ; j 1 ; x ) ;     i = 1 , 2 ; (102)

q A ( 4 ) [ 4 + i ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 4 ; j 1 ; x ) ; α ] ≜ ∂ R ( 3 ) [ j 3 ; j 2 ; j 1 ; U ( 3 ) ; A ( 3 ) ; α ] ∂ a ( 3 ) ( i ; j 2 ; j 1 ; x ) ;       i = 1 , 2 , 3 , 4 ; (103)

{ 〈 A ( 4 ) ( 8 ; x ) , V M ( 4 ) [ 8 × 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) 〉 3 } α 0 = { 〈 A ( 4 ) ( 8 ; x ) , Q V ( 4 ) [ 8 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ; δ α ] 〉 3 } α 0 ,       x ∈ Ω x . (104)

A M ( 4 ) [ 8 × 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] ≜ [ V M ( 4 ) ( 8 × 8 ; U ( 4 ) ; α ) ] * = ( { [ V M ( 3 ) ( 4 × 4 ) ] * } † { [ V M 21 ( 4 ) ( 4 × 4 ) ] * } † 0 [ 4 × 4 ] { [ V M 22 ( 3 ) ( 4 × 4 ) ] * } † ) . (105)

2.5. The Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5<sup>th</sup>-CASAM-N)

Each of the 4th-order sensitivities R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] will be assumed to satisfy the conditions stated in Equations (6) and (7) for each j 1 , j 2 , j 3 , j 4 = 1 , ⋯ , T P . Hence, the 1st-order total G-differential of R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] will exist and will be linear in the variations V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ≜ δ U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) and δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) in a neighborhood around the nominal values of the parameters and the respective state functions. By definition, the 1st-order total G-differential of R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] is given by the following expression:

{ δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ; δ A ( 4 ) ; δ α ] } α 0 ≜ d d ε { R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) + ε V ( 4 ) ; A ( 4 ) + ε δ A ( 4 ) ; α + ε δ α } ε = 0 ≜ { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ; δ α ] } d i r + { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ] } i n d . (106)

In Equation (106), the quantity { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; δ α ] } d i r denotes the “direct-effect term,” which comprises all of the dependencies on the vector δ α of parameter variations and has the following expression:

{ δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; δ α ] } d i r ≜ { ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] ∂ α δ α } α 0 = ∑ j 5 = 1 T P { ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; α ] ∂ α j 5 } α 0 δ α j 5 . (107)

In Equation (106), the quantity { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ; δ A ( 4 ) ] } i n d denotes the “indirect-effect term,” which comprises all of the dependencies on the vectors V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) and δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) of variations in the state functions U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) and A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ; this indirect-effect term is defined as follows:

{ δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ; δ A ( 4 ) ] } i n d ≜ { ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) } α 0 V ( 4 ) ( 8 ; j 2 ; j 1 ; x )     + { ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) } α 0 δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) , (108)

where:

∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ≜ ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ u ( x ) v ( 1 ) ( x ) + ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ a ( 1 ) ( x ) δ a ( 1 ) ( x ) + ∑ i = 1 2 ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ a ( 2 ) ( i ; j 1 ; x ) δ a ( 2 ) ( i ; j 1 ; x )

+ ∑ i = 1 4 ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ a ( 3 ) ( i ; j 2 ; j 1 ; x ) δ a ( 3 ) ( i ; j 2 ; j 1 ; x ) , (109)

and

∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ≜ ∑ i = 1 8 ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ] ∂ a ( 4 ) ( i ; j 3 ; j 2 ; j 1 ; x ) δ a ( 4 ) ( i ; j 3 ; j 2 ; j 1 ; x )   . (110)

The direct-effect term { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; δ α ] } d i r can be computed immediately. On the other hand, the indirect-effect term { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ; δ A ( 4 ) ] } i n d can be computed only after having determined the vectors V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) and δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) . The vectors V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) and δ A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) are the solution of the 5th-Level Variational Sensitivity System (5th-LVSS), which is obtained by concatenating the 4th-LVSS defined by Equations (82) and (83) together with the G-differentiated 4th-LASS, i.e., the G-differentiated Equations (97) and (98). Thus, the 5th-LVSS has the following expression:

{ V M ( 5 ) [ 2 4 × 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ] V ( 5 ) ( 2 4 ; x ) } α 0 = { Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] } α 0 ,     x ∈ Ω x , (111)

{ B V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; V ( 5 ) ( 2 4 ; x ) ; α ; δ α ] } α 0 = 0 [ 2 4 ] ;     x ∈ ∂ Ω x ( α 0 ) ; (112)

where:

U ( 5 ) ( 2 4 ; x ) ≜ ( U ( 4 ) ( 8 ; x ) A ( 4 ) ( 8 ; x ) ) ;           V ( 5 ) ( 2 4 ; x ) ≜ δ U ( 5 ) ( 2 4 ; x ) ≜ ( δ U ( 4 ) ( 8 ; x ) δ A ( 4 ) ( 8 ; x ) ) = [ v ( 1 ) ( x ) , δ a ( 1 ) ( x ) , δ a ( 2 ) ( 1 ; j 1 ; x ) , δ a ( 2 ) ( 2 ; j 1 ; x ) , δ a ( 3 ) ( 1 ; j 2 ; j 1 ; x ) , ⋯ , δ a ( 3 ) ( 4 ; j 2 ; j 1 ; x ) , δ a ( 4 ) ( 1 ; j 3 ; j 2 ; j 1 ; x ) , ⋯ , δ a ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) ] † ; (113)

V M ( 5 ) ( 2 4 × 2 4 ) ≜ ( V M ( 4 ) ( 2 3 × 2 3 ) 0 [ 2 3 × 2 3 ] V M 21 ( 5 ) ( 2 3 × 2 3 ) V M 22 ( 5 ) ( 2 3 × 2 3 ) ) ; (114)

V M 21 ( 5 ) ( 2 3 × 2 3 ) ≜ − ∂ Q A ( 4 ) [ 8 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] ∂ U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) + ∂ { A M ( 4 ) [ 8 × 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) } ∂ U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; (115)

V M 22 ( 5 )   ( 2 3 × 2 3 ) ≜ A M ( 4 ) [ 8 × 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] ; (116)

Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] ≜ ( Q V ( 4 ) [ 8 ; U ( 4 ) ( 4 ; x ) ; α ; δ α ] Q 2 ( 5 ) [ 8 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] ) ≡ { q V ( 5 ) [ 1 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] , ⋯ , q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] } † ; (117)

Q 2 ( 5 ) [ 8 ; U ( 5 ) ( 2 4 ; j 3 ; j 2 ; j 1 ; x ) ; α ; δ α ] ≜ ∂ Q A ( 4 ) [ 8 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] ∂ α ∂ α − ∂ { A M ( 4 ) [ 8 × 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ] A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) } ∂ α ∂ α   ; (118)

B V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; j 3 ; j 2 ; j 1 ; x ) ; V ( 5 ) ( 2 4 ; j 3 ; j 2 ; j 1 ; x ) ; α ; δ α ] ≜ ( B V ( 4 ) [ 8 ; U ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; V ( 4 ) ( 8 ; j 2 ; j 1 ; x ) ; α ; δ α ] δ B A ( 4 ) [ 8 ; U ( 5 ) ( 2 4 ; j 3 ; j 2 ; j 1 ; x ) ; V ( 5 ) ( 2 4 ; j 3 ; j 2 ; j 1 ; x ) ; α ; δ α ] ) (119)

The quantities which appear in Equations (113)-(119) are evaluated at the nominal values of the parameters and respective state functions, but the notation { } α 0 , which indicates this evaluation, has been omitted, in order to simplify the notation.

The quantities q V ( 5 ) [ i ; U ( 5 ) ( 2 4 ; j 3 ; j 2 ; j 1 ; x ) ; α ; δ α ] , i = 1 , ⋯ , 2 4 , are linear in the parameter variations δ α i , i = 1 , ⋯ , T P , and can therefore be written in the following form:

q V ( 5 ) [ i ; U ( 5 ) ; α ; δ α ] ≜ ∑ j 5 = 1 T P s V ( 5 ) [ i ; j 5 ; ⋯ ; j 1 ; U ( 5 ) ; α ] δ α j 5 ;     i = 1 , ⋯ , 2 4 . (120)

The variational matrix V M ( 5 ) ( 2 4 × 2 4 ) comprises 4 block-matrices, each block comprising ( 8 × 8 ) T D 2 components/elements. Thus, the matrix V M ( 5 ) ( 2 4 × 2 4 ) comprises a total of ( 2 4 × 2 4 ) T D 2 components/elements. Each of the vectors V ( 5 ) ( 2 4 ; x ) , Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] and B V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; V ( 5 ) ( 2 4 ; x ) ; α ; δ α ] comprises 2 4 T D -dimensional vectors, as shown in their respective definitions.

The source-term on the right-side of the 5th-LVSS, cf. Equation (111), actually depends on the indices j 1 , j 2 , j 3 , j 4 , j 5 = 1 , ⋯ , T P . Hence, the 5th-LVSS would need to be solved T P 5 times in order to obtain each of the variational functions V ( 5 ) ( 2 4 ; j 1 , j 2 , j 3 , j 4 , j 5 ; x ) , which is unrealistic for large-scale systems comprising many parameters. Since the 5th-LVSS is never actually solved but is only used to construct the corresponding adjoint sensitivity system, the actual dependence of the 5th-LVSS on the indices j 1 , j 2 , j 3 , j 4 , j 5 = 1 , ⋯ , T P has been suppressed.

The 5th-CASAM-N methodology, which will presented in the remainder of this Section, circumvents the need for solving the 5th-LVSS by deriving an alternative expression for the indirect-effect term defined in Equation (108), in which the function V ( 5 ) ( 2 4 ; x ) is replaced by a 5th-level adjoint function which is independent of parameter variations. This 5th-level adjoint function will be the solution of a 5th-Level Adjoint Sensitivity System (5th-LASS) which will be constructed by applying the same principles as those used for constructing the 1st-LASS, 2nd-LASS, 3rd-LASS, and the 4th-LASS. The Hilbert space appropriate for constructing the 5th-LASS will be denoted as H 5 ( Ω x ) and comprises as elements block-vectors of the same form as V ( 5 ) ( 2 4 ; x ) . Thus, a generic block-vector in H 5 ( Ω x ) will have the structure Ψ ( 5 ) ( 2 4 ; x ) ≜ [ ψ ( 5 ) ( 1 ; x ) , ⋯ , ψ ( 5 ) ( 2 4 ; x ) ] † ∈ H 5 ( Ω x ) , comprising 24 TD-dimensional vectors of the form ψ ( 5 ) ( i ; x ) ≜ [ ψ 1 ( 5 ) ( i ; x ) , ⋯ , ψ T D ( 5 ) ( i ; x ) ] † ∈ H 1 ( Ω x ) , i = 1 , ⋯ , 2 4 . The inner product of two vectors Ψ ( 5 ) ( 2 4 ; x ) ∈ H 5 ( Ω x ) and Φ ( 5 ) ( 2 4 ; x ) ∈ H 5 ( Ω x ) in the Hilbert space H 5 ( Ω x ) will be denoted as 〈 Ψ ( 4 ) ( 8 ; x ) , Φ ( 4 ) ( 8 ; x ) 〉 5 and defined as follows:

〈 Ψ ( 5 ) ( 2 4 ; x ) , Φ ( 5 ) ( 2 4 ; x ) 〉 5 ≜ ∑ i = 1 2 4 〈 ψ ( 5 ) ( i ; x ) , φ ( 5 ) ( i ; x ) 〉 1 . (121)

The inner product defined in Equation (121) is continuous in α , in a neighborhood around α 0 . Using the definition of the inner product defined in Equation (121), construct the inner product of Equation (111) with a vector A ( 5 ) ( 2 4 ; x ) ≜ [ a ( 5 ) ( 1 ; x ) , ⋯ , a ( 5 ) ( 2 4 ; x ) ] † ∈ H 5 ( Ω x ) to obtain the following relation:

{ 〈 A ( 5 ) ( 2 4 ; x ) , V M ( 5 ) [ 2 4 × 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ] V ( 5 ) ( 2 4 ; x ) 〉 5 } α 0 = { 〈 A ( 5 ) ( 2 4 ; x ) , Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] 〉 5 } α 0 ,     x ∈ Ω x . (122)

The inner product on the left-side of Equation (122) is further transformed by using the definition of the adjoint operator to obtain the following relation:

{ 〈 A ( 5 ) ( 2 4 ; x ) , V M ( 5 ) [ 2 4 × 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ] V ( 5 ) ( 2 4 ; x ) 〉 5 } α 0 = { 〈 V ( 5 ) ( 2 4 ; x ) , A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) A ( 5 ) ( 2 4 ; x ) 〉 3 } α 0 + { [ P ( 5 ) ( U ( 5 ) ; A ( 5 ) ; V ( 5 ) ; α ) ] ∂ Ω x } α 0 , (123)

where:

A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) ≜ [ V M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) ] * = ( { [ V M ( 4 ) ( 2 3 × 2 3 ) ] * } † { [ V M 21 ( 4 ) ( 2 3 × 2 3 ) ] * } † 0 [ 2 3 × 2 3 ] { [ V M 22 ( 4 ) ( 2 3 × 2 3 ) ] * } † ) , (124)

and where { [ P ( 5 ) ( U ( 5 ) ; A ( 5 ) ; V ( 5 ) ; α ) ] ∂ Ω x } α 0 denotes the corresponding bilinear concomitant on the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions. The adjoint matrix-valued operator A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) comprises ( 2 4 × 2 4 ) T D 2 components/elements, while the adjoint function A ( 5 ) ( 2 4 ; x ) ∈ H 5 ( Ω x ) comprises ( 2 4 × T D ) components/elements.

The domain of the adjoint matrix-operator A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) is specified by requiring that the function A ( 5 ) ( 2 4 ; x ) ∈ H 5 ( Ω x ) satisfies adjoint boundary/initial conditions denoted as follows:

{ B A ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; j 2 ; j 1 ; x ) ; A ( 4 ) ( 2 4 ; x ) ; α ] } α 0 = 0 [ 2 4 ] ,   x ∈ ∂ Ω x ( α 0 ) . (125)

The 5th-level adjoint boundary/initial conditions represented by Equation (125) are determined by requiring that: 1) they must be independent of unknown values of V ( 5 ) ( 2 4 ; x ) ; 2) the substitution of the boundary and/or initial conditions represented by Equations (112) and (125) into the expression of { [ P ( 5 ) ( U ( 5 ) ; A ( 5 ) ; V ( 5 ) ; α ; δ α ) ] ∂ Ω x } α 0 must cause all terms containing unknown values of V ( 5 ) ( 2 4 ; x ) to vanish.

Implementing the boundary/initial conditions represented by Equations (112) and (125) into Equation (123) will transform the later relation into the following form:

{ 〈 V ( 5 ) ( 2 4 ; x ) , A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) A ( 5 ) ( 2 4 ; x ) 〉 3 } α 0 = { 〈 A ( 5 ) ( 2 4 ; x ) , V M ( 5 ) [ 2 4 × 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ] V ( 5 ) ( 2 4 ; x ) 〉 5 } α 0 − { [ P ^ ( 5 ) ( U ( 5 ) ; A ( 5 ) ; V ( 5 ) ; α ; δ α ) ] ∂ Ω x } α 0 , (126)

where { [ P ^ ( 5 ) ( U ( 5 ) ; A ( 5 ) ; V ( 5 ) ; α ; δ α ) ] ∂ Ω x } α 0 denotes residual boundary terms which may have not vanish automatically. The right-side of Equation (122) is now used in the first term on the right-side of Equation (126) to obtain the following relation:

{ 〈 V ( 5 ) ( 2 4 ; x ) , A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) A ( 5 ) ( 2 4 ; x ) 〉 5 } α 0 = { 〈 A ( 5 ) ( 2 4 ; x ) , Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] 〉 5 } α 0 − { [ P ^ ( 5 ) ( U ( 5 ) ; A ( 5 ) ; V ( 5 ) ; α ; δ α ) ] ∂ Ω x } α 0 . (127)

The definition of the 5th-level adjoint function A ( 5 ) ( 2 4 ; x ) ≜ [ a ( 5 ) ( 1 ; x ) , ⋯ , a ( 5 ) ( 2 4 ; x ) ] † ∈ H 5 ( Ω x ) is now completed by requiring that the left-side of Equation (127) and the right-side of Equation (108) represent the “indirect-effect term” { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ; δ A ( 4 ) ] } i n d , for each of the indices j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 ; j 4 = 1 , ⋯ , j 3 . Hence, there will be T P ( T P + 1 ) ( T P + 2 ) ( T P + 3 ) / 24 distinct 5th-level adjoint functions A ( 5 ) ( 2 4 ; x ) ≜ [ a ( 5 ) ( 1 ; x ) , ⋯ , a ( 5 ) ( 2 4 ; x ) ] † ∈ H 5 ( Ω x ) , each corresponding to one combination of the indices j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 ; j 4 = 1 , ⋯ , j 3 . Each of these distinct 5th-level adjoint functions will correspond to a specific ( j 1 , j 2 , j 3 , j 4 ) -dependent indirect-effect term.

The left-side of Equation (127) will be identical to the right-side of Equation (108) by requiring that the following relation be satisfied by the 5th-level adjoint functions A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) ≜ [ a ( 4 ) ( 1 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) , ⋯ , a ( 4 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) ] † , for each value of the indices j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 ; j 4 = 1 , ⋯ , j 3 :

A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) A ( 5 ) ( 2 4 ; j 4 , ⋯ , j 1 ; x ) = Q A ( 5 ) [ 2 4 ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] , (128)

where:

Q A ( 5 ) [ 2 4 ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ≜ { q A ( 5 ) [ 1 ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] , ⋯ , q A ( 4 ) [ 8 ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] } † j 1 = 1 , ⋯ , T P ;     j 2 = 1 , ⋯ , j 1 ;   j 3 = 1 , ⋯ , j 2 ;   j 4 = 1 , ⋯ , j 3 ; (129)

q A ( 5 ) [ 1 ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ≜ ∂ R ( 4 ) [ j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] / ∂ u ( x ) ; (130)

q A ( 5 ) [ 2 ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ≜ ∂ R ( 4 ) [ j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] / ∂ a ( 1 ) ( x ) ; (131)

q A ( 5 ) [ 2 + i ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ≜ ∂ R ( 4 ) [ j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] / ∂ a ( 2 ) ( i ; j 1 ; x ) ;     i = 1 , 2 ; (132)

q A ( 5 ) [ 4 + i ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ≜ ∂ R ( 4 ) [ j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ∂ a ( 3 ) ( i ; j 2 ; j 1 ; x ) ;       i = 1 , 2 , 3 , 4 ; (133)

q A ( 5 ) [ 8 + i ; j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ≜ ∂ R ( 4 ) [ j 4 , ⋯ , j 1 ; U ( 5 ) ; α ] ∂ a ( 4 ) ( i ; j 3 ; j 2 ; j 1 ; x ) ;       i = 1 , ⋯ , 8 ; (134)

The boundary conditions to be satisfied by each of the 5th-level adjoint functions A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) ≜ [ a ( 4 ) ( 1 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) , ⋯ , a ( 4 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) ] † are those represented by Equation (125). The system of equations represented by Equations (128) and (125) will be called the 5th-Level Adjoint Sensitivity System (5th-LASS); its solution, A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) , will be called the 5th-level adjoint function. Using the equations underlying the 5th-LASS and Equation (127) in Equation (108) yields the following expression for the indirect-effect term:

{ δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 4 ) ; A ( 4 ) ; α ; V ( 4 ) ; δ A ( 4 ) ] } i n d ≜ { 〈 A ( 5 ) ( 2 4 ; x ) , Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] 〉 5 } α 0 − { [ P ^ ( 5 ) ( U ( 5 ) ; A ( 5 ) ; α ; δ α ) ] ∂ Ω x } α 0 ≡ { δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; A ( 4 ) ; α ; δ α ] } i n d . (135)

As the identity in Equation (135) indicates, the dependence of the indirect-effect term on the function V ( 5 ) ( 2 4 ; x ) has been replaced by the dependence on the adjoint function A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) , for each j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 ; j 4 = 1 , ⋯ , j 3 . Replacing the expression obtained in Equation (135) for the indirect-effect term together with the expression for the direct-effect term provided in Equation (107) yields the following expression for the total differential defined by Equation (106):

{ δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; A ( 5 ) ; α ; δ α ] } α 0 = { ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; α ] ∂ α δ α } α 0 − { [ P ^ ( 5 ) ( U ( 5 ) ; A ( 5 ) ; α ; δ α ) ] ∂ Ω x } α 0 { 〈 A ( 5 ) ( 2 4 ; x ) , Q V ( 5 ) [ 2 4 ; U ( 5 ) ( 2 4 ; x ) ; α ; δ α ] 〉 5 } α 0 . (136)

In component form, the total differential expressed by Equation (136) can be written in the following form, for each j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 ; j 4 = 1 , ⋯ , j 3 :

{ δ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; A ( 5 ) ; α ; δ α ] } α 0 = ∑ j 5 = 1 T P { R ( 5 ) [ j 5 ; j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; A ( 5 ) ; α ] } α 0 δ α j 5 , (137)

where the quantity R ( 5 ) [ j 5 ; j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; A ( 5 ) ; α ] denotes the fifth-order sensitivity of the generic scalar-valued response R [ u ( x ) ; α ] with respect to any five model parameters α j 1 , α j 2 , α j 3 , α j 4 , α j 5 , and has the following expression:

R ( 5 ) [ j 5 ; j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; A ( 5 ) ; α ] ≜ ∂ R ( 4 ) [ j 4 ; j 3 ; j 2 ; j 1 ; U ( 5 ) ; α ] ∂ α j 5 − [ P ^ ( 5 ) ( U ( 5 ) ; A ( 5 ) ; α ; δ α ) ] ∂ Ω x ∂ α j 5   + ∑ i = 1 2 4 〈 a ( 5 ) ( i ; j 4 , ⋯ , j 1 ; x ) , s V ( 5 ) [ i ; j 5 ; ⋯ ; j 1 ; U ( 5 ) ; α ] 〉 1 ≡ ∂ 5 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 ∂ α j 4 ∂ α j 5 . (138)

As Equations (125) and (128) indicate, solving the 5th-LASS provides the 5th-level adjoint function A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) , for each of the indices j 1 = 1 , ⋯ , T P ; j 2 = 1 , ⋯ , j 1 ; j 3 = 1 , ⋯ , j 2 ; j 4 = 1 , ⋯ , j 3 . In turn, the availability of A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) enables the exact and efficient computation of all of the partial fifth-order sensitivities, ∂ 5 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 ∂ α j 4 ∂ α j 5 .

The adjoint matrix A M ( 5 ) ( 2 4 × 2 4 ; U ( 5 ) ; α ) is block-diagonal; therefore, solving the 5th-LASS is equivalent to solving five times the 1st-LASS, with five different source terms. The 5th-LASS was designated as the “fifth-level” rather than “fifth-order” adjoint sensitivity system since the 5th-LASS does not involve any explicit 2nd-order, 3rd-order, 4th-order and/or 5th-order G-derivatives of the operators underlying the original system but involves the inversion of the operators similar to those that needed to be inverted for solving the 1st-LASS.

By solving the 5th-LASS T P ( T P + 1 ) ( T P + 2 ) ( T P + 3 ) / 24 times, the 5th-order mixed sensitivities ∂ 5 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 ∂ α j 4 ∂ α j 5 will be computed four times, in four different ways using distinct adjoint functions. Consequently, the multiple symmetries inherent to the fifth-order sensitivities provide an intrinsic numerical verification that the components of the 1st-, 2nd-, 3rd-, 4th-, and 5th-level adjoint functions are computed accurately.

The structure of the 5th-LASS enables full flexibility for prioritizing the computation of the 5th-order sensitivities. The computation of the 5th-order sensitivities would be prioritized based on the relative magnitudes of the 4th-order sensitivities, so that the unimportant 5th-order sensitivities can be deliberately neglected while knowing the error incurred by neglecting them.

3. Conclusions

This work has presented the mathematical framework of the “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (5th-CASAM-N), which generalizes and extends all of the previous works performed to date on this subject. The qualifier “comprehensive” distinguishes this mathematical framework from previous works in that the 5th-CASAM-N enables the exact and efficient computation of response sensitivities not only to internal model parameters but also enables the computation of sensitivities to uncertain boundaries of the system’s domain of definition, thus enabling the quantification of uncertainties stemming from manufacturing tolerances.

The mathematical framework of the 5th-CASAM-N encompasses and builds upon the frameworks of all of the lower-order sensitivity analysis frameworks for nonlinear systems, namely:

1) The 1st-CASAM-N, which enables the exact and efficient computation of the 1st-order sensitivities. A single large-scale computation is needed for solving the 1st-Level Adjoint Sensitivity System (1st-LASS) to obtain the 1st-level adjoint sensitivity function a ( 1 ) ( x ) . Subsequently, the function a ( 1 ) ( x ) is used in inexpensive quadrature formulas to obtain exactly and efficiently the model response sensitivities to all model parameters α j 1 (including boundary and initial conditions), j 1 = 1 , ⋯ , T P , where TP denotes the total number of parameters.

2) The 2nd-CASAM-N, which enables the exact and efficient computation of the 2nd-order sensitivities. After solving the 2nd-LASS to obtain the 2nd-level adjoint sensitivity function A ( 2 ) ( 2 ; j 1 ; x ) , 1 ≤ j 1 , j 2 ≤ T P , the 2nd-order sensitivities can be computed selectively, in the priority order pre-established by the user. If the 2nd-LASS is solved TP-times, the 2nd-order mixed sensitivities ∂ 2 R / ∂ α j 2 ∂ α j 1 will be computed twice, in two different ways, in terms of two distinct 2nd-level adjoint functions. Consequently, the symmetry property enjoyed by the second-order sensitivities provides an intrinsic (numerical) verification that the functions A ( 2 ) ( 2 ; j 1 ; x ) and a ( 1 ) ( x ) are computed accurately.

3) The 3rd-CASAM-N, which enables the exact and efficient computation of the 3rd-order sensitivities. After solving the 3rd-LASS to obtain the 3rd-level adjoint sensitivity function A ( 3 ) ( 4 ; j 2 , j 1 ; x ) , 1 ≤ j 1 , j 2 ≤ T P , the 3rd-order sensitivities can be computed selectively, in the priority order pre-established by the user. If the 3rd-LASS is solved T P ( T P + 1 ) / 2 times, then each of the 3rd-order mixed sensitivities ∂ 3 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 will be computed three times, each time using distinct adjoint functions. Consequently, the symmetry property enjoyed by the third-order sensitivities provides an intrinsic verification of the numerical computation of the various adjoint sensitivity functions.

4) The 4th-CASAM-N, which enables the exact and efficient computation of the 4th-order sensitivities. After solving the 4th-LASS to obtain the 4th-level adjoint sensitivity function A ( 4 ) ( 8 ; j 3 ; j 2 ; j 1 ; x ) , 1 ≤ j 1 , j 2 , j 3 ≤ T P , the 4th-order sensitivities can be computed selectively, in the priority order pre-established by the user. If the 4th-LASS is solved T P ( T P + 1 ) ( T P + 2 ) / 6 times, then each of the 4th-order mixed sensitivities ∂ 4 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 ∂ α j 4 will be computed four times, each time using distinct adjoint functions. Consequently, the symmetry property enjoyed by the fourth-order sensitivities provides an intrinsic verification of the numerical computation of the various adjoint sensitivity functions.

5) The 5th-CASAM-N, which enables the exact and efficient computation of the 5th-order sensitivities. After solving the 5th-LASS to obtain the 5th-level adjoint sensitivity function A ( 5 ) ( 2 4 ; j 4 ; j 3 ; j 2 ; j 1 ; x ) , 1 ≤ j 1 , j 2 , j 3 , j 4 ≤ T P , the 5th-order sensitivities can be computed selectively, in the priority order pre-established by the user. If the 5th-LASS is solved T P ( T P + 1 ) ( T P + 2 ) ( T P + 3 ) / 24 times, then each of the 5th-order mixed sensitivities ∂ 5 R [ u ( x ) ; α ] / ∂ α j 1 ∂ α j 3 ∂ α j 3 ∂ α j 4 ∂ α j 5 will be computed five times, in five different ways, using distinct adjoint functions. Consequently, the multiple symmetries inherent to the fifth-order sensitivities provide an intrinsic numerical verification that the components of the 1st-, 2nd-, 3rd-, 4th-, and 5th-level adjoint functions are computed accurately. The structure of the 5th-LASS enables full flexibility for prioritizing the computation of the 5th-order sensitivities. The computation of the 5th-order sensitivities would be prioritized based on the relative magnitudes of the 4th-order sensitivities, so that the unimportant 5th-order sensitivities can be deliberately neglected while knowing the error incurred by neglecting them.

The 5th-CASAM-N provides a fundamental step towards overcoming the curse of dimensionality in sensitivity and uncertainty analysis. A paradigm illustrative application to a Bernoulli model with uncertain parameters and boundaries will be presented in an accompanying work [28].

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

Cite this paper

Cacuci, D.G. (2022) Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N): I. Mathematical Framework. American Journal of Computational Mathematics, 12, 44-78. https://doi.org/10.4236/ajcm.2022.121005

ReferencesCacuci, D.G. (1981) Sensitivity Theory for Nonlinear Systems: I. Nonlinear Functional Analysis Approach. Journal of Mathematical Physics, 22, 2794-2802. https://doi.org/10.1063/1.525186Cacuci, D.G. (2003) Sensitivity and Uncertainty Analysis: Theory. Vol. 1, Chapman & Hall/CRC, Boca Raton, 285 p.Cacuci, D.G. (2015) Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) for Computing Exactly and Efficiently First-and Second-Order Sensitivities in Large-Scale Linear Systems: I. Computational Methodology. Journal of Computational Physics, 284, 687-699. https://doi.org/10.1016/j.jcp.2014.12.042Cacuci, D.G. (2016) Second-Order Adjoint Sensitivity Analysis Methodology for Large-Scale Nonlinear Systems: I. Theory. Nuclear Science and Engineering, 184, 16-30. https://doi.org/10.13182/NSE16-16Cacuci, D.G. (2018) The Second-Order Adjoint Sensitivity Analysis Methodology. Taylor & Francis/CRC Press, Boca Raton, 305 p.Cacuci, D.G., Fang, R. and Favorite, J.A. (2019) Comprehensive Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) Applied to a Subcritical Experimental Reactor Physics Benchmark: I. Effects of Imprecisely Known Microscopic Total and Capture Cross Sections. Energies, 12, Article No. 4219. https://doi.org/10.3390/en12214219Fang, R. and Cacuci, D.G. (2019) Comprehensive Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) Applied to a Subcritical Experimental Reactor Physics Benchmark: II. Effects of Imprecisely Known Microscopic Scattering Cross Sections. Energies, 12, Article No. 4114. https://doi.org/10.3390/en12214114Cacuci, D.G., Fang, R., Favorite, J.A., Badea, M.C. and Di Rocco, F. (2019) Comprehensive Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) Applied to a Subcritical Experimental Reactor Physics Benchmark: III. Effects of Imprecisely Known Microscopic Fission Cross Sections and Average Number of Neutrons per Fission. Energies, 12, Article No. 4100. https://doi.org/10.3390/en12214100Fang, R. and Cacuci, D.G. (2020) Comprehensive Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) Applied to a Subcritical Experimental Reactor Physics Benchmark. IV: Effects of Imprecisely Known Source Parameters. Energies, 13, Article No. 1431. https://doi.org/10.3390/en13061431Fang, R. and Cacuci, D.G. (2020) Comprehensive Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) Applied to a Subcritical Experimental Reactor Physics Benchmark: V. Computation of 2nd-Order Sensitivities Involving Isotopic Number Densities. Energies, 13, Article No. 2580. https://doi.org/10.3390/en13102580Cacuci, D.G., Fang R. and Favorite, J.A. (2020) Comprehensive Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) Applied to a Subcritical Experimental Reactor Physics Benchmark: VI. Overall Impact of 1st-and 2nd-Order Sensitivities. Energies, 13, Article No. 1674. https://doi.org/10.3390/en13071674Valentine, T.E. (2006) Polyethylene-Reflected Plutonium Metal Sphere Subcritical Noise Measurements, SUB-PU-METMIXED-001. International Handbook of Evaluated Criticality Safety Benchmark Experiments, NEA/NSC/DOC(95)03/I-IX, Organization for Economic Co-Operation and Development, Nuclear Energy Agency, Paris.Cacuci, D.G. and Fang, R. (2020) Third-Order Adjoint Sensitivity Analysis of an OECD/NEA Reactor Physics Benchmark: I. Mathematical Framework. American Journal of Computational Mathematics (AJCM), 10, 503-528. https://doi.org/10.4236/ajcm.2020.104029 https://www.scirp.org/journal/ajcmFang, R. and Cacuci, D.G. (2020) Third-Order Adjoint Sensitivity Analysis of an OECD/NEA Reactor Physics Benchmark: II. Computed Sensitivities. American Journal of Computational Mathematics (AJCM), 10, 529-558. https://doi.org/10.4236/ajcm.2020.104030 https://www.scirp.org/journal/ajcm.Fang, R. and Cacuci, D.G. (2020) Third-Order Adjoint Sensitivity Analysis of an OECD/NEA Reactor Physics Benchmark: III. Response Moments. American Journal of Computational Mathematics (AJCM), 10, 559-570. https://doi.org/10.4236/ajcm.2020.104031 https://www.scirp.org/journal/ajcm.Cacuci, D.G (2021) Fourth-Order Comprehensive Adjoint Sensitivity Analysis (4th-CASAM) of Response-Coupled Linear Forward/Adjoint Systems. I. Theoretical Framework. Energies, 14, Article No. 3335. https://doi.org/10.3390/en14113335Cacuci, D.G. and Fang, R. (2021) Fourth-Order Adjoint Sensitivity Analysis of an OECD/NEA Reactor Physics Benchmark: I. Mathematical Expressions and CPU-Time Comparisons for Computing 1st-, 2nd-and 3rd-Order Sensitivities. American Journal of Computational Mathematics (AJCM), 11, 94-132. https://doi.org/10.4236/ajcm.2021.112009Cacuci, D.G. and Fang, R. (2021) Fourth-Order Adjoint Sensitivity Analysis of an OECD/NEA Reactor Physics Benchmark: II. Mathematical Expressions and CPU-Time Comparisons for Computing 4th-Order Sensitivities. American Journal of Computational Mathematics (AJCM), 11, 133-156. https://doi.org/10.4236/ajcm.2021.112010Fang, R. and Cacuci, D.G. (2021) Fourth-Order Adjoint Sensitivity and Uncertainty Analysis of an OECD/NEA Reactor Physics Benchmark: I. Computed Sensitivities, Journal of Nuclear Energy, 2, 281-308. https://doi.org/10.3390/jne2030024Fang, R. and Cacuci, D.G. (2022) Fourth-Order Adjoint Sensitivity and Uncertainty Analysis of an OECD/NEA Reactor Physics Benchmark: II. Computed Response Uncertainties. Journal of Nuclear Energy, 3, 1-16. https://doi.org/10.3390/jne3010001Cacuci, D.G. (2021) The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems (nth-CASAM-L): I. Mathematical Framework. Energies, 14, Article No. 8314. https://doi.org/10.3390/en14248314Bellman, R.E. (1957) Dynamic Programming. Rand Corporation, Princeton University Press, Princeton. Republished: Bellman, R.E. (2003) Dynamic Programming. Courier Dover Publications, Mineola.Cacuci, D.G. (2022) The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (nth-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume I: Linear Systems. Springer, New York, Berlin, (In Print).Cacuci, D.G. (2020) The First-Order Comprehensive Sensitivity Analysis Methodology (1st-CASAM) for Scalar-Valued Responses: I. Theory. American Journal of Computational Mathematics (AJCM), 10, 275-289. https://doi.org/10.4236/ajcm.2020.102015Cacuci, D.G. (2021) First-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1st-CASAM) for Computing Efficiently the Exact Response Sensitivities for Physical Systems with Imprecisely Known Boundaries and Parameters: General Theory and Illustrative Paradigm Applications. Annals of Nuclear Energy, 151, Article ID: 107913. https://doi.org/10.1016/j.anucene.2020.107913Cacuci, D.G. (2021) First-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1st-CASAM) for Responses at Critical Points in Coupled Nonlinear Systems. I: Mathematical Framework. Fluids, 6, Article No. 33. https://doi.org/10.3390/fluids6010033Cacuci, D.G. (2020) Second-Order Adjoint Sensitivity Analysis Methodology for Computing Exactly Response Sensitivities to Uncertain Parameters and Boundaries of Linear Systems: Mathematical Framework. American Journal of Computational Mathematics, 10, 329-354. https://doi.org/10.4236/ajcm.2020.103018Cacuci, D.G. (2022) The Fourth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (4th-CASAM-N): I. Mathematical Framework. Journal of Nuclear Energy, 3, 37-71. https://doi.org/10.3390/jne3010004Cacuci, D.G. (2022) Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N): II. Paradigm Application to a Bernoulli Model Comprising Uncertain Parameters. American Journal of Computational Mathematics, Submitted.