ajcmAmerican Journal of Computational Mathematics2161-12112161-1203Scientific Research Publishing10.4236/ajcm.2024.141002ajcm-132058ArticlePhysicsMathematicsIntroducing the nth-Order Features Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (nth-FASAM-N): I. Mathematical FrameworkCacuciDan Gabriel1 Center for Nuclear Science and Energy, University of South Carolina, Columbia, USA
The author declares no conflicts of interest regarding the publication of this paper.
This work presents the “nth-Order Feature Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (abbreviated as “nth-FASAM-N”), which will be shown to be the most efficient methodology for computing exact expressions of sensitivities, of any order, of model responses with respect to features of model parameters and, subsequently, with respect to the model’s uncertain parameters, boundaries, and internal interfaces. The unparalleled efficiency and accuracy of the nth-FASAM-N methodology stems from the maximal reduction of the number of adjoint computations (which are considered to be “large-scale” computations) for computing high-order sensitivities. When applying the nth-FASAM-N methodology to compute the second- and higher-order sensitivities, the number of large-scale computations is proportional to the number of “model features” as opposed to being proportional to the number of model parameters (which are considerably more than the number of features).When a model has no “feature” functions of parameters, but only comprises primary parameters, the nth-FASAM-N methodology becomes identical to the extant nth CASAM-N (“nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems”) methodology. Both the nth-FASAM-N and the nth-CASAM-N methodologies are formulated in linearly increasing higher-dimensional Hilbert spaces as opposed to exponentially increasing parameter-dimensional spaces thus overcoming the curse of dimensionality in sensitivity analysis of nonlinear systems. Both the nth-FASAM-N and the nth-CASAM-N are incomparably more efficient and more accurate than any other methods (statistical, finite differences, etc.) for computing exact expressions of response sensitivities of any order with respect to the model’s features and/or primary uncertain parameters, boundaries, and internal interfaces.
Computation of High-Order SensitivitiesSensitivities to Features of Model ParametersSensitivities to Domain BoundariesAdjoint Sensitivity Systems1. Introduction
The computational model of a physical system comprises the following fundamental conceptual components: (a) a well-posed system of equations that relate the system’s independent variables and parameters to the system’s state (i.e., dependent) variables; (b) nominal or mean values for the system parameters, along with information regarding the probability distributions, moments thereof, inequality and/or equality constraints that define the range of variations of the system’s parameters; (c) one or several quantities, customarily referred to as system responses (or objective functions, or indices of performance), which are computed using the mathematical model. Since the physical processes themselves are seldom known precisely and since most of the model’s parameters stem from experimental procedures that are also subject to imprecisions and/or uncertainties, the results predicted by these models are also imprecise, being affected by the uncertainties underlying the respective model. The analysis of the accuracy of responses computed by models relies fundamentally on the functional derivatives (also called “sensitivities”) of the respective responses with respect to the imprecisely known parameters underlying the computational model. Such sensitivities are needed for many purposes, including: (i) understanding the model by ranking the importance of the various parameters; (ii) performing “reduced-order modeling” by eliminating unimportant parameters and/or processes; (iii) quantifying the uncertainties induced in a model response due to model parameter uncertainties; (iv) performing “model validation,” by comparing computations to experiments to address the question “does the model represent reality?” (v) prioritizing improvements in the model; (vi) performing data assimilation and model calibration as part of forward “predictive modeling” to obtain best-estimate predicted results with reduced predicted uncertainties; (vii) performing inverse “predictive modeling”; (viii) designing and optimizing the system.
Response sensitivities are computed by using either deterministic or statistical methods. The “statistical methods” construct an approximate response distribution (often called “response surface”) in the parameters space by performing many “forward” computations using the model with altered parameter values, and subsequently use scatter plots, regression, rank transformation, correlations, and/or so-called “partial correlation analysis,” in order to identify approximate expectation values, variances and covariances for the responses. These statistical quantities are subsequently used to construct quantities that play the role of (approximate) first-order response sensitivities. Thus, statistical methods commence with “uncertainty analysis” and subsequently attempt an approximate “sensitivity analysis” of the approximately computed model response (called a “response surface”) in the phase-space of the parameters under consideration. The currently popular statistical methods for uncertainty and sensitivity analysis are broadly categorized as variance-based methods (see, e.g., [1][2][3]), sampling-based methods (see, e.g., [4][5]), or Bayesian methods (see, e.g., [6]). Various variants of the statistical methods for uncertainty and sensitivity analysis are reviewed in the book edited by Saltarelli et al. [7]. The conclusions that emerge from examining these statistical methods are as follows:
1) The main advantage of using statistical methods for uncertainty and sensitivity analysis is that they are conceptually easy to implement.
2) Even first-order sensitivities cannot be computed exactly.
3) Statistical methods are subject to the curse of dimensionality and have not been developed for producing higher-order sensitivities.
4) Since the response sensitivities and parameter uncertainties are amalgamated, inherently and inseparably, within the results produced by statistical methods, improvements in parameter uncertainties cannot be directly propagated to improve response uncertainties; rather, the entire set of simulations and statistical post-processing must be repeated anew.
5) A “fool-proof” statistical method for analyzing correctly models involving highly correlated parameters does not seem to exist currently so particular care must be used when interpreting regression results obtained using such models.
The simplest deterministic method for computing response sensitivities is to use finite-difference schemes in conjunction with re-computations using the model with “judiciously chosen” altered parameter values. Evidently, such methods can at best compute approximate values of a very limited number of sensitivities. The earliest deterministic methods that could compute more exactly the values of first-order sensitivities include the “Green’s function method” [8], the “forward sensitivity analysis methodology” [9], and the “direct method” [10], which rely on analytical or numerical differentiation of the computational model under investigation to compute local response sensitivities exactly. However, for a computational model comprising many parameters, the conventional deterministic methods become impractical for computing sensitivities higher than first-order because they are subject to the “curse of dimensionality,” a term coined by Belmann [11] to describe phenomena in which the number of computations increases exponentially in the respective phase-space. In the particular case of sensitivity analysis using conventional deterministic methods, the number of large-scale computations increases exponentially in the phase-space of the model parameter as the order of sensitivities increases.
It is known that the “adjoint method of sensitivity analysis” has been the most efficient method for computing exactly first-order sensitivities, since it requires a single large-scale (adjoint) computation for computing all of the first-order sensitivities, regardless of the number of model parameters. The idea underlying the computation of response sensitivities with respect to model parameters using adjoint operators was first used by Wigner [12] to analyze first-order perturbations in nuclear reactor physics and shielding models based on the linear neutron transport (or diffusion) equation, as subsequently described in textbooks on these subjects (see, e.g., [13][14][15][16][17]). Cacuci [9] is given credit (see, e.g., [18][19]) for having conceived the rigorous mathematical framework of the “1st-order adjoint sensitivity analysis methodology” for generic large-scale nonlinear (as opposed to linearized) systems involving generic operator responses, and for having introduced these principles to the earth, atmospheric and other sciences.
Cacuci [20][21] has extended his 1st-order adjoint sensitivity analysis methodology to enable the comprehensive computation of 2nd-order sensitivities of model responses to model parameters (including imprecisely known domain boundaries and interfaces) for large-scale linear and nonlinear systems. The unparalleled efficiency of the 2nd-order adjoint sensitivity analysis methodology for linear systems [20] was demonstrated by Cacuci and Fang [22] by applying this methodology to compute exactly the 21,976 first-order sensitivities and 482,944,576 second-order sensitivities (of which 241,483,276 are distinct from each other) for an OECD/NEA reactor physics benchmark [23]. This benchmark is modeled by the neutron transport equation involving 21,976 uncertain parameters, the solving of which is representative of “large-scale computations.” The neutron transport equation was solved using the software package PARTISN [24] in conjunction with the MENDF71X cross section library [25], which comprises 618-group cross sections based on ENDF/B-VII.1 nuclear data [26]. The spontaneous fission source has been computed using the code SOURCES4C [27].
Contrary to the widely held belief that second- and higher-order sensitivities are negligible for reactor physics systems, Cacuci and Fang [22] found that many 2nd-order sensitivities of the OECD benchmark’s response to the benchmark’s uncertain parameters were much larger than the largest 1st-order ones, which motivated the investigation of the largest 3rd-order sensitivities, many of which were found to be even larger than the 2nd-order ones. This finding has motivated the development of the mathematical framework for determining and computing the 4th-order sensitivities, many of which were found to be larger than the 3rd-order ones. This sequence of findings has motivated the development by Cacuci [28] of the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (which is abbreviated as “nth-CASAM-L”). The “nth-CASAM-L” mathematical framework was developed specifically for linear systems because important model responses produced by such systems are various Lagrangian functionals such as eigenvalues or “CONTRIBUTONS” [29], which depend simultaneously on both the forward and adjoint state functions governing the respective linear system. Such responses can occur only for linear systems because responses in nonlinear systems can only depend on the system’s forward state functions, since nonlinear operators do not admit adjoint operators. The nth-CASAM-L overcomes the curse of dimensionality in sensitivity analysis of linear systems, enabling the efficient computation of exactly-determined expressions of arbitrarily high-order sensitivities of a generic system response (that can depend on both the forward and adjoint state functions) with respect to all of the parameters that characterize the physical system.
In parallel with developing the nth-CASAM-L, Cacuci [30] has also developed the nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (nth-CASAM-N). Just like the nth-CASAM-L, the nth-CASAM-N is also formulated in linearly increasing higher-dimensional Hilbert spaces (as opposed to exponentially increasing parameter-dimensional spaces), thus overcoming the curse of dimensionality in sensitivity analysis of nonlinear systems, enabling the most efficient computation of exactly-determined expressions of arbitrarily high-order sensitivities of generic nonlinear system responses with respect to model parameters, uncertain boundaries and internal interfaces in the model’s phase-space.
Recently, Cacuci [31] has introduced the “Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (2nd-FASAM-N), which enables a considerable reduction (by comparison to the 2nd-CASAM-N) of the number of large-scale computations needed to compute the second-order sensitivities of a model response with respect to the model parameters. The construction of the 2nd-FASAM-N is based on the same principles as those underlying the construction of the 2nd-CASAM-N. This fact indicates that the principles which enabled the construction of the nth-CASAM-N methodology [30] could also be used to generalize the 2nd-FASAM-N to enable the most efficient computation of the exact expressions of arbitrarily-high (“nth”) order sensitivities of model responses with respect to features/functions of model parameters. This is indeed the case, as will be shown in Section 2 of this work, which presents the construction of the general mathematical framework underlying the nth-FASAM-N. This construction uses “mathematical induction” mirroring the construction of the nth-CASAM-N methodology [30]. Section 3 concludes this work by discussing the significance of the nth-CASAM-N and preparing the groundwork for a paradigm illustrative application to the Nordheim-Fuchs reactor dynamics/safety model [32][33] to be presented in the accompanying “Part 2” [34].
2. The nth-Order Feature/Functions Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (nth-FASAM-N) Methodology
The generic mathematical model of a nonlinear system which comprises uncertain (i.e., imprecisely known) model parameters, domain boundaries and interfaces, is the same as was introduced by Cacuci [30] and is reproduced in Appendix A, for convenient referencing.
The validity of the mathematical methodology underlying the nth-FASAM-N methodology will be established in this Section by using the “proof by mathematical induction” comprising the usual steps, as follows:
1) Surmise the general pattern underlying the nth-FASAM-N methodology for an arbitrarily high-order denoted as “n.” It is expected that the general pattern underlying the nth-FASAM-N can be surmised based on the expected similarities with the general pattern underlying the nth-CASAM-N [30].
2) Prove that the general pattern underlying the nth-FASAM-N is valid for the lowest values of n, i.e., n = 1, which should reduce to the mathematical framework underlying the 1st-FASAM-N, which is reproduced from Cacuci [31] in Appendix B.
3) Assuming that the pattern underlying the nth-FASAM-N is valid for an arbitrarily high-order, n, prove that this pattern is also valid for n+1, i.e., for the (n+1)th-FASAM-N.
2.1. Establishing the Mathematical Framework of the nth-Order Feature Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (nth-FASAM-N)
Comparing the mathematical framework of the 1st-FASAM-N [31] to the framework of the 1st-CASAM-N [30] suggests that the component “features” f i ( α ) , i = 1 , ⋯ , T F , of the vector-valued “feature function” f ( α ) ≜ [ f 1 ( α ) , ⋯ , f T F ( α ) ] † , where TF denotes the total number of the components, play within the 1st-FASAM-N the same role as played by the components α j , j = 1 , ⋯ , T P , of the “vector of primary model parameters” α ≜ ( α 1 , ⋯ , α T P ) † within the framework of the 1st-CASAM-N. It is important to underscore at the outset that the total number of model parameters is by definition much larger than the total number of components of the feature function f ( α ) , i.e., T P ≫ T F ; TF coincides with TP if the mathematical/computational model under consideration possesses no “features.”
An examination of the pattern underlying the nth-CASAM-N methodology [30] indicates that the nth-order sensitivity of the model’s response R [ u ( x ) ; f ( α ) ] with respect to the components f j 1 , ⋯ , f j n of the “feature” function f ( α ) ≜ [ f j 1 ( α ) , ⋯ , f j n ( α ) ] † , for each j 1 , ⋯ , j n = 1 , ⋯ , T F is expected to have the functional form R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ( α ) ] ≜ ∂ n R [ u ( x ) ; f ( α ) ] / ∂ f j n ⋯ ∂ f j 1 . It can also be surmised that the nth-order sensitivity R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ( α ) ] stems from the total first-order G-differential of each of the (n-1)th-order sensitivities, each of which is expected to have the following expression for j 1 , ⋯ , j n − 1 = 1 , ⋯ , T F :
Analogous with the pattern underlying the nth-CASAM-Nmethodology [30], it is surmised that the total first-order G-differential of the (n-1)th-order sensitivity R ( n − 1 ) [ j n − 1 ; ⋯ ; j 1 ; U ( n − 1 ) ; A ( n − 1 ) ; f ( α ) ] with respect to the feature-function f ( α ) is expected to have the following expression:
The vectors V ( n − 1 ) ( 2 n − 2 ; j n − 3 ; ⋯ ; j 1 ; x ) ≜ δ U ( n − 1 ) ( 2 n − 2 ; j n − 3 ; ⋯ ; j 1 ; x ) and δ A ( n − 1 ) ( 2 n − 2 ; j n − 2 ; ⋯ ; j 1 ; x ) are the solution of the following nth-Level Variational Sensitivity System (nth-LVSS), which is obtained by concatenating the (n-1)th-LVSS together with the G-differentiated (n-1)th-Level Adjoint Sensitivity System, for j 1 , ⋯ , j n − 1 = 1 , ⋯ , T F :
The various matrices and vectors appearing in Equations (4) and (5) are defined as follows:
1) The variational matrix V M ( n ) [ 2 n − 1 × 2 n − 1 ; U ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) ; f ] comprises ( 2 n − 1 × 2 n − 1 ) block-matrices, each comprising TD2 components/elements, defined as follows:
The matrix V M ( n ) [ 2 n − 1 × 2 n − 1 ; U ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) ; f ] comprises a total of ( 2 n − 1 × 2 n − 1 ) T D 2 components/elements, as indicated by its first argument. The submatrix V M ( n − 1 ) [ 2 n − 2 × 2 n − 2 ; x ] is defined recursively, while the remaining submatrices are defined as follows:
2) The vector V ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) is defined recursively, below, and comprises 2 n − 1 blocks of TD-dimensional vectors [thus comprising a total of ( 2 n − 1 × T D ) components/elements]; this fact is indicated by the first argument of each of these quantities.
3) The vector U ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) has the same structure as the vector V ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) , comprising 2 n − 1 blocks of TD-dimensional vectors, as defined below.2
4) The vector Q V ( n ) [ 2 n − 1 ; U ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) ; f ; δ f ] also comprises 2 n − 1 blocks of TD-dimensional vectors, as defined recursively below.
5) The vector block-vector 0 [ 2 n − 1 ] comprises 2 n − 1 components, each component being a TD-dimensional vector having identically zero components.
6) The boundary terms are represented by the block-vector B V ( n ) [ 2 n − 1 ; U ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) ; V ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) ; f ; δ f ] , which is defined recursively as shown below.
The need for solving the following nth-Level Variational Sensitivity System (nth-LVSS) defined by Equations (4) and (5) is circumvented by deriving an alternative expression for the indirect-effect term defined in Equation (3), in which the function V ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) is replaced by a nth-level adjoint function which will be independent of parameter variations, and which will be denoted as A ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) ≜ [ a ( n ) ( 1 ; j n − 1 ; ⋯ ; j 1 ; x ) , ⋯ , a ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) ] † ∈ H n ( Ω x ) . The elements of the Hilbert space H n ( Ω x ) are surmised to be block-vectors of the form Ψ ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) ≜ [ ψ ( n ) ( 1 ; j n − 1 ; ⋯ ; j 1 ; x ) , ⋯ , ψ ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) ] † , comprising as elements 2 n − 1 TD-dimensional vectors of the form ψ ( n ) ( i ; x ) ≜ [ ψ 1 ( n ) ( i ; x ) , ⋯ , ψ T D ( n ) ( i ; x ) ] † ∈ H 1 ( Ω x ) , i = 1 , ⋯ , 2 n . The inner product
of two vectors Ψ ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) and Φ ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) in the
Hilbert space H n ( Ω x ) is denoted as 〈 Ψ ( n ) ( 2 n − 1 ; x ) , Φ ( n ) ( 2 n − 1 ; x ) 〉 n and defined as follows:
The nth-Level Adjoint Sensitivity System (nth-LASS) for the nth-level adjoint function A ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) is obtained by using the inner product defined in Equation (15), as follows:
where the quantity { [ P ( n ) ( U ( n ) ; A ( n ) ; V ( n ) ; α ) ] ∂ Ω x } α 0 denotes the corresponding
bilinear concomitant on the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions.
In terms of the nth-Level adjoint function A ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) , the indirect-effect term defined by Equation (3) will have the following expression:
In Equation (17), the quantity { [ P ^ ( n ) ( U ( n ) ; A ( n ) ; δ f ) ] ∂ Ω x } α 0 denotes residual
boundary terms which may have not vanished automatically after having used the boundary conditions provided in Equations (5) and (19) to eliminate from Equation (17) all unknown values of the nth-level variational function V ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) .
The quantities which appear in the definition of the nth-LASS represented by Equations (18) and (19), are defined as follows:
The final expression of the total differential expressed by Equation (17) is obtained by inserting in Equation (2) the expression for the indirect-effect term obtained in Equation (17), which yields the following expression for the nth-Order sensitivities R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ] ≡ ∂ n R [ u ( x ) ; f ] / ∂ f j n ⋯ ∂ f j 1 of the response R [ u ( x ) ; α ] with respect to the parameters α j 1 , ⋯ , α j n , for j 1 = 1 , ⋯ , T F ; ⋯ ; j n − 1 = 1 , ⋯ , j n − 2 :
R(n)[jn;⋯;j1;U(n);A(n);f]≡∂nR[u(x);f]/∂fjn⋯∂fj1=∂R(n−1)[jn−1;⋯;j1;U(n−1)(x);A(n−1)(x);f]∂fjn−[∂P^(n)(U(n);A(n);δf)]∂Ωx∂fjn+∑i=12n−1〈a(n)(i;jn−1;⋯;j1;x),sV(n)[i;jn;⋯;j1;U(n)(x);f]〉1.≜∫λ1(α)ω1(α)...∫λTI(α)ωTI(α)S(n)[jn;⋯;j1;U(n)(2n−1;x);A(n)(2n−1;x);f]dx1⋯dxTI.
2.2. The Particular Form of the n
<sup>th</sup>
-FASAM-N Framework for
<italic>n</italic>
= 1
Setting n = 1 into the mathematical framework of the nth-FASAM-N conjectured in Section 2.1 above yields the following expressions for the particular case n = 1:
The 1st-level adjoint sensitivity function A ( 1 ) ( 2 0 ; x ) ≡ a ( 1 ) ( x ) is the solution of the following 1st-LASS obtained by setting n=1 in Equations (18) and (19):
Comparing the expressions obtained above in Equations (26)-(33) with the expressions obtained independently, from first principles, in Appendix B reveals that the corresponding expressions are identical to each other, which proves the correctness of the conjectured general expressions obtained within the nth-FASAM-N methodology for the particular case n = 1.
2.3. Derivation of the Mathematical Framework of the (n+1)
<sup>th</sup>
-FASAM-N
It will be assumed that, for each j 1 , ⋯ , j n = 1 , ⋯ , T F , the 1st-order total G-differential of the nth-order sensitivities R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ] will exist and will be linear in the variations
V(n)(2n−1;jn−2;⋯;j1;x)≜δU(n−1)(2n−1;jn−2;⋯;j1;x)
δ A ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) in a neighborhood around the nominal values of the “feature functions,” the parameters, and the respective state functions. By definition, the 1st-order total G-differential of R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ] is given by the following expression:
where the quantity { δ R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ; V ( n ) ; δ A ( n ) ] } i n d denotes the “indirect-effect term” which is defined as follows:
The vectors V ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) ≜ δ U ( n ) ( 2 n − 1 ; j n − 2 ; ⋯ ; j 1 ; x ) and
δ A ( n ) ( 2 n − 1 ; j n − 1 ; ⋯ ; j 1 ; x ) , which are needed in order to evaluate the indirect-effect term { δ R ( n ) [ j n ; ⋯ ; j 1 ; U ( n ) ; A ( n ) ; f ; V ( n ) ; δ A ( n ) ] } i n d , are the solutions of the following (n+1)th-Level Variational Sensitivity System, which is obtained by concatenating the nth-LVSS defined by Equations (4) and (5) together with the G-differentiated nth-LASS, for j 1 , ⋯ , j n = 1 , ⋯ , T P :
Solving the (n+1)th-LVSS would require O ( T P n + 1 ) large-scale computations, which is unrealistic for large-scale systems comprising many parameters. The (n+1)th-FASAM-N circumvents the need for solving the (n+1)th-LVSS by deriving an alternative expression for the indirect-effect term defined in Equation , in which the function V ( n + 1 ) ( 2 n ; j n − 1 ; ⋯ ; j 1 ; x ) is replaced by a (n+1)th-level adjoint function, denoted as A ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) ≜ [ a ( n + 1 ) ( 1 ; j n ; ... ; j 1 ; x ) , ⋯ , a ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) ] † ∈ H n + 1 ( Ω x ) , which is independent of parameter variations. The elements of the Hilbert space H n + 1 ( Ω x ) are block-vectors of the form
Ψ ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) ≜ [ ψ ( n + 1 ) ( 1 ; j n ; ⋯ ; j 1 ; x ) , ⋯ , ψ ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) ] † , comprising 2 n TD-dimensional block-vectors of the form
ψ ( n + 1 ) ( i ; x ) ≜ [ ψ 1 ( n + 1 ) ( i ; x ) , ⋯ , ψ T D ( n + 1 ) ( i ; x ) ] † ∈ H 1 ( Ω x ) , i = 1 , ⋯ , 2 n + 1 . The inner
product of two vectors Ψ ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) and Φ ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) in the Hilbert space H n + 1 ( Ω x ) is denoted as 〈 Ψ ( n + 1 ) ( 2 n ; x ) , Φ ( n + 1 ) ( 2 n ; x ) 〉 n + 1
The (n+1)th-Level Adjoint Sensitivity System [abbreviated as: (n+1)th-LASS ] for the (n+1)th-level adjoint function A ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) is obtained by using the inner product defined in Equation (47), as follows:
where the quantity { [ P ( n + 1 ) ( U ( n + 1 ) ; A ( n + 1 ) ; V ( n + 1 ) ; f ) ] ∂ Ω x } α 0 denotes the corres-
ponding bilinear concomitant on the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions.
In terms of the (n+1)th-level adjoint function A ( n + 1 ) ( 2 n ; j n ; ⋯ ; j 1 ; x ) , the indirect-effect term defined by Equation (35) will have the following expression:
In Equation (49), the quantity { [ P ^ ( n + 1 ) ( U ( n + 1 ) ; A ( n + 1 ) ; δ f ) ] ∂ Ω x } α 0 denotes
residual boundary terms which may have not vanished automatically after having used the boundary conditions provided in Equations (37) and (51) to eliminate from Equation (48) all unknown values of the (n+1)th-level variational function V ( n + 1 ) ( 2 n ; j n − 1 ; ⋯ ; j 1 ; x ) .
The quantities that appear in the definition of the (n+1)th-LASS represented by Equations (50) and (51), are defined as follows:
The final expression of the total differential expressed by Equation (17) is obtained by inserting in Equation (34) the expression for the indirect-effect term obtained in Equation (49), which yields the following expression for the nth-order sensitivities R ( n + 1 ) [ j n + 1 ; ⋯ ; j 1 ; U ( n + 1 ) ; A ( n + 1 ) ; f ] ≡ ∂ n + 1 R [ u ( x ) ; f ( α ) ] / ∂ f j n + 1 ⋯ ∂ f j 1 of the response R [ u ( x ) ; f ( α ) ] with respect to the components f j 1 , ⋯ , f j n + 1 of the “features function” f ( α ) , for j 1 = 1 , ⋯ , T F ; ⋯ ; j n = 1 , ⋯ , j n − 1 :
The expression obtained in Equation (57) is identical with the expression that would be obtained by replacing the index n with (n+1) in the expression obtained in Equation (25), thus completing the proof, by mathematical induction, of the validity/correctness of the conjectured general expressions underlying the nth-FASAM-N.
3. Conclusions
This work has presented the “nth-Order Feature Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (abbreviated as “nth-FASAM-N”), which is the most efficient methodology for computing exact expressions of sensitivities of model responses with respect to features of model parameters and, subsequently, with respect to the model parameters themselves. This efficiency stems from the reduction of the number of adjoint computations (which are considered to be “large-scale” computations), by comparison to the extant [30] high-order adjoint sensitivity analysis methodology nth-CASAM-N (the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems”), as follows:
1) Comparing the mathematical framework of the nth-FASAM-N methodology presented in this work to the framework of the nth-CASAM-N methodology [30] indicates that the components f i ( α ) , i = 1 , ⋯ , T F , of the “feature function” f ( α ) ≜ [ f 1 ( α ) , ⋯ , f T F ( α ) ] † play within the nth-FASAM-N the same role as played by the components α j , j = 1 , ⋯ , T P , of the “vector of primary model parameters” α ≜ ( α 1 , ⋯ , α T P ) † within the framework of the nth-CASAM-N [30]. It is paramount to underscore, at the outset, that the total number of model parameters is always larger (usually by a wide margin) than the total number of components of the feature function f ( α ) , i.e., T P ≫ T F .
2) The 1st-FASAM-N and the 1st-CASAM-N methodologies require a single large-scale “adjoint” computations for solving the 1st-LASS (1st-Level Adjoint Sensitivity System), so they are equally efficient for computing the exact expressions of the first-order sensitivities of a model response to the model’s uncertain parameters, boundaries, and internal interfaces.
3) For computing the exact expressions of the second-order response sensitivities with respect to the primary model’s parameters, the 2nd-FASAM-N methodology [31] requires as many large-scale “adjoint” computations as there are “feature functions of parameters” f i ( α ) , i = 1 , ⋯ , T F (where TF denotes the total number of feature functions) for solving the left-side of the 2nd-LASS with TF distinct sources on its right-side. By comparison, the 2nd-CASAM-N methodology [30] requires TP (where TP denotes the total number of model parameters) large-scale computations for solving the same left-side of the 2nd-LASS but with TP distinct sources. Since T F ≪ T P , the 2nd-FASAM-N methodology is considerably more efficient than the 2nd-CASAM-N methodology for computing the exact expressions of the second-order sensitivities of a model response to the model’s uncertain parameters, boundaries, and internal interfaces.
4) For computing the exact expressions of the third-order response sensitivities with respect to the primary model’s parameters, it can be deduced from the general theory underlying the 3rd-FASAM-N presented in this work that, for n = 3 , the 3rd-FASAM-N requires at most T F ( T F + 1 ) / 2 large-scale “adjoint” computations while the 3rd-CASAM-N methodology [30] requires at most. The same computational-count of ”large-scale computations” caries over when computing the higher-order sensitivities, i.e., the formula for calculating the “number of large-scale adjoint computations” is formally the same for both the nth-FASAM-N and the nth-CASAM-N methodologies, but the “variable” in the formula for determining the number of adjoint computations for the nth-FASAM-N methodology is TF (i.e., the total number of feature functions) while the counterpart for the formula for determining the number of adjoint computations for the nth-CASAM-N is methodology is TP (i.e., the total number of model parameters). Since T F ≪ T P , it follows that the higher the order of computed sensitivities, the more efficient the nth-FASAM-N methodology becomes by comparison to the nth-CASAM-N methodology.
5) When a model has no “feature” functions of parameters, but only comprises primary parameters, the nth-FASAM-N methodology becomes identical to the nth-CASAM-N methodology [30].
Both the nth-FASAM-N and the nth-CASAM-N methodologies are formulated in linearly increasing higher-dimensional Hilbert spaces as opposed to exponentially increasing parameter-dimensional spaces thus overcoming the curse of dimensionality in sensitivity analysis of nonlinear systems. Both the nth-FASAM-N and the nth-CASAM-N methodologies are incomparably more efficient and more accurate than any other methods (statistical, finite differences, etc.) for computing exact expressions of response sensitivities (of any order) with respect to the model’s uncertain parameters, boundaries, and internal interfaces.
The question of “when to stop computing progressively higher-order sensitivities?” has been addressed by Cacuci [28][30] in conjunction with the question of convergence of the Taylor-series expansion of the response in terms of the uncertain model parameters [see Equations (65)‒(67) in Appendix A]. This Taylor-series expansion is the fundamental premise for obtaining meaningful (convergent) expressions provided by the “propagation of errors” methodology, as originally proposed heuristically by Tukey [35], for the cumulants of the model response distribution in the phase-space of model parameters. The convergence of this Taylor-series, which depends on both the response sensitivities to parameters and the uncertainties associated with the parameter distribution, must be ensured. This can be done by ensuring that the combination of parameter uncertainties and response sensitivities are sufficiently small to fall inside the radius of convergence of this Taylor-series expansion.
The companion work “Part 2” [34] will present an illustrative paradigm application of the nth-FASAM-N methodology to the Nordheim-Fuchs reactor dynamics/safety model [32][33].
Appendix A: Mathematical Modeling of a Generic Nonlinear System Comprising Functions (“Features”) of Uncertain Parameters and Boundaries
The mathematical model that underlies the numerical evaluation of a process and/or state of a physical system comprises equations that relate the system’s independent variables and parameters to the system’s state/dependent variables. These coupled equations, which are in general nonlinear, can be represented generically in operator form as follows:
The results computed using a mathematical model are customarily called “model responses” (or “system responses” or “objective functions” or “indices of performance”). As has been discussed by Cacuci (2022, 2023a, 2023b), all responses can be fundamentally analyzed in terms of the following generic integral representation:
where S [ u ( x ) ; g ( α ) ; h ( α ) ; x ] is a suitably differentiable nonlinear function of u ( x ) and of α .
Without loss of generality, the quantities which appear in Equations (58)-(60) can be considered to be real-valued, having the following meanings:
1) Matrices are denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “ ≜ ” will be used to denote “is defined as” or “is by definition equal to.” Transposition will be indicated by a dagger ( † ) superscript. The equalities in this work are considered to hold in the weak (“distributional”) sense. Both sides of Equations (58)-(60) may contain “generalized functions/functionals”, particularly Dirac-distributions and derivatives thereof.
2) The TP-dimensional column-vector α ≜ ( α 1 , ⋯ , α T P ) † ∈ ℝ T P represents the “vector of primary model parameters” and has components denoted as α 1 , ⋯ , α T P , where TP denotes the “total number of parameters” involved in the model under consideration. These model parameters usually stem from processes that are external to the system under consideration and afflicted by uncertainties (i.e., they are not known precisely). The known characteristics of the model parameters usually includes their nominal (expected/mean) values and, possibly, higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis) of their unknown distribution; these characteristics are usually determined from experimental data and/or processes external to the physical system under consideration. Occasionally, just the lower and the upper bounds may be known for some model parameters, expressed by inequality and/or equality constraints that delimit the ranges of the system’s parameters are known. Without loss of generality, the imprecisely known model parameters can be considered to be real-valued scalar quantities. It is important to note that the components of the vector α include not only parameters that appear in in Equations (58) and (59), but also include parameters that may specifically occur only in the definition of the model’s response provided in Equation (60). The nominal parameter values will be denoted as α 0 ≜ [ α 1 0 , ⋯ , α i 0 , ⋯ , α T P 0 ] † ; the superscript “0” will be used throughout this monograph to denote “nominal” or “mean” values.
3) The TI-dimensional column vector x ≜ ( x 1 , ⋯ , x T I ) † ∈ ℝ T I comprises the model’s independent variables, denoted as x i , i = 1 , ⋯ , T I , where the sub/superscript “TI” denotes the “total number of independent variables.” The vector x ∈ ℝ T I is considered to be defined on a phase-space domain denoted as Ω x ( x ) ≜ { λ i ( α ) ≤ x i ≤ ω i ( α ) ; i = 1 , ⋯ , T I } , including the particular cases when λ i ( α ) = − ∞ , ω i ( α ) = ∞ for some independent variables x i , i = 1 , ⋯ , T I . The domain boundary ∂ Ω x [ λ ( α ) ; ω ( α ) ] ≜ { λ i ( α ) ∪ ω i ( α ) , i = 1 , ⋯ , T I } of Ω x [ f ( α ) ] is defined to comprise the set of all of the endpoints λ i ( α ) , ω i ( α ) , i = 1 , ⋯ , T I . For subsequent mathematical developments, it is convenient to consider that the endpoints λ i ( α ) , ω i ( α ) , i = 1 , ⋯ , T I are components of column-vectors λ ( α ) ≜ [ λ 1 ( α ) , ⋯ , λ T I ( α ) ] † and ω ( α ) ≜ [ ω 1 ( α ) , ⋯ , ω T I ( α ) ] † , respectively These endpoints depend on the physical system’s geometrical dimensions, which may be imprecisely known because of manufacturing tolerances, and are considered therefore to be components of the vector α ≜ ( α 1 , ⋯ , α T P ) † ∈ ℝ T P of primary model parameters. Furthermore, the boundary-endpoints λ i ( α ) , ω i ( α ) , i = 1 , ⋯ , T I , may also depend on the parameters that define the material properties of the respective medium. For example, in models based on diffusion theory, the boundary conditions for materials facing air/vacuum are imposed on a physics-based mathematical construct called the “extrapolated boundary” of the respective spatial domain. The “extrapolated boundary” depends both on the imprecisely known physical dimensions of the system’s materials and also on the material’s properties, such as atomic number densities and microscopic transport cross sections. Therefore, the boundary end-points can be considered, in general, to be functions of (some of) the primary model parameters.
4) The components u i ( x ) , i = 1 , ⋯ , T D of the TD-dimensional column vector u ( x ) ≜ [ u 1 ( x ) , ⋯ , u T D ( x ) ] † represent the model’s dependent variables (also called “state functions”); the abbreviation “TD” denotes “total number of dependent variables.”
5) The vector g ( α ) ≜ [ g 1 ( α ) , ⋯ , g T G ( α ) ] is a TG-dimensional vector having components g i ( α ) , i = 1 , ⋯ , T G , which are real-valued functions of (some of) the primary model parameters α ∈ ℝ T P . The quantity TG denotes the total number of such functions which appear exclusively in the definition of the model’s underlying equations. Such functions customarily appear in models in the form of correlations that describe “features” of the system under consideration, such as material properties, flow regimes. etc. Usually, the number of functions g i ( α ) is considerably smaller than the total number of model parameters, i.e., T G ≪ T P . For example, the numerical model (Cacuci and Fang, 2023) of the OECD/NEA reactor physics benchmark (Valentine 2006) comprises 21,976 uncertain primary model parameters (including microscopic cross sections and isotopic number densities) but the neutron transport equation, which is solved to determine the neutron flux distribution within the benchmark, does not use these primary parameters directly but instead uses several hundreds of “group-averaged macroscopic cross sections” which are functions/features of the microscopic cross sections and isotopic number densities (which in turn are uncertain quantities that would be components of the vector of primary model parameters). In particular, a component g j ( α ) may simply be one of the primary model parameters α j , i.e., g j ( α ) ≡ α j .
6) The TD-dimensional column vector N [ u ( x ) ; x ; g ( α ) ] ≜ ( N 1 , ⋯ , N T D ) † comprises components N i [ u ( x ) ; x ; g ( α ) ] , i = 1 , ⋯ , T D , which are operators (including differential, difference, integral, distributions, and/or finite or infinite matrices) acting nonlinearly (in general) on the dependent variables u ( x ) , the independent variables x and on the functions g ( α ) of model parameters α .
7) The TD-dimensional column vector Q [ u ( x ) ; x ; g ( α ) ] ≜ ( q 1 , ⋯ , q T D ) † , having components q i [ u ( x ) ; x ; g ( α ) ] , i = 1 , ⋯ , T D , denotes inhomogeneous source terms, which usually depend nonlinearly on uncertain parameters α .
8) The components of B [ u ( x ) ; g ( α ) ; x ] are nonlinear operators, while the components of C [ x , g ( α ) ] represent inhomogeneous boundary sources, all defined on the boundary ∂ Ω x .
9) The integral representation of the response provided in Equation (60) can represent “averaged” and/or “point-valued” quantities in the phase-space of independent variables. For example, if R [ u ( x ) ; f ( α ) ] represents the computation or the measurement (which would be a “detector-response”) of a quantity of interest at a point x d in the phase-space of independent variables, then S [ u ( x ) ; g ( α ) ; h ( α ) ; x ] would contain a Dirac-delta functional of the form δ ( x − x d ) . Responses that represent “differentials/derivatives of quantities” would contain derivatives of Dirac-delta functionals in the definition of S [ u ( x ) ; g ( α ) ; h ( α ) ; x ] . The vector h ( α ) ≜ [ h 1 ( α ) , ⋯ , h T H ( α ) ] , having components h i ( α ) , i = 1 , ⋯ , T H , which appears among the arguments of the function S [ u ( x ) ; g ( α ) ; h ( α ) ; x ] , represents functions of primary parameters that often appear solely in the definition of the response but do not appear in the mathematical definition of the model, i.e., in Equations (58) and (59). The quantity TH denotes the total number of such functions which appear exclusively in the definition of the model’s response. Evidently, the response will depend directly and/or indirectly (through the “feature”-functions) on all of the primary model parameters. This fact has been indicated in Equation (60) by using the vector-valued function f ( α ) as an argument in the definition of the response R [ u ( x ) ; f ( α ) ] to represent the concatenation of all of the “features” of the model and response under consideration. The vector f ( α ) of “model features” is thus defined as follows:
As defined in Equation (61), the quantity TF denotes the total number of “feature functions of the model’s parameters” which appear in the definition of the nonlinear model’s response and underlying equations.
Solving Equations (58) and (59) at the nominal parameter values, α 0 ≜ [ α 1 0 , ⋯ , α i 0 , ⋯ , α T P 0 ] † , provides the “nominal solution” u 0 ( x ) ; which means that the vectors u 0 ( x ) and α 0 satisfy the following equations:
Using the nominal parameter values α 0 together with the “nominal solution” u 0 ( x ) in Equation (60) yields the nominal value of the response, namely:
In view of Equation (64), each model response R k [ f ( α ) ] , k = 1 , ⋯ , T R , where TR denotes the “total number of responses,” can be considered to depend directly on the feature functions f ( α ) , and would therefore admit a Taylor-series expansion around the nominal value f 0 ≜ f ( α 0 ) , having the following form:
where δ f j ≜ [ f j ( α ) − f j 0 ] ; f j 0 ≜ f j ( α 0 ) ; j = 1 , ⋯ , T F . The “sensitivities of the model response with respect to the (feature) functions” are naturally defined as being the functional derivatives of R k [ f ( α ) ] with respect to the components (“features”) f j ( α ) of f ( α ) . The notation { · } f 0 indicates that the quantity enclosed within the braces is to be evaluated at the nominal values f 0 ≜ f ( α 0 ) . Since T F ≪ T P , the computations of the functional derivatives of R k [ f ( α ) ] with respect to the functions f j ( α ) , which appear in Equation (65), will be considerably less expensive computationally than the computation of the functional derivatives involved in the Taylor-series of the response with respect to the model parameters. The functional derivatives of the response with respect to the parameters can be obtained from the functional derivatives of the response with respect to the “feature” functions f j ( α ) by simply using the chain rule, i.e.:
and so on. The evaluation/computation of the functional derivatives
∂ f i 1 ( α ) / ∂ α j 1 , ∂ 2 f i 1 ( α ) / ∂ α j 1 ∂ α j 2 , etc., does not require computations involving
the model, and is therefore computationally trivial by comparison to the evaluation of the functional derivatives (“sensitivities”) of the response with respect to either the functions (“features”) f j ( α ) or the model parameters α i , i = 1 , ⋯ , T P .
The range of validity of the Taylor-series shown in Equation (65) is defined by its radius of convergence. The accuracy—as opposed to the “validity”—of the Taylor-series in predicting the value of the response at an arbitrary point in the phase-space of model parameters depends on the order of sensitivities retained in the Taylor-expansion: the higher the respective order, the more accurate the respective response value predicted by the Taylor-series. In the particular cases when the response happens to be a polynomial function of the “feature” functions f j ( α ) , the Taylor series is actually exact.
In turn, the functions f i ( α ) can also be formally expanded in a multivariate Taylor-series around the nominal (mean) parameter values α 0 , namely:
The domain of validity of the Taylor-series in Equation (67) is defined by its own radius of convergence.
Appendix B: The 1
<sup>st</sup>
-FASAM-N: First-Order Function/Feature Adjoint Sensitivity Analysis Methodology for Nonlinear Systems
The 1st-order G-differential, denoted as δ R [ u 0 ( x ) ; f 0 ( α ) ; v ( 1 ) ( x ) ; δ f ( α ) ] , of the response R [ u ( x ) ; f ( α ) ] at a point ( u 0 ; f 0 ) in the phase-space of dependent variables u ( x ) and feature-functions f ( α ) is defined as follows:
The necessary and sufficient conditions for the 1st-order G-differential δ R [ u 0 ( x ) ; f 0 ( α ) ; v ( 1 ) ( x ) ; δ f ( α ) ] to be linear in the variations v ( 1 ) ≜ δ u and δ f (and hence admit partial G-derivatives with respect to u and f ) at a point ( u 0 ; f 0 ) in the phase-space of dependent variables and feature-functions are as follows:
i) R [ u ( x ) ; f ( α ) ] satisfies the following inequality:
‖R[u0+εv(1);f0+ε(δf)]−R(u0;f0)‖≤k‖ε(u0;f0)‖,k<∞
ii) R [ u ( x ) ; f ( α ) ] satisfies the following condition for a scalar ε and vectors v 2 , f 2 :
In practice, the relations provided in Equations (69) and (70) are seldom used directly since the computation of the expression on the right-side of Equation (68) reveals immediately if the respective expression is linear (or not) in the vectors v ( 1 ) ( x ) and/or δ f ( α ) .
Numerical methods (e.g., Newton’s method and variants thereof) for solving Equations (58) and (59) also require the existence of the first-order G-derivatives of the original model equations. Therefore, the conditions provided in Equations (69) and (70) are henceforth considered to be satisfied by the model responses and also by the operators underlying the physical system modeled by Equations (58)-(60), which implies that all of the operators/functions considered in this work admit G-derivatives.
When the 1st-order G-variation δ R [ u 0 ( x ) ; f 0 ( α ) ; v ( 1 ) ( x ) ; δ f ( α ) ] satisfies the conditions provided in (69) and (70), it can be written as follows:
The notation on the left-side of Equation (73) represents the inner product between two vectors (comprising an implied multiplication of a vector with a transposed vector), but the symbol “( † )” which indicates “transposition” has been omitted in order to keep the notation as simple as possible. “Daggers” indicating transposition will also be omitted in other inner products, whenever possible, while avoiding ambiguities.
The direct-effect term can be computed after having solved Equations (58) and (59) to obtain the nominal values, u 0 ( x ) , of the dependent variables. On
the other hand, the quantity { d R [ u ( x ) ; f ( α ) ; v ( 1 ) ( x ) ] } i n d defined in Equation
(71) comprises only variations in the state functions −and is therefore called the “indirect-effect term” being defined as follows:
The “indirect-effect” term induces variations in the response through the variations in the state functions, which are, in turn, caused by the parameter variations through the equations underlying the model. The indirect-effect term can be quantified only after having determined the variations v ( 1 ) ( x ) in terms of the variations δ g , δ λ , and δ ω .
The first-order relationship between the vectors v ( 1 ) ( x ) and the variations δ g , δ λ , and δ ω is determined by solving the equations obtained by applying the definition of the G-differential to Equations (58) and (59), which yields the following equations:
Carrying out the differentiations with respect to ε in Equations (80) and (81), and setting ε = 0 in the resulting expressions yields the following equations:
In Equations (82) and (83), the superscript “(1)” indicates “1st-Level” and the various quantities which appear in these equations are defined as follows:
The system of equations comprising Equations (82) and (83) is called the “1st-Level Variational Sensitivity System” (1st-LVSS). The solution, v ( 1 ) ( x ) , of the 1st-LVSS will be a function of the variations δ g , δ λ , and δ ω . Hence, when the function v ( 1 ) ( x ) is introduced into the expression of the indirect-effect term defined in Equation (78), it will introduce dependencies of the response sensitivities on δ g , δ λ , and δ ω , which will be in addition to the dependencies displayed by the direct-effect term defined in Equation (72). As Equations (82) and (83) indicate, every parameter variation δ α j 1 , j 1 = 1 , ⋯ , T P , will induce, directly or indirectly, a variations in the model’s state variables. In principle, therefore, for every parameter variation δ α j 1 , j 1 = 1 , ⋯ , T P , there would correspond a solution v ( 1 ) ( j 1 ; x ) , j 1 = 1 , ⋯ , T P , of the 1st-LVSS. Thus, if the effect of every parameter variation were of interest, then the 1st-LVSS would need to be solved TP times, with distinct right-sides and boundary conditions for each parameter variation δ α j 1 , which would require at least TP large-scale computations. If only variations induced by the functions δ g , δ λ , and/or δ ω were of interest, then fewer than TP large-scale computations would be required.
However, solving the 1st-LVSS can be avoided altogether by using the ideas underlying the “adjoint sensitivity analysis methodology” as originally conceived by Cacuci (1981) and subsequently generalized by Cacuci (2022, 2023a) to enable the computation of arbitrarily high-order response sensitivities to model parameters. Thus, the need for computing the vectors v ( 1 ) ( j 1 ; x ) , j 1 = 1 , ⋯ , T P , is eliminated by expressing the indirect-effect term defined in Equation in terms of the solutions of the “1st-Level Adjoint Sensitivity System” (1st-LASS), which is constructed by introducing a (real) Hilbert space denoted as H 1 ( Ω x ) , endowed with an inner product of two vectors w ( 1 ) ( x ) ∈ H 1 and w ( 2 ) ( x ) ∈ H 1 which is denoted as 〈 w ( 1 ) , w ( 2 ) 〉 1 and defined as follows:
In Equation (87), the “dagger” ( † ), which indicates “transposition,” has been omitted to simplify the notation for the scalar product
w(1)(x)⋅w(2)(x)≜∑i=1TDwi(1)(x)wi(2)(x)
The 1st-LASS is now constructed by considering a vector a ( 1 ) ( x ) ∈ H 1 , which is an element in H 1 ( Ω x ) but is otherwise arbitrary at this stage, and by using Equation (87) to form the inner product of a ( 1 ) ( x ) ∈ H 1 with the relation provided in Equation (82) to obtain:
where [ P ( 1 ) ( u ; g ; λ ; ω ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x denotes the associated bilinear concomitant evaluated on the domain’s boundary ∂ Ω x , and where
A ( 1 ) ( u ; g ) ≜ [ N ( 1 ) ( u ; g ) ] * denotes the operator formally adjoint to N ( 1 ) ( u ; g ) .
The symbol [ ] ∗ indicates “formal adjoint” operator.
The first term on the right-side of Equation (89) is now required to represent the indirect-effect term defined in Equation (78) by imposing the following relationship:
The domain of A ( 1 ) ( u ; g ) is determined by selecting appropriate adjoint boundary and/or initial conditions, which will be denoted in operator form as:
{bA(1)(u;a(1);g)}α0=0,x∈∂Ωx.
The above boundary conditions for the adjoint operator A ( 1 ) ( u ; g ) are obtained by imposing the following requirements:
i) they must be independent of unknown values of v ( 1 ) ( x ) and δ g ;
ii) the substitution of the boundary and initial conditions represented by Equations (83) and (91) into the expression of { [ P ( 1 ) ( u ; g ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x } α 0 must cause all terms containing unknown values of v ( 1 ) ( x ) to vanish.
Using the adjoint and forward variational boundary conditions represented by Equations (91) and (83) into (89) reduces the bilinear concomitant
{ [ P ( 1 ) ( u ; g ; a ( 1 ) ; v ( 1 ) ) ] ∂ Ω x } α 0 to a residual term denoted as
{ [ P ^ ( 1 ) ( u ; g ; λ ; ω ; a ( 1 ) ; δ g ; δ λ ; δ ω ) ] ∂ Ω x } α 0 , which will contain boundary terms involving only known values of δ g , δ λ ; δ ω ; g , u , and a ( 1 ) The residual term { [ P ^ ( 1 ) ( u ; g ; λ ; ω ; a ( 1 ) ; δ g ; δ λ ; δ ω ) ] ∂ Ω x } α 0 is linear in δ g , δ λ ; δ ω , and
Equations (90) and (91) are called the 1st-Level Adjoint Sensitivity System (1st-LASS). The solution, a ( 1 ) ( x ) ∈ H 1 ( Ω x ) , of the 1st-LASS is called the 1st-level adjoint function.
The results obtained in (89) and (90) are now replaced in (78) to obtain the following expression of the indirect-effect term as a function of a ( 1 ) ( x ) :
Replacing in Equation (72) the result obtained in Equation (93) together with the expression for the direct-effect term provided in Equation (74) yields the following expression for the first-order G-differential of the response R [ u ( x ) ; f ( α ) ] :
where R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; f ( α ) ] ≜ ∂ R [ u ( x ) ; f ( α ) ] / ∂ f j 1 denotes the 1st-Order sensitivity of the response R [ u ( x ) ; f ( α ) ] with respect to the components f j 1 of the “feature” function f ( α ) ≜ [ g ( α ) ; h ( α ) ; λ ( α ) ; ω ( α ) ] † .
Each1st-order sensitivity R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; f ( α ) ] is obtained by identifying the expression that multiplies the corresponding component of the “feature” variations δ g , δ h , δ λ , and δ ω , respectively, on the right-side of Equation (94). After having obtained the 1st-level adjoint function a ( 1 ) ( x ) ∈ H 1 , each of the 1st-order sensitivities R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; f ( α ) ] of the response R [ u ( x ) ; f ( α ) ] with respect to the components of the functions g , h , λ , and ω can be computed inexpensively by using quadrature formulas to evaluate the inner products involving a ( 1 ) ( x ) ∈ H 1 in Equation . The only large-scale computation needed for obtaining all of the first-order sensitivities is the computation of the 1st-Level adjoint function a ( 1 ) ( x ) ∈ H 1 , which is obtained by solving numerically the 1st-Level Adjoint Sensitivity System (1st-LASS) represented by Equations (90) and (91).
It is very important to note that the 1st-LASS is independent of all variations δ g , δ h , δ λ , and δ ω (or, equivalently, of all parameter variations δ α j 1 , j 1 = 1 , ⋯ , T P ) and therefore needs to be solved only once, regardless of the number of model parameters under consideration. Furthermore, since the 1st-LASS is linear in a ( 1 ) ( x ) , solving it requires less computational effort than solving the original model, which is nonlinear in u ( x ) . The first-order sensitivity R ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; f ( α ) ] can be represented formally in the following integral form:
The functions S ( 1 ) [ j 1 ; u ( x ) ; a ( 1 ) ( x ) ; f ( α ) ] are subsequently used for determining the exact expressions of the second-order sensitivities of the response with respect to the components of the function f ( α ) of model parameters.
ReferencesCukier, R.I., Levine, H.B. and Shuler, K.E. (1978) Nonlinear Sensitivity Analysis of Multiparameter Model Systems. Journal of Computational Physics, 26, 1-42. https://doi.org/10.1016/0021-9991(78)90097-9 10.1016/0021-9991(78)90097-9https://doi.org/10.1016/0021-9991(78)90097-9Cukier, R.I.Levine, H.B.Shuler, K.E.1978Nonlinear Sensitivity Analysis of Multiparameter Model SystemsJournal of Computational Physics99917810.1016/0021-9991(78)90097-9Hora, S.C. and Iman, R.L. (1986) A Comparison of Maximum/Bounding and Bayesian/Monte Carlo for Fault Tree Uncertainty Analysis. Technical Report SAND85-2839, Sandia National Laboratories, Albuquerque. https://doi.org/10.2172/5824798 10.2172/5824798https://doi.org/10.2172/5824798Hora, S.C.Iman, R.L.Laboratories, A1986A Comparison of Maximum/Bounding and Bayesian/Monte Carlo for Fault Tree Uncertainty AnalysisTechnical Report SAND85-283910.2172/5824798Sobol, I.M. (1993) Sensitivity Analysis for Nonlinear Mathematical Models. Ath. Model.Comput. Exp, 1, 407-414.Sobol, I.M.1993Sensitivity Analysis for Nonlinear Mathematical ModelsAth. Model. Comput. Exp1Iman, R.L., Helton, J.C. and Campbell, J.E. (1981) An Approach to Sensitivity Analysis of Computer Models, Part 1. Introduction, Input Variable Selection and Preliminary Variable Assessment. Journal of Quality Technology, 13, 174-183. https://doi.org/10.1080/00224065.1981.11978748 10.1080/00224065.1981.11978748https://doi.org/10.1080/00224065.1981.11978748Iman, R.L.Helton, J.C.Campbell, J.E.Models, PIntroduction, I1981An Approach to Sensitivity Analysis of Computer Models, Part 1Introduction1310.1080/00224065.1981.11978748Iman, R.L., Helton, J.C. and Campbell, J.E. (1981) An Approach to Sensitivity Analysis of Computer Models, Part 2. Ranking of Input Variables, Response Surface Validation, Distribution Effect and Technique Synopsis. Journal of Quality Technology, 13, 232-240. https://doi.org/10.1080/00224065.1981.11978763 10.1080/00224065.1981.11978763https://doi.org/10.1080/00224065.1981.11978763Iman, R.L.Helton, J.C.Campbell, J.E.Models, PVariables, RValidation, D1981An Approach to Sensitivity Analysis of Computer Models, Part 2Ranking of Input Variables1310.1080/00224065.1981.11978763Rios Insua, D. (1990) Sensitivity Analysis in Multiobjective Decision Making. In: Rios Insua, D., Eds., Sensitivity Analysis in Multi- Objective Decision Making, Springer, Berlin, 74-126. https://doi.org/10.1007/978-3-642-51656-6_3 10.1007/978-3-642-51656-6_3https://doi.org/10.1007/978-3-642-51656-6_3Insua, D.Insua, D.Making, S1990Sensitivity Analysis in Multiobjective Decision MakingIn: Rios Insua7410.1007/978-3-642-51656-6_3Saltarelli, A., Chan, K. and Scott, E.M. (2000) Sensitivity Analysis. John Wiley & Sons, Chichester.Saltarelli, A.Chan, K.Scott, E.M.Sons, C2000Sensitivity AnalysisJohn Wiley & SonsKramer, M.A., Calo, J.M. and Rabitz, H. (1981) An Improved Computational Method for Sensitivity Analysis: Green’s Function Method with “AIM”. Applied Mathematical Modelling, 5, 432-442 https://doi.org/10.1016/S0307-904X(81)80027-3 10.1016/S0307-904X(81)80027-3https://doi.org/10.1016/S0307-904X(81)80027-3Kramer, M.A.Calo, J.M.Rabitz, H.1981An Improved Computational Method for Sensitivity Analysis: Green’s Function Method with “AIM”Applied Mathematical Modelling510.1016/S0307-904X(81)80027-3Cacuci, D.G. (1981) Sensitivity Theory for Nonlinear Systems: I. Nonlinear Functional Analysis Approach. Journal of Mathematical Physics, 22, 2794-2802. https://doi.org/10.1063/1.525186 10.1063/1.525186https://doi.org/10.1063/1.525186Cacuci, D.G.1981Sensitivity Theory for Nonlinear Systems: INonlinear Functional Analysis Approach. Journal of Mathematical Physics2210.1063/1.525186Dunker, A.M. (1984) The Decoupled Direct Method for Calculating Sensitivity Coefficients in Chemical Kinetics. The Journal of Chemical Physics, 81, 2385-2393. https://doi.org/10.1063/1.447938 10.1063/1.447938https://doi.org/10.1063/1.447938Dunker, A.M.1984The Decoupled Direct Method for Calculating Sensitivity Coefficients in Chemical KineticsThe Journal of Chemical Physics8110.1063/1.447938Bellman, R.E. (1957) Dynamic Programming. Rand Corporation, Princeton University Press, Princeton.Bellman, R.E.Corporation, PPress, P1957Dynamic ProgrammingRand CorporationWigner, E.P. (1945) Effect of Small Perturbations on Pile Period. Chicago Report CP-G-3048, Chicago.Wigner, E.P.1945Effect of Small Perturbations on Pile PeriodChicago Report CP-G-3048Weiberg, A.M. and Wigner, E.P. (1958) The Physical Theory of Neutron Chain Reactors. University of Chicago Press, Chicago.Weiberg, A.M.Wigner, E.P.Press, C1958The Physical Theory of Neutron Chain ReactorsUniversity of Chicago PressWeisbin, C.R., et al. (1978) Application of Sensitivity and Uncertainty Methodology to Fast Reactor Integral Experiment Analysis. Nuclear Science and Engineering, 66, 307-333. https://doi.org/10.13182/NSE78-3 10.13182/NSE78-3https://doi.org/10.13182/NSE78-3Weisbin, C.R.1978Application of Sensitivity and Uncertainty Methodology to Fast Reactor Integral Experiment AnalysisNuclear Science and Engineering6610.13182/NSE78-3Williams, M.L. (1986) Perturbation Theory for Nuclear Reactor Analysis. In: Ronen, Y., Ed., Handbook of Nuclear Reactor Calculations, CRC Press, Boca Raton, 63-188.Williams, M.L.Ronen, Y.Calculations, CPress, B1986Perturbation Theory for Nuclear Reactor AnalysisIn: Ronen63Shultis, J.K. and Faw, R.E. (2000) Radiation Shielding. American Nuclear Society, Illinois.Shultis, J.K.Faw, R.E.Society, I2000Radiation ShieldingAmerican Nuclear SocietyStacey, W.M. (2001) Nuclear Reactor Physics. John Wiley & Sons, New York.Stacey, W.M.Sons, N2001Nuclear Reactor PhysicsJohn Wiley & SonsPráger, T. and Kelemen, F.D. (2014) Adjoint Methods and Their Application in Earth Sciences. In: Faragó, I., Havasi, Á. and Zlatev, Z., Eds., Advanced Numerical Methods for Complex Environmental Models: Needs and Availability, Bentham Science Publishers, Oak Park, 203-275. https://doi.org/10.2174/9781608057788113010011 10.2174/9781608057788113010011https://doi.org/10.2174/9781608057788113010011Kelemen, F.D.Zlatev, Z.Availability, BPublishers, O2014Adjoint Methods and Their Application in Earth SciencesIn: Faragó20310.2174/9781608057788113010011Luo, Z., Wang, X. and Liu, D. (2020) Prediction on the Static Response of Structures with Large-Scale Uncertain-But-Bounded Parameters Based on the Adjoint Sensitivity Analysis. Structural and Multidisciplinary Optimization, 61, 123-139. https://doi.org/10.1007/s00158-019-02349-w 10.1007/s00158-019-02349-whttps://doi.org/10.1007/s00158-019-02349-wLuo, Z.Wang, X.Liu, D.2020Prediction on the Static Response of Structures with Large-Scale Uncertain-But-Bounded Parameters Based on the Adjoint Sensitivity AnalysisStructural and Multidisciplinary Optimization6110.1007/s00158-019-02349-wCacuci, D.G. (2015) Second-Order Adjoint Sensitivity Analysis Methodology for Computing Exactly and Efficiently First-and Second-Order Sensitivities in Large-Scale Linear Systems: I. Computational Methodology. Journal of Computational Physics, 284, 687-699. https://doi.org/10.1016/j.jcp.2014.12.042 10.1016/j.jcp.2014.12.042https://doi.org/10.1016/j.jcp.2014.12.042Cacuci, D.G.2015Second-Order Adjoint Sensitivity Analysis Methodology for Computing Exactly and Efficiently First-and Second-Order Sensitivities in Large-Scale Linear Systems: IComputational Methodology. Journal of Computational Physics28410.1016/j.jcp.2014.12.042Cacuci, D.G. (2016) Second-Order Adjoint Sensitivity Analysis Methodology for Large-Scale Nonlinear Systems: I. Theory. Nuclear Science and Engineering, 184, 16-30. https://doi.org/10.13182/NSE16-16 10.13182/NSE16-16https://doi.org/10.13182/NSE16-16Cacuci, D.G.2016Second-Order Adjoint Sensitivity Analysis Methodology for Large-Scale Nonlinear Systems: ITheory. Nuclear Science and Engineering18410.13182/NSE16-16Cacuci, D.G. and Fang, R. (2023) The n th-Order Comprehensive Adjoint Sensitivity Analysis Methodology (nth-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume II: Application to a Large-Scale System. Springer Nature Switzerland, Cham.Cacuci, D.G.Fang, R.Analysis, VSwitzerland, C2023The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (nth-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume II: Application to a Large-Scale SystemSpringer Nature SwitzerlandValentine, T.E. (2006) Polyethylene-Reflected Plutonium Metal Sphere Subcritical Noise Measurements, SUB-PU-METMIXED-001. International Handbook of Evaluated Criticality Safety Benchmark Experiments, NEA/NSC/DOC(95)03/I-IX, Organization for Economic Co-Operationand Development, Nuclear Energy Agency, Paris.Valentine, T.E.Measurements, SExperiments, NI-IX, ODevelopment, NAgency, P2006Polyethylene-Reflected Plutonium Metal Sphere Subcritical Noise Measurements, SUB-PU-METMIXED-001International Handbook of Evaluated Criticality Safety Benchmark ExperimentsAlcouffe, R.E., Baker, R.S., Dahl, J.A., Turner, S.A. and Ward, R. (2008) PARTISN: A Time-Dependent, Parallel Neutral Particle Transport Code System. LA-UR-08-07258, Los Alamos National Laboratory, Los Alamos.Alcouffe, R.E.Baker, R.S.Dahl, J.A.Turner, S.A.Ward, R.Time-Dependent, PLaboratory, L2008PARTISN: A Time-Dependent, Parallel Neutral Particle Transport Code SystemLA-UR-08-07258Conlin, J.L., Parsons, D.K., Gardiner, S.J., Gray, M., Lee, M.B. and White, M.C. (2013) MENDF71X: Multigroup Neutron Cross-Section Data Tables Based upon ENDF/B-VII.1X. Los Alamos National Laboratory Report LA-UR-15-29571, Los Alamos National Laboratory, Los Alamos. https://doi.org/10.2172/1063914 10.2172/1063914https://doi.org/10.2172/1063914Conlin, J.L.Parsons, D.K.Gardiner, S.J.Gray, M.Lee, M.B.White, M.C.Laboratory, L2013MENDF71X: Multigroup Neutron Cross-Section Data Tables Based upon ENDF/B-VII1X. Los Alamos National Laboratory Report LA-UR-15-2957110.2172/1063914Chadwick, M.B., Herman, M., Obložinský, P., Dunn, M.E., Danon, Y., Kahler, A.C., Smith, D.L., Pritychenko, B., Arbanas, G., Brewer, R., et al. (2011) ENDF/B-VII.1: Nuclear Data for Science and Technology: Cross Sections, Covariances, Fission Product Yields and Decay Data. Nuclear Data Sheets, 112, 2887-2996. https://doi.org/10.1016/j.nds.2011.11.002 10.1016/j.nds.2011.11.002https://doi.org/10.1016/j.nds.2011.11.002Chadwick, M.B.Herman, M.Dunn, M.E.Danon, Y.Kahler, A.C.Smith, D.L.Pritychenko, B.Arbanas, G.Brewer, R.Sections, C2011ENDF/B-VII1: Nuclear Data for Science and Technology: Cross Sections11210.1016/j.nds.2011.11.002Wilson, W.B., Perry, R.T., Shores, E.F., Charlton, W.S., Parish, T.A., Estes, G.P., Brown, T.H., Arthur, E.D., Bozoian, M., England, T.R., et al. (2002) SOURCES4C: A Code for Calculating (α, n), Spontaneous Fission, and Delayed Neutron Sources and Spectra. Proceedings of the American Nuclear Society/ Radiation Protection and Shielding Division 12 th Biennial Topical Meeting, Santa Fe, 14-18 April 2002.Wilson, W.B.Perry, R.T.Shores, E.F.Charlton, W.S.Parish, T.A.Estes, G.P.Brown, T.H.Arthur, E.D.Bozoian, M.England, T.R.Meeting, S2002SOURCES4C: A Code for Calculating (α, n), Spontaneous Fission, and Delayed Neutron Sources and SpectraProceedings of the American Nuclear Society/Radiation Protection and Shielding Division 12th Biennial Topical Meeting14Cacuci, D.G. (2022) The n th-Order Comprehensive Adjoint Sensitivity Analysis Methodology (n th-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume I: Linear Systems. Springer, New York.Cacuci, D.G.Analysis, VSpringer, N2022The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (nth-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume I: Linear SystemsSpringerWilliams, M.L. and Engle, W.W. (1977) The Concept of Spatial Channel Theory Applied to Reactor Shielding Analysis. Nuclear Science and Engineering, 62, 92-104. https://doi.org/10.13182/NSE77-A26941 10.13182/NSE77-A26941https://doi.org/10.13182/NSE77-A26941Williams, M.L.Engle, W.W.1977The Concept of Spatial Channel Theory Applied to Reactor Shielding AnalysisNuclear Science and Engineering6210.13182/NSE77-A26941Cacuci, D.G. (2023) The n th-Order Comprehensive Adjoint Sensitivity Analysis Methodology (nth-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume III: Nonlinear Systems. Springer Nature Switzerland, Cham.Cacuci, D.G.Analysis, VSwitzerland, C2023The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (nth-CASAM): Overcoming the Curse of Dimensionality in Sensitivity and Uncertainty Analysis, Volume III: Nonlinear SystemsSpringer Nature SwitzerlandCacuci, D.G. (2023) Computation of High-Order Sensitivities of Model Responses to Model Parameters. II: Introducing the Second-Order Adjoint Sensitivity Analysis Methodology for Computing Response Sensitivities to Functions/Features of Parameters. Energies, 16, Article 6356. https://doi.org/10.3390/en16176356 10.3390/en16176356https://doi.org/10.3390/en16176356Cacuci, D.G.2023Computation of High-Order Sensitivities of Model Responses to Model ParametersII: Introducing the Second-Order Adjoint Sensitivity Analysis Methodology for Computing Response Sensitivities to Functions/Features of Parameters. Energies16635610.3390/en16176356Hetrick, D.L. (1993) Dynamics of Nuclear Reactors. American Nuclear Society, Inc., La Grange Park, 164-174.Hetrick, D.L.Society, I1993Dynamics of Nuclear ReactorsAmerican Nuclear Society164Lamarsh, J.R. (1966) Introduction to Nuclear Reactor Theory. Adison-Wesley Publishing Co., Reading, 491-492.Lamarsh, J.R.1966Introduction to Nuclear Reactor TheoryAdison-Wesley Publishing Co.491Cacuci, D.G. (2024) Introducing the n th-Order Features Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (n th-FASAM-N): II. Illustrative Application.Cacuci, D.G.2024Introducing the nth-Order Features Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (nth-FASAM-N): IITukey, J.W. (1957) The Propagation of Errors, Fluctuations and Tolerances. Technical Reports No. 10-12, Princeton University, Princeton.Tukey, J.W.Errors, FUniversity, P1957The Propagation of Errors, Fluctuations and TolerancesTechnical Reports No. 10-12